Proxies for AI Agents

Residential is the usual choice for an agent that browses many different sites. Residential and mobile share one balance, so one agent can use both: residential for research, mobile for the few targets that need it. The API prices are per 1,000 successful results; the volume price applies from 10,000 results in one purchase.

A provider's success rate holds only for the targets it was measured on, so test on yours: 100 URLs, a pass mark set in advance, about 200 MB of traffic for a full browser run. The gap between an easy target and a real one is large. In Proxyway's Proxy Market Research 2026 (13 providers, data collected mostly in March and April 2026), the median residential success rate was 99.28% on a test page that does not block requests, and 74.43% on popular targets (Amazon, Google and Instagram); the median response time rose from 0.93 s to 3.21 s. An agent's targets look like the second test, not the first. Proxyway, a benchmark site, publishes named results every year: Proxy Market Research 2026, Web Scraping API Report 2025 and Search APIs in 2026: Overview & Benchmarks (titles as listed on its research page). ProxyField's own figure is a 99.6% success rate, according to our service data; your agent's sites decide what you get, so run the test.

1. Put 100 URLs your agent really visits in agent_urls.txt, spread over the sites it uses most.
2. Set your pass mark before you run, for example 95 of 100 pages with status 200 and the content you expect.
3. Run the script below once per provider, with the same URLs and at the same hour.
4. Compare the pass count, the median time, and the traffic the provider's dashboard shows for the run.

import statistics, time, requests

PROXY = "http://<login>-country-us:<password>@gate.proxyfield.io:7000"
urls = [u.strip() for u in open("agent_urls.txt") if u.strip()][:100]
ok, times, body_bytes = 0, [], 0
for url in urls:
    t = time.monotonic()
    try:
        r = requests.get(url, proxies={"http": PROXY, "https": PROXY}, timeout=30)
        times.append(time.monotonic() - t)
        body_bytes += len(r.content)
        if r.status_code == 200 and "<title" in r.text.lower():   # replace with your own content check
            ok += 1
    except requests.RequestException as e:
        print("failed:", url, type(e).__name__)
median = statistics.median(times) if times else 0
print(f"{ok}/{len(urls)} passed, median {median:.2f} s, {body_bytes / 1e6:.1f} MB of page bodies")

The last line prints <passed>/100 passed, median <seconds> s, <MB> MB of page bodies: compare the first number with the pass mark you set in step 2. The run fetches HTML only; at about 2 MB per full page, a browser run of the same 100 URLs is about 200 MB, about $0.59 at $2.95/GB, well inside the 1 GB Easy Trial. ProxyField's dashboard shows usage and remaining gigabytes in real time.

The choice is how much of the fetching you want to own:

• Proxies: your agent's HTTP tool or browser connects through the gateway, and you write the rendering, retries and parsing yourself. You get full control and pay per GB.
• A scraper API: the agent sends a URL and gets the page back. The API handles IPs, retries and, if you ask for it, JavaScript rendering, and you pay per successful result.

What 1,000 pages cost each way, at about 2 MB per full page (an example size, not a measurement):

Proxies charge for bytes, so a page that loads only HTML costs a fraction of a full page; the API charges per result, so page weight does not change the bill, and blocked or timed-out requests are not billed.

ProxyField's Web Scraping API is in development. It is priced at $0.50 per 1,000 successful results over plain HTTP ($0.20 from 10,000 results in one purchase) and $2.00 with JavaScript rendering ($1.00 from 10,000), billed on successful results only. Its design is in the API documentation, and access is through Contact sales. The full comparison: scraper API vs proxies.

An agent's web requests look like a scraper's to the websites it visits, and usually like an unusual one:

• Server IPs. Agents run on cloud servers, and many sites rate-limit or challenge cloud-provider IP ranges on sight.
• Bursts. An agent may open 20 pages in parallel to answer one question, far beyond a person's pace.
• Shared addresses. Many agents, or many users of one agent product, leave from the same few IPs, which quickly earn a bad reputation.
• Wrong location. An agent in a US data center sees US prices, results and availability, even when the user asked about Germany.

A proxy fixes the address and the location. It does not fix bursts or a headless browser's tells; those need pacing and a well-configured browser (see what proxies can and cannot fix).

The model never touches the website; tools do. Set the proxy on each tool that reaches the web, and leave the rest alone.

Two minimal setups, with the gateway and a username example (an exit in New York City, United States):

# HTTP tool: every call leaves from a new residential IP in New York
import requests

PROXY = "http://<login>-country-us-city-new_york:<password>@gate.proxyfield.io:7000"

def fetch_page(url: str) -> str:
    r = requests.get(url, proxies={"http": PROXY, "https": PROXY}, timeout=30)
    r.raise_for_status()
    return r.text

# Browser tool: one sticky IP in New York for a 10-minute task
from playwright.sync_api import sync_playwright

with sync_playwright() as p:
    browser = p.chromium.launch(proxy={
        "server": "http://gate.proxyfield.io:7000",
        "username": "<login>-country-us-city-new_york-session-task42-ttl-600",
        "password": "<password>",
    })
    page = browser.new_page()
    page.goto("https://api.ipify.org")
    print(page.inner_text("body"))
    browser.close()

The same for an agent written in Node, with Puppeteer and one sticky IP in Germany:

const puppeteer = require("puppeteer");

(async () => {
  const browser = await puppeteer.launch({ args: ["--proxy-server=http://gate.proxyfield.io:7000"] });
  const page = await browser.newPage();
  await page.authenticate({ username: "<login>-country-de-session-task43-ttl-600", password: "<password>" });
  await page.goto("https://api.ipify.org");
  console.log(await page.evaluate(() => document.body.innerText));
  await browser.close();
})();

For tools you did not write, such as a local MCP server in Python, set the proxy in the process environment and keep the model's API direct with NO_PROXY (Python requests and httpx read all three variables):

export HTTP_PROXY="http://<login>-country-us:<password>@gate.proxyfield.io:7000"
export HTTPS_PROXY="$HTTP_PROXY"
export NO_PROXY="api.openai.com,api.anthropic.com"

Agent frameworks: where each one takes the proxy

Agent libraries that drive their own browser have their own proxy setting. From each tool's documentation:

# Browser Use: the agent's browser on one sticky US IP for 30 minutes
from browser_use import Agent, Browser
from browser_use.browser import ProxySettings

browser = Browser(proxy=ProxySettings(
    server="http://gate.proxyfield.io:7000",
    username="<login>-country-us-session-agent01-ttl-1800",
    password="<password>",
))
# agent = Agent(task="...", llm=..., browser=browser)

# Crawl4AI: every crawl through a German residential IP
from crawl4ai import AsyncWebCrawler, CrawlerRunConfig, ProxyConfig

run_config = CrawlerRunConfig(proxy_config=ProxyConfig(
    server="http://gate.proxyfield.io:7000",
    username="<login>-country-de",
    password="<password>",
))
# async with AsyncWebCrawler() as crawler:
#     result = await crawler.arun("https://example.com", config=run_config)

{
  "browser": {
    "launchOptions": {
      "proxy": {
        "server": "http://gate.proxyfield.io:7000",
        "username": "<login>-country-us-session-mcp01-ttl-1800",
        "password": "<password>"
      }
    }
  }
}

Save the last block as config.json and start the server with npx @playwright/mcp@latest --config config.json; the agent's browser then opens https://api.ipify.org on the proxy's IP, not your server's.

Expose fetch_page (or the browser) to the agent as a tool in whichever framework you use; the framework does not need to know a proxy exists. The username takes <login>-country-<cc>-region-<region>-city-<city>-session-<id>-ttl-<seconds>, in lowercase with underscores, and every part after the login is optional. Full setup for each library: Playwright, Python, Node.js. How agents reach scraping tools through MCP: web scraping MCP servers.

Match the IP to the task, not to the agent:

• Independent lookups (50 product pages, 20 search results): a new IP on every request spreads the load.
• A task that moves through one site (search, filter, open, add to cart, sign in): one sticky IP for the whole task, so the site sees one visitor. Set a ttl longer than the task, and keep the session busy: an IP that carries no traffic for 120 s is replaced, which matters when the agent stops to call the model.
• Several agents in parallel: give each agent, or each task, its own session ID, and never let two agents share a sticky IP on the same site. That keeps one agent's mistakes from blocking the others.
• Retries: a sticky IP is held while its exit node is up; if the node drops, the IP may change and a broken connection is not restored. Let the agent restart the step on a new IP, and after 2 or 3 failures report the failure instead of looping.

Cap concurrency per website (a few requests at a time per domain) even though threads are unlimited on our side. Sites judge the pace, not your plan.

A web-capable agent usually needs 4 steps, and each has a tool:

The scraper APIs are in development and offered through Contact sales. They are paid from the same crypto balance as proxies, with no subscription.

Websites are setting rules for AI traffic. Cloudflare announced pay per crawl on July 1, 2025, in private beta. A site owner on Cloudflare can allow an AI crawler, charge it per request, or block it. A crawler that has not agreed to pay gets 402 Payment Required with a crawler-price header. A crawler that will pay sends crawler-exact-price on its retry, or crawler-max-price up front; if the site's price is at or below that maximum, the answer is 200 OK with a crawler-charged header (Cloudflare's post). More sites also list AI crawlers by name in robots.txt.

For an agent working for a user, the safe rules are simple:

• respect robots.txt and the site's terms;
• identify your crawler honestly where the site asks;
• keep a human-scale pace;
• do not use proxies to get past a block the site set on purpose.

Collect public pages only. ProxyField's acceptable use policy allows collecting public web data, monitoring and testing, and forbids access to data behind logins you are not entitled to. The legal side is in is AI scraping legal.

A 50-page research task at about 2 MB a page is about 100 MB, which is about $0.30 at $2.95/GB. The 2 MB is an example size, not a measurement, so weigh your own pages.

An agent that fetches HTML or JSON instead of full pages uses a fraction of that. Blocking images and media in the agent's browser makes the same GB go further. The tier rate applies to the whole purchase; prices for every volume are on pricing. Traffic never expires, so an agent that runs in bursts does not waste a monthly plan.