Set up Screaming Frog and ScrapeBox with ProxyField
- In Screaming Frog, open File → Settings → Proxy (on macOS, Screaming Frog SEO Spider → Settings → Proxy) and turn the proxy on.
- Enter
gate.proxyfield.io, port7000, your username and password, click OK and restart the SEO Spider. - Crawl one URL and check that it returns 200, then set Max URI/s and threads under Configuration → Speed.
- In ScrapeBox, open the proxy manager (Manage), load the line from the clipboard and test it.
- Tick Use Proxies next to the harvester and run your search and check tasks.
The line for ScrapeBox's proxy manager, in its host:port:username:password form, here for a Canberra IP held for 60 seconds.
gate.proxyfield.io:7000:<login>-country-au-region-australian_capital_territory-city-canberra-session-he9f2qga-ttl-60:<password>Screaming Frog's own guide to the proxy setting: SEO Spider Configuration: Proxy (Screaming Frog).
| Tool | What it accepts | ProxyField setup | Rotate or stick |
|---|---|---|---|
| Screaming Frog SEO Spider | 1 proxy server, HTTP, with a login | host gate.proxyfield.io, port 7000, your username and password; with the IP allowlist, the port you set up in the dashboard | a new IP per request for public sites; a sticky session (30 s to 36 h) for your own site |
| ScrapeBox | HTTP and SOCKS proxies, host:port:user:pass | gate. | a new IP per request for harvesting and checks |
Labels can differ slightly between versions.
On this page 8 sections
One gateway line for every SEO tool
| Field | ProxyField value |
|---|---|
| Line format | host:: gate. |
| Host and port | gate.proxyfield.io, 7000 with username and password; with the IP allowlist, the port you set up in the dashboard |
| Login | your username with the location, as in <login>-country-us-city-new_, and your password; or your IP on the allowlist. Outside the US, the two-letter country code in lowercase: -country-de |
| Protocols | HTTP, HTTPS and SOCKS5 on the same host, port, username and password |
| IP behaviour | a new IP on every request, or a sticky session from 30 s to 36 h (20 min without ttl) |
| Threads | unlimited concurrent threads |
| Targeting | country, region, city, at the same price per GB |
Region, session and session time go in the same username: -region-<region>, -session-<id> and -ttl-<seconds> (30 to 129,600 seconds), as in <login>-country-us-region-new_york-city-new_york-session-he9f2qga-ttl-600; every part after the login is optional. Residential is the default; add -mobile after the login for mobile IPs: <login>-mobile-country-us. Your credentials, the allowlist and the configurator (it generates as many proxy lines as you need) are in your ProxyField account.
Screaming Frog: set up the proxy
- Open the proxy settings: File → Settings → Proxy on Windows and Linux, Screaming Frog SEO Spider → Settings → Proxy on macOS (Configuration → System → Proxy in older versions).
- Turn on the proxy server option and enter
gate.proxyfield.ioand port7000(with the IP allowlist, the port you set up in the dashboard).Screenshot Screaming Frog proxy settings with gate.proxyfield.io and port 7000 - Enter your ProxyField username and password. With your IP on the allowlist, leave the login empty.
- Click OK and restart the SEO Spider: the proxy setting takes effect only after a restart.
- Crawl one URL and check that it returns 200 before you start the full crawl.
Read moreShow less
Screaming Frog takes only one proxy server, and it has no exceptions: either all HTTP and HTTPS traffic goes through the proxy or none does. That one server is the ProxyField gateway, and the gateway gives each request a new IP, so a single entry spreads a crawl over many IPs.
Check the connection
| What you see | What it means | Fix |
|---|---|---|
| The first URL returns 200 | Requests go through the proxy | Start the crawl |
| An authentication error on every URL | Wrong username or password, or a typo in the location part | Copy the credentials from your ProxyField account again; with the allowlist, check that this machine's IP is on it |
| Every URL times out at once | Wrong host or port, or the proxy setting not applied | gate.proxyfield.io, port 7000 for username and password, or the port you set up in the dashboard for the allowlist; restart the SEO Spider |
| 429 or 503 that grow during the crawl | The site limits the crawl rate | Lower Max URI/s and threads |
| 403 on your own site | Your firewall or CDN blocks the crawl | Crawl from one sticky session and allowlist it, or ask the site owner to allow the crawl |
| Pages in the wrong country or language | No country in the username | Add -country-<cc> to the username |
Gateway error messages: 407 Proxy Authentication Required for a wrong login or password, an IP not on the allowlist or a malformed location, 402 Payment Required when no traffic is left and 502 Bad Gateway when no exit could be connected; a restricted site gets no separate response, and a 403 comes from the site itself. Status codes are explained in proxy errors. To test a line outside Screaming Frog, use the proxy checker (up to 100 proxies per check; your list and results are not stored).
Screaming Frog on a large site
The proxy decides where requests come from; the speed settings decide how hard you hit the site. For a large crawl:
| Setting | Where | Start with |
|---|---|---|
| Max URI/s | Configuration → Speed | 1 URL per second; raise it only while 429 and 503 stay near zero |
| Max threads | Configuration → Speed | the default 5; the gateway takes unlimited threads, but the site sets the pace |
| Storage mode | File → Settings → Storage Mode | database storage on an SSD (the default); memory mode suits sites under 500k URLs |
| Memory | File → Settings → Memory Allocation | at least 2 GB below your machine's physical memory |
| Scope | Configuration → Include / Exclude | crawl the site in sections |
| Sessions | the proxy username | a new IP per request for a public site; -session-<id>-ttl-<seconds> (up to 129,600 s, 36 h) when the site ties a crawl to one IP |
Traffic is billed per GB, with no fee per thread or per proxy. A crawl of HTML pages uses far less traffic than one that renders JavaScript and loads images, so turn off image and resource crawling you do not need. An IP in a sticky session that sits idle for 120 seconds is replaced, and when its node drops the IP may change, so a resumable crawl is safer than one long session.
ScrapeBox: harvester, keyword scraper and checks
- In ScrapeBox, open the proxy manager (Manage).
- Load the proxy from the clipboard in ScrapeBox's
host:port:user:passform:gate.proxyfield.io:7000:<login>:<password>.Screenshot ScrapeBox proxy manager with the ProxyField gateway line loaded - Test it in the proxy manager and keep it when it passes.
- Tick Use Proxies next to the harvester, and set the connections in the harvester settings.
- Run the harvester, keyword scraper, index checker or link checker; each request leaves from a new IP.
ScrapeBox takes HTTP and SOCKS proxies with a login. HTTP with your username and password covers its search and check tasks; SOCKS5 runs on the same gate.proxyfield.io:7000 with the same login. ScrapeBox's proxy harvester collects free public proxies; they are shared and slow, and harvests that run on them often fail. A paid gateway removes the step of harvesting and testing lists.
Keep ScrapeBox's comment poster and other posting add-ons off ProxyField proxies: ScrapeBox used for spam is not allowed.
Tools the acceptable use policy does not allow
ProxyField's acceptable use policy does not allow spam tools: GSA Search Engine Ranker, and ScrapeBox used for spam. It also rules out spam, fake accounts and fake engagement. So this page sets up Screaming Frog, and ScrapeBox for search and check tasks, and ProxyField proxies are not for GSA Search Engine Ranker. For rank tracking and search research across regions, see SEO proxies.
Which proxies to use with SEO software
| Task | Proxy | Price |
|---|---|---|
| ScrapeBox harvesting, keyword scrapes, index and rank checks | Rotating residential | $2.95/GB for 1 GB down to $1.95/GB from 1,000 GB |
| Mobile rankings and mobile versions of pages | Mobile 5G/4G | $3.45/GB for 1 GB down to $2.45/GB from 1,000 GB |
| A Screaming Frog crawl that must come from one fixed IP | Dedicated ISP | from $2.29 per IP per 30 days |
| Screaming Frog crawls of sites that accept datacenter IPs, such as your own | Rotating datacenter | $0.29/GB down to $0.22/GB |
Residential and mobile share one balance; per-IP products are 30-day terms you renew in the dashboard. The pool is clean for Google, checked and segmented.
If you need Google results rather than proxies, the Google SERP API is designed to return the results page as JSON, with the AI Overview, the People Also Ask block and the local pack parsed, at $1.20 per 1,000 successful results, or $0.80 from 10,000 results in one purchase. It is in development: see the API documentation and Contact sales. To look at one query by hand, use the SERP checker.
Get started
- Sign up with your email and buy the Easy Trial: 1 GB for $2.95, in USDT, BTC, ETH or 10 other coins; there is no minimum purchase and no subscription.
- Copy your login, or add your IP to the allowlist, in your ProxyField account; access opens once the payment is confirmed.
- Enter
gate.proxyfield.io:7000with your username and password in Screaming Frog or ScrapeBox as above; with the IP allowlist, entergate.proxyfield.ioand the port you set up in the dashboard.
The traffic never expires and all purchases are final. Our 24/7 live chat helps with setup and integration. See pricing per GB.
Proxies for Screaming Frog and ScrapeBox FAQ
Which proxies work with SEO tools like ScrapeBox or Screaming Frog?
ProxyField rotating residential proxies work with ScrapeBox and Screaming Frog from $1.95/GB ($2.95 for 1 GB): one gateway entry, gate.proxyfield.io on port 7000 with your login, gives each request a new IP. Use mobile IPs for mobile rankings, a dedicated ISP IP (from $2.29 per 30 days) when a crawl must come from one address, and rotating datacenter ($0.29/GB) for sites that accept datacenter IPs.
How do I configure proxies in Screaming Frog for crawling a large site?
Enter gate.proxyfield.io and port 7000 under File → Settings → Proxy, add your login and restart the SEO Spider. Then start at 1 URL per second and the default 5 threads under Configuration → Speed, use database storage on an SSD, and raise the speed only while 429 and 503 responses stay near zero.
How do I set up a proxy in Screaming Frog?
Open File → Settings → Proxy on Windows or Linux, or Screaming Frog SEO Spider → Settings → Proxy on a Mac, turn the proxy on, and enter the host, port, username and password. Click OK and restart: the setting works only after a restart.
Does Screaming Frog support rotating proxies?
Screaming Frog takes one proxy server, not a list. A rotating gateway fills that one slot: ProxyField's gate.proxyfield.io:7000 gives every request a new IP, so the crawl is spread over many IPs.
Can I add proxy exceptions in Screaming Frog?
No. Either all HTTP and HTTPS traffic goes through the proxy or none does. To crawl a site without the proxy, turn the proxy setting off and restart.
Why is Screaming Frog not using my proxy?
Most often the SEO Spider was not restarted after the proxy was saved. If it was, check the host gate.proxyfield.io, the port 7000 and your login, and crawl one URL to see whether it returns 200.
What proxy should I use for Screaming Frog?
An authenticated HTTP proxy. ProxyField rotating residential from $1.95/GB suits crawls of public sites from another country; rotating datacenter from $0.29/GB suits sites that accept datacenter IPs, such as your own.
Should a Screaming Frog crawl keep one IP?
Only when the site ties the crawl to one IP, for example your own site behind a firewall. Then add -session-<id>-ttl-<seconds> to the username for up to 36 hours, or use a dedicated ISP IP from $2.29 per 30 days.
Can I use ProxyField proxies with GSA Search Engine Ranker?
No. ProxyField's acceptable use policy does not allow spam tools, and GSA Search Engine Ranker is one of them. Screaming Frog, and ScrapeBox for search and check tasks, are set up on this page.
Why do I need a proxy for ScrapeBox?
The harvester, keyword scraper and index checker send many queries to search engines, which challenge or block one busy IP. Proxies spread those queries across many IPs and countries.
Show 10 more questionsShow fewer questions
What are the best ScrapeBox proxies?
ProxyField rotating residential proxies, $2.95/GB for 1 GB down to $1.95/GB, suit ScrapeBox's harvester and checks: one line, a new IP per request. Rotating datacenter at $0.29/GB down to $0.22/GB costs less per GB and fits sites that accept datacenter IPs.
Where can I buy ScrapeBox proxies?
ProxyField sells them per GB with no subscription: start with the Easy Trial, 1 GB for $2.95, paid in USDT, BTC, ETH or 10 other coins. Access opens once the payment is confirmed, and the traffic never expires.
How many proxies do I need for ScrapeBox?
With ProxyField's rotating gateway, one line covers all connections, each with its own IP. Set the connection count in ScrapeBox by how fast the search engine tolerates queries, not by proxy count.
Should I use rotating or static proxies for ScrapeBox?
Rotating for harvesting and checks, where each query should come from a new IP. A sticky session (30 s to 36 h) or a dedicated ISP IP only helps a task that must keep one address for a sequence of requests.
Can I use private proxies with ScrapeBox?
Yes. ScrapeBox loads any proxy with a login in host:port:user:pass form. ProxyField needs your login or an allowlisted IP, and dedicated ISP IPs cost from $2.29 per IP per 30 days if you need fixed ones.
Where should ScrapeBox get its proxies from?
From a paid provider rather than ScrapeBox's proxy harvester: harvested public proxies are shared and slow, and lists of them have to be tested and refreshed. A paid gateway is one line with a login.
Does ScrapeBox support SOCKS5 proxies?
Yes, ScrapeBox takes SOCKS proxies as well as HTTP ones. ProxyField runs HTTP, HTTPS and SOCKS5 on the same gate.proxyfield.io:7000 with the same username and password.
What if my ScrapeBox proxies get blocked?
Slow down: fewer connections and a delay between queries to the same engine. With a rotating gateway, a blocked IP is replaced on the next request, so persistent blocks point to the query rate, not the proxies.
What is ScrapeBox used for?
Collecting URLs and keywords from search engines, checking indexing, links and rankings, and other bulk SEO tasks. It also has posting add-ons; keep them off ProxyField proxies, since ScrapeBox used for spam is not allowed.
What is Screaming Frog used for?
Crawling websites for technical SEO audits: broken links, redirects, titles, meta descriptions, canonicals and more. A proxy lets it crawl from another country or spread a large crawl over many IPs.