Selenium Proxy: Authenticated Setup in Python and Java

Test the gateway before you touch Selenium: one cURL command tells you whether the credentials and targeting work.

curl -x gate.proxyfield.io:7000 -U "<login>-country-us-city-new_york:<password>" https://api.ipify.org

You should see an IP that is not yours. Check the country with https://ipinfo.io/json or test your proxy in the ProxyField checker. If cURL fails, Selenium will fail too; the cURL setup page covers the errors.

To set up a proxy in Selenium without credentials, add your public IP to the IP allowlist (IP whitelist) in the dashboard, then pass the gateway to Chrome with --proxy-server on the port you set up in the dashboard (<port> in the code). Chrome needs no username, so nothing else is required.

# pip install selenium   (Selenium Manager downloads the driver)
from selenium import webdriver

options = webdriver.ChromeOptions()
options.add_argument("--proxy-server=http://gate.proxyfield.io:<port>")  # <port>: the port you set up in the dashboard
driver = webdriver.Chrome(options=options)
driver.get("https://api.ipify.org?format=json")
print(driver.find_element("tag name", "body").text)
driver.quit()

The allowlist only works from the IP you added. A laptop or a CI runner whose IP changes needs Method 2 or 3. The allowlist takes up to 100 IPv4 addresses per account; with the allowlist, the product, location and session are set per port in the dashboard.

A small Manifest V3 extension sets the proxy and answers Chrome's auth challenge, so the username and password never touch the command line. It uses chrome.proxy and chrome.webRequest.onAuthRequired with the webRequestAuthProvider permission (Chrome webRequest API, Chrome proxy API).

Create a folder pf-auth with two files. manifest.json:

{
  "manifest_version": 3,
  "name": "ProxyField auth",
  "version": "1.0",
  "permissions": ["proxy", "webRequest", "webRequestAuthProvider"],
  "host_permissions": ["<all_urls>"],
  "background": { "service_worker": "background.js" }
}

background.js (it reads the credentials from config.js, which the Python helper writes):

importScripts("config.js");

chrome.proxy.settings.set({
  value: {
    mode: "fixed_servers",
    rules: {
      singleProxy: { scheme: "http", host: self.PF.host, port: self.PF.port }
    }
  },
  scope: "regular"
});

chrome.webRequest.onAuthRequired.addListener(
  (details, callback) => {
    if (details.isProxy) {
      callback({ authCredentials: { username: self.PF.username, password: self.PF.password } });
    } else {
      callback({});
    }
  },
  { urls: ["<all_urls>"] },
  ["asyncBlocking"]
);

The Python helper writes config.js and loads the extension:

import json
import os
import pathlib
from selenium import webdriver

EXT_DIR = pathlib.Path("pf-auth").resolve()

def write_credentials(username, password):
    config = {"host": "gate.proxyfield.io", "port": 7000,
              "username": username, "password": password}
    (EXT_DIR / "config.js").write_text("self.PF = " + json.dumps(config) + ";\n")

write_credentials(os.environ["PF_USER"], os.environ["PF_PASS"])  # e.g. <login>-country-us

options = webdriver.ChromeOptions()
options.binary_location = os.environ["CFT_CHROME"]  # path to Chrome for Testing, see the note
options.add_argument(f"--load-extension={EXT_DIR}")
driver = webdriver.Chrome(options=options)
driver.get("https://api.ipify.org?format=json")
print(driver.find_element("tag name", "body").text)
driver.quit()

Keep the credentials in environment variables (PF_USER, PF_PASS), not in the script.

Version note. Branded Google Chrome ignores --load-extension since Chrome 137. Chrome for Testing and Chromium still load it, so point options.binary_location at one of them. Extensions also load in the new headless mode (--headless=new).

Selenium for Python (4.32 and later) has a network auth handler over WebDriver BiDi, with no extension and no MITM certificate. Turn on BiDi, point Chrome at the gateway, and register the credentials with driver.network.add_auth_handler() (Selenium BiDi docs). On port 7000 the gateway asks for the login with a 407 that carries Proxy-Authenticate: Basic, on CONNECT too.

import os
from selenium import webdriver

options = webdriver.ChromeOptions()
options.enable_bidi = True
options.add_argument("--proxy-server=http://gate.proxyfield.io:7000")
driver = webdriver.Chrome(options=options)

handler_id = driver.network.add_auth_handler(os.environ["PF_USER"], os.environ["PF_PASS"])
driver.get("https://api.ipify.org?format=json")
print(driver.find_element("tag name", "body").text)

driver.network.remove_auth_handler(handler_id)
driver.quit()

It loads no extension, so branded Chrome's --load-extension limit does not apply to it; check the exit IP on the first page it opens. Older Selenium releases lack driver.network; upgrade with pip install -U selenium.

Point Selenium at the gateway once: every new connection gets a new IP, with no proxy list to cycle through and no restart. Chrome keeps connections open, so several loads of the same site can share one IP; to force a new IP, start a new driver. More on how this works: rotating proxies.

Check it in one driver:

for _ in range(3):
    driver.get("https://api.ipify.org?format=json")
    print(driver.find_element("tag name", "body").text)

If all three loads show the same IP, Chrome reused its open connection to the gateway; a new driver opens a new connection and gets a new IP.

For a sticky IP per job, put a session in the username. Sticky sessions run from 30 s to 36 h, with presets of 1, 5, 10 and 30 min and 1, 3 and 6 h. Region, session and session time go in the same username: -region-<region>, -session-<id> and -ttl-<seconds> (30 to 129,600 seconds, 20 minutes by default), as in <login>-country-us-region-new_york-city-new_york-session-he9f2qga-ttl-600; every part after the login is optional. Residential is the default; add -mobile after the login for mobile IPs: <login>-mobile-country-us. To change the session per job:

• Method 2: call write_credentials() with the new username and start a new driver.
• Method 3: start a new driver and register the new username with add_auth_handler().
• Method 1: there is no username: with the allowlist, the session is set per port in the dashboard.

Country and city go in the username: <login>-country-us for the United States, <login>-country-us-city-new_york for New York. Targeting covers country, region (state) and city, and in the US all 50 states. All username parameters are in the docs.

The same gateway serves residential proxies and mobile 5G/4G proxies. ASN, ZIP and carrier targeting are not offered.

In Java, add --proxy-server to ChromeOptions with your IP on the allowlist; <port> is the port you set up in the dashboard. Maven: org.seleniumhq.selenium:selenium-java:4.51.0.

import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.chrome.ChromeOptions;

public class ProxyFieldSelenium {
    public static void main(String[] args) {
        ChromeOptions options = new ChromeOptions();
        options.addArguments("--proxy-server=http://gate.proxyfield.io:<port>");
        WebDriver driver = new ChromeDriver(options);
        try {
            driver.get("https://api.ipify.org?format=json");
            System.out.println(driver.findElement(By.tagName("body")).getText());
        } finally {
            driver.quit();
        }
    }
}

The org.openqa.selenium.Proxy object does the same through capabilities:

import org.openqa.selenium.Proxy;

Proxy proxy = new Proxy();
proxy.setHttpProxy("gate.proxyfield.io:<port>");
proxy.setSslProxy("gate.proxyfield.io:<port>");
ChromeOptions options = new ChromeOptions();
options.setProxy(proxy);
WebDriver driver = new ChromeDriver(options);

For a username and password in Java, load the Method 2 extension the same way: options.setBinary("/path/to/chrome-for-testing") and options.addArguments("--load-extension=/abs/path/pf-auth").

Firefox takes the proxy through preferences, including SOCKS5 with remote DNS. Use the allowlist, or the Method 3 auth handler for a username and password: Firefox supports WebDriver BiDi natively. Port 7000 below is for a username and password; with the allowlist, put the port you set up in the dashboard in its place.

from selenium import webdriver

options = webdriver.FirefoxOptions()
options.set_preference("network.proxy.type", 1)  # manual proxy configuration
options.set_preference("network.proxy.http", "gate.proxyfield.io")
options.set_preference("network.proxy.http_port", 7000)
options.set_preference("network.proxy.ssl", "gate.proxyfield.io")
options.set_preference("network.proxy.ssl_port", 7000)
driver = webdriver.Firefox(options=options)
driver.get("https://api.ipify.org?format=json")
print(driver.find_element("tag name", "body").text)
driver.quit()

For SOCKS5 proxies, replace the four http/ssl lines:

options.set_preference("network.proxy.socks", "gate.proxyfield.io")
options.set_preference("network.proxy.socks_port", 7000)
options.set_preference("network.proxy.socks_version", 5)
options.set_preference("network.proxy.socks_remote_dns", True)

SOCKS5 takes the same username and password. Setting the proxy by hand in the browser instead? See Firefox proxy settings and Chrome proxy settings.

selenium-wire passes user:pass@host:port to the upstream proxy through its own local proxy, but the project has had no release since 5.1.0 (October 15, 2022), and its author archived the GitHub repository (PyPI). It can still install and run, but it gets no fixes; prefer Method 1, 2 or 3.

# pip install selenium-wire
import os
from urllib.parse import quote
from seleniumwire import webdriver

proxy = f"http://{quote(os.environ['PF_USER'], safe='')}:{quote(os.environ['PF_PASS'], safe='')}@gate.proxyfield.io:7000"
seleniumwire_options = {"proxy": {"http": proxy, "https": proxy, "no_proxy": "localhost,127.0.0.1"}}
driver = webdriver.Chrome(seleniumwire_options=seleniumwire_options)
driver.get("https://api.ipify.org?format=json")
print(driver.find_element("tag name", "body").text)
driver.quit()

quote() URL-encodes special characters in the password (p@ss becomes p%40ss). selenium-wire decrypts HTTPS with its own CA certificate (a man-in-the-middle setup), so the browser must trust that certificate. On recent Python versions it may also need pip install "blinker<1.8".

Turn off images to cut what you pay: billing is per GB of traffic, and images make up much of a full page load.

options.add_experimental_option(
    "prefs", {"profile.managed_default_content_settings.images": 2}
)
driver = webdriver.Chrome(options=options)

The dashboard shows real-time usage and the gigabytes you have left, so you can compare a run before and after the change.

options.add_argument("--user-agent=...") sets the user agent (more in our scraping headers guide ); sites still read navigator.webdriver and the browser fingerprint, which our browser fingerprinting guide covers .

A proxy changes the IP, its location and its reputation. It spreads requests over many IPs, which cuts IP rate-limit blocks. It does not solve CAPTCHAs, and it does not hide a headless browser's fingerprint or TLS signature. Our anti-bot guide explains how sites detect bots and what proxies can and cannot fix.

The gateway answers 407 Proxy Authentication Required for a wrong login or password, an IP not on the allowlist or a malformed location, 402 Payment Required when no traffic is left and 502 Bad Gateway when no exit could be connected; a restricted site gets no separate response, and a 403 comes from the site itself. All gateway responses: proxy errors.

Using another framework? See the Playwright and Puppeteer setup pages. For Selenium itself (locators, waits, Grid), our Selenium web scraping tutorial covers the basics.