Linux Proxy Setup

A provider works well on a Linux scraping server when its gateway speaks HTTP and SOCKS5 on one port, gives a new residential IP on every request, holds one IP when a job needs it, and lets you allowlist the server: then curl, wget, Python, Scrapy and Playwright use it with nothing to install. ProxyField meets each point below. Check any provider you test the same way, with the command in the last column, before you buy volume.

export http_proxy="http://<login>:<password>@gate.proxyfield.io:7000"
export https_proxy="http://<login>:<password>@gate.proxyfield.io:7000"
export no_proxy="localhost,127.0.0.1,::1,.local"
curl https://api.ipify.org

The last command prints the proxy's IP. Some programs read only the uppercase names, so add them for HTTPS_PROXY and NO_PROXY:

export HTTPS_PROXY="$https_proxy" NO_PROXY="$no_proxy"

curl reads http_proxy only in lowercase and ignores an uppercase HTTP_PROXY on purpose; HTTPS_PROXY, ALL_PROXY and NO_PROXY work in either case (everything curl: proxy environment variables). Set HTTP_PROXY as well only for a tool that needs it, such as some Go programs.

Pick the location in the username: <login>-country-us-city-new_york:<password> for New York. In the URL form, encode special characters in the password (p@ss becomes p%40ss). On a shared server, keep the password out of your shell history: allowlist the server's IP in your ProxyField account, drop <login>:<password>@ from the URL and use the port you set up in the dashboard instead of 7000.

Which file to use depends on how the shell starts (GNU Bash manual: Bash Startup Files):

• A new terminal window runs an interactive non-login bash, which reads ~/.bashrc.
• An SSH session or a text console runs a login shell, which reads ~/.bash_profile, ~/.bash_login or ~/.profile, the first one it finds. On Debian and Ubuntu, ~/.profile sources ~/.bashrc, so one file covers both; on Fedora and RHEL, ~/.bash_profile sources ~/.bashrc. Check yours with grep bashrc ~/.profile ~/.bash_profile.
• zsh reads ~/.zshrc.

cat >> ~/.bashrc <<'EOF'
export http_proxy="http://<login>:<password>@gate.proxyfield.io:7000"
export https_proxy="$http_proxy"
export no_proxy="localhost,127.0.0.1,::1,.local"
EOF
source ~/.bashrc
env | grep -i proxy   # the variables this shell now has

bash -i -c 'env | grep -i proxy'     # a new terminal (interactive shell)
bash -il -c 'env | grep -i proxy'    # an SSH or console login (interactive login shell)

# no password: <port> is the port you set up in the dashboard for this server's allowlisted IP
sudo tee /etc/profile.d/proxy.sh >/dev/null <<'EOF'
export http_proxy="http://gate.proxyfield.io:<port>"
export https_proxy="$http_proxy"
export no_proxy="localhost,127.0.0.1,::1,.local"
EOF

Package managers keep their own setting. On Debian and Ubuntu, create /etc/apt/apt.conf.d/95proxy:

Acquire::http::Proxy "http://<login>:<password>@gate.proxyfield.io:7000/";
Acquire::https::Proxy "http://<login>:<password>@gate.proxyfield.io:7000/";

Then sudo apt update downloads through the proxy. On Fedora, RHEL, CentOS Stream, Rocky and AlmaLinux, add to the [main] section of /etc/dnf/dnf.conf (/etc/yum.conf on older systems):

proxy=http://gate.proxyfield.io:7000
proxy_username=<login>
proxy_password=<password>

sudo drops your environment variables by default, so sudo pip install or sudo curl skip a proxy set with export. Use sudo -E, or add Defaults env_keep += "http_proxy https_proxy no_proxy" with visudo.

git config --global http.proxy "http://<login>:<password>@gate.proxyfield.io:7000"

To remove it from git later, run this on its own:

git config --global --unset http.proxy

A systemd service doesn't read your shell. For Docker pulls, create /etc/systemd/system/docker.service.d/http-proxy.conf:

[Service]
Environment="HTTP_PROXY=http://<login>:<password>@gate.proxyfield.io:7000"
Environment="HTTPS_PROXY=http://<login>:<password>@gate.proxyfield.io:7000"
Environment="NO_PROXY=localhost,127.0.0.1"

Then sudo systemctl daemon-reload && sudo systemctl restart docker. The same Environment= lines work for any service of your own.

A scraper on a server needs no proxy list: every request to gate.proxyfield.io:7000 gets a new residential IP, with unlimited concurrent threads. Check it with five requests:

for i in 1 2 3 4 5; do
  curl -s -x gate.proxyfield.io:7000 -U "<login>-country-us:<password>" https://api.ipify.org; echo
done
# five lines, five different IPs

Add a session id to hold one IP, for example for a login and the pages behind it:

for i in 1 2 3; do
  curl -s -x gate.proxyfield.io:7000 -U "<login>-country-us-session-job42-ttl-1800:<password>" https://api.ipify.org; echo
done
# the same IP three times, held for 1,800 s

A session lasts 30 s to 36 h (ttl-30 to ttl-129600), 20 minutes when you set no ttl; an IP left idle for 120 s is replaced. The pool has 412M+ residential IPs in 195+ countries and 21,000+ cities, with targeting by country, region and city, including all 50 US states, at the same price per GB.

The same URL goes into your scraper's code:

# Python requests
import requests
proxy = "http://<login>-country-us:<password>@gate.proxyfield.io:7000"
print(requests.get("https://api.ipify.org", proxies={"http": proxy, "https": proxy}, timeout=30).text)

# Scrapy: per request
yield scrapy.Request(url, meta={"proxy": "http://<login>-country-us:<password>@gate.proxyfield.io:7000"})

Full examples with retries and concurrency: Python and Node.js. ProxyField residential proxies cost $2.95/GB for 1 GB, $2.79 from 10 GB, $2.65 from 50 GB, $2.49 from 100 GB, $2.29 from 250 GB, $2.09 from 500 GB and $1.95 from 1,000 GB; the tier rate applies to the whole purchase, there is no subscription and the traffic never expires.

Most tools take a SOCKS5 URL in the same variables; socks5h lets the proxy resolve hostnames, so DNS lookups happen at the proxy's location, while socks5 resolves them on your machine (curl's --proxy option):

export all_proxy="socks5h://<login>:<password>@gate.proxyfield.io:7000"
curl https://api.ipify.org

SOCKS5 takes the same username and password. apt (1.6 and later, in Acquire::http::Proxy) and dnf (in proxy=, through libcurl) also accept a socks5h:// URL, but the HTTP proxy of section 3 is the simpler and better-documented choice for package downloads. When SOCKS5 is the better choice is covered on SOCKS5 proxies.

The finished setup, then the explanation. The end of /etc/proxychains4.conf:

strict_chain
proxy_dns
[ProxyList]
socks5 <gateway IPv4> 7000 <login> <password>

and the run line: proxychains4 -q <command>.

Proxychains-ng preloads a library into the program (LD_PRELOAD) that replaces its connect() and DNS calls, so the program's TCP connections go through the proxy even when it has no proxy setting of its own. Install it:

sudo apt install proxychains4        # Debian, Ubuntu, Kali
sudo dnf install proxychains-ng      # Fedora; RHEL family from EPEL

getent ahostsv4 gate.proxyfield.io | head -1    # the first column is the IPv4 address

strict_chain
proxy_dns
[ProxyList]
socks5 <gateway IPv4> 7000 <login> <password>

proxychains4 curl https://api.ipify.org                                  # prints the chain, then the proxy's IP
proxychains4 -q -f ~/.proxychains/proxychains.conf python3 scraper.py    # -q quiet, -f your own config

On GNOME, open Settings → Network → Network Proxy, choose Manual, and enter gate.proxyfield.io and 7000 for HTTP and HTTPS; on KDE Plasma, System Settings → Network → Proxy. Browsers that follow the desktop ask for the username and password on the first page. The full Ubuntu desktop walk-through is on Ubuntu proxy setup. For a proxy only inside one browser, see Firefox or Chrome.

env | grep -i proxy                 # what this shell will use
curl -s https://api.ipify.org       # the exit IP: not yours
curl -s https://ipinfo.io/json      # its city and country
curl -sv https://example.com -o /dev/null 2>&1 | grep -i "proxy\|connect"

Run the IP check twice: with a rotating username the IP changes on every request; with a sticky session it stays for the session length you set. For a whole list of proxy lines, use the proxy checker: up to 100 proxies per check, and your list and results are not stored. The -x and -U options and curl's own error codes are covered on cURL with a proxy.

unset http_proxy https_proxy all_proxy HTTP_PROXY HTTPS_PROXY ALL_PROXY no_proxy NO_PROXY

Then remove the lines from ~/.bashrc, ~/.profile, /etc/profile.d/proxy.sh, /etc/environment, the apt or dnf file and any systemd drop-in you added.

Gateway error messages: 407 Proxy Authentication Required for a wrong login or password, an IP not on the allowlist or a malformed location, 402 Payment Required when no traffic is left and 502 Bad Gateway when no exit could be connected; a restricted site gets no separate response, and a 403 comes from the site itself. Status codes are explained in proxy errors. Our 24/7 live chat helps with setup and integration.

For scraping code, see Python, Node.js and wget. Use ProxyField for lawful work such as public web data collection, price and SEO monitoring and testing, within our acceptable use policy: public pages only, following each site's terms.

Start with the Easy Trial: 1 GB for $2.95, with no minimum purchase and no subscription. The traffic never expires and all purchases are final. See pricing per GB.