Tutorial11 min read

What Is an HTTP Proxy?

Short answer

In simple terms, an HTTP proxy is a middleman server for web traffic: your browser or script sends each request to the proxy, the proxy sends it on to the website from its own IP address and passes the response back, so the site sees the proxy's IP instead of yours. For HTTPS sites it opens an encrypted tunnel with the CONNECT method and relays the traffic without reading it. In scraping, HTTP proxies spread requests over many IPs and locations so no single address hits a site's rate limit; in testing, an intercepting proxy on your own machine (Burp Suite or mitmproxy on 127.0.0.1:8080) lets you view, pause, edit and replay the requests an app sends. ProxyField runs HTTP proxies from $1.95/GB; start with the Easy Trial: 1 GB for $2.95.

residential IPs
412M+
from, traffic never expires
$1.95/GB
Easy Trial
1 GB for $2.95
On this page 12 sections
  1. HTTP proxy: the definition
  2. What HTTP proxies are used for in scraping and testing
  3. Scraping through an HTTP proxy: a Python example
  4. Testing with an intercepting proxy
  5. How an HTTP proxy works
  6. What a site can see: the proxy headers
  7. Authentication: the Proxy-Authorization header
  8. The HTTP proxy URL format and where you enter it
  9. Types of HTTP proxies
  10. Is an HTTP proxy secure?
  11. HTTP proxy or SOCKS5?
  12. What Is an HTTP Proxy? FAQ
You → proxy → website → proxy → you

HTTP proxy: the definition

NIST defines a proxy as "software that receives a request from a client, then sends a request on the client's behalf to the desired destination" (NIST glossary, from SP 800-86). An HTTP proxy is that software for web requests: it speaks HTTP, the protocol of the web (MDN: Proxy servers and tunneling).

  • What it is: an address, host and port, such as gate.proxyfield.io:7000, often with a username and password.
  • What the site sees: the proxy's IP, not yours.
  • What the proxy sees: the full request for http:// pages; only the host name for https:// pages.
  • What "HTTP" names: the protocol the proxy speaks, not the IP behind it. Residential, mobile and datacenter proxies can all be HTTP proxies.

What HTTP proxies are used for in scraping and testing

JobWhat the proxy doesTypical setup
Web scrapingSends requests from many IPs, so no single address crosses a site's rate limitA rotating proxy: a new IP on every request; see proxies for web scraping
Multi-step scraping (logins you own, carts, pagination)Keeps one IP for a sequence of requestsA sticky session, for example 10 minutes
Checking content by locationShows search results, prices and ads as a reader in another country or city sees themA proxy targeted by country or city
Testing a site from another regionLoads your own site or app from a foreign networkA browser or test runner with a proxy setting
Security and debugging testsIntercepts requests so you can view, pause, edit and replay themAn intercepting proxy on 127.0.0.1:8080
Company networksFilters, logs and caches employees' web trafficSet by the IT team, often through a PAC file

Scraping through an HTTP proxy: a Python example

Every scraping library takes an HTTP proxy as one URL. In Python requests, the same http:// proxy URL goes under both keys, and HTTPS pages pass through it in a CONNECT tunnel:

Python
# pip install requests
import requests

proxy = "http://<login>-country-us-city-new_york:<password>@gate.proxyfield.io:7000"
proxies = {"http": proxy, "https": proxy}

for _ in range(3):
    r = requests.get("https://api.ipify.org", proxies=proxies, timeout=30)
    print(r.status_code, r.text)   # 200 and the exit IP the site sees, not yours

What to expect: 200 and an IP in New York on each line. With per-request rotation, each request can leave from a different IP; a sticky session keeps one IP for 30 seconds to 36 hours. A wrong password returns 407 Proxy Authentication Required instead (fixes). More: Python requests with a proxy and Scrapy proxy setup.

Easy Trial: 1 GB for $2.95.
Try for the price of 1 GB, no subscription, traffic never expires.

Testing with an intercepting proxy

An intercepting proxy is an HTTP proxy that runs on your own machine and holds each request until you forward it. Burp Suite's Proxy tool and the open-source mitmproxy both listen on 127.0.0.1:8080 by default: you point a browser or app at that address, and every request appears in the tool, where you can change a header, a cookie or a form field before it reaches the server, or resend it with changes.

A typical check: a login form sends username=alex&password=.... With interception on, you change username to a long string or a quote character, forward the request, and see whether the server validates the input or returns an error that leaks details. Do this only on systems you own or are authorized to test.

Shell
mitmproxy --listen-port 8080
# in a second terminal:
curl -x http://127.0.0.1:8080 --cacert ~/.mitmproxy/mitmproxy-ca-cert.pem https://example.com

To read HTTPS, the tool decrypts and re-encrypts the traffic with its own certificate authority, which you install on the test device (mitmproxy serves it at http://mitm.it once the device uses the proxy). That is the difference from a normal HTTP proxy, which only tunnels HTTPS. For regional tests rather than inspection, give the browser a remote proxy instead, for example in Playwright:

Python
from playwright.sync_api import sync_playwright

with sync_playwright() as p:
    browser = p.chromium.launch(proxy={"server": "http://gate.proxyfield.io:7000",
                                       "username": "<login>-country-us-city-new_york", "password": "<password>"})
    page = browser.new_page()
    page.goto("https://api.ipify.org")
    print(page.inner_text("body"))  # the IP the site sees

How an HTTP proxy works

What happens depends on whether the URL is http:// or https://.

Plain HTTP. The client sends the whole request to the proxy, with the full URL in the request line, the "absolute form" (RFC 9112, section 3.2.2):

HTTP
GET http://example.com/page HTTP/1.1
Host: example.com
Proxy-Authorization: Basic VVNFUjpQQVNT

The proxy reads the request, opens a connection to example.com, sends the request from its own IP and returns the response. Because it reads the request, it can see the URL and headers, and some proxies cache or change them.

HTTPS. The client first asks the proxy to open a tunnel:

HTTP
CONNECT example.com:443 HTTP/1.1
Host: example.com:443
Proxy-Authorization: Basic VVNFUjpQQVNT

The proxy connects to example.com on port 443 and replies 200 Connection established. From then on it copies bytes both ways, and the TLS handshake runs between your client and the site (RFC 9110, section 9.3.6). The proxy sees the host name and the amount of data, not the pages, headers or form fields. Many proxies allow CONNECT only to port 443, so a tunnel to another port may be refused (MDN).

Most sites now serve HTTPS, so most traffic through an HTTP proxy is CONNECT tunnels. The differences between an HTTP proxy, an HTTPS proxy and an "SSL proxy" are in what an SSL proxy is.

What a site can see: the proxy headers

A proxy that reads a request can add headers that tell the site a proxy is in the path:

HeaderStandardWhat it tells the site
ViaRFC 9110, section 7.6.3That the request passed a proxy, with its protocol version and name
ForwardedRFC 7239The client's IP (for=), the proxy (by=), the original host (host=) and scheme (proto=)
X-Forwarded-ForDe facto, not in an RFCThe client's IP, then each proxy's IP in order
X-Forwarded-HostDe factoThe host name the client asked for, when a reverse proxy changes it
X-Forwarded-ProtoDe factoWhether the client used http or https, when a reverse proxy changes it

A proxy can add them to plain HTTP requests, but it cannot touch the encrypted request inside a CONNECT tunnel. Proxies are graded by which of them they send, see elite proxies, and the headers your own client sends are covered in web scraping headers.

Authentication: the Proxy-Authorization header

Paid HTTP proxies check who you are in one of two ways:

  • Username and password. The client sends them in the Proxy-Authorization header, base64-encoded, on every request or CONNECT. If they are missing or wrong, the proxy answers 407 Proxy Authentication Required (RFC 9110, section 15.5.8).
  • IP allowlist. You register your server's IP with the provider, and the proxy accepts connections from it without a password. Useful for tools that cannot send credentials.

ProxyField supports both on the gateway gate.proxyfield.io:7000.

The HTTP proxy URL format and where you enter it

Most tools take an HTTP proxy as one URL:

http://USERNAME:PASSWORD@HOST:PORT

The scheme http:// describes the connection to the proxy, not the sites you visit: the same http:// proxy URL carries your https:// requests through CONNECT. Command-line tools and many libraries also read the HTTP_PROXY and HTTPS_PROXY environment variables; both usually point to the same http:// proxy URL.

Shell
curl -x gate.proxyfield.io:7000 -U "<login>-country-us-city-new_york:<password>" https://api.ipify.org

This request goes through a ProxyField residential IP in New York and prints the IP the site sees. Setup for other tools: curl with a proxy, Python proxies and Chrome proxy settings.

The other place you meet an "HTTP proxy" field is your operating system or browser: Windows, macOS and Linux each have a proxy setting that takes a host and port, see what proxy settings are. Company networks often set it with a PAC file instead, a small JavaScript file served as application/x-ns-proxy-autoconfig whose FindProxyForURL function picks a proxy per URL:

JavaScript
function FindProxyForURL(url, host) {
  if (dnsDomainIs(host, ".intranet.example")) return "DIRECT";
  return "PROXY proxy.example.com:8080; DIRECT";   // try the proxy, fall back to a direct connection
}

More in PAC files and automatic proxy configuration.

Types of HTTP proxies

The word covers three different jobs:

TypeWho sets it upWhat it does
Forward proxyYou, in your browser, app or scriptSends your requests out from its IP. Proxy services for scraping and IP rotation are forward proxies, and so are intercepting proxies for testing.
Reverse proxyThe website's ownerSits in front of the site's servers and receives visitors' requests: load balancing, caching, compression, TLS termination. You do not configure it.
Transparent proxyA network: an office, a school, an ISPIntercepts web traffic without any setting on your device, usually to filter or cache. See what a transparent proxy is.

Is an HTTP proxy secure?

An HTTP proxy does not add encryption. For HTTPS sites that does not matter much: the TLS tunnel runs from your client to the site, and the proxy cannot read the content. For plain http:// sites, the proxy, and anyone on the network between you and the proxy, can read and change the traffic. The proxy password also travels unencrypted, base64-encoded, unless the connection to the proxy itself uses TLS.

So the risk is mostly the operator. A paid provider with clear rules is a different case from a free proxy found in a public list, which may log or alter what passes through it. For privacy on an untrusted network, a VPN is the better tool: proxy vs VPN.

HTTP proxy or SOCKS5?

An HTTP proxy speaks web traffic only; a SOCKS5 proxy relays any TCP connection without reading it. For browsers, scrapers and APIs, HTTP is the simplest choice; for SSH or apps that are not web clients, SOCKS5 is the one that works. The full comparison is in SOCKS5 vs HTTP proxy.

Need HTTP proxies by country or city? ProxyField HTTP proxies run on 412M+ residential IPs in 195+ countries, with a new IP per request or sticky sessions up to 36 h, over HTTP, HTTPS and SOCKS5. Residential traffic costs $2.95/GB for 1 GB and falls to $1.95/GB from 1,000 GB, with no subscription. Start with the Easy Trial: 1 GB for $2.95. Traffic never expires, and all purchases are final.

What Is an HTTP Proxy? FAQ

What is an HTTP proxy server?

ProxyField's gateway, gate.proxyfield.io on port 7000, is one example: an HTTP proxy server is the machine at a host and port that accepts your web requests and sends them on from its own IP. You enter that host, the port and, if required, a username and password in your browser, app or script.

What is an HTTP proxy in simple terms, and what is it used for in scraping and testing?

An HTTP proxy is a middleman server that forwards your web requests from its own IP address and returns the responses, so the website sees the proxy's IP instead of yours. In scraping it spreads requests over many IPs and locations so no single address hits a site's rate limit; in testing, an intercepting proxy such as Burp Suite or mitmproxy on 127.0.0.1:8080 lets you view, edit and replay the requests an app sends.

What does HTTP proxy mean?

It means a proxy that speaks the web's protocol, HTTP: you give your browser or app its address and port, and it sends your web requests through it. In network settings, the "HTTP proxy" field takes that host, port and, if required, a username and password.

How does an HTTP proxy work?

For a plain HTTP page, the client sends the full request to the proxy, which makes it from its own IP and returns the response. For an HTTPS page, the client sends a CONNECT request, the proxy opens a TCP connection to the site, usually on port 443, and then relays encrypted bytes it cannot read.

What is proxy tunneling?

Proxy tunneling is when a proxy opens a raw connection to a destination and relays bytes without reading them, instead of forwarding individual requests. HTTP proxies do it with the CONNECT method, which is how HTTPS sites and other encrypted protocols pass through them.

What are HTTP proxies commonly used for?

Web scraping and data collection, checking search results, prices and ads by country or city, testing sites from other regions, and filtering or caching traffic on company networks. Developers also run local intercepting proxies on 127.0.0.1:8080 to inspect what an app sends.

What is an intercepting proxy used for in testing?

It holds each request from your browser or app so you can read it, change a header, cookie or form field, and forward or replay it, which shows how the server handles altered input. Burp Suite and mitmproxy both listen on 127.0.0.1:8080 by default; use them only on systems you own or are authorized to test.

What are the benefits of HTTP proxies?

They work with almost every browser, library and scraping tool without extras, handle both HTTP and HTTPS sites, and support username and password or IP allowlist authentication. With a provider's pool behind them, they also give you many IPs and locations from one address.

Can HTTP proxies be used for web scraping?

Yes; they are the standard choice, because every scraping library accepts an HTTP proxy URL, such as http://<login>:<password>@gate.proxyfield.io:7000 in Python requests. What decides whether pages load is the IP behind the proxy and your request rate, so spread requests over many IPs and keep a steady pace.

Is an HTTP proxy secure?

It does not add encryption, but HTTPS sites stay encrypted end to end through its CONNECT tunnel, so the proxy sees only which hosts you visit. Plain HTTP pages and the proxy password can be read on the way, so use a provider you trust and HTTPS targets.

Show 5 more questionsShow fewer questions

What is an HTTP proxy URL?

It is the proxy's address written as one string: http://USERNAME:PASSWORD@HOST:PORT. The http:// scheme describes the connection to the proxy, so the same URL also carries your HTTPS requests.

Which headers does an HTTP proxy add to a request?

Depending on the proxy, Via, Forwarded (RFC 7239) or X-Forwarded-For, which can carry your IP and the proxy's; reverse proxies also send X-Forwarded-Host and X-Forwarded-Proto. A proxy can add them to plain HTTP requests, but not to the encrypted request inside an HTTPS CONNECT tunnel.

What are the different types of HTTP proxies?

By role: forward proxies, which you configure to send your requests out; reverse proxies, which sit in front of a website's servers; and transparent proxies, which a network applies without any setting on your device. By IP: residential, mobile, ISP or datacenter.

Do HTTP proxies cache everything?

No. Only some proxies cache, mostly company and ISP proxies, and only plain HTTP responses that the site marks as cacheable. HTTPS traffic in a CONNECT tunnel cannot be cached, and proxy services for data collection usually do not cache at all.

Does an HTTP proxy always resolve DNS remotely?

Usually, yes: the client sends the host name in the request line or the CONNECT request, and the proxy looks it up. The exception is a client that resolves the name itself and sends the proxy an IP address, which some tools do when configured to.

Easy Trial: 1 GB for $2.95.

Try for the price of 1 GB, no subscription, traffic never expires.