What Is a Headless Browser?

"Head" here means the graphical user interface: the window, tabs, address bar and buttons. A headless browser has the same engine underneath (the HTML parser, the CSS layout, the JavaScript engine, the network stack) and leaves out only the part you look at. Chrome's documentation puts it as running "the browser in an unattended environment, without any visible UI" (Chrome for Developers: Chrome Headless mode, page last updated 2024-10-21).

"Headless mode" is a switch, not a different product: the same Chrome or Firefox binary runs with a window or without one. "Headful" (or "headed") is the opposite, the normal mode with a window.

What can still differ. New headless Chrome is the same browser, but some behaviour still depends on the tool and its settings (Playwright: browsers):

• User agent: headless Chrome reports HeadlessChrome instead of Chrome in its user-agent string unless the script sets another one.
• Automation flag: under Puppeteer, Playwright or Selenium, navigator.webdriver is true, as the W3C WebDriver standard defines for a browser under remote control.
• Which headless: Playwright's default Chromium headless is chrome-headless-shell, the old implementation, not the full new headless; the chromium channel opts into new headless (Playwright docs; see the library table below).
• Window size and fonts: a server with no display has its own default viewport and installed fonts, so screenshots can differ from your laptop.

A headless run is not lighter than an HTTP request: it still downloads scripts, styles and often images and fonts, and holds the page in memory like any browser.

Start headless Chrome on each system:

Because the page is fully rendered, a headless browser sees what a visitor sees after the scripts have run, including content that a plain HTTP request never gets, such as lists loaded by JavaScript after the first response.

Testing is the best-known job; these are the others, each with the tool and the call that does it, and what the browser costs you compared with a plain HTTP client.

Scraping with a headless browser follows the same rules as any scraping: respect each site's terms, access controls and the law where you work; see is web scraping legal?.

The terminal versions of the screenshot, DOM and PDF jobs need no library at all:

google-chrome --headless --screenshot=page.png https://example.com
google-chrome --headless --dump-dom https://example.com > page.html
google-chrome --headless --print-to-pdf --no-pdf-header-footer https://example.com

These flags are in Chrome's Headless command-line reference.

Headless testing means running browser tests without a visible window, which is how test suites run on CI servers. The test opens the app, performs the steps a user would and checks the result, exactly as in a headed run; only the display is missing. Teams usually develop and debug tests headed, then run them headless in the pipeline. Playwright Test, Selenium with a test framework and Puppeteer with Jest all support both modes with one setting.

Chrome has run headless since Chrome 59 in 2017, and its headless mode changed in two steps (Chrome for Developers: Chrome Headless mode):

• Chrome 112 (2023): the new headless mode. Headless Chrome is now the real Chrome browser, with every feature, that "creates, but doesn't display, any platform windows".
• Chrome 132.0.6793.0: the old headless implementation left the Chrome binary and lives on as a separate program, chrome-headless-shell, downloadable from Chrome for Testing (Chrome blog).

So "chrome headless shell" is the old, lighter headless browser: faster to start and smaller, but not identical to Chrome. New headless is the full browser. Bugs in headless mode are reported at goo.gle/headless-bug.

In the automation libraries (from Puppeteer's docs and Playwright: browsers):

Playwright installs only the shell with npx playwright install --only-shell (Playwright: browsers).

Firefox runs headless with the -headless flag, and Selenium, Playwright and Puppeteer can all launch it that way. Puppeteer has supported stable Firefox since v23.0.0 (2024), over WebDriver BiDi (Puppeteer: supported browsers). Firefox matters when a site behaves differently in Gecko, Firefox's engine, or when your tests must cover more than Chromium.

firefox -headless --screenshot https://example.com

PhantomJS, a well-known headless browser of the 2010s, is no longer developed; modern headless Chrome and Firefox replaced it. Which one to pick for scraping: best headless browsers.

Advantages

• See pages exactly as a browser renders them, including JavaScript content.
• Run on servers and in containers with no display.
• Skip painting to a screen, and run several in parallel on one machine.
• Make tests and scrapers repeatable and fast to rerun.

Disadvantages

• Far heavier than HTTP requests: more CPU, memory and bytes per page.
• Slower per page, because they wait for scripts and rendering.
• Harder to debug without a window; run headed while you write the script.
• Websites can tell automated browsers from people by many signals, so a headless browser alone does not stop rate limits or blocks; see browser fingerprinting and scraping.

Only when the data is not in the HTML the server sends. Check the page source (Ctrl+U) or the network tab first: if the data is in the HTML, or in a JSON request the page makes, an HTTP client is faster and uses a fraction of the traffic. Use a headless browser for pages that build their content with JavaScript, need clicks or scrolling, or require a login to your own account.

A headless browser takes a proxy at launch, like any browser. With Playwright:

from playwright.sync_api import sync_playwright

with sync_playwright() as p:
    browser = p.chromium.launch(headless=True, proxy={
        "server": "http://gate.proxyfield.io:7000",
        "username": "<login>-country-us-city-new_york", "password": "<password>"})
    page = browser.new_page()
    page.goto("https://api.ipify.org?format=json")
    print(page.inner_text("body"))
    browser.close()

<login>-country-us-city-new_york asks for an exit IP in New York. Region, session and session time go in the same username: -region-<region>, -session-<id> and -ttl-<seconds> (30 to 129,600 seconds, 20 minutes by default), as in <login>-country-us-region-new_york-city-new_york-session-he9f2qga-ttl-600; every part after the login is optional. Setup guides: Playwright, Puppeteer, Selenium.

Browser pages download more bytes than plain requests, so block images, fonts and media you do not need: the traffic on a residential proxy is billed per GB. ProxyField residential proxies cost $2.95/GB for 1 GB and fall to $1.95/GB from 1,000 GB, with a new IP per request or sticky sessions from 30 s to 36 h, and no subscription; traffic never expires, and all purchases are final. Start with residential proxies and the Easy Trial: 1 GB for $2.95.