What Is Screen Scraping?

Screen scraping means a program copies information from an application's visible output and turns it into data. The program does not ask the application for the data in a machine-readable form; it reads what the application shows, the way a person would, only faster and without typing mistakes.

The term is older than the web. It started with mainframe terminals: programs read the 80-column text screens of a host system and copied fields into newer software. Today it covers three kinds of source:

• Desktop and legacy applications with no export, no database access and no API.
• Terminal screens of mainframe and AS/400 systems, read through a terminal emulator.
• Web pages as displayed, read from the rendered page or a screenshot instead of the HTML the server sent.

"Screen scrape", "screen scraper" and "screen scraping software" all refer to this. On websites, people often say screen scraping when they mean web scraping: a web scraper usually reads the HTML the server sends, a screen scraper what the browser has rendered. The difference is covered in our guide on what web scraping is.

A screen scraper reads the display with one of four methods, from the most reliable to the least, and usually drives the interface between reads.

Operating the interface. Most screen scrapers also act like a user: they open a screen, type into a search box, click "next", scroll to load more rows or log in, and wait for the new content to draw before they read again. This is where screen scraping overlaps with robotic process automation (RPA); RPA tools and their screen scraping steps are covered in RPA Web Scraping .

After reading, the scraper does what every scraper does: it maps the values to fields, cleans them (a displayed "$19.99" becomes the number 19.99) and writes them to a CSV file, a spreadsheet or a database.

To screen scrape a website, you load it in a real browser, let its scripts run, and read what the browser displays: the text of the rendered elements, or a screenshot.

The DOM is what you read. The browser turns every page into the Document Object Model (MDN), a tree of elements such as headings, table cells and buttons. A scraper finds elements in that tree by tag, id or CSS selector and copies their text: in a browser console, document.querySelectorAll("h1, h2"); in Playwright, page.locator("h1, h2"); in Selenium, driver.find_elements(By.CSS_SELECTOR, "h1, h2") (see Selenium's locator strategies). Selectors in depth: HTML Scraping With CSS Selectors and XPath.

A headless browser driven by Playwright, Puppeteer or Selenium does both kinds of read. Reading the rendered elements is far more accurate than OCR, so keep screenshots for content that exists only as an image, such as a chart or a price drawn on a canvas.

This Python example opens a page through the ProxyField gateway with Playwright, saves a screenshot and reads the text of every heading from the DOM. Replace <login> and <password> with your credentials; <login>-country-us-city-new_york asks for an exit IP in New York.

# pip install playwright && playwright install chromium
from playwright.sync_api import sync_playwright

with sync_playwright() as p:
    browser = p.chromium.launch(proxy={"server": "http://gate.proxyfield.io:7000",
                                       "username": "<login>-country-us-city-new_york", "password": "<password>"})
    page = browser.new_page()
    page.goto("https://example.com")
    page.screenshot(path="page.png", full_page=True)   # the screen as an image
    print(page.locator("h1, h2").all_inner_texts())     # the screen as text, read from the DOM
    browser.close()

If the data is in the HTML the server sends, skip the browser and read the HTML directly: it is faster and uses less traffic. Our guide to scraping dynamic web pages shows how to tell which case you have. Region targeting and sticky sessions take further username settings: -region-<region>, -session-<id> and -ttl-<seconds> (30 to 129,600 seconds, 20 minutes by default) after the location, as in <login>-country-us-region-new_york-city-new_york-session-he9f2qga-ttl-600.

To screen scrape with OCR in Python, capture a screen region with mss, clean the image with Pillow, read it with the Tesseract engine through pytesseract, then parse the value you need. Four steps:

1. Install the packages and the engine. pytesseract is only a wrapper; it does not install Tesseract.

Check the engine with tesseract --version. If Python cannot find it, pytesseract raises pytesseract.pytesseract.TesseractNotFoundError: tesseract is not installed or it's not in your PATH. See README file for more information.

2. Capture, 3. clean and OCR, 4. parse:

import re
import mss
import pytesseract
from PIL import Image, ImageOps

# Windows only, if tesseract.exe is not on PATH:
# pytesseract.pytesseract.tesseract_cmd = r"C:\Program Files\Tesseract-OCR\tesseract.exe"

region = {"left": 100, "top": 200, "width": 600, "height": 80}   # pixels on screen

# 2. Capture: one mss instance, reused for repeated grabs
with mss.mss() as sct:
    shot = sct.grab(region)            # sct.monitors[1] is the first monitor, [0] all of them
    img = Image.frombytes("RGB", shot.size, shot.bgra, "raw", "BGRX")

# 3. Clean the image (grayscale, enlarge 2x, black and white), then OCR it
img = ImageOps.grayscale(img)
img = img.resize((img.width * 2, img.height * 2))
img = img.point(lambda p: 255 if p > 160 else 0)   # use ImageOps.invert(img) first for light text on dark
text = pytesseract.image_to_string(img, config="--psm 6")
print(repr(text))                      # 'Total: $19.99 Items: 3\n'

# 4. Parse the value you need
m = re.search(r"\$\s?(\d[\d,]*\.\d{2})", text)
price = float(m.group(1).replace(",", "")) if m else None
print(price)                           # 19.99

We ran steps 3 and 4 with Tesseract 5.3.4 on a rendered image of "Total: $19.99 Items: 3"; the printed output is in the comments.

• Page segmentation mode: --psm 6 reads one uniform block of text; --psm 7 a single line. pytesseract.image_to_data() returns each word with its position and a confidence score, which helps you reject misreads.
• On Linux, mss and Pillow's ImageGrab need an X11 display; on a server, run the app in a virtual display such as Xvfb. ImageGrab.grab(bbox=(left, top, right, bottom)) works on Windows and macOS too, but mss is faster for repeated grabs.
• Check the output: OCR confuses 0 and O or 1 and l, and loses table structure, so validate numbers before anyone relies on them.

If the text is in the page's HTML or DOM, do not OCR it: read the elements as in the Playwright example. Where a desktop application exposes its controls, a UI automation library reads the exact text and is the better first choice.

Screen scraping is used where the data exists only on a screen.

• Legacy system migration. Copying records out of an old application before it is switched off, when it has no export.
• Integration with mainframes. Letting a modern web app read and write a host system's screens, a long-standing use in insurance, government and banking.
• RPA workflows. Software robots that copy values between applications in back-office work, such as invoice entry.
• Financial account aggregation. Apps that logged in to a bank's website with the customer's credentials and read the balances; regulators have been moving this to bank APIs (see below).
• Visual checks of websites. Screenshots that confirm how a page, an ad or a price looks in a given country or city, as in ad verification.
• Content that is drawn, not written. Prices or figures rendered as images or on a canvas.

Banking is where screen scraping is most regulated. In the EU, the second Payment Services Directive (PSD2, Directive (EU) 2015/2366) and its technical standard on strong customer authentication (Delegated Regulation (EU) 2018/389) require banks to give licensed third parties access through a dedicated interface, usually an API, and to identify themselves, applying from September 2019. In the UK, the Open Banking standard set up bank APIs for the same purpose. In the US, the Consumer Financial Protection Bureau finalized its Personal Financial Data Rights rule under section 1033 of the Dodd-Frank Act in October 2024 to move account data access to developer interfaces, and opened a reconsideration of the rule in August 2025; check the rule's current status on the CFPB's rule page. The common thread: where an API exists, regulators prefer it to screen scraping with stored passwords.

These four overlap, so it helps to separate what each one reads and does.

RPA uses screen scraping as one step; OCR is one technique a screen scraper may use. Web scraping is usually more reliable than screen scraping, because page code is more structured than pixels. The tools for each are compared in best web scraping tools.

Screen scraping is the method of last resort because:

• It breaks with the interface. Move a field or rename a button and the scraper reads the wrong value or nothing.
• It is slow. It waits for screens to draw, and OCR takes time on every image.
• OCR makes mistakes, so numbers need validation before anyone relies on them.
• It often needs credentials. Scrapers that log in as a user hold passwords, which is a security and compliance risk.
• It loads the source system like a very fast user, which matters on old hosts and on websites with rate limits.

Use an API, an export or the page's HTML when any of them exists, and keep screen scraping for the data that has no other way out.

Yes. A website cannot see your screenshots, but it sees the browser that loads the page: how many requests come from one IP, how fast they come, and whether the browser's fingerprint looks automated. Screen scraping a site at volume needs the same care as any web scraping: polite pacing, realistic browser settings and a varied IP footprint. Rotating proxies give each page load a new IP, or keep one IP for a sticky session when a flow spans several screens. Scrape public data only, and check the rules first in is web scraping legal.

ProxyField residential proxies work with Playwright, Puppeteer and Selenium over HTTP, HTTPS and SOCKS5, with country, region and city targeting and sticky sessions from 30 s to 36 h. Setup guides: Playwright, Puppeteer, Selenium. You pay per GB with no subscription; the traffic never expires, and all purchases are final.