Data Parsing vs Web Scraping

Web scraping is the automated collection of data from web pages, from the first request to the stored row: build the URL list, fetch each page (rendering it in a browser if its content needs JavaScript), parse it, extract and clean the fields, and store them. For the full explainer, see what web scraping is.

Data parsing is converting data from one format into a structure a program can work with. For web pages, an HTML parser reads the markup and builds a tree of elements, the same kind of tree a browser builds. Your code then selects the elements it needs from the tree, with CSS selectors or XPath, and reads their text or attributes. JSON and XML parsers do the same for API responses and feeds.

In everyday use, "parsing" often covers both building the tree and selecting the fields. Strictly, the second part is extraction, and the clean-up after it (currency signs, date formats, units) is normalization.

A scraper runs the steps in this order:

Keeping steps 2 and 3 apart pays off: if you save the raw HTML, you can fix a parsing bug and re-parse yesterday's pages without fetching them again. This Python example does exactly that; we ran it without the proxy on October 23, 2026 against books.toscrape.com, a public scraping sandbox, and it parsed 20 books from the saved page.

# pip install requests beautifulsoup4 lxml
import pathlib, requests
from bs4 import BeautifulSoup

proxy = "http://<login>-country-us-city-new_york:<password>@gate.proxyfield.io:7000"

# Scraping: fetch the page and keep the raw bytes
raw = requests.get("https://books.toscrape.com/", proxies={"http": proxy, "https": proxy}, timeout=30).content
pathlib.Path("page.html").write_bytes(raw)

# Parsing: build the tree from the saved file (no network) and extract fields
soup = BeautifulSoup(pathlib.Path("page.html").read_bytes(), "lxml")
books = [{"title": a["title"], "price": p.get_text(strip=True)}
         for a, p in zip(soup.select("article.product_pod h3 a"),
                         soup.select("article.product_pod p.price_color"))]
print(len(books), books[0])

The deeper guide to HTML parsing, with selector examples, is HTML scraping.

• Selectors return nothing after a site update. The layout changed: update the selectors, and alert when a field comes back empty.
• Garbled characters, such as "£" instead of "£". The text was decoded with the wrong encoding; pass the raw bytes to the parser and let it detect the encoding.
• Numbers in local formats, such as 1.234,56 in German. Normalize per locale before you store.
• Malformed HTML. Use a lenient parser such as lxml or html5lib; browsers tolerate bad markup, and so must your parser.
• The data is not in the HTML at all. That is a fetching problem, not a parsing one: the page builds the content with JavaScript, so fetch it with a headless browser or call the JSON request the page makes. See scraping dynamic web pages.
• A block page instead of the product page. Also a fetching problem: the parser finds no fields because the site served a CAPTCHA or an error. Check status codes and page titles before parsing.

Build your own when you scrape a few sites whose fields you know well; selectors are a few lines each. Buy, or use a scraping API that returns parsed fields, when you need many sites of one type, such as search engines or marketplaces, and do not want to maintain a parser for each. ProxyField's Google SERP API returns the results page as JSON, with the AI Overview, the People Also Ask block and the local pack parsed, for $1.20 down to $0.80 per 1,000 successful results.

A proxy changes where your requests come from, so it solves fetching problems: per-IP rate limits, location-specific content and blocks on IP ranges. It does nothing for parsing, which runs on data you already have. ProxyField residential proxies give a new IP on every request or a sticky session from 30 s to 36 h, with country, region and city targeting, over HTTP, HTTPS and SOCKS5. You pay per GB with no subscription; the traffic never expires, and all purchases are final. What a job costs: proxies for web scraping.