Scrape Airline Data: Public Fares from Airline Websites
To scrape airline data from airline websites, ProxyField's Web Scraping API fetches public fare search, fare calendar and schedule pages for $2.00 per 1,000 successful results with the JS rendering airline sites need, from $1.00 at volume, or $0.50 per 1,000 (from $0.20) over plain HTTP. You pay only for successful results; to start, contact sales, or run your own fare scraper on residential proxies with the Easy Trial: 1 GB for $2.95.
Through the sales team on our 24/7 live chat
| Fetch | Entry | Volume, from |
|---|---|---|
| HTTP | $0.50 | $0.20 |
| JS rendering | $2.00 | $1.00 |
Billed on successful results only.
Airline data on public pages
Scraping airline data means reading what an airline's own website shows a traveler before booking: the fares for a route and date, the fare families behind them and the timetable. It differs from scraping travel agencies and metasearch sites, which show many airlines at once and are covered by the flight scraper and the Google Flights scraper. The Web Scraping API fetches an airline page and returns it; your code takes the fields out. ProxyField sells no separate airline or fare API with pre-parsed fields.
| Airline page | What the public page shows |
|---|---|
| Fare search results | Flights for a route and date: times, flight numbers, stops, duration, fare per passenger, seats-left notes |
| Fare calendar | The lowest fare shown for each day or month on a route |
| Fare families | What each fare includes: bags, seat choice, changes, refundability, miles |
| Timetable and route map | Routes flown, days of operation, flight times |
| Flight status | Departure and arrival times, delays, gates |
| Fee pages | Published bag, seat and change fees |
Booking steps that ask for passenger details, and everything inside a frequent flyer account, are out of scope: ProxyField's acceptable use policy forbids accessing data behind logins you are not entitled to. The API returns the page itself as HTML, Markdown or plain text, or a JSON envelope with the status code, the final URL, the headers, the cookies and the content; it has no parsed airline fields, so your parser reads them.
Why the same flight shows many fares
An airline prices a seat by route, date, time of booking, cabin and fare family, number of passengers and point of sale, the country and currency of the site the search comes from. The same London–New York flight can cost something different on the airline's UK, US and German sites. To record what each market sees, search from that country: ProxyField residential proxies offer targeting by country, region (state) and city, with 412M+ residential IPs in 195+ countries, such as the United Kingdom or Germany. In a Web Scraping API request, country sets the country the request comes from (the exit IP), as a two-letter code such as de; the countries it accepts: the proxy pool's 195+ countries.
A useful fare record therefore keeps the search with the price: route, departure date, cabin, passengers, fare family, currency, the country searched from and the time of the check.
Why airlines fight fare scrapers
Every search an airline website answers costs the airline money: its reservation and shopping systems have to price each request, and much of an airline's selling runs through global distribution systems (GDS) and other channels that charge per booking or per transaction. Scrapers send many searches and book nothing, which raises the look-to-book ratio, searches per booking, and the cost of every sale. That is why airlines put strong bot protection on their search pages, limit how often one visitor can search and, in some cases, take scrapers to court (see the legal section below).
Airline APIs or scraping?
Airlines sell through their own channels to accredited travel sellers: GDS connections and NDC APIs, built on IATA's New Distribution Capability standard, give travel agencies and booking sites live fares under a commercial agreement. Some airlines also run developer portals for schedules or flight status. Without such an agreement, the fares a traveler sees are on the public pages.
| Item | GDS or NDC access | Web Scraping API |
|---|---|---|
| Who can use it | Accredited travel sellers, under contract | ProxyField customers |
| Purpose | Selling and booking tickets | Reading public fare and schedule pages |
| Fares | The airline's distributed offers | What a traveler in a given market sees |
| Billing | Per the agreement | Per 1,000 successful results |
If you sell tickets, ask the airline or a GDS for access first.
What you send
You send the URL of an airline's fare search, calendar or timetable page, with JS rendering, and get the page back. The API takes a POST to https://api.proxyfield.io/v1/scrape with your API key in an Authorization: Bearer <api_key> header and a JSON body: url, render (false for plain HTTP, true for JS rendering), country and output (html, markdown, text or json). Airline sites build fare results with scripts, so with rendering wait_for takes a CSS selector to wait for, such as the element that holds the results list. Parameter reference: API documentation.
# The request the Web Scraping API takes (access through Contact sales)
curl https://api.proxyfield.io/v1/scrape \
-H "Authorization: Bearer <api_key>" \
-H "Content-Type: application/json" \
-d '{"url": "https://www.example-airline.com/<fare-search-url-from-your-browser>", "render": true, "country": "gb", "output": "json"}'Example response (shortened; content is the page's HTML):
{
"metadata": {"id": "req_5965aa58c657ca5f", "api": "web_scrape", "status": "success", "billed": true},
"status_code": 200,
"final_url": "https://www.example-airline.com/<fare-search-url-from-your-browser>",
"content_type": "text/html; charset=utf-8",
"content": "<!DOCTYPE html><html lang=\"en-GB\">…</html>"
}Pricing: $2.00 per 1,000 rendered airline pages, from $1.00
Airline sites build fare results with scripts after the page loads, so budget for JS rendering: $2.00 per 1,000 successful results, from $1.00 at volume. Static pages such as fee tables and route lists cost $0.50 per 1,000 over plain HTTP, from $0.20. You pay only for successful results. Billed: a page the airline site answered with a 2xx status. Not billed: blocked requests, timeouts and errors on our side; a blocked request is retried on new IPs, and the retries are not billed. Every response says whether it was billed. A target's 404 or 410, and a sign-in page the site serves with a 200 status, count as billed results. The volume price starts at 10,000 successful results in one purchase. For availability and access, contact sales through the sales team on our 24/7 live chat.
One result is one page: a search that needs a second page for later flights counts twice. Payment and balance for the API: the same crypto balance as proxies, which never expires; there is no subscription or monthly fee. All prices: pricing.
Through the sales team on our 24/7 live chat
| Workload (30-day month) | Pages a month | JS rendering at $2.00 | At $1.00 (volume) |
|---|---|---|---|
| 30 routes × 60 departure dates, daily | 54,000 | $108 | $54 |
| 100 routes, fare calendar for the next 6 months, daily | 18,000 | $36 | $18 |
| Both | 72,000 | $144 | $72 |
When an airline site refuses a request
Airline search pages answer automated traffic with a challenge page, a 403 or a 429, or show an error instead of fares when one IP searches too often. With your own scraper, use a real browser, spread searches over rotating residential IPs, keep one sticky session (30 s to 36 h) for a search that loads in several steps and pace requests per route. With the Web Scraping API you pay only for successful results. Error codes explained: errors.
Scrape airline fares with Python and Playwright
This script opens an airline's fare search page in a real browser through a residential IP from the market you price and prints the prices on the page. Put in the URL your browser shows after a search on that airline's site.
import re
from playwright.sync_api import sync_playwright
URL = "https://www.example-airline.com/<fare-search-url-from-your-browser>"
with sync_playwright() as p:
browser = p.chromium.launch(proxy={
"server": "http://gate.proxyfield.io:7000",
"username": "<login>-country-us-city-new_york", "password": "<password>"})
page = browser.new_page(locale="en-US")
page.goto(URL, wait_until="networkidle", timeout=90000)
# Each airline site differs: check the selectors on a live page first
print(re.findall(r"[$£€]\s?\d[\d,]*", page.inner_text("body"))[:10])
browser.close()-
Run one search on the airline's site and copy the results URL: route, date, cabin, passengers.
-
Open it through
gate.proxyfield.io:7000with an IP from the market you price. -
Wait for the results, then read flight, times, fare family and price.
-
Store each fare with the search, currency, country and time of the check.
Access opens once the payment is confirmed; the gateway, credentials or allowlist are in your account. The username format is <login>-country-us-city-new_york; country keys outside the US, such as the United Kingdom or Germany: the two-letter country code in lowercase, as in -country-de; other tokens, including the sticky session: -region-<region>, -session-<id> and -ttl-<seconds> (30 to 129,600 seconds, 20 minutes by default) after the location, as in <login>-country-us-region-new_york-city-new_york-session-he9f2qga-ttl-600. Residential proxies cost $2.95/GB for 1 GB and from $1.95/GB at volume, with unlimited concurrent threads; the traffic never expires, and all purchases are final. More on Playwright.
What teams use airline data for
- Fare monitoring: price history per route, date and market, straight from the airline.
- Revenue and pricing research: how a competitor prices a route and its fare families.
- Ancillary research: what bags, seats and changes cost on each fare.
- Schedule and network analysis: routes, frequencies and flight times.
- Travel content and alerts built on public fares.
Booking or holding seats through a scraper is not data collection; fraud and fake accounts are forbidden under ProxyField's acceptable use policy.
Airlines' terms and the law
Most airlines restrict automated collection in their website terms, and airlines have taken fare scrapers to court in the US and the EU: Ryanair v PR Aviation (2015) at the EU Court of Justice, and Ryanair's CFAA case against Booking.com in Delaware, whose 2024 jury verdict the judge set aside in January 2025. Both are followed in Web Scraping Legal News. Collect public fares only, keep personal data out, and ask for distribution access when you sell an airline's tickets. For the wider question, read Is Web Scraping Legal? US, EU and UK Rules (2026).
ProxyField's rule for every site: public pages only, and follow the site's terms.
This is not legal advice.
More scrapers
- Flight scraper: fares on travel agencies and metasearch sites.
- Google Flights scraper and Expedia scraper on the same Web Scraping API.
- Hotel data scraping: hotel rates by date and market.
- All scrapers: scrapers.
Scrape Airline Data FAQ
24/7 support by live chat, including help with setup and integration.
How much does it cost to scrape fare data?
ProxyField's Web Scraping API costs $2.00 per 1,000 successful airline pages with JS rendering, from $1.00 at volume, and you pay only for successful results. Checking 30 routes for 60 departure dates every day is 54,000 pages a month: $108, or $54 at the volume price.
How do I scrape airline data?
Run a search on the airline's site, copy the results URL, then open it in a real browser through a residential IP from the market you price and read flight, times, fare family and price. Repeat per route and date, a few seconds apart, and store each fare with its search and time.
What is fare scraping?
Fare scraping is collecting the ticket prices that airline and travel websites show for a route and date, usually on a schedule to build price history. It is how fare trackers, price alerts and airline pricing teams see the market.
Can I scrape airline websites directly?
Yes, their fare search, calendar and timetable pages are public, but they are among the most guarded pages on the web. Expect scripts that build results after loading and strict bot checks, so use a real browser or a scraping API with JS rendering.
What is a flight fares scraper API?
A service that takes a fare search URL, fetches the page through its own IPs and returns it, so you get fares without running browsers and proxies yourself. ProxyField's is its Web Scraping API, at $2.00 per 1,000 rendered pages.
What data can I extract from airline fare pages?
Flights for the route and date with times, flight numbers, stops and duration, the fare per passenger for each fare family, what each fare includes, and the lowest fares in the calendar. Fee pages add bag, seat and change fees.
How do I start scraping fare data?
Pick a few routes and dates, search them once by hand on the airline's site to see how the results page loads, then automate the same search in a browser through a proxy. Start small and add routes once the parsing holds.
Will I get blocked when scraping fare data?
Often, if you search fast from one IP: airline sites answer with a challenge page, a 403 or a 429. Rotating residential IPs, a real browser, one sticky session per search and a slow pace per route keep blocks down.
Do I need an account to scrape fare data?
No. Fare searches, calendars and timetables are public, and a scraper should not sign in. Frequent flyer accounts and booking steps with passenger details are out of scope.
Can I geo-target flight fare requests to a specific country or city?
With your own scraper, yes: ProxyField residential proxies target country, region and city, so a search can come from the market whose fares you record. Airlines price by point of sale, so the country matters more than the city.
Show 7 more questionsShow fewer questions
Do airline websites show different fares by country?
They can: airlines price by point of sale, the country and currency of the site a search comes from, so the same flight may cost something different on its UK, US and German sites. Record the country you searched from with every fare.
How does unauthorized scraping add to airlines' GDS and API costs?
Each fare search costs an airline computing and, through distribution channels, transaction fees, while scrapers search without booking. That raises the look-to-book ratio and the cost of each sale, which is why airlines block heavy search traffic.
What output formats does a flight fares scraper return?
ProxyField's Web Scraping API returns the page; your parser turns it into the records you store, such as JSON rows or a CSV file. Keep one record per fare with the search details next to the price.
How fast is a flight fares scraper at scale?
Speed depends on how many searches run in parallel and on the airline's bot checks. With your own scraper on ProxyField proxies you get unlimited concurrent threads.
Can I try an airline fare scraper before paying for volume?
For the API, contact sales with your routes, dates and markets; you are billed only for successful results. To test your own scraper, start with the Easy Trial: 1 GB of residential traffic for $2.95, with no subscription, and the traffic never expires.
Can I scrape baggage and seat fees?
Published fee tables and the fare family comparison on the results page are public and can be collected. Seat maps and fees shown only inside the booking flow, after passenger details, are out of scope.
Should I scrape airline sites or travel sites for fares?
Airline sites show the airline's own fares and fare families, including fares some airlines sell only there; travel sites show many airlines in one search. Fare trackers usually combine both and keep the source with each price.
Related pages
Start collecting public pages with the Web Scraping API
Billed on successful results only. Contact sales for the Web Scraping API and the volume you need, or run your own scraper on residential proxies. 24/7 live chat helps with setup and integration.