Flight Scraper: Travel Fares and Hotels via API
A good flight scraper depends on the fares you need: for fares you will book or show as bookable, use an official API (Skyscanner's Travel API under a commercial agreement, Amadeus for Developers or Duffel); to monitor public fares on Skyscanner, Kayak, Momondo and Priceline, scrape the public search pages. ProxyField gives you two ways to do the second: run your own scraper today on ProxyField residential proxies, starting with the Easy Trial at 1 GB for $2.95, or use the Web Scraping API, designed to return each rendered page for $2.00 per 1,000 successful results ($1.00 from 10,000 results in one purchase), which is in development: Contact sales.
Through the sales team on our 24/7 live chat
| Fetch | Entry | Volume, from |
|---|---|---|
| HTTP | $0.50 | $0.20 |
| JS rendering | $2.00 | $1.00 |
Billed on successful results only.
| You want | Use | Price | Available |
|---|---|---|---|
| Bookable fares, fare rules, a price check before payment | Skyscanner Travel API, Amadeus for Developers, Duffel | the provider's own terms | from each provider, after approval where it applies |
| Public fares by route, date and country, with your own scraper | ProxyField residential proxies + the Playwright script on this page | $2.95/GB for 1 GB, from $1.95/GB at volume | today: Easy Trial, 1 GB for $2.95 |
| Public fare pages returned to you rendered, with your own parser | ProxyField Web Scraping API | $2.00 per 1,000 rendered pages, $1.00 from 10,000 | in development: Contact sales |
Start today, on proxies. Buy the Easy Trial (1 GB for $2.95; no subscription, the traffic never expires), put your login in the Playwright script and run it. How many searches 1 GB covers depends on the page weight: a results page that loads about 2 MB with its scripts (an example input, not a measurement; measure your own target) means about 500 page loads per GB, so about $5.90 per 1,000 pages at $2.95/GB and a third less at $1.95/GB. Blocking images and fonts in the browser cuts the weight.
Which route fits: partner API, parsed data or a scraper
| You need | The route that fits |
|---|---|
| To sell or book fares, or show them as bookable | The airline's or OTA's own API under a partner agreement, or a booking API such as Amadeus for Developers or Duffel |
| Skyscanner's own inventory as data | Skyscanner's Travel API: "access to our Flights APIs is through a commercial agreement only" |
| Ready-parsed flight fields with no parser of your own | A vendor that sells a parsed flight dataset; ProxyField does not sell one |
| Public fares as seen from many countries, with your own parser | ProxyField's Web Scraping API ($2.00 per 1,000 rendered pages, from $1.00), or your own scraper on residential proxies from $1.95/GB |
Whichever route you take, the sites' terms come first: many travel sites and airlines restrict automated collection (see Scrape travel sites or use partner APIs? below). Our rule for every site: public pages only, follow the site's terms.
What a flight scraper collects from public pages
A flight scraper reads the fares that online travel agencies (OTAs) and metasearch sites show any visitor for a route and a date. The Web Scraping API fetches the URL and returns the page; your code pulls the fares out of it. It has no parsed flight or hotel fields.
| Travel page | What the public page shows |
|---|---|
| Flight search results (Skyscanner, Kayak, Momondo, Priceline) | Airlines, departure and arrival times, stops, duration, fare per passenger, the seller behind each fare |
| Date grids and fare calendars | The lowest fare shown for each day or month on a route |
| Hotel search results (Booking.com, Trivago, Priceline, Google Hotels) | Hotel name, stars, review score, price for the dates, taxes note, location |
| Metasearch offers (Trivago, Skyscanner) | The same flight or room priced by several sellers |
Account pages, bookings and member-only rates sit behind a login and are out of scope: ProxyField's acceptable use policy forbids accessing data behind logins you are not entitled to.
Scrape travel sites or use partner APIs?
To sell, book or show fares as bookable, use a partner or booking API; to monitor public prices, compare markets or do research, a scraper on public pages fits, within each site's terms. A partner API gives you fare rules, availability and a price you can confirm before booking; a scraper gives you what a visitor sees, which may be cached, personalised or out of date a minute later.
Read moreShow less
| Your product | Better route | Why | Cost |
|---|---|---|---|
| Booking app or OTA | Airline or OTA partner API, or a booking API (Amadeus for Developers, Duffel) | You need bookable offers, fare rules and a price check before payment | the provider's terms or a commercial agreement |
| Metasearch site that shows other sites' fares | A licence from each site | Airlines such as Ryanair license price comparison only in writing | the licence |
| Fare monitoring and price history | Scraper on public pages, within the terms | You need what a visitor sees, per market, over time | Web Scraping API $2.00 per 1,000 rendered pages ($1.00 from 10,000), or residential proxies from $1.95/GB |
| Market and pricing research | Scraper on public pages, within the terms | You compare markets and sellers; nothing is sold or booked | as above |
The limits of scraped fares:
- Prices vary by country, currency, dates and device, so a scraped fare is a reading, not an offer. Confirm the price on the seller's site before you show it as bookable.
- Pages change. Travel sites rebuild their result pages often, and a parser breaks with them.
- Terms apply. Ryanair's Terms of Use bar the use of "any automated system or software ... to extract any data" from its site for commercial purposes (clause 3), allow price comparison sites access to its data only under a written licence "for the sole purpose of price comparison" (clause 2), and reserve the right to block and to bring legal proceedings (clauses 4 and 9) (Ryanair terms). Frontier Airlines' Terms of Use say under "No Scraping" that "website scraping," "data scraping" or "screen scraping" of its site "is strictly prohibited". Skyscanner's Terms of Service restrict automated collection too.
ProxyField does not sell a fare or booking API. More on the trade-off: web scraping vs API.
Why one flight shows many prices
The fare you scrape depends on who asks. Travel sites price by point of sale (the country the search comes from), currency, travel and search dates, and sometimes device, so a London–New York fare seen from the UK, Germany and the US can differ. To collect what each market sees, send the search from that country: ProxyField residential proxies offer targeting by country, region (state) and city, with 412M+ residential IPs in 195+ countries, for example the United Kingdom or Germany. On a Web Scraping API request, country sets the country of the exit IP as a two-letter code such as de, and it accepts the proxy pool's 195+ countries. Mobile web and app prices can differ again; mobile proxies send the request over carrier networks.
How a request works
You send a search URL and the Web Scraping API returns the page, rendered in a browser when you set render: true. This is the API's design (API documentation): a POST to https://api.proxyfield.io/v1/scrape with your API key in an Authorization: Bearer <api_key> header and a JSON body with url, render, country, output (html, markdown, text or json) and, for rendered pages, wait_for, a CSS selector the page must show before it is returned.
Read moreShow less
# The request the Web Scraping API is designed to take (access through Contact sales)
curl https://api.proxyfield.io/v1/scrape \
-H "Authorization: Bearer <api_key>" \
-H "Content-Type: application/json" \
-d '{"url": "https://www.kayak.com/flights/JFK-LHR/2026-12-10?sort=price_a", "render": true, "country": "gb", "output": "json"}'Example response (shortened; content is the rendered page's HTML):
{
"metadata": {"id": "req_2f9d84c1e07ab365", "api": "web_scrape", "status": "success", "billed": true},
"status_code": 200,
"final_url": "https://www.kayak.com/flights/JFK-LHR/2026-12-10?sort=price_a",
"content_type": "text/html; charset=utf-8",
"content": "<!DOCTYPE html><html lang=\"en-GB\">…</html>"
}Parse the fares into rows with Python
The API returns the page, so the parsing is yours. This turns the JSON response into rows of fare, departure and arrival time, one per result card. Read the card selector from the page you target in your browser's developer tools; the one below is a placeholder.
import re
import requests
from bs4 import BeautifulSoup
URL = "https://www.kayak.com/flights/JFK-LHR/2026-12-10?sort=price_a"
CARD = "div.result-card" # placeholder: copy the result card's selector from the page
resp = requests.post(
"https://api.proxyfield.io/v1/scrape",
headers={"Authorization": "Bearer <api_key>"},
json={"url": URL, "render": True, "wait_for": CARD, "country": "gb", "output": "json"},
timeout=200,
)
data = resp.json()
print("billed:", data["metadata"]["billed"], "status:", data["status_code"])
soup = BeautifulSoup(data["content"], "html.parser")
rows = []
for card in soup.select(CARD):
text = card.get_text(" ", strip=True)
fare = re.search(r"[$£€]\s?\d[\d,.]*", text)
times = re.findall(r"\b\d{1,2}:\d{2}\b", text)
rows.append({
"fare": fare.group(0) if fare else None,
"depart": times[0] if times else None,
"arrive": times[1] if len(times) > 1 else None,
"country": "gb",
})
print(rows[:5])What rows looks like, one row per result card (illustrative values to show the format, not real fares):
| fare | depart | arrive | country |
|---|---|---|---|
| £412 | 08:15 | 20:05 | gb |
| £438 | 10:40 | 22:55 | gb |
| £455 | 18:30 | 06:20 | gb |
At scale, 200 routes × 30 departure dates checked daily is 180,000 such pages a month: $360 at the entry JS price, or $180 at the volume price (table below).
Many travel pages also load their results as JSON in the background, which is easier to read than the HTML: see how to scrape hidden APIs and embedded JSON.
Pricing: $0.002 per rendered page, $0.001 at volume
Most flight search pages build their results with scripts, so they need JS rendering: $2.00 per 1,000 successful results, which is $0.002 per page, or $1.00 per 1,000 ($0.001 per page) from 10,000 results in one purchase. Hotel pages and some listings carry their data in the HTML, and plain HTTP costs $0.50 per 1,000, or $0.20 from 10,000 results.
You pay only for successful results. Billed: the page the travel site returns, including a 404 or 410 and a sign-in page served with a 200 status, as is the market norm. Not billed: error responses, blocked and timed-out requests and our internal retries. Every response says whether it was billed. One result is one page: a search that loads more results on a second page counts twice.
The API is paid from the same crypto balance as proxies, which never expires; there is no subscription, plan or monthly fee. The API is in development; for access, contact sales through the sales team on our 24/7 live chat. All prices: pricing.
Through the sales team on our 24/7 live chat
| Workload (30-day month) | Pages a month | JS at $2.00 | JS at $1.00 (volume) | HTTP at $0.50 | HTTP at $0.20 (volume) |
|---|---|---|---|---|---|
| 200 routes × 30 departure dates, daily | 180,000 | $360 | $180 | $90 | $36 |
| 50 routes × 14 dates × 3 countries, daily | 63,000 | $126 | $63 | $31.50 | $12.60 |
| 1,000 hotels × 2 date pairs, daily | 60,000 | $120 | $60 | $30 | $12 |
Scrape Skyscanner with Python
The short answer: apply for Skyscanner's Travel API (an x-api-key, with published defaults of 100 create-search and 500 poll calls a minute on Live Prices); if you collect public pages instead, use one sticky session per search, several seconds between searches, back off on 429 and 403, and stop at the third refusal. What not to do: switch IPs to keep going after a refusal, or try to defeat a challenge; ProxyField does not solve CAPTCHAs.
Decide in this order:
-
Ask for Skyscanner's Travel API first
Access is "through a commercial agreement only": you apply to Skyscanner's partnerships team, and an approved key goes in the
x-api-keyheader of calls topartners.api.skyscanner.net. Its published default limits are 100 "create search" and 500 "poll search" calls a minute on the Live Prices API, and 500 a minute on the Indicative Prices API. Respect them with a queue and backoff. -
If you collect public pages instead, read Skyscanner's Terms of Service first: they restrict automated collection. Our rule is public pages only, follow the site's terms.
-
Pace your own scraper so it behaves like a careful visitor: one search at a time per session, a pause of several seconds between searches, one sticky session per search so its pages load from one IP, and a full stop when the site refuses you.
Read moreShow less
| Refusal | What it means | What to change |
|---|---|---|
429 Too Many Requests | you searched faster than the site allows | wait as long as Retry-After asks, or 8 s, then 16 s, then 32 s; run fewer searches in parallel |
403 Forbidden | the site refused this client | slow down and check your headers; after the third refusal, stop and review |
| A challenge page instead of results | the site wants proof of a human visitor | stop the run; do not try to defeat it |
Skyscanner's help centre explains its blocks: Why have I been blocked from accessing the Skyscanner website?
This function fetches one page through ProxyField's gateway on one sticky session, backs off on 429 and 403 with growing pauses, and stops after three refusals instead of retrying from new IPs:
import time
import requests
def fetch_politely(url, country="gb", session="sky01", attempts=3, pause=8):
user = f"<login>-country-{country}-session-{session}-ttl-600"
proxy = f"http://{user}:<password>@gate.proxyfield.io:7000"
delay = pause
for _ in range(attempts):
r = requests.get(url, proxies={"http": proxy, "https": proxy},
headers={"Accept-Language": "en-GB"}, timeout=60)
if r.status_code in (403, 429):
wait = r.headers.get("Retry-After")
time.sleep(float(wait) if wait and wait.isdigit() else delay)
delay *= 2 # 8 s, 16 s, 32 s
continue
r.raise_for_status()
time.sleep(pause) # the gap before the next search
return r.text
raise RuntimeError(f"{url}: refused {attempts} times; stop and review before you continue")Skyscanner's result pages build their fares with scripts, so for them use the Playwright script below or the API's JS rendering; the pacing rules stay the same. Status codes explained: 429 Too Many Requests. More: web scraping without getting blocked in Python.
Scrape flight prices with Python and ProxyField proxies
This script opens one Kayak search in a real browser through a residential IP in each of two countries and prints the prices it finds. Playwright runs the page's scripts, which a plain HTTP request would not.
import re
from playwright.sync_api import sync_playwright
URL = "https://www.kayak.com/flights/JFK-LHR/2026-12-10?sort=price_a"
with sync_playwright() as p:
for country in ("us", "gb"):
browser = p.chromium.launch(proxy={
"server": "http://gate.proxyfield.io:7000",
"username": f"<login>-country-{country}-session-kyk01-ttl-600",
"password": "<password>"})
page = browser.new_page()
page.goto(URL, wait_until="networkidle", timeout=90_000)
prices = re.findall(r"[$£€]\s?\d[\d,]*", page.inner_text("body"))
print(country, prices[:10])
browser.close()Access opens once the payment is confirmed; the gateway, credentials or allowlist are in your account. The username takes -region-<region>, -city-<city>, -session-<id> and -ttl-<seconds> (30 to 129,600 seconds, 20 minutes by default) after the country, as in <login>-country-us-region-new_york-city-new_york-session-he9f2qga-ttl-600.
-
Build the search URL: route, dates, passengers, sort order.
-
Open it through
gate.proxyfield.io:7000with the country you want to price from. -
Wait for the results, then read airline, times, stops and fare from each result card.
-
Store fare, currency, country and the time of the check, and repeat on a schedule, paced as above.
Residential proxies cost $2.95/GB for 1 GB and from $1.95/GB at volume, with unlimited concurrent threads; the traffic never expires, and all purchases are final. More on browser automation: Playwright.
What teams use travel data for
- Fare monitoring: price history per route, date and market.
- Hotel rate checks: how a hotel's price on Booking.com or Trivago compares across dates and sellers.
- Revenue and pricing research for airlines, hotels and travel agencies.
- Market research: routes, demand signals and seasonality.
Booking, holding seats or reselling inventory through a scraper is not data collection; fraud and fake accounts are forbidden under ProxyField's acceptable use policy.
Legal questions
Most travel sites restrict automated collection in their terms, including Skyscanner's Terms of Service, Booking.com's Terms of Service and Ryanair's Terms of Use, and travel companies have taken scrapers to court. In the EU, Ryanair v PR Aviation (C-30/14, 2015) held that a site's terms can limit the use of data that database rights do not protect. Collect public fares only, keep personal data out, and get a licence or partner access when you resell or display a site's content. For the wider question, read Is Web Scraping Legal? US, EU and UK Rules (2026).
ProxyField's rule for every site: public pages only, and follow the site's terms. This is not legal advice.
More scrapers
- Google Flights scraper and airline websites.
- Hotel prices, Airbnb, Expedia and travel reviews, on the same Web Scraping API.
- Proxies for travel fare aggregation: running your own fare collection.
- Google SERP API: Google results for travel keywords, $1.20 per 1,000 successful results, or $0.80 from 10,000 results in one purchase.
- All scrapers: scrapers.
Flight Scraper FAQ
24/7 support by live chat, including help with setup and integration.
What's a good flight price scraper or API that collects fares from sites like Skyscanner, Momondo and Priceline?
ProxyField residential proxies let you run your own fare scraper today, starting with the Easy Trial at 1 GB for $2.95, and ProxyField's Web Scraping API is designed to return those sites' public search pages rendered for $2.00 per 1,000 successful results ($1.00 from 10,000); the API is in development, so access is through Contact sales. To book or show fares as bookable, use an official API instead: Skyscanner's Travel API under a commercial agreement, Amadeus for Developers or Duffel. Check each site's terms before you collect.
How do I scrape Skyscanner fares with Python without running into its rate limits and bot checks?
Apply for Skyscanner's Travel API first: its published defaults are 100 create-search and 500 poll calls a minute, with an x-api-key header. For public pages, read Skyscanner's Terms of Service, which restrict automated collection; if you go ahead, pace searches several seconds apart on one sticky session, back off on 429 and 403 (8 s, 16 s, 32 s), and stop at the third refusal rather than switching IPs.
Is it better to scrape travel sites for fares or get them through airline and OTA partner APIs?
To sell, book or show fares as bookable, a partner or booking API is better: it gives fare rules, availability and a price you can confirm, and airlines such as Ryanair forbid commercial scraping and license price comparison only in writing. To monitor public prices or research markets, a scraper on public pages fits, within each site's terms.
How much does it cost to scrape flight prices?
ProxyField's Web Scraping API costs $2.00 per 1,000 successful results with JS rendering, which most flight search pages need, or $1.00 from 10,000 results; pages that need no scripts cost $0.50 per 1,000 over plain HTTP, or $0.20. Checking 200 routes on 30 departure dates every day is 180,000 pages a month: $360 at the entry JS price, $180 at the volume price.
What is a flight scraper?
A tool or script that collects the fares, airlines, times and stops that travel sites show for a route and a date. It reads the public search results of OTAs and metasearch sites, usually on a schedule, so you can track how fares move.
How do I scrape flight prices?
Build the search URL for a route and date, load it with a browser or an API that runs JavaScript, and read the fare, airline, times and stops from each result. Send the search from the country whose prices you want, pace it, and repeat it on a schedule to build a price history.
Can I collect flight prices as seen from another country?
Yes: send the search through an IP in that country, because travel sites price by point of sale, currency and sometimes device. ProxyField residential proxies offer targeting by country, region (state) and city, with 412M+ residential IPs in 195+ countries.
How do I scrape flight data with Python?
Use Playwright, because flight results load through scripts: open the search URL through a proxy, wait for the results and read the result cards, as in the code on this page. Requests and Beautiful Soup work on pages that carry their data in the HTML, or on the HTML a rendering API returns.
What flight ticket data can a scraper collect?
The public results show the airline, flight times, stops, duration, fare per passenger, fare conditions in brief and the site selling the ticket. Booking a ticket, holding a seat or reading your account's bookings is not part of scraping.
What travel data can be scraped?
Public travel pages show flight fares and schedules, hotel prices and availability for given dates, room types, review scores and the sellers behind each offer. Member rates and anything behind a login are out of scope.
Show 16 more questionsShow fewer questions
What is travel data scraping?
The automated collection of public fares, hotel rates and availability from travel sites. Teams run it on a schedule across routes, dates and countries, then compare the results over time.
Why do travel businesses need web scraping?
Fares and room rates change many times a day and differ by market, so airlines, hotels, agencies and travel startups watch them to set their own prices. Scraped data also feeds deal alerts, demand research and route planning.
How do I scrape and extract data from travel websites?
Fetch the search or listing page, rendered if it builds results with scripts, then parse the fields you need into rows: route or hotel, date, price, currency, seller and the time of the check. Many travel pages also load their results as JSON in the background, which is easier to read than the HTML.
What is a Booking scraper API?
An API that fetches Booking.com search and hotel pages for you, so you send a URL and get the page back. ProxyField's Web Scraping API is designed to do this for any URL from $0.50 per 1,000 successful results; there is no separate Booking.com endpoint with parsed fields.
Does Booking.com have an API?
Booking.com gives API access to approved partners, such as affiliates and connectivity providers, under its partner terms. Check its partner pages for what is open to your business before you plan to collect its pages.
How do I scrape Booking.com hotel prices?
Open a search results URL with the destination, check-in, check-out and guest count, then read each hotel's name, score and price for those dates. Prices change with dates and the visitor's country, so fix both in every check, and follow Booking.com's terms.
How do I scrape Skyscanner?
Start with Skyscanner's Travel API, which it opens through a commercial agreement. For public pages, read its Terms of Service first, then load a search URL with JS rendering, wait for the results, read the fares and sellers, and pace searches several seconds apart.
What is the rate limit for scraping Skyscanner?
For its partner Travel API, Skyscanner publishes defaults of 100 create-search and 500 poll calls a minute on Live Prices, and 500 a minute on Indicative Prices. For its public website it publishes no number; it refuses clients that search too fast, so keep several seconds between searches and stop when refused.
What is a Priceline scraper API?
An API that returns Priceline's flight, hotel and car pages for your own parser. ProxyField has no separate Priceline endpoint; the Web Scraping API is designed to fetch those URLs at $2.00 per 1,000 successful results with JS rendering, or $1.00 from 10,000 results.
How do I scrape Momondo?
Momondo belongs to the same group as Kayak and builds its flight results with scripts, so load the search URL with JS rendering or Playwright and read the fares once the results settle. Momondo runs separate sites per country, and each shows its own prices.
What is a Trivago scraper API?
An API that fetches Trivago's hotel comparison pages, where one hotel is priced by several booking sites. With ProxyField you send the Trivago URL to the Web Scraping API and parse the offers yourself.
How do I scrape Orbitz?
Treat it like any OTA: open a flight or hotel search URL with JS rendering and read the results. Fix the dates, passengers and the country you search from, so that repeated checks stay comparable.
How do I scrape Google Hotels?
Google Hotels pages build their price lists with scripts, so fetch them with JS rendering or a browser and read the hotel, the dates and each seller's price. The country of the request decides the currency and the offers shown.
How much does it cost to scrape Google Hotels?
With ProxyField's Web Scraping API, a Google Hotels page with JS rendering costs $2.00 per 1,000 successful results, or $1.00 from 10,000 results. Checking 500 hotels once a day is 15,000 pages a month: $30, or $15 at the volume price.
Is a fly scraper the same as a flight scraper?
In search, yes: "fly scraper" is mostly a shorter way of asking for a flight scraper, a tool that collects fares from travel sites. Everything on this page applies to it.
Is travel fare scraping legal?
Collecting public fares is common, but travel sites and airlines restrict automated access in their terms, and in the EU a site's terms can limit the use of its data (Ryanair v PR Aviation, 2015). Collect public pages only, keep personal data out and get partner access or a licence when you resell or display a site's content. This is not legal advice.
Related pages
Start collecting public pages with the Web Scraping API
Billed on successful results only. Contact sales for the Web Scraping API and the volume you need, or run your own scraper on residential proxies. 24/7 live chat helps with setup and integration.