proxyfield
Start with 1 GB
Site scraper

Google News Scraper: Headlines by Edition, With Real Article URLs

HTTP $0.50JS rendering $2.00 per 1,000 results or your own scraper from $1.95/GB

To scrape Google News headlines for one country's edition with Python, fetch its RSS feed, such as news.google.com/rss/search?q=<query>&hl=fr&gl=FR&ceid=FR:fr, where hl is the language, gl the country and ceid the pair. Parse the feed with feedparser, then turn each item's news.google.com/rss/articles/… link into the publisher's URL with a decoder such as googlenewsdecoder, because following the redirect ends on a Google page. ProxyField residential proxies give the scraper an IP in the edition's country from $1.95/GB, and a feed is a few kilobytes; start with the Easy Trial, 1 GB for $2.95. To send feed URLs instead, ProxyField's Web Scraping API is designed for $0.50 per 1,000 feeds, or $0.20 from 10,000 results in one purchase. It is in development, so access is through Contact sales.

Get residential proxies from $1.95/GB

Contact sales for the Web Scraping API

Web Scraping API per 1,000 successful results
Web Scraping API prices per 1,000 successful results
FetchEntryVolume, from
HTTP$0.50$0.20
JS rendering$2.00$1.00

Billed on successful results only.

Your own scraper on residential proxies
from $1.95/GB Easy Trial: 1 GB for $2.95
gate.proxyfield.io:7000
Data

Scrape one Google News edition with Python

This function builds the feed URL for an edition from two arguments, fetches it through a ProxyField IP in that country and parses it with feedparser. It then resolves each Google link to the publisher's article URL and keeps the Google link as a fallback. The worked example is the French edition.

Read moreShow less
Shell
pip install feedparser requests googlenewsdecoder   # googlenewsdecoder needs Python 3.11+
Python
# google_news.py: one edition's headlines with publisher URLs
from urllib.parse import urlencode, urlparse
import feedparser, requests
from googlenewsdecoder import gnewsdecoder

PROXY = "http://<login>-country-fr-session-gn1-ttl-600:<password>@gate.proxyfield.io:7000"

def feed_url(query=None, hl="fr", country="FR"):
    edition = {"hl": hl, "gl": country, "ceid": f"{country}:{hl.split('-')[0]}"}
    if query:                                    # search feed
        return "https://news.google.com/rss/search?" + urlencode({"q": query, **edition}, safe=":")
    return "https://news.google.com/rss?" + urlencode(edition, safe=":")   # top stories

def google_news(query=None, hl="fr", country="FR", limit=20, resolve=True):
    resp = requests.get(feed_url(query, hl, country), timeout=30,
                        proxies={"http": PROXY, "https": PROXY},
                        headers={"User-Agent": "Mozilla/5.0"})
    resp.raise_for_status()
    feed = feedparser.parse(resp.content)
    for e in feed.entries[:limit]:
        google_link = e.link
        url, error = None, None
        if resolve:
            r = gnewsdecoder(google_link, interval=1, proxy=PROXY, timeout=15)
            if r.get("success") and "google." not in urlparse(r["decoded_url"]).netloc:
                url = r["decoded_url"]
            else:
                error = r.get("message", "resolved to a Google URL")
        yield {"id": e.get("id"), "title": e.title.rsplit(" - ", 1)[0], "published": e.get("published"),
               "publisher": e.get("source", {}).get("title"),
               "publisher_site": e.get("source", {}).get("href"),
               "url": url, "google_link": google_link, "error": error}

if __name__ == "__main__":
    for item in google_news("énergie renouvelable when:1d", hl="fr", country="FR"):
        print(item["published"], "|", item["publisher"], "|", item["title"])
        print("   ", item["url"] or f"unresolved: {item['error']}")

feed_url("énergie renouvelable when:1d", "fr", "FR") builds https://news.google.com/rss/search?q=%C3%A9nergie+renouvelable+when:1d&hl=fr&gl=FR&ceid=FR:fr, and feed_url(None, "en-GB", "GB") builds the UK top-stories feed https://news.google.com/rss?hl=en-GB&gl=GB&ceid=GB:en. Each record looks like this:

Python
{"id": "CBMi…", "title": "<headline>", "published": "Mon, 26 Oct 2026 08:00:00 GMT",
 "publisher": "Le Monde", "publisher_site": "https://www.lemonde.fr",
 "url": "https://www.lemonde.fr/…", "google_link": "https://news.google.com/rss/articles/CBMi…?oc=5", "error": None}

  1. Build the edition hl (fr, en-GB, de), gl (FR, GB, DE) and ceid (FR:fr, GB:en, DE:de). Keep the three in agreement, or the feed comes back mixed or empty.

  2. Fetch through an IP in that country -country-fr in the ProxyField username. A sticky session (-session-<id>-ttl-600) keeps one IP for the feed and the link decoding.

  3. Parse feedparser gives title, link, published, id (the item's GUID) and source (publisher name and site). Google appends " - " to each title, which the function strips.

  4. Resolve and store keep url when the decoder succeeds and the host is not Google's. Otherwise store google_link with the error. Key records by id so the next run skips items you already have.

The parsing above was run on 2026-10-27 against a saved Google News RSS item, with the decoder's success and failure answers stubbed. Run it against a live feed before you depend on it. The function needs requests, feedparser and googlenewsdecoder 0.2.2 (MIT license). More on proxies in Python: Python.

Get the real article URLs, not Google's

Every item's <link> is a Google News address of the form news.google.com/rss/articles/CBMi…, not the publisher's page. Following it with requests or curl -L does not get you there: the redirects end on a news.google.com page, and the last hop to the publisher happens in the browser's JavaScript. The GNews documentation says it plainly: "Google requires JavaScript execution to resolve them." Base64-decoding the CBMi… ID does not work for current links either.

What a decoder does (from the googlenewsdecoder source, version 0.2.2):

  1. GET news.google.com/rss/articles/<id> and read two attributes from the page, data-n-a-ts (a timestamp) and data-n-a-sg (a signature).

  2. POST the ID, timestamp and signature to news.google.com/_/DotsSplashUi/data/batchexecute and read the publisher URL from the answer. Pass a list of links to gnewsdecoder() and they share one POST.

  3. Return {"success": True, "decoded_url": …} or {"success": False, "message": …}. A batchexecute answer with no URL comes back as success: False, so check it every time, as the function above does.

Read moreShow less

Your options, and what each costs you:

Way to get the publisher URLHowWatch for
googlenewsdecoderTwo requests per link (or one POST for a list), plain HTTPAn undocumented endpoint: re-test after any change on Google's side
GNews with its Playwright extra (pip install "gnews[playwright]", then playwright install chromium)Opens each link in a headless ChromiumIts docs give about 70–80% success and say it silently returns the Google URL on failure; Chromium is about 150 MB
Keep source onlyThe publisher's name and site from each RSS itemNo article URL, but no extra requests
  • Traffic each link's Google page weighs far more than the feed, so resolving costs most of your traffic. Resolve each item once, cache the result by GUID, and resolve only the items you keep.
  • After resolving fetching the article itself is a request to the publisher's site, where its own bot rules apply. That job is covered by the news scraper.

Editions, feed types and limits

FeedURLWhat it holds
Top stories of an editionnews.google.com/rss?hl=de&gl=DE&ceid=DE:deThe edition's front page
Searchnews.google.com/rss/search?q=<query>&hl=en-US&gl=US&ceid=US:enItems for a query; operators when:1h, when:1d, when:7d, site: work as in the search box
Topicnews.google.com/rss/headlines/section/topic/BUSINESS?hl=en-GB&gl=GB&ceid=GB:enA section of an edition (WORLD, NATION, BUSINESS, TECHNOLOGY, SCIENCE, HEALTH, SPORTS, ENTERTAINMENT)
Web pagenews.google.com/search?q=<query>&hl=en-US&gl=US&ceid=US:enThe same stories as cards, with images and story clusters; needs JS rendering

Common editions: US:en (hl=en-US&gl=US), GB:en (hl=en-GB&gl=GB), DE:de, FR:fr, ES:es, IT:it. Fetch each from its own country: ProxyField has 61.2M+ residential IPs in the United States, 16.6M+ in the United Kingdom, 20.2M+ in Germany and 8.3M+ in France.

Limits readers hit (Google publishes none of this, so re-check):

  • A feed returns up to about 100 items and has no page parameter. To go deeper, split the query by time (when:1d) or by outlet (site:lemonde.fr) and merge the feeds by GUID.
  • Search feeds are not sorted by time and can include items several days old. Add when:1h or when:1d when you need fresh headlines.
  • The feed is not a supported product: no key, no contract, and the format can change without notice.
Compare

Is there a Google News API?

Google offers no public API for reading Google News. Its News Search API was deprecated in 2011 and has since been shut down, and the RSS feeds are the open format it still publishes. The News tab of Google Search is a separate view of Google's results. ProxyField's Google SERP API is designed to return the Top stories block of a normal search, at $1.20 per 1,000 successful results; the News tab is not part of it.

ItemGoogle News RSS with your own scraperProxyField Web Scraping API (in development)
AccessOpen feed, no keyThrough Contact sales
OutputXML you fetch and parseThe feed or page, for your parser
IPsYours, or ProxyField residential proxies from $1.95/GBProxyField's pool
BillingProxy traffic per GBPer 1,000 successful results

What you send to the Web Scraping API

The Web Scraping API is designed to fetch a Google News feed or page and return it; your code takes the fields out and resolves the links. ProxyField sells no separate Google News API with pre-parsed fields. As designed:

  • The request a POST to https://api.proxyfield.io/v1/scrape, with your API key in an Authorization: Bearer <api_key> header and a JSON body: - url; - render (false for RSS, true for the web pages); - country, the exit IP's country as a two-letter code; it accepts the proxy pool's 195+ countries; - output (html, markdown, text or json).
  • The response the feed itself, or a JSON envelope with the status code, the final URL, the headers, the cookies and the content.

Parameter reference: API documentation.

Shell
# The request as designed (in development; access through Contact sales)
curl https://api.proxyfield.io/v1/scrape \
  -H "Authorization: Bearer <api_key>" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://news.google.com/rss/search?q=voitures+electriques+when:1d&hl=fr&gl=FR&ceid=FR:fr", "render": false, "country": "fr", "output": "json"}'

Example response (shortened; content is the feed's XML):

JSON
{
  "metadata": {"id": "req_7c3e19a4b2d05f86", "api": "web_scrape", "status": "success", "billed": true},
  "status_code": 200,
  "final_url": "https://news.google.com/rss/search?q=voitures+electriques+when:1d&hl=fr&gl=FR&ceid=FR:fr",
  "content_type": "application/xml; charset=utf-8",
  "content": "<?xml version=\"1.0\" encoding=\"UTF-8\"?><rss version=\"2.0\">…</rss>"
}
Pricing

Pricing: $0.50 per 1,000 feeds, $0.20 from 10,000

A Google News RSS feed is one plain HTTP result. The Web Scraping API is priced at $0.50 per 1,000 successful results, or $0.20 from 10,000 results in one purchase. Web pages with JS rendering cost $2.00 per 1,000, or $1.00 from 10,000.

You pay only for successful results:

Workload (30-day month)
Workload (30-day month), Feeds a month, Plain HTTP at $0.50, At $0.20 (from 10,000)
Workload (30-day month)Feeds a monthPlain HTTP at $0.50At $0.20 (from 10,000)
200 brand and keyword feeds, every hour144,000$72$28.80
10 topics × 5 editions, every 30 minutes72,000$36$14.40
1,000 keyword feeds, once a day30,000$15$6
  • Billed a feed or page Google answered with a 2xx status, and a target's 404 or 410.
  • Not billed blocked requests, timeouts, errors on our side and our internal retries on new IPs.

Every response says whether it was billed. For availability and access, contact sales through the sales team on our 24/7 live chat.

The API is paid from the same crypto balance as proxies, which never expires; there is no subscription or monthly fee. With your own scraper, residential proxies cost $2.95/GB for 1 GB down to $1.95/GB from 1,000 GB; the traffic never expires, and all purchases are final. All prices: pricing.

Contact sales

Through the sales team on our 24/7 live chat

When Google refuses a request

Google answers too many requests from one IP with a 429 or a "sorry" page instead of the feed. Space your fetches, cache a feed for a few minutes instead of fetching it again, and spread editions over IPs from their own countries. The interval argument of gnewsdecoder adds a pause after each link's page fetch, for large batches. The pool is clean for Google, checked and segmented. Access to the gateway opens once the payment is confirmed; the credentials or allowlist are in your account.

Use cases

What teams use Google News data for

  • Media monitoring every new headline that names your brand, product or people, edition by edition.
  • PR reporting which outlets covered a launch, and when.
  • Market and competitor tracking news about competitors, suppliers or a sector.
  • Trading and risk signals headline counts and timing by company or topic.
  • Research datasets coverage of a topic across outlets and editions over time.
Terms and law

Google's terms and the law

Google's terms restrict automated access to its services, and the headlines and snippets belong to their publishers, so store headlines, links and times rather than republishing article text. The legal side, for Google News as well as Search, Maps and Images, is in Is It Legal to Scrape Google?; the wider rules are in Is Web Scraping Legal? US, EU and UK Rules (2026).

ProxyField's rule for every site: public pages only, and follow the site's terms.

This is not legal advice.

More scrapers

More scrapers

  • News scraper: the articles themselves, from thousands of news sites.
  • Google SERP API: the Google results page as JSON, with the AI Overview, People Also Ask and the local pack parsed.
  • Google Trends scraper: search interest over time for the topics in the news.
  • All scrapers: scrapers.
FAQ

Google News Scraper FAQ

24/7 support by live chat, including help with setup and integration.

How do I scrape Google News headlines for one country's edition and language with Python?

ProxyField's script on this page does it in 3 steps: build the feed URL with hl, gl and ceid (for France hl=fr&gl=FR&ceid=FR:fr), fetch it through an IP in that country, and parse it with feedparser into title, publisher, time and link. Keep the three edition parameters in agreement, or the feed comes back mixed or empty. A ProxyField IP in the edition's country costs from $1.95/GB, and one feed is a few kilobytes.

How do I get the real article URL from a Google News RSS link?

Use a decoder such as googlenewsdecoder: gnewsdecoder(link) reads the signature and timestamp from the link's Google page and asks Google's batchexecute endpoint for the publisher URL. It returns success and decoded_url, or success: False with a message. Store the Google link as a fallback, because the endpoint is undocumented and can change.

How much does it cost to scrape Google News?

ProxyField's Web Scraping API is priced at $0.50 per 1,000 successful Google News RSS feeds over plain HTTP, or $0.20 from 10,000 results in one purchase, and $2.00 per 1,000 rendered web pages. Checking 200 keyword feeds every hour is 144,000 feeds a month: $72, or $28.80 at the volume price. The API is in development; access is through Contact sales.

What is a Google News scraper?

A Google News scraper collects the headlines Google News lists for a query, topic or edition, with each item's publisher, time and link. It reads the RSS feeds or the news.google.com pages; the articles themselves stay on the publishers' sites.

Is it possible to scrape Google News?

Yes: Google News publishes RSS feeds for searches, topics and each edition's top stories, which any HTTP client can fetch, and the web pages can be loaded in a browser. Space your requests, because Google answers too many from one IP with a 429 or a "sorry" page.

How do I scrape Google News?

Build a search or topic feed URL with the edition parameters, fetch it on a schedule, parse the XML items, resolve the links you keep and store them keyed by GUID. For more queries or editions, spread the requests over residential IPs from each edition's country, or send the feed URLs to a scraping API.

How do I scrape Google News with Python?

Fetch the RSS feed with requests through a proxy, parse it with feedparser, and resolve each link with googlenewsdecoder; the function on this page returns title, publisher, time, publisher URL and the Google link. The feed needs no browser.

How do I scrape Google News for a specific country or language?

Set the edition in the URL: hl for the language, gl for the country and ceid for the pair, such as ceid=GB:en or ceid=DE:de. Fetch it from an IP in the same country; ProxyField targets country, region and city at the same price per GB.

How many results does a Google News RSS feed return?

Up to about 100 items, and there is no page or offset parameter. To collect more, split the query by time with when: or by outlet with site:, then merge the feeds by GUID.

Show 11 more questionsShow fewer questions

What happened to the Google News API?

Google deprecated its News Search API in 2011, and it has since been shut down. Since then, the open way to read Google News by program is its RSS feeds, or scraping the web pages.

Is there an official Google News API?

No, not for reading Google News. Google publishes RSS feeds for searches, topics and editions, and the News tab of Google Search is part of the Google results page rather than a separate API.

What data can I extract from Google News?

Headlines, publisher names, publication times, links, the story cluster each headline belongs to and, on the web pages, thumbnails. The full article text, author and body are on the publisher's page, not on Google News.

What data can I extract from Google News RSS?

Each item has a title (with " - " appended), a Google link, a publication date, a GUID, the source with its site address and a description listing related coverage of the same story. The publisher's own article URL is not in the feed; you resolve it from the link.

Do I need an account to scrape Google News RSS?

No. The feeds are public and need no Google account or API key. A scraper should never sign in to Google to collect them.

How do I get structured JSON from Google News RSS?

Parse each item of the feed into a record with title, link, publication time, GUID and source, add the resolved URL, and write the list with json.dump. ProxyField's Web Scraping API returns the feed; turning it into JSON takes a few lines of code, as on this page.

Which programming languages can I use to scrape Google News RSS?

Any language with an HTTP client and an XML parser: Python, Node.js, Go, Java, PHP or Ruby. RSS is a standard format, so ready-made feed parsers exist for most of them; resolving the article links needs a decoder or a browser in any language.

How does pricing work for a Google News scraper API?

On ProxyField's Web Scraping API you pay per 1,000 successful results: one feed or page fetched is one result. A feed of the day's headlines for one query costs $0.50 per 1,000 fetches, or $0.20 from 10,000 results in one purchase.

How often can I scrape Google News?

As often as you need fresh headlines. Cache a feed for a few minutes instead of fetching it again, and spread the checks over the hour rather than sending them in one burst.

Can I try a Google News scraper before paying for volume?

For the API, contact sales with your queries, editions and frequency; you are billed only for successful results. To test your own scraper, start with the Easy Trial: 1 GB of residential traffic for $2.95, with no subscription, and the traffic never expires.

How do I scrape Google News search results?

Fetch the search feed news.google.com/rss/search?q=<query> with the edition parameters, or the search page with JS rendering if you want the cards as a browser shows them. Use operators such as when:1d to keep each result list to recent items.

Keep reading

Related pages

Collect Google News pages with the Web Scraping API

Billed on successful results only. Contact sales for the Web Scraping API and the volume you need, or run your own scraper on residential proxies. 24/7 live chat helps with setup and integration.

Contact sales Residential proxies from $1.95/GB