MLS Scraper: Public IDX and Broker Listing Pages via API
You can view public IDX listing pages without being a licensed agent, but not copy them at scale: MLS rules limit IDX data to consumers' personal, non-commercial use, so a database, app or resale product needs a licensed feed or written permission. Within that, ProxyField's Web Scraping API is designed to fetch the public pages where MLS listings reach the web, such as broker IDX search pages and portal listings, for $0.50 per 1,000 successful results over plain HTTP, $0.20 from 10,000 results in one purchase, or $2.00 per 1,000 with JS rendering. It never enters the MLS itself, a members-only database; its rule is public pages only, following each site's terms. The API is in development: contact sales, or run your own scraper on 61.2M+ US residential IPs today with the Easy Trial: 1 GB for $2.95.
Through the sales team on our 24/7 live chat
| Fetch | Entry | Volume, from |
|---|---|---|
| HTTP | $0.50 | $0.20 |
| JS rendering | $2.00 | $1.00 |
Billed on successful results only.
Can I get MLS listing data without being a licensed agent, by scraping public IDX listing pages?
Yes for viewing, no for copying at scale. The bottom line:
- Allowed reading public IDX and portal pages, and noting facts (price, beds, status) for your own decision.
- Needs a licensed feed or written permission a database, app, lead tool, resale product or AI dataset built on listings.
- Never an MLS login that isn't yours, members' areas, getting around logins or blocks, and reusing listing photos and agents' remarks.
Read moreShow less
Anyone can read the public IDX and portal pages that show MLS listings. Copying them at scale is a different matter. Two layers of rules apply: NAR's IDX policy is the rule set that MLSs and their broker members follow, and each MLS's own IDX rules (Stellar MLS, below, is one example) are what IDX sites must show you and what bind your use as a consumer. The rules differ from one MLS to the next, so read the terms on the IDX site you use. These five sources set the limits:
| Question | Short answer | Source |
|---|---|---|
| What is IDX? | The MLS's system that lets brokers show each other's listings: "IDX policy gives MLS participants the ability to authorize limited electronic display of their listings by other participants." | NAR: Internet Data Exchange (IDX) |
| What may a consumer do with IDX listings? | Typical MLS rules make sites state that IDX information is "exclusively for consumers' personal non-commercial use" and may not be used "for any purpose other than to identify prospective properties consumers may be interested in purchasing." | Stellar MLS rules, Article 19.24: Consumer use of IDX listings |
| Do IDX sites have to stop scrapers? | Yes, under typical MLS rules: each participant "shall make reasonable efforts to avoid 'scraping' of the MLS GRID Data by third parties", must monitor its site for it, and must report suspected scraping to the MLS. | Stellar MLS rules, Article 19.29: Security of IDX listings |
| Is reading a public page "unauthorized access"? | In hiQ Labs v. LinkedIn (9th Cir., April 18, 2022), the court upheld a preliminary injunction against LinkedIn, finding serious questions about whether the federal Computer Fraud and Abuse Act lets a site stop the collection of publicly available profiles. That ruling covers the CFAA only, at the injunction stage: the site's terms, copyright and state law still apply. | Ninth Circuit opinion, No. 17-16783 |
| Are listing photos and remarks protected? | Copyright protects original works such as photographs and written descriptions, not facts such as price, beds or square footage. | U.S. Copyright Office: What does copyright protect? |
If you are not an agent and need listing data for a product, the licensed routes are:
-
Ask the local MLS whether it licenses data to non-members or to vendors (many license feeds through the RESO Web API standard).
-
Work through a broker a participating broker can approve you as its vendor for an IDX or VOW feed under the MLS's rules.
-
Buy from an authorized data provider that holds MLS licenses for your area.
Whichever route you take, never use someone else's MLS login or collect from a members' area: it breaks the MLS's rules and ProxyField's acceptable use policy, which forbids accessing data behind logins you are not entitled to. When you collect public pages, store facts (price, size, status and dates) rather than photos and agents' remarks.
ProxyField's rule for every site: public pages only, and follow the site's terms. More in Is It Legal to Scrape eBay, Walmart, Zillow and Other Marketplaces? and Is Web Scraping Legal? US, EU and UK Rules (2026). This is not legal advice.
What an MLS is, and which part of it is public
A multiple listing service (MLS) is a database that local real estate associations run for their member brokers and agents, who share their listings in it. The United States has several hundred of them, each covering its own area, and the full records sit behind a members' login. Part of every listing reaches the public: brokers show other brokers' listings on their own websites through IDX (Internet Data Exchange), and the big portals show listings they receive from the MLSs.
| Where MLS listings appear publicly | Example | What the public page shows |
|---|---|---|
| Broker and agent IDX sites | a brokerage's "search homes" pages | Address, price, beds, baths, square footage, status, photos, listing brokerage, MLS number |
| National portals | Realtor.com, Zillow, Redfin listing pages | The same facts plus price and tax history where shown |
| Canadian MLS® listings | REALTOR.ca | Price, address, building and land details, listing brokerage |
| Public records (not the MLS) | county assessor and recorder sites | Owner of record, assessed value, sale history |
Which fields an IDX page shows is set by the local MLS's display rules, so the same listing can look different on two broker sites. A listing being public does not give you the right to collect it: each site's terms and the MLS's consumer-use rules still apply. The API is designed to return the page itself as HTML, Markdown or plain text, or a JSON envelope with the status code, the final URL, the headers, the cookies and the content; it has no parsed listing fields, so your parser reads them.
MLS data feeds or a scraper?
The MLS's own route to the data is a feed: the RESO Web API, IDX and VOW feeds give licensed brokers and the vendors they approve the listings in a standard format, under the MLS's license. If you are a broker, an agent or a vendor working for one, ask your MLS for a feed first. A scraper reads only what the public pages show: it needs no MLS login, but it gives fewer fields, and the site's terms and the MLS's display rules still apply to what you collect.
| Item | MLS feed (RESO Web API, IDX, VOW) | Web Scraping API |
|---|---|---|
| Who can use it | MLS members and approved vendors | ProxyField customers |
| Data | Full listing records the license allows | Public IDX and portal pages |
| Access | The MLS's license and data agreement | No MLS login; the site's terms and the MLS's display rules apply |
| Commercial reuse | As the license allows | Needs the site's or the MLS's permission |
| Cost | MLS and vendor fees | $0.50 per 1,000 successful results, from $0.20 |
How the Web Scraping API fetches a listing page
You send the URL of a public listing or search page and choose plain HTTP or JS rendering; the API returns the page. This is the API's design (in development, access through Contact sales): a POST to https://api.proxyfield.io/v1/scrape with your API key in an Authorization: Bearer <api_key> header and a JSON body: url, render (false for plain HTTP, true for JS rendering), country and output (html, markdown, text or json). For IDX widgets and map searches that load listings with scripts, wait_for takes a CSS selector to wait for before the rendered page is returned. Parameter reference: API documentation.
# The request the Web Scraping API takes (access through Contact sales)
curl https://api.proxyfield.io/v1/scrape \
-H "Authorization: Bearer <api_key>" \
-H "Content-Type: application/json" \
-d '{"url": "https://www.example-brokerage.com/listing/123-main-st", "render": false, "country": "us", "output": "json"}'Example response (shortened; content is the page's HTML):
{
"metadata": {"id": "req_5b80e4c1f93a27d6", "api": "web_scrape", "status": "success", "billed": true},
"status_code": 200,
"final_url": "https://www.example-brokerage.com/listing/123-main-st",
"content_type": "text/html; charset=utf-8",
"content": "<!DOCTYPE html><html lang=\"en\">…</html>"
}| Choice | Price per 1,000 successful results | Use it for |
|---|---|---|
| Plain HTTP | $0.50, or $0.20 from 10,000 results | Listing pages whose facts are in the HTML, often with schema.org markup |
| JS rendering | $2.00, or $1.00 from 10,000 results | IDX search widgets and map searches that load listings with scripts |
Pricing: $0.50 per 1,000 listing pages, from $0.20
MLS listing pages cost the Web Scraping API's price: $0.50 per 1,000 successful results over plain HTTP, or $0.20 from 10,000 results in one purchase, and $2.00 per 1,000 with JS rendering, or $1.00 from 10,000. You pay only for successful results.
Billed: a page the listing site returned, including a 404 or 410 and a sign-in page the site serves with a 200 status (the market norm). Not billed: error responses, blocked and timed-out requests, and our internal retries. Every response says whether it was billed. Contact sales through the sales team on our 24/7 live chat.
At the volume price the weekly status check costs $8 over plain HTTP. Payment and balance: the same crypto balance as proxies, which never expires; there is no subscription or monthly fee. All prices: pricing.
Through the sales team on our 24/7 live chat
| Workload (30-day month) | Results | Plain HTTP at $0.50 | JS rendering at $2.00 |
|---|---|---|---|
| 200 new listings a day in one metro | 6,000 | $3 | $12 |
| 1,000 sold listings for comparables, once a month | 1,000 | $0.50 | $2 |
| 10,000 active listings, weekly status check (4 runs) | 40,000 | $20 | $80 |
Why listing sites are hard to scrape
Two things make MLS listings slow to collect. First, there is no single site: each broker runs an IDX site on one of many vendors' templates, so a parser written for one rarely works on the next, and the portals each have their own layout. Second, portals and IDX vendors watch for automated traffic: fast request rates and datacenter IPs get a block page or a challenge. Pick a few sources that cover your area, write one parser per source, and with your own scraper rotate US residential IPs and keep the rate low. For Zillow specifically, see the Zillow scraper.
What teams use MLS listing data for
- Comparables recent sales and active listings near a property, for pricing and appraisal checks.
- Market reports new listings, inventory, days on market and median prices by city or ZIP code.
- Investment screening price per square foot and price cuts across a metro.
- Lead timing for services movers, lenders and contractors watching new and sold listings in their area.
Agent and owner names are personal data, so collect them only where your purpose needs them; spam is forbidden under ProxyField's acceptable use policy. For a product built on listings, use a licensed feed or written permission (see above).
Scrape IDX listing pages with Python and ProxyField proxies
Many listing pages describe the home in schema.org markup, a JSON block in the HTML meant for search engines. Python can read it without a browser; save the HTML of pages that carry no such block and write a parser for that source.
import json
import requests
from bs4 import BeautifulSoup
proxy = "http://<login>-country-us-city-new_york:<password>@gate.proxyfield.io:7000"
url = "https://www.example-brokerage.com/listing/123-main-st" # a public IDX listing page
resp = requests.get(url, proxies={"http": proxy, "https": proxy},
headers={"User-Agent": "Mozilla/5.0", "Accept-Language": "en-US"},
timeout=30)
soup = BeautifulSoup(resp.text, "html.parser")
found = False
for tag in soup.find_all("script", type="application/ld+json"):
try:
data = json.loads(tag.string or "")
except ValueError:
continue
for item in data if isinstance(data, list) else [data]:
print(item.get("@type"), item.get("name"), item.get("offers", {}))
found = True
if not found:
open("listing.html", "w").write(resp.text) # no markup: inspect and write a parserProxyField residential proxies cost $2.95/GB for 1 GB and from $1.95/GB at volume, with no subscription; the traffic never expires, and all purchases are final. In the United States you get 61.2M+ residential IPs in all 50 states and 2,800+ cities, with state and city targeting, a new IP on every request or a sticky session from 30 s to 36 h. More on US proxies.
More scrapers
- Zillow scraper public Zillow search, property and sold pages.
- Real estate scraper Redfin, Realtor.com, Rightmove, idealista and other portals.
- Google Maps scraper local businesses such as brokerages, $1.50 per 1,000 successful results, from $1.00.
- Web Scraping API any URL, plain HTTP or with JS rendering, from $0.20 per 1,000 successful results.
- All scraper APIs
MLS Scraper FAQ
24/7 support by live chat, including help with setup and integration.
What is an MLS scraper API?
ProxyField has no separate MLS API; its Web Scraping API, in development, is designed to fetch the public pages that show MLS listings, such as broker IDX sites and portals, for $0.50 per 1,000 successful results over plain HTTP, or $0.20 from 10,000 results. It does not enter the MLS itself, which is a members-only database.
Can I get MLS listing data without being a licensed agent, for example by scraping public IDX listing pages?
Yes for viewing, no for copying at scale: you can view public IDX and portal pages without a license, but MLS rules typically limit IDX information to consumers' personal, non-commercial use, and require IDX sites to make reasonable efforts to stop third-party scraping (Stellar MLS Article 19.29). A database, app or resale product built on listings needs a licensed feed, through the MLS, a broker or an authorized data provider, or written permission. Rules differ by MLS; this is not legal advice.
Can I scrape MLS listings?
Only the public pages where MLS listings appear, such as broker IDX sites and portals, and only as each site's terms allow; the MLS's consumer-use rules limit reuse. The MLS system behind a members' login is off limits without your own membership and its rules.
What is MLS data?
MLS data is the listing information that brokers share in a multiple listing service: address, price, status, home facts, photos, remarks, showing details and the listing brokerage. Local real estate associations run the MLSs, and each covers its own area.
Is there an official MLS API?
There is no single MLS API, because there are several hundred MLSs. Most offer data feeds, often through the RESO Web API standard, to their members and the vendors those members approve, under a license.
What is an MLS data scraper?
A tool or script that collects listing data from public pages that show MLS listings, such as IDX search results and portal listing pages. It reads price, address, home facts and status, usually on a schedule, so you can track the market.
How do I scrape MLS data?
Choose public sources that cover your area, such as a few broker IDX sites or a portal, and collect their search and listing URLs. Fetch each page through rotating US residential IPs or a scraping API, read the facts, often from the page's schema.org markup, and repeat on a schedule.
What data can I get from public MLS listings?
Usually the address, price, beds, baths, square footage, lot size, listing status, days on market, photos and the listing brokerage, as the local display rules allow. Showing instructions, agent-only remarks and full sale histories stay inside the MLS.
How much does it cost to scrape MLS listings?
On ProxyField's Web Scraping API, 6,000 listing pages a month cost $3 over plain HTTP, and a weekly check of 10,000 listings, 40,000 pages a month, costs $20, or $8 at the $0.20 price from 10,000 results in one purchase. Your own scraper on residential proxies starts at 1 GB for $2.95.
Is scraping MLS data legal?
Using someone else's MLS login or scraping a members' area breaks the MLS's rules. For public pages, the Ninth Circuit's 2022 hiQ v. LinkedIn ruling left serious doubt that the CFAA reaches the collection of public data, but each site's terms, copyright in photos and remarks, and state law still apply. Store facts rather than republishing content; this is not legal advice.
Show 9 more questionsShow fewer questions
How do I get started with MLS data?
If you are a broker or work for one, ask your MLS about an IDX or RESO Web API feed. If not, start with the public listing pages in your area: test a few sources, write a parser for each, and fetch them through a scraping API or residential proxies.
Can I get data from any MLS?
Each MLS licenses its own data to its own members, so a feed from one does not cover another's area. Public pages are the common ground: portals show listings from many MLSs, and broker IDX sites show their local one.
Is MLS data available for immediate download?
Not from ProxyField: it sells the Web Scraping API and proxies, not ready-made datasets. Some vendors sell listing data under MLS licenses; otherwise you collect the public pages yourself.
In what formats can I receive MLS data?
MLS feeds use standard formats, today mostly the RESO Web API with JSON responses. A scraper gets the web page instead, and your parser turns its fields into rows, JSON or a spreadsheet.
Can I receive MLS data on an ongoing basis?
Yes: schedule your scraper or API calls, for example daily for new listings and weekly for status changes, and store each result with the date of the check. A feed from your MLS updates continuously but needs a license.
Can I geo-target MLS requests to a specific country or city?
With your own scraper, yes: ProxyField residential proxies target the country, region (state) and city, with 61.2M+ residential IPs in the United States across all 50 states and 2,800+ cities. Pick the state or city of the market you study, and keep a few seconds between requests per IP.
Do MLS and IDX sites block scrapers?
Portals and many IDX vendors do: they challenge or block fast traffic and datacenter IPs. Rotate US residential IPs, keep a few seconds between requests per IP and send normal browser headers.
Is MLS data compliant with privacy regulations?
Listings are mostly facts about homes, but agent names, phone numbers and sometimes owner names are personal data. Under the CCPA in California, and GDPR where it applies, keep only the personal data your purpose needs and never use it for spam.
What is an IDX website?
An IDX website is a broker's or agent's site that shows other brokers' MLS listings under the MLS's Internet Data Exchange rules. It is the most common public place to see MLS listings, and its display rules decide which fields appear.
Related pages
Collect MLS pages with the Web Scraping API
Billed on successful results only. Contact sales for the Web Scraping API and the volume you need, or run your own scraper on residential proxies. 24/7 live chat helps with setup and integration.