Wayfair Scraper: Public Product and Price Pages via API
ProxyField's Web Scraping API works as a Wayfair scraper: it fetches public wayfair.com product, search and category pages for $0.50 per 1,000 successful results over plain HTTP, from $0.20 at volume, and $2.00 per 1,000 with JS rendering for pages that build in the browser. A "press and hold" page is Wayfair's bot check: it appears when requests come fast from one IP, from a datacenter range or from a client that doesn't behave like a browser, it holds no product data, and the API doesn't bill it. Contact sales to use the API, or run your own Wayfair scraper on ProxyField residential proxies from the Easy Trial: 1 GB for $2.95, with no subscription.
Through the sales team on our 24/7 live chat
| Fetch | Entry | Volume, from |
|---|---|---|
| HTTP | $0.50 | $0.20 |
| JS rendering | $2.00 | $1.00 |
Billed on successful results only.
What a Wayfair scraper collects
Wayfair sells furniture and home goods from thousands of suppliers, so the same sofa can appear in many colors, sizes and prices under one product page. A scraper reads what a logged-out shopper sees: the Web Scraping API returns the page, and your parser picks the fields. ProxyField sells no separate Wayfair API with pre-parsed fields.
| Page | URL pattern | Public data |
|---|---|---|
| Product page | wayfair.com/<department>/pdp/<name>-<sku>.html | Title, brand, SKU, price, list price and sale, options (color, size, material), stock messages, delivery estimate, rating, review count, dimensions and specifications |
| Search results | wayfair.com/keyword.php?keyword=<query> | Products in ranking order, prices, sale and sponsored badges, ratings |
| Category pages | wayfair.com/<department>/sb0/<category>-c<id>.html | Products of a category, filters, prices, page after page |
| Reviews | on the product page | Star rating, review text, date, customer photos |
Delivery dates and shipping costs depend on the delivery ZIP code the shopper enters on the site. The API returns the page itself as HTML, Markdown or plain text, or a JSON envelope with the status code, the final URL, the headers, the cookies and the content; it has no parsed Wayfair fields, so your parser reads them.
Plain HTTP or JS rendering for Wayfair
You send a Wayfair URL and choose the mode; the API returns the page. The API is in development, and this is its design (access through Contact sales). It takes a POST to https://api.proxyfield.io/v1/scrape with your API key in an Authorization: Bearer <api_key> header and a JSON body: url, render (false for plain HTTP, true for JS rendering), country and output (html, markdown, text or json). For pages that build in the browser, wait_for takes a CSS selector to wait for with rendering, so the product block is on the page before it comes back. Each account runs 20 concurrent requests (higher limits through sales), and a batch takes up to 1,000 URLs with a signed callback when the job is done. Parameter reference: API documentation.
# The request the Web Scraping API takes (access through Contact sales)
curl https://api.proxyfield.io/v1/scrape \
-H "Authorization: Bearer <api_key>" \
-H "Content-Type: application/json" \
-d '{"url": "https://www.wayfair.com/furniture/pdp/example-sofa-w000000000.html", "render": false, "country": "us", "output": "json"}'Example response (shortened; content is the page's HTML):
{
"metadata": {"id": "req_c93722909b64627f", "api": "web_scrape", "status": "success", "billed": true},
"status_code": 200,
"final_url": "https://www.wayfair.com/furniture/pdp/example-sofa-w000000000.html",
"content_type": "text/html; charset=utf-8",
"content": "<!DOCTYPE html><html lang=\"en-US\">…</html>"
}| Mode | Price per 1,000 successful results | When it fits Wayfair |
|---|---|---|
| Plain HTTP | $0.50, from $0.20 at volume | Pages that arrive with the product data in the HTML |
| JS rendering | $2.00, from $1.00 at volume | Pages that arrive as an empty shell and fill in in the browser, and option prices that change with a click |
Wayfair is a script-heavy site, so test 50 pages of each type over plain HTTP before you pick the mode for the whole run.
Pricing: $0.50 per 1,000 Wayfair pages, from $0.20
You pay only for successful results. The volume price starts at 10,000 successful results in one purchase.
Billed: a page Wayfair answered with a 2xx status. Not billed: blocked requests, timeouts and errors on our side; a blocked request is retried on new IPs, and the retries are not billed. Every response says whether it was billed. A target's 404 or 410, and a sign-in page the site serves with a 200 status, count as billed results. Contact sales through the sales team on our 24/7 live chat.
Payment and balance for the API: the same crypto balance as proxies, which never expires; there is no subscription or monthly fee. All prices: pricing.
Through the sales team on our 24/7 live chat
| Workload (30-day month) | Pages a month | HTTP $0.50 | HTTP $0.20 | JS $2.00 | JS $1.00 |
|---|---|---|---|---|---|
| 2,000 products, price and stock daily | 60,000 | $30 | $12 | $120 | $60 |
| 25,000 products, once a month | 25,000 | $12.50 | $5 | $50 | $25 |
| 30 categories, first 10 pages, weekly (4 runs) | 1,200 | $0.60 | $0.24 | $2.40 | $1.20 |
Wayfair's own API
Wayfair's developer portal (developer.wayfair.io) is built for businesses that sell through Wayfair, such as suppliers managing their own orders, inventory and shipments. It is not a public catalog feed of other sellers' prices, search rankings or reviews, so market data on Wayfair comes from the public pages.
| Item | Wayfair supplier API | Web Scraping API |
|---|---|---|
| Who can use it | Wayfair suppliers | ProxyField customers |
| Data | Your own orders, inventory, shipments | Any public product, search or category page |
| Other sellers' prices and rankings | No | Yes, as shown on the page |
| Cost | Set by Wayfair | $0.50 per 1,000 successful results, from $0.20 |
Why Wayfair shows a "press and hold" page, and how to trigger it less
The "press and hold" page is a bot check, not a scraper error. Wayfair shows it when the traffic looks automated: many fast requests from one IP, a datacenter IP range, missing cookies or JavaScript, or headers no browser sends. The page holds no product data, so count it as a failed fetch. A residential IP lowers how often you get it but doesn't rule it out: residential traffic that behaves like a bot is checked too. Wayfair's pages preconnect to perimeterx.net, the bot-defense service of HUMAN Security (formerly PerimeterX), whose challenge asks the visitor to press and hold a button (seen in wayfair.ie's response headers on 2026-10-27). Even one plain request from a server can be refused: a single request to wayfair.com/terms and one to wayfair.ie, both from a server, got HTTP 429 Too Many Requests that day.
Read moreShow less
What lowers the rate, in order:
-
One new IP per request for listing and search pages
ProxyField's default is a new residential IP on every request:
<login>-country-us:<password>@gate.proxyfield.io:7000. -
A sticky session for one product browse or one delivery ZIP
Add
-session-<id>-ttl-600to keep one IP for 10 minutes (20 minutes if you leave outttl, up to 36 h). An IP idle for 120 s gets a new one, so keep the session busy or start a new one. -
A few seconds between requests per IP
Scale out over many IPs in parallel (threads are unlimited) instead of speeding up one connection.
-
A real browser for pages that arrive as an empty shell
Use JS rendering on the API ($2.00 per 1,000 successful results, from $1.00) or the same gateway in Playwright, so cookies and scripts run as they do for a shopper.
-
On a challenge, retry once or twice on a new session, then stop
Lower the rate before you go on. Don't loop on challenged URLs and don't automate the button: repeated failures keep the block in place.
| Item | Your own scraper on ProxyField proxies | Web Scraping API |
|---|---|---|
| A "press and hold" page | Traffic you paid for per GB, with no data in it | Not billed; the API retries on new IPs, and the retries are not billed |
| What you tune | IP rotation, sessions, pacing, rendering | Mode (render) and the URLs you send |
| Price | $2.95/GB for 1 GB, from $1.95/GB | $0.50 per 1,000 successful results, from $0.20 |
Collect public pages only, and follow Wayfair's terms.
What teams use Wayfair data for
- Competitive pricing for furniture and home-goods brands that also sell on Wayfair or their own store.
- Sale and promotion tracking: when prices drop, by how much, for how long.
- Assortment research: what Wayfair lists in a category, by material, size and price band.
- Review analysis: what buyers say about quality, assembly and delivery.
For prices across many stores, see price scraping and the ecommerce scraper.
Scrape Wayfair with Python and ProxyField proxies
This script fetches one product page through a US residential IP and reads the schema.org data (JSON-LD), if the page carries it; check one page of each type by hand first, since Wayfair can also embed product data in other scripts. A challenge page has no product data, so the script retries twice on a fresh session, with a pause, and then stops instead of storing empty rows.
Read moreShow less
import json
import random
import string
import time
import requests
from bs4 import BeautifulSoup
url = "https://www.wayfair.com/furniture/pdp/example-sofa-w000000000.html"
headers = {"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64)",
"Accept-Language": "en-US,en;q=0.9"}
def proxies(session):
p = f"http://<login>-country-us-session-{session}-ttl-600:<password>@gate.proxyfield.io:7000"
return {"http": p, "https": p}
def product_rows(html):
rows = []
for tag in BeautifulSoup(html, "html.parser").select('script[type="application/ld+json"]'):
try:
data = json.loads(tag.string or "{}")
except json.JSONDecodeError:
continue
for item in data if isinstance(data, list) else [data]:
if item.get("@type") == "Product":
offer = item.get("offers") or {}
offer = offer[0] if isinstance(offer, list) else offer
rating = item.get("aggregateRating") or {}
rows.append((item.get("sku"), item.get("name"), offer.get("price"), rating.get("ratingValue")))
return rows
rows = []
for attempt in range(3): # the first try and 2 retries, each on a new session (a new IP)
session = "".join(random.choices(string.ascii_lowercase + string.digits, k=8))
resp = requests.get(url, proxies=proxies(session), headers=headers, timeout=30)
rows = product_rows(resp.text)
if rows:
break
challenged = "press & hold" in resp.text.lower() or resp.status_code in (403, 429)
print(f"try {attempt + 1}: status {resp.status_code}, challenge page: {challenged}")
time.sleep(15 * (attempt + 1))
print(rows or "no product data: a challenge page, or the page needs JS rendering; slow down before the next run")For pages that build in the browser, run the same gateway in Playwright or Puppeteer. ProxyField has 61.2M+ residential IPs in the United States, 16.6M+ in the United Kingdom and 5.5M+ in Canada, a new IP on every request or a sticky session from 30 s to 36 h, and unlimited concurrent threads.
| Residential traffic in one purchase | Price per GB |
|---|---|
| 1-9 GB (Easy Trial: 1 GB) | $2.95 |
| 10-49 GB | $2.79 |
| 50-99 GB | $2.65 |
| 100-249 GB | $2.49 |
| 250-499 GB | $2.29 |
| 500-999 GB | $2.09 |
| 1,000+ GB | $1.95 |
The tier rate applies to the whole purchase, and country, state and city targeting cost the same per GB. The traffic never expires, and all purchases are final.
Wayfair's terms and the law
Wayfair's terms of use (wayfair.com/terms) set the rules for automated access to its site, and sites like it commonly restrict robots and scrapers, so read them before you scrape, even public pages; whether a scraper also breaks a law is a separate question. Prices, options and ratings are facts, while product photos, descriptions and review texts can be protected by copyright, so keep facts and short excerpts. More in Is It Legal to Scrape eBay, Walmart, Zillow and Other Marketplaces? and Is Web Scraping Legal? US, EU and UK Rules (2026). This is not legal advice.
ProxyField's rule for every site: public pages only, and follow the site's terms.
More scrapers
- Home Depot scraper API, Best Buy scraper and Etsy scraper on the same Web Scraping API.
- Amazon scraper API: Amazon product and search pages, $0.60 per 1,000 successful results, from $0.30.
- Google Shopping scraper: one product's offers across stores.
- All scrapers: scrapers.
Wayfair Scraper FAQ
24/7 support by live chat, including help with setup and integration.
What does a Wayfair scraper do?
ProxyField's Wayfair scraper is the Web Scraping API used on Wayfair pages: it fetches public product, search and category pages for $0.50 per 1,000 successful results, from $0.20 at volume. Any Wayfair scraper, whether a script or a service, collects prices, options, stock messages and ratings from wayfair.com for tracking and research.
What happens when I send a Wayfair URL to a scraper API?
You send a Wayfair URL; the API fetches it through its own IPs, over plain HTTP or with JS rendering, and returns the page for your parser. You decide which URLs to send and how often, and you are billed per 1,000 successful results.
How much does it cost to scrape Wayfair?
On ProxyField's Web Scraping API, $0.50 per 1,000 successful results over plain HTTP, from $0.20 at volume, or $2.00 per 1,000 with JS rendering, from $1.00. Checking 2,000 products a day is 60,000 pages a month: $30 over plain HTTP, $120 with JS rendering.
Does Wayfair allow scraping?
Wayfair's terms of use set the rules for automated access, so read the current terms at wayfair.com/terms before you scrape, even public pages. Whether a project also breaks a law is a separate question that depends on what you collect and where; this is not legal advice.
What types of data can be scraped from Wayfair?
Public product pages show the title, brand, SKU, price and sale price, options, stock messages, delivery estimate, rating, review count and dimensions. Search and category pages add the ranking order and badges, and reviews add the rating, text and date.
How do I get structured data from Wayfair pages?
Check the product page for schema.org data (JSON-LD) first: where it is present, it gives the name, SKU, offer price and rating as fields. Parse the visible HTML or the page's other embedded data for what it lacks, such as option prices and delivery estimates.
Does Wayfair need a headless browser to scrape?
Often for some pages: Wayfair is script-heavy, and pages that arrive as an empty shell need JS rendering, at $2.00 per 1,000 successful results on ProxyField's API. Test each page type over plain HTTP at $0.50 per 1,000 first and render only what needs it.
Why route Wayfair requests through a proxy?
So your requests come from an IP and location you choose rather than your server's datacenter IP, which Wayfair challenges sooner. For Wayfair, rotating residential proxies in the country of the site you scrape are the usual choice.
Can Wayfair detect proxies?
Wayfair checks traffic for signs of automation: datacenter IP ranges, too many requests from one IP, and clients that do not behave like a browser. Residential IPs at a slow rate per IP, with browser headers, are challenged less often than a datacenter server, but not never: traffic that behaves like a bot is checked on any IP.
Which proxy type should I use to scrape Wayfair?
Rotating residential proxies for catalog runs, with a new IP on every request, and a sticky session when you set a delivery ZIP code and read the prices for it. ProxyField residential proxies cost $2.95/GB for 1 GB and from $1.95/GB at volume.
Show 8 more questionsShow fewer questions
Can I scrape Wayfair from a specific country?
Yes: wayfair.co.uk, wayfair.ca and wayfair.ie are separate storefronts with their own prices, so fetch each from its own country. ProxyField has 16.6M+ residential IPs in the United Kingdom, 5.5M+ in Canada and 310K+ in Ireland, next to 61.2M+ in the United States.
Can I track Wayfair delivery dates by ZIP code?
Yes, by entering the ZIP code on the site in a sticky session and reading the product pages in the same session. ProxyField proxies target by country, region (state) and city; ZIP-code targeting is not offered, so the ZIP comes from the site's own setting.
What's the rate limit for scraping Wayfair?
We know of no published rate limit for Wayfair's website; it challenges traffic that looks automated. Keep each IP to a request every few seconds and scale out over many IPs in parallel, rather than speeding up one connection.
How many Wayfair requests can I run in parallel?
With ProxyField proxies, as many as your scraper handles: there are unlimited concurrent threads, with a new IP on every request. For the Web Scraping API at high volume, tell sales your expected pages per day.
How fast can a Wayfair scraper retrieve data?
Speed depends on the page type, since JS rendering takes longer per page than plain HTTP, and on how many requests you run in parallel. With your own scraper on ProxyField proxies you get unlimited concurrent threads.
Why does my Wayfair scraper keep landing on a "press and hold" page, and how do I avoid triggering it?
The page is Wayfair's bot check: your traffic looks automated (fast requests from one IP, a datacenter range, missing cookies or JavaScript), and the page holds no product data. Use a new residential IP per listing request and a sticky session (-session-<id>-ttl-600) per product browse, keep a few seconds between requests per IP, and render empty pages in a real browser. On a challenge, retry once or twice on a new session, then slow down; never loop or automate the button.
Is Python a good choice for a Wayfair scraper?
Yes: Python with requests and Beautiful Soup fetches and parses product pages, as in the example on this page, and Playwright handles the pages that need a browser. Any language with an HTTP client and an HTML parser works the same way.
Can I try Wayfair scraping before paying for volume?
For your own scraper, the Easy Trial is 1 GB of residential traffic for $2.95, with no subscription and no minimum purchase. For the Web Scraping API, contact sales with your page types and volume.
Related pages
Collect Wayfair pages with the Web Scraping API
Billed on successful results only. Contact sales for the Web Scraping API and the volume you need, or run your own scraper on residential proxies. 24/7 live chat helps with setup and integration.