How to Scrape Amazon Product Data
Fetch one Amazon product page in Python, read the title, price, rating, and images, and see why that breaks. Then get the same fields from one API call and a CSV.
A short Python script can read one public Amazon product page: title, current price, star rating, and images. This post shows that script, names the elements those values sit in, and then shows the same fields coming back from one API call you can write to CSV.
The selectors are a snapshot of the desktop page. They are not a contract. The section after the script is what actually happens when you run it on a schedule.
Fetch one product page
Install the two libraries and pick a product URL. The ASIN in this example, B08N5WRWNW, is the same sample the Amazon product API page uses.
pip install requests beautifulsoup4
requests sends the HTTP call. BeautifulSoup reads the HTML. Amazon's desktop markup is what you get in a normal browser at /dp/{ASIN}. A request with the default python-requests user agent is a common way to receive an error page instead of the listing, so the script sends a desktop browser user agent and asks for US English. That asks for the desktop page. It does not get the page past a block.
import json
import re
import requests
from bs4 import BeautifulSoup
URL = "https://www.amazon.com/dp/B08N5WRWNW"
def fetch_product_html(url):
response = requests.get(
url,
headers={
# Any current desktop browser user agent. The version is not special.
"User-Agent": (
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) "
"AppleWebKit/537.36 (KHTML, like Gecko) "
"Chrome/131.0.0.0 Safari/537.36"
),
"Accept": "text/html,application/xhtml+xml",
"Accept-Language": "en-US,en;q=0.9",
},
timeout=20,
)
body = response.text
# A block often comes back as 200, 404, or 503, with a short error
# document instead of the listing. productTitle is only on the product page.
if response.status_code != 200 or "productTitle" not in body:
raise RuntimeError(
"Amazon did not return a product page "
f"(HTTP {response.status_code}, {len(body)} bytes). "
"The body is often a robot check or the Page Not Found error."
)
return body
def text_of(node):
if node is None:
return None
value = node.get_text(" ", strip=True)
return value or None
def parse_rating(value):
# "4.6 out of 5 stars"
if not value:
return None
match = re.search(r"(\d+(?:\.\d+)?)", value)
return float(match.group(1)) if match else None
def parse_count(value):
# "84,932 ratings" -> 84932. Do not use this on the star sentence.
if not value:
return None
digits = re.sub(r"[^\d]", "", value)
return int(digits) if digits else None
def current_price(soup):
# Scoped on purpose. The first span.a-offscreen on the page is often
# a list price, a unit price, or a shipping amount.
selectors = [
"#corePriceDisplay_desktop_feature_div .priceToPay span.a-offscreen",
"#corePrice_feature_div .priceToPay span.a-offscreen",
"#corePriceDisplay_desktop_feature_div span.a-price[data-a-color='base'] span.a-offscreen",
"#corePrice_feature_div span.a-price[data-a-color='base'] span.a-offscreen",
]
for selector in selectors:
value = text_of(soup.select_one(selector))
if value:
return value
return None
def main_image_url(soup):
image = soup.select_one("#landingImage")
if image is None:
return None
hires = (image.get("data-old-hires") or "").strip()
if hires:
return hires
raw = image.get("data-a-dynamic-image")
if raw:
try:
sizes = json.loads(raw)
except json.JSONDecodeError:
sizes = {}
best_url = None
best_area = -1
for url, dims in sizes.items():
if isinstance(dims, (list, tuple)) and len(dims) >= 2:
area = int(dims[0]) * int(dims[1])
if area > best_area:
best_area = area
best_url = url
if best_url:
return best_url
return image.get("src")
def gallery_urls(html):
# The hi-res gallery is not the thumbnail strip. It is a JS object:
# colorImages.initial = [{hiRes, large, thumb}, ...]
# hiRes is null on some entries. Fall back to large.
match = re.search(
r"""['"]colorImages['"]\s*:\s*\{.*?['"]initial['"]\s*:\s*(\[.*?\])""",
html,
re.S,
)
if not match:
return []
try:
images = json.loads(match.group(1))
except json.JSONDecodeError:
return []
urls = []
for item in images:
if not isinstance(item, dict):
continue
url = item.get("hiRes") or item.get("large")
if url and url not in urls:
urls.append(url)
return urls
def parse_product(html):
soup = BeautifulSoup(html, "html.parser")
popover = soup.select_one("#acrPopover")
rating_text = text_of(soup.select_one("#acrPopover span.a-icon-alt"))
if not rating_text and popover is not None:
rating_text = popover.get("title")
ratings_text = text_of(soup.select_one("#acrCustomerReviewText"))
return {
"title": text_of(soup.select_one("#productTitle")),
"price": current_price(soup),
"rating": parse_rating(rating_text),
"ratings_total": parse_count(ratings_text),
"main_image": main_image_url(soup),
"image_urls": gallery_urls(html),
}
if __name__ == "__main__":
html = fetch_product_html(URL)
print(json.dumps(parse_product(html), indent=2))
Run it once, from your own machine, against one ASIN you are comfortable requesting. If productTitle is missing, stop. Retrying harder is the part that turns a test script into a blocking problem, which is covered below.
Where the title, price, rating, and images sit
Selectors as of November 2, 2026. This is the desktop amazon.com product page (/dp/{ASIN}) as a normal browser renders it. The mobile site and in-page experiments use different markup. Confirm every id in your own browser before you depend on it. Amazon changes them without a changelog.
On November 2, 2026, a server-side GET of https://www.amazon.com/dp/B08N5WRWNW did not receive this markup. The response was HTTP 404 and a short "Page Not Found" document (about 2 KB). An HTML comment in that document tells automated clients to contact Amazon about automated access. The listing elements below were not in the body. That response is itself the useful result: a script can get a 404 that is not a missing ASIN.
| Field | Element on the desktop page | What goes wrong |
|---|---|---|
| Title | span#productTitle | The text node has leading and trailing whitespace. Strip it. |
| Current price | span.a-offscreen inside .priceToPay, in #corePriceDisplay_desktop_feature_div or #corePrice_feature_div | The first .a-offscreen on the page is often a "was" price, a per-unit price, or shipping. The struck-through amount uses data-a-color="secondary". |
| Star rating | span.a-icon-alt inside #acrPopover. The same sentence is the popover's title. | The text is a sentence, such as "4.6 out of 5 stars", not a bare number. |
| Rating count | #acrCustomerReviewText | The label says "ratings". It includes ratings that have no written review. |
| Main image | img#landingImage | data-old-hires is the large file when it is set. data-a-dynamic-image is a JSON map of URL to [width, height]. src is often smaller. |
| Other images | colorImages.initial in a script on the page | Each object has hiRes, large, and thumb. hiRes can be null. #altImages img is the thumbnail strip (small size tokens such as _SS40_ in the URL), not the gallery you want to save. |
A few details that save a bad row:
- Older tutorials select
#priceblock_ourpriceand#priceblock_dealprice. Those ids are not the current desktop price block. A script that only looks for them returns an empty price on a normal listing. - The visible price is split across
span.a-price-symbol,span.a-price-whole, andspan.a-price-fraction.a-price-wholealso contains a decimal span, so concatenating the three is easy to get wrong. The.a-offscreenstring is the full amount, such as$499.99. - Some listings, including books and offers with no Buy Box, put the price in a different container. A missing
.priceToPaydoes not mean the request failed. Check#productTitlefirst. - The number beside the stars is not the written-review count. The written reviews are a separate total further down the page.
Why the script stops working
The parser is the small part. These are the failures that show up once the script leaves a single manual test.
The response is not the product page. Amazon answers automated clients with a robot check, a short error page, or HTTP 503. The status is sometimes 200. The November 2, 2026 fetch above was HTTP 404 with the dogs-of-Amazon error page, not a CAPTCHA form and not the listing. If you parse that HTML anyway, every field is null and it looks like a selector bug.
The layout moves. The price block has already left #priceblock_ourprice. A class such as priceToPay can be renamed, or a test can serve a different price widget to some sessions, while #productTitle still resolves. The script keeps exiting 0 and writes a blank price, or worse, the list price from the first .a-offscreen.
One IP address gets cut off. Repeat the same call from one address and Amazon stops returning the listing to that address. Fixing that means proxies, pacing, and health checks. This post does not cover how to do that. At that point you are running infrastructure, not a Python file, and the fields still break the next time the markup changes.
A page you loaded in a browser while signed in is also a different document from the one requests receives. Prices, delivery, and even which offer is featured can depend on the cookie and the delivery address. Do not debug a parser against HTML you saved from your own logged-in session and then expect a cookieless request to match it.
Terms of use, without a legal conclusion
Amazon's Conditions of Use restrict automated access, including robots and scrapers, unless Amazon has allowed it. Whether a particular script is lawful depends on what you collect, how you send the requests, and where you operate. This post is not legal advice, and it is not a reason to send the script at a catalog.
Public listing fields (title, price, rating, images) are a different situation from anything behind a login or tied to a person. Blocks, account risk, and the cost of keeping a parser alive are written up in Is Scraping Amazon Legal? Practical Alternatives in 2026. Read that before you point this script at more than a test ASIN.
The same fields from one API call
The Amazon product API returns the listing as JSON, so there is no price element to keep up with. To look at one ASIN in the browser before you write code, use the ASIN lookup tool. It does not require a key.
The REST call is GET https://rest.canopyapi.co/api/amazon/product.
- Send the key in the
API-KEYheader. - Pass exactly one of
asin,url, orgtin. Sending more than one is a validation error: "Exactly one of asin, url, or gtin must be provided." domainis optional and defaults toUS. A URL lookup takes the marketplace from the URL, so do not senddomainwithurl.- A missing or invalid key is HTTP 401. The plan limit is HTTP 402.
The 200 body is { "data": { "amazonProduct": { ... } } }. These are the fields that route returns. The numbers below are the sample published on the product page, not a live quote. Prices and ratings on a real ASIN change.
{
"data": {
"amazonProduct": {
"asin": "B08N5WRWNW",
"title": "PlayStation 5 Console",
"brand": "Sony",
"url": "https://www.amazon.com/dp/B08N5WRWNW",
"isPrime": true,
"isNew": true,
"isInStock": true,
"price": {
"symbol": "$",
"value": 499.99,
"currency": "USD",
"display": "$499.99"
},
"mainImageUrl": "https://m.media-amazon.com/images/I/51vHwT8xOyL._AC_SL1500_.jpg",
"imageUrls": [],
"rating": 4.7,
"ratingsTotal": 84932,
"featureBullets": [
"Example bullet. Live text comes from the current listing."
]
}
}
}
How those map to the page you were parsing:
titleis the product title.brand,url, andasincome with it.subtitleanditemHighlightsare included when the listing has them.itemHighlightsis the short line under the title, up to 125 characters.priceis the Buy Box price:symbol, numericvalue,currency, anddisplay(the string shoppers see, such as$499.99).pricecan be null when the item is unavailable.isInStock,isPrime, andisNewdescribe that same Buy Box offer.ratingis the average star rating.ratingsTotalcounts ratings with and without a written review, which is the number next to the stars, not the written-review total.mainImageUrlis the main image.imageUrlsis the rest of the gallery and does not repeat the main image.- Also on the object when Amazon has them:
featureBullets(the About this item lines),technicalSpecifications(nameandvalue),categories(id,name,url),seller(sellerIdandname, omitted in some cases such as when Amazon is the seller), andcoupon(label).
Field notes are on the Amazon product API page and in the API docs. Every account includes 100 free requests each month, then usage is billed per request starting at $0.01, with automatic volume discounts as usage grows. There is no monthly minimum. The call above is one request. Nested fields such as sales estimates, stock estimates, and A+ content count as an additional request when you ask for them. This route does not ask for those. Create a key from the dashboard after you sign in.
import requests
response = requests.get(
"https://rest.canopyapi.co/api/amazon/product",
params={"asin": "B08N5WRWNW", "domain": "US"},
headers={"API-KEY": "YOUR_API_KEY"},
timeout=30,
)
response.raise_for_status()
product = response.json()["data"]["amazonProduct"]
price = product.get("price") or {}
print(product["title"], price.get("display"), product.get("rating"))
Vendor comparisons are separate pages, so this one can stay a tutorial. The product overview is the Amazon scraper API page. General scraping APIs are in the best Amazon scraper API comparison.
Write the rows to CSV
csv is in the standard library. DictWriter quotes titles that contain commas. price is read with .get because it is null when the Buy Box has no price. imageUrls is joined with a space so the gallery stays in one column.
import csv
import requests
API_KEY = "YOUR_API_KEY"
ASINS = ["B08N5WRWNW"]
rows = []
for asin in ASINS:
response = requests.get(
"https://rest.canopyapi.co/api/amazon/product",
params={"asin": asin, "domain": "US"},
headers={"API-KEY": API_KEY},
timeout=30,
)
response.raise_for_status()
product = response.json()["data"]["amazonProduct"]
price = product.get("price") or {}
rows.append(
{
"asin": product.get("asin"),
"title": product.get("title"),
"brand": product.get("brand"),
"price_display": price.get("display"),
"price_value": price.get("value"),
"currency": price.get("currency"),
"rating": product.get("rating"),
"ratings_total": product.get("ratingsTotal"),
"in_stock": product.get("isInStock"),
"main_image": product.get("mainImageUrl"),
"image_urls": " ".join(product.get("imageUrls") or []),
"url": product.get("url"),
}
)
with open("products.csv", "w", newline="", encoding="utf-8") as handle:
writer = csv.DictWriter(handle, fieldnames=list(rows[0].keys()))
writer.writeheader()
writer.writerows(rows)
The dict from parse_product can go through the same writer if you already have HTML for a page. You will not get brand, isInStock, or a numeric price from the selectors above. Those are on the API object: price.value is a number, and price.display is the string.
Questions people ask while parsing the page
Why did the request return Page Not Found for an ASIN that exists?
Because that response is a common refusal, not a lookup result. On November 2, 2026 a datacenter GET of a real /dp/ URL came back as HTTP 404 with a short error page and no productTitle. The ASIN was still a valid sample. Check the body before you change a selector. A robot check and an HTTP 503 are the other forms of the same failure.
Which price element is the current price?
The .a-offscreen text inside .priceToPay, in #corePriceDisplay_desktop_feature_div or #corePrice_feature_div. If .priceToPay is absent, use span.a-price[data-a-color="base"] span.a-offscreen in that same block. Skip data-a-color="secondary". That is the struck-through list price. Skip the first .a-offscreen on the page. It is often a different amount.
Is the number next to the stars the number of written reviews?
No. #acrCustomerReviewText is a ratings count. It includes ratings with no written review. The API field for that count is ratingsTotal. The written-review total is a different number, and this product route does not return it.
What does the free tier include?
100 requests every month, then $0.01 per request, with automatic volume discounts and no monthly minimum. One product lookup is one request. Asking for sales estimates, stock estimates, or A+ content on a query that supports them counts as another request. The product call on this page does not ask for those fields.