{"id":20709913,"url":"https://github.com/oxylabs/how-to-scrape-amazon-prices","last_synced_at":"2025-05-16T03:06:09.132Z","repository":{"id":229157406,"uuid":"775509078","full_name":"oxylabs/how-to-scrape-amazon-prices","owner":"oxylabs","description":"A code for extracting best-selling items, search results, and currently available deals from Amazon using Python and Oxylabs E-Commerce Scraper API.","archived":false,"fork":false,"pushed_at":"2025-04-09T08:38:37.000Z","size":73,"stargazers_count":480,"open_issues_count":0,"forks_count":6,"subscribers_count":7,"default_branch":"main","last_synced_at":"2025-04-19T02:14:50.366Z","etag":null,"topics":["amazon","amazon-scraper","api","python","python-scraper","scraper-api","web-scraper","web-scraping"],"latest_commit_sha":null,"homepage":"https://oxylabs.io/products/scraper-api/ecommerce","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/oxylabs.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2024-03-21T14:21:53.000Z","updated_at":"2025-04-17T17:01:24.000Z","dependencies_parsed_at":"2024-03-22T11:02:40.992Z","dependency_job_id":"5044bd6e-e212-4c40-9197-90d5ac009bc0","html_url":"https://github.com/oxylabs/how-to-scrape-amazon-prices","commit_stats":null,"previous_names":["oxylabs/how-to-scrape-amazon-prices"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oxylabs%2Fhow-to-scrape-amazon-prices","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oxylabs%2Fhow-to-scrape-amazon-prices/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oxylabs%2Fhow-to-scrape-amazon-prices/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oxylabs%2Fhow-to-scrape-amazon-prices/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/oxylabs","download_url":"https://codeload.github.com/oxylabs/how-to-scrape-amazon-prices/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254459088,"owners_count":22074605,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["amazon","amazon-scraper","api","python","python-scraper","scraper-api","web-scraper","web-scraping"],"created_at":"2024-11-17T02:08:58.968Z","updated_at":"2025-05-16T03:06:04.124Z","avatar_url":"https://github.com/oxylabs.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# How to Scrape Amazon Prices\n\n[![Oxylabs promo code](https://raw.githubusercontent.com/oxylabs/amazon-scraper/refs/heads/main/Amazon-Scraper-API-1090x275.png)](https://oxylabs.go2cloud.org/aff_c?offer_id=7\u0026aff_id=877\u0026url_id=112)\n\n[![](https://dcbadge.vercel.app/api/server/eWsVUJrnG5)](https://discord.gg/Pds3gBmKMH)\n\n  * [Free Amazon Prices Scraper](#free-amazon-prices-scraper)\n  * [Prerequisites](#prerequisites)\n  * [Installation](#installation)\n  * [Retrieving the URL of an Amazon page to scrape prices from](#retrieving-the-url-of-an-amazon-page-to-scrape-prices-from)\n  * [Scraping Amazon prices](#scraping-amazon-prices)\n  * [Retrieved data](#retrieved-data)\n  * [Notes](#notes)\n  * [Scraping with Oxylabs API](#scraping-with-oxylabs-api)\n  * [The complete code](#the-complete-code)\n\nFollow this quick and easy tutorial to scrape public Amazon pricing data; in the first part of the tutorial, we'll show how you can do it for free. If you need the pricing data on a bigger scale, please refer to the second part of the tutorial; there, we demonstrate how to do it with [Oxylabs API](https://oxylabs.io/products/scraper-api/web). \n\n### Free Amazon Prices Scraper\n\nA free tool used to get Amazon product prices for a provided Amazon department page.\n\n### Prerequisites\n\nTo run this tool, you need to have Python 3.11 installed in your system.\n\n### Installation\n\nOpen up a terminal window, navigate to this repository and run this command:\n\n```make install```\n\n### Retrieving the URL of an Amazon page to scrape prices from\n\nFirst off, open up Amazon and select a department from which you want to scrape prices for products. \n\nFor this example, we'll be using the `Camera \u0026 Photo` department.\n\n\u003cimg width=\"993\" alt=\"image\" src=\"https://github.com/user-attachments/assets/0ee1f37b-d99a-4cb7-b699-9b6f6c95bfff\" /\u003e\n\n\nAfter the page loads, simply copy the URL in the browser and save it. We'll need it for scraping price data.\n\n### Scraping Amazon prices\n\nTo get prices from products listed on the department page you chose, simply run this command in your terminal:\n\n```make scrape URL=\"\u003camazon_department_page_url\u003e\"```\n\nWith the URL we retrieved earlier, the command would look like this:\n\n```make scrape URL=\"https://www.amazon.com/s?i=specialty-aps\u0026bbn=16225009011\u0026rh=n%3A%2116225009011%2Cn%3A502394\u0026ref=nav_em__nav_desktop_sa_intl_camera_and_photo_0_2_6_3\"```\n\nMake sure to surround the URL with quotation marks, otherwise the tool might have trouble parsing it.\n\nAfter running the command, your terminal should look something like this:\n\n\u003cimg width=\"1163\" alt=\"image\" src=\"https://github.com/user-attachments/assets/eb0e83f9-d995-4c48-8cc8-db7635c55feb\" /\u003e\n\nIf a listed product is out of stock, the tool will notify you with a dedicated message. The product will be skipped if that is the case.\n\n### Retrieved data\n\nAfter the tool has finished running, you should see a file named `amazon_prices.csv` in your directory.\n\nThe generated CSV file contains data with these columns inside it:\n\n- `title` - The title of the product.\n- `url` - The URL pointing to the product's Amazon page.\n- `price` - The price of the product.\n- `currency` - The currency that the product is sold in.\n\nThe data should look something like this:\n\n\u003cimg width=\"717\" alt=\"image\" src=\"https://github.com/user-attachments/assets/4e00fecc-8176-4248-9b63-0bc6dd382905\" /\u003e\n\n### Notes\n\nIn case the code doesn't work or your project is of bigger scale, please refer to the second part of the tutorial. There, we showcase how to scrape public data with Oxylabs Scraper API.\n\n### Scraping with Oxylabs API\n\nHere's the process of scraping best-selling items, search results, and currently available deals from Amazon using Python and Oxylabs [E-Commerce Scraper API](https://oxylabs.io/products/scraper-api/ecommerce) (a part of Web Scraper API). You can claim a **1-week free trial** by registering on the [dashboard](https://dashboard.oxylabs.io/).\n\nFor a detailed walkthrough with explanations and visuals, check our [blog post](https://oxylabs.io/blog/scraping-amazon-prices).\n\n## The complete code\n\n```python\nimport requests\nimport pandas as pd\n\nUSERNAME = \"USERNAME\"\nPASSWORD = \"PASSWORD\"\n\n\ndef parse_price_results(results):\n    return [\n        {\n            \"price\": result[\"price\"],\n            \"title\": result[\"title\"],\n            \"currency\": result[\"currency\"],\n        }\n        for result in results\n    ]\n\n\ndef get_best_seller_results(category_id):\n    payload = {\n        \"source\": \"amazon_bestsellers\",\n        \"domain\": \"com\",\n        \"query\": category_id,\n        \"start_page\": 1,\n        \"parse\": True,\n    }\n    response = requests.post(\n        \"https://realtime.oxylabs.io/v1/queries\",\n        auth=(USERNAME, PASSWORD),\n        json=payload,\n    )\n    response.raise_for_status()\n    results = response.json()[\"results\"][0][\"content\"][\"results\"]\n    return parse_price_results(results)\n\n\ndef get_search_results(query):\n    payload = {\n        \"source\": \"amazon_search\",\n        \"domain\": \"com\",\n        \"query\": query,\n        \"start_page\": 1,\n        \"parse\": True,\n    }\n    response = requests.post(\n        \"https://realtime.oxylabs.io/v1/queries\",\n        auth=(USERNAME, PASSWORD),\n        json=payload,\n    )\n    response.raise_for_status()\n    results = response.json()[\"results\"][0][\"content\"][\"results\"][\"organic\"]\n    return parse_price_results(results)\n\n\ndef get_deals_results(url):\n    payload = {\n        \"source\": \"amazon\",\n        \"url\": url,\n        \"parse\": True,\n    }\n    response = requests.post(\n        \"https://realtime.oxylabs.io/v1/queries\",\n        auth=(USERNAME, PASSWORD),\n        json=payload,\n    )\n    response.raise_for_status()\n    results = response.json()[\"results\"][0][\"content\"][\"results\"][\"organic\"]\n    return parse_price_results(results)\n\n\ndog_food_category_id = \"2975359011\"\n\nbest_seller_results = get_best_seller_results(dog_food_category_id)\nbest_seller_df = pd.DataFrame(best_seller_results)\nbest_seller_df.to_csv(\"best_seller.csv\")\n\nsearch_results = get_search_results(\"couch\")\nsearch_df = pd.DataFrame(search_results)\nsearch_df.to_csv(\"search.csv\")\n\ndeal_url = \"https://www.amazon.com/s?i=sporting\u0026rh=n%3A3400371%2Cp_n_deal_type%3A23566064011\u0026s=exact-aware-popularity-rank\u0026pf_rd_i=10805321\u0026pf_rd_m=ATVPDKIKX0DER\u0026pf_rd_p=bf702ff1-4bf6-4c17-ab26-f4867bf293a9\u0026pf_rd_r=ER3N9MGTCESZPZ0KRV8R\u0026pf_rd_s=merchandised-search-3\u0026pf_rd_t=101\u0026ref=s9_acss_bw_cg_SODeals_3e1_w\"\n\ndeal_results = get_deals_results(deal_url)\ndeal_df = pd.DataFrame(deal_results)\ndeal_df.to_csv(\"deals.csv\")\n```\n\n## Final word\n\nCheck our [documentation](https://developers.oxylabs.io/scraper-apis/web-scraper-api/amazon) for all of the API parameters found in this guide.\n\nIf you have any questions, feel free to contact us at support@oxylabs.io or via the live chat on our [homepage](https://oxylabs.io/).\n\nLooking to scrape more other Amazon data? [Amazon Review Scraper](https://github.com/oxylabs/amazon-review-scraper), [Amazon ASIN Scraper](https://github.com/oxylabs/amazon-asin-scraper), [Bypass Amazon CAPTCHA](https://github.com/oxylabs/how-to-bypass-amazon-captcha), [Scraping Amazon Product Data](https://github.com/oxylabs/how-to-scrape-amazon-product-data)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Foxylabs%2Fhow-to-scrape-amazon-prices","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Foxylabs%2Fhow-to-scrape-amazon-prices","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Foxylabs%2Fhow-to-scrape-amazon-prices/lists"}