{"id":21201789,"url":"https://github.com/aorosoeon/ecommerce_parcing","last_synced_at":"2026-04-15T15:34:33.798Z","repository":{"id":262120137,"uuid":"886265493","full_name":"aorosoeon/ecommerce_parcing","owner":"aorosoeon","description":"An example of ecommerce parcing project through Selenium, FastAPI, and APScheduler. It saves product's name, price, link, and stock availability in a csv while handling the website's pagination. It can be activated via \"/scrape\" FastAPI endpoint and then keeps running by APScheduler.","archived":false,"fork":false,"pushed_at":"2024-11-18T02:22:50.000Z","size":126,"stargazers_count":1,"open_issues_count":1,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-07-28T14:50:19.202Z","etag":null,"topics":["apscheduler","automation","ecommerce","fastapi","python","selenium"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/aorosoeon.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-11-10T16:15:29.000Z","updated_at":"2025-01-03T19:10:35.000Z","dependencies_parsed_at":"2024-11-10T17:31:08.098Z","dependency_job_id":"abc4a53a-75cd-4add-95cd-f68b2e109ef5","html_url":"https://github.com/aorosoeon/ecommerce_parcing","commit_stats":null,"previous_names":["aorosoeon/ecommerce_parcing"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/aorosoeon/ecommerce_parcing","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aorosoeon%2Fecommerce_parcing","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aorosoeon%2Fecommerce_parcing/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aorosoeon%2Fecommerce_parcing/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aorosoeon%2Fecommerce_parcing/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/aorosoeon","download_url":"https://codeload.github.com/aorosoeon/ecommerce_parcing/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aorosoeon%2Fecommerce_parcing/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31848003,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-15T15:24:51.572Z","status":"ssl_error","status_checked_at":"2026-04-15T15:24:39.138Z","response_time":63,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["apscheduler","automation","ecommerce","fastapi","python","selenium"],"created_at":"2024-11-20T20:11:08.933Z","updated_at":"2026-04-15T15:34:33.781Z","avatar_url":"https://github.com/aorosoeon.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# ecommerce_parcing\n\nThis architecture is an example of an e-commerce website parcing.\n\nHere is a quick video of how it works: https://youtu.be/Tu2o3SrzJnM. If you are curious to see what the output looks like, open scraped_data.csv in this repo.\n\nQuick setup:\n- pip install -r requirements.txt\n- Download the repo\n- Substitute chromedriver_placeholder with actual chromedriver\n- Fill in login_info_placeholder.py and change it's name to login_info.py\n- Line 13 in scraper.py - provide the right path\n- Lines 38 and 43 in scraper.py - remove limitation if needed\n- Type in your terminal \"uvicorn fastapi_file:app --host 0.0.0.0 --port 8000\"\n- Type in any other device's browser under the same network \"http://IP_OF_THE_LOCAL_COMPUTER:8000/scrape\"\n- Once it's done, it scrapes data in csv and repeats this procedure after a specific interval (customizable in fastapi_file.py)\n\n\u003cimg src=\"imgs/architecture.png\"\u003e\n\nUPD: scraper.py logs errors now as well\n\nDisclaimer: This project includes code for a one-time web parcing example, created solely for educational purposes. It is not intended for continuous or automated scraping of any website. Please ensure you adhere to the terms of service of any website you interact with, as this code is provided as an example and not for production use. Sail.ca website was chosen randomly for this project and the only goal of this code is educational.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faorosoeon%2Fecommerce_parcing","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Faorosoeon%2Fecommerce_parcing","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faorosoeon%2Fecommerce_parcing/lists"}