{"id":27949773,"url":"https://github.com/justsxm/deals-scraper","last_synced_at":"2025-07-04T11:07:15.430Z","repository":{"id":158134983,"uuid":"439413307","full_name":"JustSxm/Deals-Scraper","owner":"JustSxm","description":"Deals Scraper is a Canadian tool to find good deals on websites like Facebook Marketplace, Kijiji, Ebay, Amazon and Lespacs","archived":false,"fork":false,"pushed_at":"2024-04-29T04:22:28.000Z","size":10115,"stargazers_count":93,"open_issues_count":2,"forks_count":17,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-07-04T11:07:14.695Z","etag":null,"topics":["amazon","deals","ebay","facebook","kijiji","lespacs","python","scraper","scraping"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/JustSxm.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2021-12-17T17:37:01.000Z","updated_at":"2025-06-18T17:49:23.000Z","dependencies_parsed_at":null,"dependency_job_id":"89caa8fc-af62-49ad-a752-e5f1789a472b","html_url":"https://github.com/JustSxm/Deals-Scraper","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/JustSxm/Deals-Scraper","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JustSxm%2FDeals-Scraper","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JustSxm%2FDeals-Scraper/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JustSxm%2FDeals-Scraper/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JustSxm%2FDeals-Scraper/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/JustSxm","download_url":"https://codeload.github.com/JustSxm/Deals-Scraper/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JustSxm%2FDeals-Scraper/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":263499191,"owners_count":23476021,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["amazon","deals","ebay","facebook","kijiji","lespacs","python","scraper","scraping"],"created_at":"2025-05-07T15:30:16.874Z","updated_at":"2025-07-04T11:07:15.421Z","avatar_url":"https://github.com/JustSxm.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\n![Python](https://img.shields.io/badge/Python-FFD43B?style=for-the-badge\u0026logo=python\u0026logoColor=blue)\n\n#### Please star the repo if you found it useful! Thank you!\n\n# Deals Scraper\n\n\u003cdetails\u003e\n\u003csummary\u003eTable of Contents\u003c/summary\u003e\n \n- [Features](#features)  \n- [Requirement](#Requirement)  \n- [Configuration](#Configuration)  \n  - [Facebook](#FacebookConfiguration)  \n  - [Kijiji](#KijijiConfiguration)  \n  - [Ebay](#EbayConfiguration)  \n  - [Lespacs](#LespacsConfiguration)  \n  - [Amazon](#AmazonConfiguration)  \n- [Running](#Running)  \n- [More information about each site](#Moreinformationabouteachsite)  \n  - [Facebook](#Facebook)  \n  - [Kijiji](#Kijiji)  \n  - [Ebay](#Ebay)  \n  - [Lespacs](#Lespacs)  \n  - [Amazon](#Amazon)  \n- [Acknowledgements](#Acknowledgements)  \n- [Contributors](#Contributors)  \n- [License](#License)\n \n\u003c/details\u003e\n\nDeals Scraper is a Canadian tool to find good deals on websites like Facebook Marketplace, Kijiji, Ebay, Amazon and Lespacs\n\n\n\u003ca name=\"features\"/\u003e\n\n## Features\n\n- Zooming fast\n- Specify a price range (Min \u0026 Max) per scraper\n- Blacklist keywords from the scraping\n- Strict mode so only ads containing your keywords are picked\n- Schedule the project to run on a recurring basis\n- Easily add your own website if you know what you're doing\n\n\u003ca name=\"Requirement\"/\u003e\n\n## Requirement\n\n - [Python](https://www.python.org/)\n\u003ca name=\"Installation\"/\u003e\n\n## Installation\n\n\nClone the repo\n```bash\n  git clone https://github.com/JustSxm/Deals-Scraper.git\n```\n\nInstall dependencies\n```bash\n  pip install -r requirement.txt\n```\n\u003ca name=\"Configuration\"/\u003e\n\n## Configuration\n\u003ca name=\"DefaultConfiguration\"/\u003e\n\n## Default Configuration\nOpen the config file (config.ini) and find the section named DEFAULT\n```INI\n[DEFAULT]\nKeywords = airpods pro\nExclusions = case\nStrictMode = False\nInterval = 1\n```\n- Keywords: This config sets the keywords that the software will search for.\n- Exclusions: This config sets the words or phrases that the software will ignore.\n- StrictMode: This config determines whether the software will strictly match the keywords and exclusions or be more flexible in its search.\n- Interval: This config sets the time interval for how often the software will search for the keywords in minutes.\n\u003ca name=\"FacebookConfiguration\"/\u003e\n\n### Facebook\nOpen the config file (config.ini) and find the section named FACEBOOK\n```INI\n[FACEBOOK]\nEnabled = True\nCityId = \nMinPrice = 0\nMaxPrice = 1000\nSortBy = distance_ascend\n; best_match, price_ascend, price_descend, distance_ascend, creation_time_descend\n```\n- Enabled: This config determines whether the Facebook module is enabled (True) or disabled (False).\n- CityId: This config sets the ID for the desired city or location for the Facebook search.\n- MinPrice: This config sets the minimum price range for the Facebook search.\n- MaxPrice: This config sets the maximum price range for the Facebook search.\n- SortBy: This config sets the sorting method for the Facebook search results. Available options include \"best_match\" (sorted by Facebook's relevance algorithm), \"price_ascend\" (sorted by price in ascending order), \"price_descend\" (sorted by price in descending order), \"distance_ascend\" (sorted by distance in ascending order), and \"creation_time_descend\" (sorted by time in descending order).\n\nTo find your CityId:\n- go to facebook and search for your city (Ex: https://www.facebook.com/search/top?q=toronto)\n- Click on \"Places\" \u003cbr\u003e\n![Image of Place button](https://i.imgur.com/6uquG7S.png)\n- Click on your city \u003cbr\u003e\n![Image of Toronto City](https://i.imgur.com/6ipzFG5.png)\n- Copy your city's id \u003cbr\u003e\n![Image of Toronto City's URL](https://i.imgur.com/sCmi2jW.png)\n\nAfter make sure to login to Facebook, this scraper uses your browser to scrape, it doesn't connect automatically.\n\u003ca name=\"KijijiConfiguration\"/\u003e\n\n## Kijiji\nOpen the config file (config.ini) and find the section named Kijiji\n```INI\n[KIJIJI]\nEnabled = True\nCityUrl = \nIdentifier = \nMinPrice = 20\nMaxPrice = 100\nType = ownr\n; ownr, delr, all\n```\n- Enabled: This config determines whether the Kijiji module is enabled (True) or disabled (False).\n- CityUrl: This config sets the URL for the Kijiji website for the desired city or location.\n- Identifier: The identifier with the city url.\n- MinPrice: This config sets the minimum price range for the Kijiji search.\n- MaxPrice: This config sets the maximum price range for the Kijiji search.\n- Type: This config sets the type of Kijiji ads to search for, which can be \"ownr\" (owner-sold ads), \"delr\" (dealer-sold ads), or \"all\" (both types).\n\nTo find your CityId and Identifier:\n- go to kijiji and search for something random with ads results\n- Copy your city's id and identifier \u003cbr\u003e\n![Image of Kijiji City's URL](https://i.imgur.com/6e3UUnv.png)\n\u003ca name=\"EbayConfiguration\"/\u003e\n\n## Ebay\nOpen the config file (config.ini) and find the section named Ebay\n```INI\n[EBAY]\nEnabled = True\nMinPrice = 20\nMaxPrice = 100\n```\n- Enabled: This config determines whether the eBay module is enabled (True) or disabled (False).\n- MinPrice: This config sets the minimum price range for the eBay search.\n- MaxPrice: This config sets the maximum price range for the eBay search.\n\u003ca name=\"LespacsConfiguration\"/\u003e\n\n## Lespacs\nThe LesPACs configuration provided in this repository is not currently implemented due to the website either undergoing a rewrite or implementing security measures to prevent web scraping.\nUnfortunately, without more information from LesPACs themselves, it is not possible to provide an ETA for when the configuration will be functional again.\n\u003ca name=\"AmazonConfiguration\"/\u003e\n\n## Amazon\nIf you are looking to use the Amazon configuration in this repository, please note that it is not included in the current version and is not planned to be included in the future. This is due to the implementation of anti-scraping measures on the Amazon website, such as CAPTCHAs, which make it difficult or impossible to retrieve data using a web scraper. As such, the Amazon configuration provided in previous versions of the repository may no longer be functional.\n\u003ca name=\"Running\"/\u003e\n\n## Running\n\nRun the python script\n\n```bash\n  python main.py\n```\n\n\u003ca name=\"Moreinformationabouteachsite\"/\u003e\n\n## More information about each site\n\u003ca name=\"Facebook\"/\u003e\n\n#### Facebook\n\nFacebook uses a city to look around as it is not international, you can find the id by looking for your city on facebook and copy the id of their page.\n(usually facebook.com/..../place/id)\n\nFacebook is a good website for scraping.\n\u003ca name=\"Kijiji\"/\u003e\n\n#### Kijiji\n\nKijiji is a good website for scraping\n\u003ca name=\"Ebay\"/\u003e\n#### Ebay\n\nEbay is an okay website for scraping\n\u003ca name=\"Amazon\"/\u003e\n\n#### Amazon\n\nSince Amazon is a vast website, it is way harder to find new ads and to precise what we want, therefore you will most likely get garbage from it than what you're actually looking.\nIt could be fine if you're looking for the cheapest price for a \"popular\" item\n\nAmazon is a bad website for scraping\n\u003ca name=\"Lespacs\"/\u003e\n\n#### Lespacs\n\nLespacs is just like kijiji except it is more Quebec centered than Canada, therefore it can be a bad site for scraping if you are not from Quebec, otherwise it is a pretty good one\n\nOutside of Quebec: Lespacs is a bad website for scraping\n\nInside of Quebec: Lespacs is a good website for scraping\n\n\n\n\u003ca name=\"Acknowledgements\"/\u003e\n\n## Acknowledgements\n\nThis project was made with the help of [scrapy](https://github.com/scrapy/scrapy)\n\u003ca name=\"Contributors\"/\u003e\n## Contributors\nREADME - [ChatGPT](https://chat.openai.com/chat)\n\u003ca name=\"License\"/\u003e\n## License\n\n[MIT](https://choosealicense.com/licenses/mit/)\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjustsxm%2Fdeals-scraper","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjustsxm%2Fdeals-scraper","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjustsxm%2Fdeals-scraper/lists"}