{"id":18264718,"url":"https://github.com/mousazourob/sahib","last_synced_at":"2026-04-16T14:08:42.193Z","repository":{"id":78254046,"uuid":"288019857","full_name":"MousaZourob/Sahib","owner":"MousaZourob","description":"Python bot that scrapes Walmart's clearance page for electronics to find the best deals and post them on Twitter automatically","archived":false,"fork":false,"pushed_at":"2021-01-09T20:15:28.000Z","size":1685,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-02-14T19:53:50.930Z","etag":null,"topics":["dnspython","geckodriver","mongodb","python","selenium","webscrapping"],"latest_commit_sha":null,"homepage":"https://twitter.com/SahibBot_","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/MousaZourob.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-08-16T20:32:25.000Z","updated_at":"2021-01-09T20:15:30.000Z","dependencies_parsed_at":"2023-02-28T03:45:27.828Z","dependency_job_id":null,"html_url":"https://github.com/MousaZourob/Sahib","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MousaZourob%2FSahib","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MousaZourob%2FSahib/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MousaZourob%2FSahib/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MousaZourob%2FSahib/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/MousaZourob","download_url":"https://codeload.github.com/MousaZourob/Sahib/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247958710,"owners_count":21024821,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["dnspython","geckodriver","mongodb","python","selenium","webscrapping"],"created_at":"2024-11-05T11:15:44.322Z","updated_at":"2026-04-16T14:08:42.144Z","avatar_url":"https://github.com/MousaZourob.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Sahib\n### Overview:\n**Python** bot that scrapes Walmart's clearance page for electronics to find the best deals and post them on Twitter automatically. The bot scrapes data using the **Selenium framework** for Python, and the data is then stored in a **MongoDB** collection to be tweeted later. Using Windows Task Scheduler, the script was made into a **CRON job** to tweet 5 deals daily.\n\n### Data Flow:\n\n#### Scraping from Walmart:\n**1.** Using **Geckodriver**, a **Webdriver for Firefox** is created that opens Walmart's clearance page for electronics\n\u003cbr /\u003e\n**2.** The products are then scrapped using **Selenium**, and data such as the product's title, new and old price, and when it was scrapped is recorded\n\u003cbr /\u003e\n**3.** Calculations are done to determine the discount percentage and price, and then using **DNSPython** products are sent to be stored in a **MongoDB** collection\n\n#### Scraping from Twitter:\n**1.** Using **Geckodriver**, a **Webdriver for Firefox** is created that opens Twitter's log-in page\n\u003cbr /\u003e\n**2.** The script then checks if **cookies** saved as **JSON** objects for an older log-in exist, and if not logs in normally\n\u003cbr /\u003e\n**3.** Afterwards a connection with **MongoDB** is established, and the script finds the first 5 postings that haven't been tweeted\n\u003cbr /\u003e\n**4.** Using **Selenium** tweets are sent out over a 3 minute period, each with one of 8 template messages to publish new deals, and the posting date is saved in the database to not allow duplicate postings to occur\n\u003cbr /\u003e\n**5.** This script was turned into a **CRON job** using Windows Task Scheduler, and it runs automatically once a day to tweet out 5 deals daily\n\n## Demo:\n* To view the bot in action, click [here](https://twitter.com/SahibBot_): \n![Demo](https://user-images.githubusercontent.com/66835262/104045431-8c008080-51ac-11eb-9d31-7537516b84c5.png)\n\n\n## Libraries and Frameworks Used: \n* **Selenium framework for Python:** https://selenium-python.readthedocs.io/\n* **Geckodriver:** https://github.com/mozilla/geckodriver/releases\n* **MongoDB for Python:** https://www.mongodb.com/blog/post/getting-started-with-python-and-mongodb\n* **DNSPython:** https://pypi.org/project/dnspython/\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmousazourob%2Fsahib","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmousazourob%2Fsahib","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmousazourob%2Fsahib/lists"}