{"id":25428805,"url":"https://github.com/sukundev/twitter-scrape","last_synced_at":"2025-05-14T08:35:14.218Z","repository":{"id":240649193,"uuid":"803197704","full_name":"SukunDev/twitter-scrape","owner":"SukunDev","description":" Twitter Scrape is a Python bot designed for scraping Twitter data based on specified keywords. It collects information such as tweet links, text, engagement metrics, and more. The bot stores the data in MongoDB, with functionality to export it to CSV. Setup involves cloning the repository, installing dependencies, configuring credentials, and run","archived":false,"fork":false,"pushed_at":"2025-02-04T01:03:03.000Z","size":4922,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-02-17T01:47:34.961Z","etag":null,"topics":["python","python-scraping","scrape","twitter","twitter-bot"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/SukunDev.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-05-20T09:03:43.000Z","updated_at":"2025-02-04T01:10:37.000Z","dependencies_parsed_at":"2025-02-04T02:29:12.830Z","dependency_job_id":null,"html_url":"https://github.com/SukunDev/twitter-scrape","commit_stats":null,"previous_names":["sukundev/twitter-scrape"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SukunDev%2Ftwitter-scrape","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SukunDev%2Ftwitter-scrape/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SukunDev%2Ftwitter-scrape/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SukunDev%2Ftwitter-scrape/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/SukunDev","download_url":"https://codeload.github.com/SukunDev/twitter-scrape/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254104737,"owners_count":22015533,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["python","python-scraping","scrape","twitter","twitter-bot"],"created_at":"2025-02-17T01:48:08.268Z","updated_at":"2025-05-14T08:35:14.210Z","avatar_url":"https://github.com/SukunDev.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Twitter Scrape\n\nBot untuk scraping data dari Twitter.\n\n## Installasi\n\n### Persiapan\n\nSebelum memulai, pastikan Anda telah menginstal dan menambahkan ke dalam environment variable:\n\n- [MongoDB](https://www.mongodb.com/try/download/community)\n- [Mongo Shell](https://www.mongodb.com/try/download/shell)\n- [Mongo Database Tools](https://www.mongodb.com/try/download/database-tools)\n\n### Cloning Repository\n\n```bash\ngit clone https://github.com/SukunDev/twitter-scrape.git\ncd twitter-scrape\n```\n\n### Instalasi di Command Prompt (CMD)\n\n```bash\npython -m venv .venv\n.venv\\Scripts\\activate\npip install -r requirements.txt\n```\n\n### Instalasi di Git Bash\n\n```bash\npython -m venv .venv\nsource .venv/Scripts/activate\npip install -r requirements.txt\n```\n\n## Migrasi Database\n\n### Import Database\n\n```bash\nmongorestore ./database_backup/bot_scrape --db bot_scrape\n```\n\n### Export Database\n\n```bash\nmongodump -d bot_scrape -o database_backup\n```\n\n### Export ke CSV\n\n```bash\nmongoexport --host localhost --db bot_scrape --collection tweet --type=csv \\\n--out film_horor_collection.csv --fields tweet_link,full_text,bookmark_count, \\\nfavorite_count,quote_count,reply_count,retweet_count,lang,user,entities, \\\nextended_entities,created_at\n```\n\n## Menjalankan Program\n\nSebelum menjalankan program, edit terlebih dahulu file `main.py`:\n\n```python\nfrom scrape import Scrape\n\nif __name__ == \"__main__\":\n    # Ganti dengan username dan password Twitter Anda\n    scrape = Scrape(username=\"\u003cusername_twitter\u003e\", password=\"\u003cpassword_twitter\u003e\", keywords=\"film horor\")\n    scrape.run()\n```\n\nSetelah itu, jalankan program dengan perintah:\n\n```bash\npython main.py\n```\n\n## Kontribusi\n\nKami menerima kontribusi dari siapa saja! Jika Anda ingin berkontribusi, ikuti langkah berikut:\n\n1. Fork repository ini.\n2. Buat branch baru untuk fitur atau perbaikan Anda.\n3. Commit perubahan Anda dengan deskripsi yang jelas.\n4. Push ke repository Anda dan buat pull request ke repository utama.\n\n## Lisensi\n\nProyek ini dilisensikan di bawah [MIT License](LICENSE).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsukundev%2Ftwitter-scrape","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsukundev%2Ftwitter-scrape","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsukundev%2Ftwitter-scrape/lists"}