https://github.com/sukundev/twitter-scrape
Twitter Scrape is a Python bot designed for scraping Twitter data based on specified keywords. It collects information such as tweet links, text, engagement metrics, and more. The bot stores the data in MongoDB, with functionality to export it to CSV. Setup involves cloning the repository, installing dependencies, configuring credentials, and run
https://github.com/sukundev/twitter-scrape
python python-scraping scrape twitter twitter-bot
Last synced: 11 months ago
JSON representation
Twitter Scrape is a Python bot designed for scraping Twitter data based on specified keywords. It collects information such as tweet links, text, engagement metrics, and more. The bot stores the data in MongoDB, with functionality to export it to CSV. Setup involves cloning the repository, installing dependencies, configuring credentials, and run
- Host: GitHub
- URL: https://github.com/sukundev/twitter-scrape
- Owner: SukunDev
- License: mit
- Created: 2024-05-20T09:03:43.000Z (almost 2 years ago)
- Default Branch: main
- Last Pushed: 2025-02-04T01:03:03.000Z (about 1 year ago)
- Last Synced: 2025-02-17T01:47:34.961Z (about 1 year ago)
- Topics: python, python-scraping, scrape, twitter, twitter-bot
- Language: Python
- Homepage:
- Size: 4.69 MB
- Stars: 1
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Twitter Scrape
Bot untuk scraping data dari Twitter.
## Installasi
### Persiapan
Sebelum memulai, pastikan Anda telah menginstal dan menambahkan ke dalam environment variable:
- [MongoDB](https://www.mongodb.com/try/download/community)
- [Mongo Shell](https://www.mongodb.com/try/download/shell)
- [Mongo Database Tools](https://www.mongodb.com/try/download/database-tools)
### Cloning Repository
```bash
git clone https://github.com/SukunDev/twitter-scrape.git
cd twitter-scrape
```
### Instalasi di Command Prompt (CMD)
```bash
python -m venv .venv
.venv\Scripts\activate
pip install -r requirements.txt
```
### Instalasi di Git Bash
```bash
python -m venv .venv
source .venv/Scripts/activate
pip install -r requirements.txt
```
## Migrasi Database
### Import Database
```bash
mongorestore ./database_backup/bot_scrape --db bot_scrape
```
### Export Database
```bash
mongodump -d bot_scrape -o database_backup
```
### Export ke CSV
```bash
mongoexport --host localhost --db bot_scrape --collection tweet --type=csv \
--out film_horor_collection.csv --fields tweet_link,full_text,bookmark_count, \
favorite_count,quote_count,reply_count,retweet_count,lang,user,entities, \
extended_entities,created_at
```
## Menjalankan Program
Sebelum menjalankan program, edit terlebih dahulu file `main.py`:
```python
from scrape import Scrape
if __name__ == "__main__":
# Ganti dengan username dan password Twitter Anda
scrape = Scrape(username="", password="", keywords="film horor")
scrape.run()
```
Setelah itu, jalankan program dengan perintah:
```bash
python main.py
```
## Kontribusi
Kami menerima kontribusi dari siapa saja! Jika Anda ingin berkontribusi, ikuti langkah berikut:
1. Fork repository ini.
2. Buat branch baru untuk fitur atau perbaikan Anda.
3. Commit perubahan Anda dengan deskripsi yang jelas.
4. Push ke repository Anda dan buat pull request ke repository utama.
## Lisensi
Proyek ini dilisensikan di bawah [MIT License](LICENSE).