https://github.com/onome-joseph/x-profile-scrapping
Python-based web scraper that automates X (Twitter) login and extracts profile details.
https://github.com/onome-joseph/x-profile-scrapping
beautifulsoup4 python-scraping selenium-webdriver twitter
Last synced: 2 months ago
JSON representation
Python-based web scraper that automates X (Twitter) login and extracts profile details.
- Host: GitHub
- URL: https://github.com/onome-joseph/x-profile-scrapping
- Owner: Onome-Joseph
- License: mit
- Created: 2025-02-03T01:01:15.000Z (4 months ago)
- Default Branch: main
- Last Pushed: 2025-03-08T06:18:28.000Z (2 months ago)
- Last Synced: 2025-03-08T06:28:15.344Z (2 months ago)
- Topics: beautifulsoup4, python-scraping, selenium-webdriver, twitter
- Language: Python
- Homepage:
- Size: 10.7 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# **X Automation and Scrapping using Selenium & Bs4**
This project is a **Python-based web scraper** that extracts X profile details using **Selenium and BeautifulSoup4**. The script automates **X login**, navigates to user profiles, and retrieves:
- **Bio**
- **Following Count**
- **Followers Count**
- **Location**
- **Website (if available)**
## **The extracted data is stored in a **CSV file** for further analysis.**
---
- **Automated Login:** Logs into X using provided credentials.
- **Scrapes Multiple Profiles:** Reads links from a CSV file and extracts profile details.
- **Handles Restricted Profiles:** Ensures authentication before scraping.
- **Error Handling:** Skips missing elements to avoid script failure.
---
## **Installation & Setup**
### **1 Clone the Repository**
```bash
git clone https://github.com/Onome-Joseph/X-Profile-Scrapping.git
```
### **2 Install Dependencies**
```bash
pip install -r requirements.txt
```
### **3 Configure Login Credentials**
Open `X_bot.py` and **replace** the placeholders with your **X(twitter) credentials** in the config.json file.