Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/kimkykie/twitter-image-downloader
This is a tool to download images posted/retweeted from a Twitter user's timeline.
https://github.com/kimkykie/twitter-image-downloader
chromium nodejs puppeteer scraper scraping
Last synced: about 2 months ago
JSON representation
This is a tool to download images posted/retweeted from a Twitter user's timeline.
- Host: GitHub
- URL: https://github.com/kimkykie/twitter-image-downloader
- Owner: Kimkykie
- Created: 2021-04-08T11:50:35.000Z (almost 4 years ago)
- Default Branch: master
- Last Pushed: 2024-08-01T14:58:30.000Z (5 months ago)
- Last Synced: 2024-08-01T16:05:14.363Z (5 months ago)
- Topics: chromium, nodejs, puppeteer, scraper, scraping
- Language: JavaScript
- Homepage:
- Size: 399 KB
- Stars: 3
- Watchers: 2
- Forks: 3
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# 🐦 Twitter Timeline Image Downloader
An advanced tool to download images from a user's Twitter timeline.
## ✨ Features
- 🔐 Automated login to Twitter
- 🖼️ Scrapes images from a specified user's timeline
- 📜 Handles pagination to fetch all available images
- 🛡️ Robust error handling and retry mechanisms
- 🚀 Efficient image downloading with duplicate prevention
- 📊 Detailed logging for better debugging and monitoring
- ⚙️ Configurable via environment variables for easy deployment
## 🏗️ Project Structure
```code
twitter-image-downloader/
├── images/
├── src/
│ ├── config/
│ │ └── config.js
│ ├── services/
│ │ ├── authService.js
│ │ ├── browserService.js
│ │ └── imageService.js
│ ├── utils/
│ │ ├── logger.js
│ │ └── fileSystem.js
│ └── index.js
├── lib/
│ └── downloader.js
├── images/
├── .env.example
├── .gitignore
├── package.json
├── package-lock.json
└── README.md
```
## 🛠️ Technologies Used
- **Node.js**: Runtime environment
- **Puppeteer**: Web scraping and automation
- **Axios**: HTTP client for image downloads
- **Winston**: Logging framework
- **Inquirer**: Command-line user interface
- **Dotenv**: Environment variable management
## 📥 Installation
### Prerequisites
Ensure you have Node.js (version 14 or higher) and npm installed on your system.
### Steps
1. Clone the repository:
```bash
git clone https://github.com/Kimkykie/twitter-image-downloader.git
cd twitter-image-downloader
```
2. Install dependencies:
```bash
npm install
```
3. Copy the `.env.example` file to `.env` and adjust the variables as needed:
```bash
cp .env.example .env
```
## 🚀 Usage
1. Start the application:
```bash
npm start
```
2. Follow the prompts to enter your Twitter credentials and the username of the account you want to download images from.
3. The script will start running and create a folder with the username you entered in the `images` folder.
4. Once complete, you'll see a "Download Complete" message in your console.
## ⚙️ Configuration
You can configure the application by modifying the `.env` file. Available options include:
- `PUPPETEER_HEADLESS`: Set to 'true' for headless mode, 'false' to see the browser (default: false)
- `PUPPETEER_SLOWMO`: Slow down Puppeteer operations by specified milliseconds (default: 50)
- `VIEWPORT_WIDTH` and `VIEWPORT_HEIGHT`: Set the browser viewport size
- Various timeout durations and URLs
## 🤝 Contributing
Contributions are welcome! Please feel free to submit a Pull Request.
## ⚠️ Disclaimer and Caution
### Risk of Account Flagging
Please be aware that using this tool may potentially lead to your Twitter account being flagged or blocked. Twitter has mechanisms in place to detect automated scraping activities, which could result in account restrictions or bans.
### Use of Sockpuppets
To mitigate risks to your primary account, consider using a sockpuppet account for scraping activities. A sockpuppet is a false online identity created for purposes such as circumventing restrictions or maintaining anonymity.
For more information on sockpuppets and their use in OSINT (Open Source Intelligence), please refer to this SANS Institute article on sockpuppets.
**Remember**: Always use this tool responsibly and in compliance with Twitter's terms of service. The developers of this tool are not responsible for any account restrictions or legal issues that may arise from its use.
## 📄 License
[MIT](https://choosealicense.com/licenses/mit/)