https://github.com/stephanakkerman/x-timeline-scraper
Scrapes X (formerly Twitter) timeline
https://github.com/stephanakkerman/x-timeline-scraper
Last synced: 8 months ago
JSON representation
Scrapes X (formerly Twitter) timeline
- Host: GitHub
- URL: https://github.com/stephanakkerman/x-timeline-scraper
- Owner: StephanAkkerman
- License: mit
- Created: 2025-09-20T17:21:43.000Z (9 months ago)
- Default Branch: main
- Last Pushed: 2025-09-21T09:35:23.000Z (9 months ago)
- Last Synced: 2025-09-21T11:33:04.376Z (9 months ago)
- Language: Python
- Size: 28.3 KB
- Stars: 0
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- Funding: .github/FUNDING.yml
- License: LICENSE
Awesome Lists containing this project
README
# X-Timeline Scraper
A Python client to scrape tweets from X (formerly Twitter) timelines using a cURL command.
---
## Introduction
This project provides a Python client to scrape tweets from X (formerly Twitter) timelines using a cURL command. It leverages asynchronous programming for efficient data retrieval and includes features for parsing tweet data.
## Table of Contents 🗂
- [Installation](#installation)
- [Usage](#usage)
- [Citation](#citation)
- [Contributing](#contributing)
- [License](#license)
## Installation ⚙️
To install the X-Timeline Scraper, you can use pip:
```bash
pip install xtimeline
```
## Usage ⌨️
To use the X-Timeline Scraper, you need to provide a cURL command that accesses the desired X timeline. The instructions can be found in [curl_example.txt](curl_example.txt). Then, you can use the `XTimelineClient` class to fetch and parse tweets.
Here's a simple example of how to use the client:
```python
import asyncio
from src.xclient import XTimelineClient
async with XTimelineClient(
"curl.txt", persist_last_id_path="state/last_id.txt"
) as xc:
tweets = await xc.fetch_tweets(update_last_id=False)
for t in tweets:
print(t.to_markdown())
```
You can also stream new tweets in real-time:
```python
import asyncio
from src.xclient import XTimelineClient
async with XTimelineClient(
"curl.txt", persist_last_id_path="state/last_id.txt"
) as xc:
async for t in xc.stream(interval_s=5.0):
print(t.to_markdown())
```
## Citation ✍️
If you use this project in your research, please cite as follows:
```bibtex
@misc{project_name,
author = {Stephan Akkerman},
title = {X-Timeline Scraper},
year = {2025},
publisher = {GitHub},
journal = {GitHub repository},
howpublished = {\url{https://github.com/StephanAkkerman/x-timeline-scraper}}
}
```
## Contributing 🛠
Contributions are welcome! If you have a feature request, bug report, or proposal for code refactoring, please feel free to open an issue on GitHub. We appreciate your help in improving this project.\
## License 📜
This project is licensed under the MIT License. See the [LICENSE](LICENSE) file for details.