https://github.com/ericlingit/weibo-trending
Scrape trending Weibo posts.
https://github.com/ericlingit/weibo-trending
Last synced: 5 months ago
JSON representation
Scrape trending Weibo posts.
- Host: GitHub
- URL: https://github.com/ericlingit/weibo-trending
- Owner: ericlingit
- License: gpl-3.0
- Created: 2022-11-07T15:07:41.000Z (over 3 years ago)
- Default Branch: main
- Last Pushed: 2022-12-05T06:56:18.000Z (over 3 years ago)
- Last Synced: 2025-10-28T02:26:58.426Z (8 months ago)
- Topics: weibo
- Language: Python
- Homepage:
- Size: 1.27 MB
- Stars: 1
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Weibo trending posts scraper
Scrap trending posts from Weibo front page.
## Usage guide
### As a library
Install
`pip install weibo-trending`
Get and parse posts
```python
from weibo_trending import get_new_posts, parse_response
resp = get_new_posts()
mblogs = parse_response(resp)
for mblog in mblogs:
print(mblog)
```
Get and parse comments
```python
from weibo_trending import get_raw_comments, parse_comments
example_post_id = "4840673008419178"
raw_cmt = get_raw_comments(example_post_id)
comments = parse_comments(raw_cmt)
for c in comments:
print(c)
```
### As a command line tool
Install
`pip install weibo-trending`
Usage
```
python -m weibo_trending --help
usage: weibo_trending [-h] [-d DIR] [-s]
Scrape and parse Weibo trending posts.
optional arguments:
-h, --help show this help message and exit
-d DIR, --dir DIR specify the output directory. Defaults to the current working directory
-s, --skip-parsing whether to skip parsing and dump the raw JSON response from Weibo
python -m weibo_trending
```
weibo_trending will save each scraped post with the following filename format:
- `weibo__.json`
- Example: `weibo_1631153043_4834313265233660.json`
Each call to weibo_trending usually saves 10 new files. If you get fewer than 10, that means the response contains one or more deleted posts. They are not saved.
## Develop
```
git clone https://github.com/ericlingit/weibo-trending.git
cd weibo-trending
python3 -m venv venv
source venv/bin/activate
pip install -U pip wheel
pip install -r requirements.txt
pip install -e .
pytest
```
### Package
```
python -m build --wheel
```
The built wheel is in `./dist/`.