https://github.com/tuvimen/disqus-scraper
A bash script for scraping comments from disqus
https://github.com/tuvimen/disqus-scraper
bash disqus json scraper
Last synced: 7 months ago
JSON representation
A bash script for scraping comments from disqus
- Host: GitHub
- URL: https://github.com/tuvimen/disqus-scraper
- Owner: TUVIMEN
- License: gpl-3.0
- Created: 2022-11-13T20:23:55.000Z (almost 3 years ago)
- Default Branch: master
- Last Pushed: 2024-04-06T15:19:45.000Z (over 1 year ago)
- Last Synced: 2025-01-31T08:33:02.442Z (8 months ago)
- Topics: bash, disqus, json, scraper
- Language: Shell
- Homepage:
- Size: 19.5 KB
- Stars: 1
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# disqus-scraper
A bash script for scraping comments from disqus.
## Requirements
- [jq](https://github.com/stedolan/jq)
## Installation
install -m 755 disqus-scraper /usr/bin
## Supported links formats
owner
owner/thread
@user
http(s)?://disqus.com/home/forum/{owner}/
http(s)?://disqus.com/home/discussion/{owner}/{thread}/
http(s)?://disqus.com/by/{user}/
## Structure
### Discussion
details.json
[0-9]+.json
### Forum
forumfeatures.json
[0-9]+.json
{discussion}/
### User
details.json
mostactiveforums.json
favorites-[0-9]+.json
comments-[0-9]+.json
threads-[0-9]+.json
followingchannels-[0-9].json
badges-[0-9].json
## Usage
disqus-scraper directory [URL]...
The script writes json files downloaded from disqus-scraper.
Get everything from user profile to directory x
disqus-scraper x @user
disqus-scraper x https://disqus.com/by/user/
Get everything from discussion to directory x
disqus-scraper x owner/thread
disqus-scraper x https://disqus.com/home/discussion/owner/thread/
Get everyting from forum to directory x
disqus-scraper x owner
disqus-scraper x https://disqus.com/home/forum/owner/