awesome-data-hoarding
How to save everything online. Tools for for scraping, saving, downloading, hoarding, archiving, etc.
https://github.com/all-the-data/awesome-data-hoarding
Last synced: 12 days ago
JSON representation
-
Processing tools
-
Quick reference
- the-eye
- ThreadReaderApp
- unblockit
- ArchiveBot
- cleanly print
- BDFRX
- Gallery-DL
- RipMe
- TumblThreeApp - pro/tumbl-three-viewer), [2](https://github.com/willsheppard/random-scripts/blob/master/TumblThree_BackupViewer.html).
- SingleFile - - Recommended!
- youtube-dl - dlp](https://github.com/yt-dlp/yt-dlp)
- Bulk-Downloader-For-Reddit
- download-all-images
- Greenshot
- amazon-orders
- unblockit
- download-all-images
-
Scraping tools
-
General purpose
-
- wget
- Chrome DevTools
- network tab
- resources tab
- Shazam - music-song-finder-for/dpacanjfikmhoddligfbehkpomnbgblf)
- La Mega, Malaga
- mwparserfromhell
- DiscordChatExporter
- youtube-dl - dlp](https://github.com/yt-dlp/yt-dlp) for Youtube and other video/audio
- Wayback-Archive - 3.0)
- StreamRipper
-
-
Techniques
-
Case studies
-
Extract playlist data from YouTube and YT Music
- <load jquery> - in>](https://stackoverflow.com/a/20495940) | `$(document).xpathEvaluate('//body/div/foo')`
- Save data out of console
- <load jquery> - in>](https://stackoverflow.com/a/20495940) | `$(document).xpathEvaluate('//body/div/foo')`
-
Keywords
python
3
downloader
3
twitter
2
tumblr
2
tumblr-backup
1
mvvm
1
dotnet
1
csharp
1
crawler
1
c-sharp
1
blog-backup
1
backup
1
reddit
1
imgur
1
gfycat
1
pixiv
1
mangadex
1
kemono
1
gallery
1
flickr
1
deviantart
1
danbooru
1
log
1
expoter
1
export
1
discord
1
chat
1
archiver
1
archival
1
web-scraping
1
receipts
1
order-history
1
cli
1
beautifulsoup
1
amazon
1
zip
1
webapp
1
web-application
1
web
1
portable
1
debugging
1
archive
1
har
1
extension
1
devtools
1
chrome
1
wikipedia
1
parser
1
mediawiki
1
wpf
1