Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

awesome-datahoarder

An Awesome & curated list of links and tools used for Data Hoarding!
https://github.com/grokskookum/awesome-datahoarder

Last synced: 3 days ago
JSON representation

Utility Scripts
- Download automation
  - Backblaze B2 sync backup script
  - rclone_dirsize
  - rm_empty_subdir - directories on Windows
  - youtube-dl_soundcloud - dl to download soundcloud playlists
  - Misc download scripts
  - void-cat-uploader
  - flac2mp3_V0.py - threaded python script to convert all flac files to mp3 V0 while keeping the directory structure
Download utilities
- Web Archiving
  - HTTrack
  - ArchiveBox - hosted web archive. Takes browser history/bookmarks/Pocket/Pinboard/etc., saves HTML, JS, PDFs, media, and more...
  - Collect
  - Heritrix - scale, archival-quality web crawler
  - wail - Click User Instigated Preservation
  - wikiteam
- General
  - aria2 - protocol & multi-source command-line download utility
  - wget - interactive download of files from the Web.
  - annie - DL alternative written in Golang
  - CrowLeer
  - curl
  - news-crawl
  - Plowshare - line tool to manage file-sharing site
  - Rclone
  - wpull - compatible web downloader and crawler
  - you-get
  - Youtube-DL - line program to download videos from YouTube and a few hundred more sites
- Application-specific
  - floatplane_ripper
  - Scribd-Downloader
  - BBCSoundDownloader
  - ChanThreadWatch - style boards and checks for updates until the thread dies
  - comics-downloader - line tool to download comicsand manga in pdf/epub/cbz/cbr from supported sites
  - gallery-dl
  - Discord-Channel-Scraper
  - dzi-dl
  - FanFicFare
  - FicSave
  - flickr_download
  - Google Images Download
  - iiif-dl - line tile downloader/assembler for IIIF endpoints/manifests
  - imgbrd-grabber
  - instaloader
  - InstaLooter - less Instagram pictures and videos downloader.
  - Instagram Scraper - scraper is a command-line application written in Python that scrapes and downloads an instagram user's photos and videos. Use responsibly.
  - RedditDownloader
  - snscrape
  - RipMe
  - tumblr-utils
  - Youtube-MA
  - PyInstaLive
- Download automation
  - bazarr
  - FlexGet
  - Jackett
  - Lidarr
  - Mylar
  - Sick-Beard
  - Radarr
  - Sonarr
Compression
- Download automation
  - peazip
  - PIGZ - threaded gzip
  - WinRAR
  - KGB Archiver
Network
- Download automation
  - NetLimiter
Content sharing
- Download automation
  - opds
  - h5ai
Data curation
- Download automation
  - diskover
  - Everything
  - FileBot
  - TeraCopy
  - tree
  - WinDirStat
  - SyncToy
  - baobab
  - beets
  - Calibre
  - fucking-weeb
  - grepWin
  - jdupes
  - MediaElch
  - MediaInfo
  - phockup
  - picard
APIs & Online tools
- Download automation
  - iqdb - service reverse image search
  - thetvdb
Hardware / Monitoring
- Download automation
  - CrystalDiskInfo
  - Hard Drive Sentinel - OS SSD and HDD monitoring and analysis software
  - smartmontools
Data recovery
- Download automation
  - PhotoRec
  - TestDisk
Local Media
- Download automation
  - MakeMKV - platform DVD ripper that supports recent blu ray DVDs. It's mostly open source, but the blu ray secret sauce is still hidden.
  - Handbrake
  - whipper - DA ripper preferring accuracy over speed. Generates .flac, .cue, and .log by default and automatically fetches metadata from musicbrainz. EAC log plugin is available.
File systems
- Download automation
  - mergerfs
File conversion
- Download automation

Programming Languages

Python 29 C++ 4 C# 4 Go 4 Java 3 JavaScript 2 Ruby 2 HTML 2 Shell 2 C 2

Ecosyste.ms: Awesome

awesome-datahoarder

Utility Scripts

Download automation

Download utilities

Web Archiving

General

Application-specific

Download automation

Compression

Download automation

Network

Download automation

Content sharing

Download automation

Data curation

Download automation

APIs & Online tools

Download automation

Hardware / Monitoring

Download automation

Data recovery

Download automation

Local Media

Download automation

File systems

Download automation

File conversion

Download automation