https://github.com/yankeexe/hf-dowloader
HuggingFace Downloader using aria2c
https://github.com/yankeexe/hf-dowloader
dataset dataset-downlaoder downloader generativeai hf huggingface model-downloader models
Last synced: 9 months ago
JSON representation
HuggingFace Downloader using aria2c
- Host: GitHub
- URL: https://github.com/yankeexe/hf-dowloader
- Owner: yankeexe
- Created: 2025-09-26T19:53:55.000Z (9 months ago)
- Default Branch: main
- Last Pushed: 2025-09-29T21:45:25.000Z (9 months ago)
- Last Synced: 2025-09-29T22:13:02.096Z (9 months ago)
- Topics: dataset, dataset-downlaoder, downloader, generativeai, hf, huggingface, model-downloader, models
- Language: Shell
- Homepage:
- Size: 2.93 KB
- Stars: 0
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# 🤗 HuggingFace Downloader
Fast parallel downloader for HuggingFace repositories using aria2c.
## Usage
```bash
./hf.sh [OPTIONS]
```
## Examples
```bash
# Download a model
./hf.sh Qwen/Qwen2.5-0.5B
# Download a dataset
./hf.sh HuggingFaceFW/fineweb-2 -ds
# Download with custom settings
./hf.sh Qwen/Qwen2.5-0.5B -x 16 -s 10 -d ./models
# Interactive file selection
./hf.sh Qwen/Qwen2.5-0.5B -i
# Exclude specific file patterns
./hf.sh Qwen/Qwen2.5-0.5B --exclude "*.sh,*.md,*.yaml"
```
## Options
- `-ds` - Download as dataset (default: disabled)
- `-i` - Interactively select files to download (default: disabled)
- `-x ` - Connections per server (default: 8)
- `-s ` - Number of splits (default: 20)
- `-k ` - Min split size (default: 1M)
- `-V ` - Verify integrity (default: true)
- `-c ` - Continue downloading (default: true)
- `-d ` - Download directory (default: repo_name)
- `--console-log-level ` - Console log level (default: error)
- `--download-result ` - Download result (default: full)
- `--summary-interval ` - Summary interval (default: 10)
- `--exclude ` - Comma-separated glob patterns to exclude files (e.g., *.sh,*.md,*.yaml)
- `--token ` - HuggingFace token for private repos
- `-h, --help` - Show help
## Requirements
- aria2c
- Python with huggingface_hub
- fzf (for interactive mode)