Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/aspadax/rustyface

Download Huggingface repositories without the need to install dependencies
https://github.com/aspadax/rustyface

ai cli dataset datasets downloader huggingface model network network-programming rust

Last synced: about 20 hours ago
JSON representation

Download Huggingface repositories without the need to install dependencies

Host: GitHub
URL: https://github.com/aspadax/rustyface
Owner: AspadaX
License: mit
Created: 2024-08-03T12:46:59.000Z (4 months ago)
Default Branch: main
Last Pushed: 2024-08-27T01:34:00.000Z (3 months ago)
Last Synced: 2024-10-31T12:15:32.564Z (20 days ago)
Topics: ai, cli, dataset, datasets, downloader, huggingface, model, network, network-programming, rust
Language: Rust
Homepage: https://github.com/AspadaX/RustyFace
Size: 260 KB
Stars: 16
Watchers: 1
Forks: 1
Open Issues: 1
Metadata Files:
- Readme: README.md
- License: LICENSE

Awesome Lists containing this project

README

# RustyFace
A command line app for downloading Huggingface repositories with Rust.

# Why using this?
RustyFace does not require installing additional dependencies such as `git` or `git lfs` etc. It aims to be lightweight and portable.
In addition to that, RustyFace is friendly to users who live in Mainland China, where HuggingFace accessibility is unstable, as this CLI app adopted a mirror that can be accessed globally.

The mirror site used in this project is `hf-mirror.com`

# How to Install and Use RustyFace
First, you need to have Rust installed. For those new to Rust, please refer to the [official installation guide](https://doc.rust-lang.org/cargo/getting-started/installation.html).

## Quickstart - Without Installation
You don't need to install Rust if you download the corresponding binaries to your platform from the Release section. That way, you can just type this command to download Huggingface repositories:
```
rustyface_windows_x86 --repository sentence-transformers/all-MiniLM-L6-v2 --tasks 4
```
- `rustyface_windows_x86` is the binary file name that you have downloaded from the Release section.
- `--repository` is followed by the `repo_id` of the repository that you want to download from HuggingFace.
- `--tasks` is followed by the number of concurrent downloads. For example, 4 means downloading 4 files at once. It is recommended to use a lower number if your network conditions do not support higher concurrency.

## Quickstart - With Installtion
If you would like to reuse the program, it is recommended to install RustyFace onto your system rather than using the binaries. Here is how you can do it.

### Install Rust
On Linux and macOS:
```
curl https://sh.rustup.rs -sSf | sh
```
On Windows, you can download the installation executable via this link: https://win.rustup.rs/

### Install RustyFace
After done installing Rust, just type this to your terminal:
```
cargo install rustyface
```

### Use RustyFace to Download Repositories
Try RustyFace out with this simple command line:
```
rustyface --repository sentence-transformers/all-MiniLM-L6-v2 --tasks 4
```
- `--repository` is followed by the `repo_id` of the repository that you want to download from HuggingFace.
- `--tasks` is followed by the number of concurrent downloads. For example, 4 means downloading 4 files at once. It is recommended to use a lower number if your network conditions do not support higher concurrency.

# Feedback & Further Development
Any participation is appreciated! Feel free to submit an issue, discussion or pull request. You can find me on WeChat: `baoxinyu2007` or Discord: `https://discord.gg/UYfZeuPy`

# License
This project is licensed under the MIT License. See the LICENSE file for details.

## Packages Used
- [clap](https://crates.io/crates/clap) for command line argument parsing.
- [futures-util](https://crates.io/crates/futures-util) for asynchronous operations.
- [indicatif](https://crates.io/crates/indicatif) for progress bars.
- [log](https://crates.io/crates/log) for logging.
- [reqwest](https://crates.io/crates/reqwest) for HTTP requests.
- [sha2](https://crates.io/crates/sha2) for SHA-256 hashing.
- [tokio](https://crates.io/crates/tokio) for asynchronous runtime.
- [fern](https://crates.io/crates/fern) for logging configuration.
- [chrono](https://crates.io/crates/chrono) for date and time handling.