{"id":20030631,"url":"https://github.com/peterding/aget-rs","last_synced_at":"2025-08-20T20:33:04.306Z","repository":{"id":80368200,"uuid":"173222915","full_name":"PeterDing/aget-rs","owner":"PeterDing","description":"Aget-rs - Fast Asynchronous Downloader with Rust 🦀","archived":false,"fork":false,"pushed_at":"2024-10-25T03:44:23.000Z","size":383,"stargazers_count":52,"open_issues_count":1,"forks_count":5,"subscribers_count":2,"default_branch":"master","last_synced_at":"2024-12-06T11:25:40.186Z","etag":null,"topics":["bittorrent","downloader","http","m3u8","m3u8-downloader","magnet","torrent"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/PeterDing.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE-APACHE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-03-01T02:43:08.000Z","updated_at":"2024-11-29T22:17:02.000Z","dependencies_parsed_at":"2024-10-25T06:33:02.664Z","dependency_job_id":"46367438-5844-4948-a667-cbbc56b877ee","html_url":"https://github.com/PeterDing/aget-rs","commit_stats":null,"previous_names":[],"tags_count":26,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PeterDing%2Faget-rs","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PeterDing%2Faget-rs/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PeterDing%2Faget-rs/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PeterDing%2Faget-rs/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/PeterDing","download_url":"https://codeload.github.com/PeterDing/aget-rs/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":230454430,"owners_count":18228392,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bittorrent","downloader","http","m3u8","m3u8-downloader","magnet","torrent"],"created_at":"2024-11-13T09:27:26.051Z","updated_at":"2024-12-19T15:09:55.306Z","avatar_url":"https://github.com/PeterDing.png","language":"Rust","readme":"\u003ch1 align=\"center\"\u003eAget-rs - Fast Asynchronous Downloader with Rust 🦀\u003c/h1\u003e\n\n[![CI](https://github.com/PeterDing/aget-rs/actions/workflows/ci.yml/badge.svg)](https://github.com/PeterDing/aget-rs/actions/workflows/ci.yml)\n\n`aget-rs` is a fast asynchronous downloader written in Rust.  \nIt requests a resource with a number of concurrent asynchronous request in a single thread.\n\nEspecially, the concurrent amount can be any positive number as your wish.\n\n`aget-rs` supports to download a **HTTP/S** link, a **M3U8** video link, a **Torrent** and a **magnet** link.\n\n## Installation\n\nYou can download the last release from https://github.com/PeterDing/aget-rs/releases\n\n## Benchmark\n\nWe conside that there is a file to download. This file has 10MB.\nThe server which hosts the file has been set a limit rate to 100KB/s, but no connecting count limit.\n\nIt will be easy to calculate the total costing time when we use 1, 10, 100 connections to request the file.\n\nIn the benchmark test, we use `nginx` to simulate the environment where a limit rate is 100KB/s for downloading.\n\nFollowing is the results of using `curl` and `aget-rs`. (For more details, you can find at [here](ci/benchmark.bash))\n\n- One connection using `curl`\n\n  ```\n  time curl http://localhost:9010/abc\n    % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current\n                                   Dload  Upload   Total   Spent    Left  Speed\n  100 10.0M  100 10.0M    0     0   100k      0  0:01:42  0:01:42 --:--:--  103k\n  real\t1m42.147s\n  user\t0m0.021s\n  sys\t0m0.035s\n  ```\n\n  **time cost: 102s**\n\n- 10 connections using `aget-rs`\n\n  ```\n  time ag http://localhost:9010/abc -s 10 -k 1m\n      File: abc\n    Length: 10.0M (10485760)\n  10.0M/10.0M 100.00% NaNG/s eta: 0s        [==================================\u003e]\n  real\t0m10.016s\n  user\t0m0.040s\n  sys\t0m0.020s\n  ```\n\n  **time cost: 10s, 10 times faster than curl**\n\n- 100 connections using `aget-rs`\n\n  ```\n  time ag http://localhost:9010/abc -s 100 -k 103k\n      File: abc\n    Length: 10.0M (10485760)\n  10.0M/10.0M 100.00% NaNG/s eta: 0s        [==================================\u003e]\n  real\t0m2.016s\n  user\t0m0.087s\n  sys\t0m0.029s\n  ```\n\n  **time cost: 2s, 50 times faster than curl**\n\n## Usage\n\n- Request a resource with default configuration\n\n  The default concurrent amount is `10` and chunk length is `1m`.\n\n  ```shell\n  ag http://cdimage.ubuntu.com/ubuntu/releases/18.10/release/ubuntu-18.10-server-amd64.iso\n  ```\n\n- Set concurrent amount and chunk length\n\n  Use `-s` or `--concurrency` to set the number of concurrent request.  \n   Use `-k` or `--chunk-size` to set the chunk length of each request.  \n   `--chunk-size` takes a literal size description, example `1k` for one Kilobyte,  \n   `2m` for two Megabyte, `1g` for Gigabyte.\n\n  ```shell\n  ag \"url of resource\" -s 20 -k 1m\n  ```\n\n- Set a path for output\n\n  Use `-o` or `--out` to set the path.  \n   If the argument is not gave, we take the last part of the url' path as the path.\n\n  ```shell\n  ag \"url of resource\" -o /path/to/file\n  ```\n\n  When download a torrent or magnet link, the path is the output directory.\n\n- Set request headers\n\n  Use `-H` to set headers.\n\n  ```shell\n  ag \"url of resource\" -H \"Cookie: key=value\" -H \"Accept: */*\"\n  ```\n\n- Set request method and data\n\n  Use `-X` or `--method` to set method for http, example, `GET`, `POST`.  \n   The default method is `GET`.  \n   With a data, using `-d` or `--data`, example, `a=b`\n\n  ```shell\n  ag \"url of resource\" -d \"a=b\"\n  ```\n\n- Download a torrent or magnet link\n\n  **Warning**: The `/path/to/outdir` directory below command must NOT exist. It will be created automatically.\n\n  ```shell\n  ag \"magnet:...\" -o /path/to/outdir\n  ag \"/path/to/torrent\" -o /path/to/outdir\n  ag \"http://example.com/some.torrent\" -o /path/to/outdir\n  ```\n\n  Use `--bt-file-regex` to only download files matching it in the torrent.\n\n  ```shell\n  ag \"magnet:...\" -o /path/to/outdir --bt-file-regex \".*\\.mp4\"\n  ```\n\n  Use `--seed` to seed the torrent after downloaded.\n\n  ```shell\n  ag \"magnet:...\" -o /path/to/outdir --seed\n  ```\n\n  Use `--bt-trackers` to specify trackers with comma as delimiter.\n\n  ```shell\n  ag \"magnet:...\" -o /path/to/outdir --bt-trackers \"udp://tracker.opentrackr.org:1337/announce,udp://opentracker.io:6969/announce\"\n  ```\n\n## Options\n\n```\nUsage: ag [OPTIONS] \u003cURL\u003e\n\nArguments:\n  \u003cURL\u003e\n\nOptions:\n  -m, --method \u003cMETHOD\u003e                Request method, e.g. GET, POST [default: GET]\n  -H, --header \u003cHEADER\u003e                Request headers, e.g. -H \"User-Agent: aget\"\n  -d, --data \u003cDATA\u003e                    Request with POST method with the data, e.g. -d \"a=b\"\n      --insecure                       Skip to verify the server's TLS certificate\n  -s, --concurrency \u003cCONCURRENCY\u003e      The number of concurrency request [default: 10]\n  -k, --chunk-size \u003cCHUNK_SIZE\u003e        The number ofinterval length of each concurrent request [default: '50m']\n  -t, --timeout \u003cTIMEOUT\u003e              Timeout(seconds) of request [default: 60]\n      --dns-timeout \u003cDNS_TIMEOUT\u003e      DNS Timeout(seconds) of request [default: 10]\n      --retries \u003cRETRIES\u003e              The maximum times of retring [default: 5]\n      --retry-wait \u003cRETRY_WAIT\u003e        The seconds between retries [default: 0]\n      --proxy \u003cPROXY\u003e                  [protocol://]host[:port] Use this proxy\n      --type \u003cTYPE\u003e                    Task type, auto/http/m3u8/bt [default: auto]\n      --bt-file-regex \u003cBT_FILE_REGEX\u003e  A regex to only download files matching it in the torrent\n      --seed                           Seed the torrent\n      --bt-trackers \u003cBT_TRACKERS\u003e      Trackers for the torrent, e.g. --bt-trackers \"udp://tracker.opentrackr.org:1337/announce\n                                       ,udp://opentracker.io:6969/announce\"\n      --bt-peer-connect-timeout \u003cBT_PEER_CONNECT_TIMEOUT\u003e\n          Peer connect timeout in seconds. [default: 10]\n      --bt-peer-read-write-timeout \u003cBT_PEER_READ_WRITE_TIMEOUT\u003e\n          Peer read/write timeout in seconds. [default: 10]\n      --bt-peer-keep-alive-interval \u003cBT_PEER_KEEP_ALIVE_INTERVAL\u003e\n          Peer keep-alive interval in seconds. [default: 120]\n      --debug                          Debug output. Print all trackback for debugging\n      --quiet                          Quiet mode. Don't show progress bar and task information. But still show the error information\n  -o, --out \u003cOUT\u003e                      The path of output for the request e.g. -o \"/path/to/file\"\n  -h, --help                           Print help\n  -V, --version                        Print version\n```\n\n## Configuration\n\nAget can be configured by a configuration file. The file locates at `~/.config/aget/config`.\nFollowing options can be set. Aget uses these options as the defaults for each command.\n\n```\nheaders = [[\"key\", \"value\"], ...]\nconcurrency = ...\nchunk_size = \"...\"\ntimeout = ...\ndns_timeout = ...\nretries = ...\nretry_wait = ...\n```\n\nIf the file does not exist, aget will use the default configuration.\n\n```toml\nheaders = [[\"user-agent\", \"aget/version\"]]\nconcurrency = 10\nchunk_size = \"50m\"\ntimeout = 60\ndns_timeout = 10\nretries = 5\nretry_wait = 0\n```\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpeterding%2Faget-rs","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpeterding%2Faget-rs","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpeterding%2Faget-rs/lists"}