Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/enobufs/node-mget
Parallel downloader powered by curl.
https://github.com/enobufs/node-mget
Last synced: 10 days ago
JSON representation
Parallel downloader powered by curl.
- Host: GitHub
- URL: https://github.com/enobufs/node-mget
- Owner: enobufs
- Created: 2018-03-29T00:04:29.000Z (over 6 years ago)
- Default Branch: master
- Last Pushed: 2018-03-31T22:02:23.000Z (over 6 years ago)
- Last Synced: 2024-09-18T07:35:24.350Z (about 2 months ago)
- Language: JavaScript
- Size: 4.9 MB
- Stars: 0
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# mget
A parallel downloader using curl. It splits a large file into multiple of segments and download
them concurrently using HTTP Range header (multiple "get"). This likely increase download speed
particularly when the TCP connections is not able to saturate available bandwidth in the path
due to some reasons which inclueds:
* Long-fat (delay x bandwidth) pipe with limited send/recv buffers.
* Small socket send buffer at server.
* Intentional rate limitting at server.
* Congestion window shrinkage due to a large packet loss in the path.
## Installation
```shell
npm install -g mget
```
## Usage
```
Usage: mget [options]
Options:
-c, --concurrency Concurrency. (default: 0 - auto)
-o, --output Output file path.)
-h, --help output usage information
```
### Example
* Using 4 concurrent HTTP GET transactions.
* Save the file as the basename of the URL. (same as curl's -O option)
```
mget -c 4 http://ipv4.download.thinkbroadband.com/5MB.zip
```
### Results
Here's an example results of the performance increase.
Concurrency: 1
```
$ time mget -c 1 http://ipv4.download.thinkbroadband.com/5MB.zip
Progress: 100%
real 0m10.607s
user 0m0.242s
sys 0m0.122s
```
Concurrency: 4
```
$ time mget -c 4 http://ipv4.download.thinkbroadband.com/5MB.zip
Progress: 100% 100% 100% 100%
real 0m6.848s
user 0m0.272s
sys 0m0.140s
```
Concurrency: 8
```
$ time mget -c 8 http://ipv4.download.thinkbroadband.com/5MB.zip
Progress: 100% 100% 100% 100% 100% 100% 100% 100%
real 0m5.668s
user 0m0.309s
sys 0m0.170s
```
## API
You can use this module inside your application.
> TODO: more details
Please see ./bin/cli.js for your reference.
## To do
* Add retries logic.
* API reference.