https://github.com/topaz1008/urls-downloader

A generic URL(s) downloader script
https://github.com/topaz1008/urls-downloader

Last synced: 3 months ago
JSON representation

A generic URL(s) downloader script

• Host: GitHub
• URL: https://github.com/topaz1008/urls-downloader
• Owner: topaz1008
• License: mit
• Created: 2023-03-08T18:53:11.000Z (almost 2 years ago)
• Default Branch: master
• Last Pushed: 2023-03-10T12:26:31.000Z (almost 2 years ago)
• Last Synced: 2024-10-10T01:27:35.452Z (3 months ago)
• Language: JavaScript
• Size: 13.7 KB
• Stars: 1
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# urls-downloader

A simple generic URL(s) downloader script using streams. Using [got](https://github.com/sindresorhus/got) under the hood.

## Installation

Download or clone this repository, then run the following command in the directory where you extracted or cloned the repository.

```

npm install

```

## Usage

Edit the `urls.txt` file and add the URLs you want to download and then run `main.js` with node. (i.e. `node main.js`)

The format for `urls.txt` is in the pattern of one URL per line, optionally the url can be followed a pipe `|` and the filename you want to save the file as.

if no pipe is specified then the filename will be the last part of the url.

e.g. `https://www.domain.com/path/to/filename.txt` will be saved as `filename.txt` in the current working directory,

while `https://www.domain.com/path/to/filename.txt|foo.txt` will be saved as `foo.txt` in the current working directory.

## Options

This script is using [got](https://github.com/sindresorhus/got) for downloading the files, so you can use any of the options that are supported by got.

Copy `options.template.json5` to `options.json5` and set/add the options you want to use with got.

Since JSON5 is used, you can add comments to the file and conform to any valid JSON5 rules.

# The `urls.txt` file

Any lines which do not start with `https` will be ignored. so comments can be included with any or no prefix.