Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/xan105/node-request
HTTP request library based around Node.js' HTTP(S) API interfaces
https://github.com/xan105/node-request
client download http https json nodejs progress promise request torrent url xml
Last synced: about 2 months ago
JSON representation
HTTP request library based around Node.js' HTTP(S) API interfaces
- Host: GitHub
- URL: https://github.com/xan105/node-request
- Owner: xan105
- License: mit
- Created: 2019-08-16T20:00:55.000Z (over 5 years ago)
- Default Branch: master
- Last Pushed: 2023-05-12T01:37:22.000Z (over 1 year ago)
- Last Synced: 2024-10-16T09:05:02.200Z (3 months ago)
- Topics: client, download, http, https, json, nodejs, progress, promise, request, torrent, url, xml
- Language: JavaScript
- Homepage:
- Size: 307 KB
- Stars: 0
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Funding: .github/FUNDING.yml
- License: LICENSE
Awesome Lists containing this project
README
About
=====
HTTP request library based around Node.js' HTTP(S) API interfaces:
- http/https
- ~~http2~~¹
- ~~undici/fetch (_included in Node.js 18_)¹~~
¹ Work in progress
Provides common features such as retry on error, following redirects, progress when downloading file, ...
This library isn't intented to compete nor replace the well known libraries such as got, axios, node-fetch, ...
This is merely educational and for informational purposes in order to learn how HTTP requests work under the hood.
This was originally created as [request-zero](https://www.npmjs.com/package/request-zero) at a time were the module `request` was the main choice and I didn't quite like it.
It had a ton of dependencies, didn't use promises and I needed something very simple.
Example
=======
Simplest call
```js
import { request } from "@xan105/request";
const res = await request("https://www.google.com");
console.log(res.body);
```
JSON
```js
import { getJSON } from "@xan105/request";
const json = await getJSON("https://jsonplaceholder.typicode.com/todos/1");
console.log(json);
/*Output:
{ userId: 1, id: 1, title: 'delectus aut autem', completed: false }
*/
//Github API
const json = await getJSON("https://api.github.com/repos/user/repo/releases/latest",{
headers: {"Accept" : "application/vnd.github.v3+json"}
});
console.log(json);
/*Output:
{ url: '...', tag_name: '0.0.0', target_commitish: 'master', ... }
*/
```
Download file(s)
```js
import { download, downloadAll } from "@xan105/request";
//Callback example to output progress in the console
function printProgress(percent, speed, file){
process.stdout.clearLine();
process.stdout.cursorTo(0);
process.stdout.write(`${percent}% @ ${speed} kb/s [${file}]`);
}
//Simple download to disk (pipe to stream)
await download(
"http://ipv4.download.thinkbroadband.com/1GB.zip",
"D:/Downloads",
printProgress
);
//Download from github ... aws redirection ... content disposition ... but custom filename
const res = await download(
"https://github.com/user/repo/releases/download/0.0.0/Setup.exe",
"D:/Downloads/",
{ filename: "supersetup.exe" },
printProgress
);
console.log(res);
/*Output:
{ status: 200, message: 'OK', headers: {...}, path: 'D:\\Downloads\\supersetup.exe' }
*/
//Download a list of files one by one
await request.download.all([
"http://ipv4.download.thinkbroadband.com/5MB.zip",
"http://ipv4.download.thinkbroadband.com/10MB.zip",
"http://ipv4.download.thinkbroadband.com/20MB.zip",
"http://ipv4.download.thinkbroadband.com/50MB.zip"],
"D:\\Downloads", printProgress);
```
Download a torrent
```js
import { download } from "@xan105/request/torrent";
download("https://webtorrent.io/torrents/sintel.torrent", "D:\\Downloads");
```
Misc
```js
import * as h1 from "@xan105/request";
//Head request
const res = await h1.head(`http://ipv4.download.thinkbroadband.com/1GB.zip`);
console.log(res);
/*Output:
{ status: 200, message: 'OK', headers: {...} }
*/
//Manually specify retry on error and redirection to follow
await request("https://steamdb.info/app/220/", { maxRetry: 2, maxRedirect: 2 });
//Upload a single file multipart/form-data
const res = await h1.upload(
"http://127.0.0.1/upload/test/",
"Hello world",
{name: "file", filename: "hello world.txt"}
);
console.log(res);
/*Output:
{ status: 200, message: 'OK', headers: {...}, body: 'ok' }
*/
```
Installation
============
```
npm install @xan105/request
```
## Optional packages
- [webtorrent](https://www.npmjs.com/package/webtorrent)
Downloading torrent
```
npm i webtorrent
```
- [xml2js](https://www.npmjs.com/package/xml2js)
XML parser
```
npm i xml2js
```
API
===
⚠️ This module is only available as an ECMAScript module (ESM) starting with version 1.0.0.
Previous version(s) are CommonJS (CJS) with an ESM wrapper.
💡 The underlying API used is determined by which namespace you import.
By default this is the http/https (h1) API.
Torrent related are under the `torrent` namespace.
```js
//Default
import * as h1 from '@xan105/request';
//http/https (h1)
import * as h1 from '@xan105/request/h1';
//http2 (h2)¹
import * as h2 from '@xan105/request/h2';
//Fetch¹
import * as fetch from '@xan105/request/fetch';
//Torrent
import * as torrent from "@xan105/request/torrent";
```
¹ Work in progress (unavailable at the moment)
## Named export
### `request(href: string, payload?: any, option?: object): Promise`
This is the core request function every other functions are helper based on this one (_except download, downloadAll and torrent_).
The response object tries to be similar whether the request failed or succeeded.
```ts
{
code: string, //HTTP or Node error code
message: string, //HTTP or Node error message (if any)
trace: string[], //URL(s) of the request (redirection)
domain: string, //url domain
sent: object, //Header sent
address?: string, //IP address
family?: string, //IPv4 or IPv6
protocol?: string, //HTTP protocol (h1, h2, ...)
security?: string, //TLS (HTTPS)
port: number, //Network port
headers?: object, //Response header
body?: string //Response body
}
```
💡 In a dual stack network, IPv4 isn't prefered over IPv6 unlike Node's default behavior (_Node < 17_ ).
💡 When making a `HEAD` request:
- The promise always resolves no matter the HTTP response code.
- **Doesn't** follow redirection **by design**.
If you need to follow the redirection you can use the headers `location` from the response and make a new `HEAD` request.
#### ⚙️ Options
| option | type | default | description |
| ----------- | ----------- | ---------------------------------- | ---------------------------------------------------------------------------------- |
| method | string | GET | HTTP method: get, post, head, etc |
| encoding | string | utf8 | Response encoding |
| timeout | number | 3000 (ms) | Time before aborting request |
| maxRedirect | number | 3 | How many redirections to follow before aborting.
Use 0 to not follow redirects |
| maxRetry | number | 0 | How many retries on error before aborting.
Use 0 to not retry at all |
| retryDelay | number | 200 (ms) | How long to wait before a retry.
Use 0 to instantly retry |
| headers | object | -> Chrome UA and UA Hint if https | Headers of your request |
| signal | AbortSignal | none | Abort signal |
### `get(url: string, option?: object): Promise`
Force the `GET` method. Since `request()` default to 'GET' you could just use `request()` directly. This is here for completeness.
### `head(url: string, option?: object): Promise`
Force the `HEAD` method.
### `getJSON(url: string, option?: object): Promise`
Parse the response body as a JSON string and return the result.
Force method to `GET` and the header `Accept` to `"application/json"`.
- alias: `getJson()`
### `postJSON(url: string, obj: object, option?: object): Promise`
Send given object payload as a JSON encoded string.
Parse the response body as a JSON string and return the result.
Force method to `POST` and the headers `Accept` and `Content-Type` to `"application/json"`.
### `getXML(url: string, option?: object): Promise`
⚠️ Requires the [xml2js](https://www.npmjs.com/package/xml2js) module.
Parse the response body as a XML string and return the result.
Force method to `GET` and the header `Accept` to `"application/xml"`.
- alias: `getXml()`
### `post(url: string, payload: unknown, option?: object): Promise`
Force method to `POST` and write/push given payload.
NB: On HTTP 301, 302, 303 redirection the method will be [changed to GET](https://developer.mozilla.org/en-US/docs/Web/HTTP/Redirections)
### `upload(url: string, payload: unknown, option?: object): Promise`
Force method to `POST` and write/push a multipart/form-data payload.
You can use option `{fieldname: string, filename: string}` to specify the form field name and the file name.
If you don't they will default respectively to 'file' and Date.now().
### `download(href: string, destDir: string, option?: object, callbackProgress?: fn): Promise`
Download file to `destDir`.
The response object is like `request()` minus `body` and with the addition of a `file` object:
```ts
{
name: string, //filename
path: string, //relative
fullPath: string //absolute
}
```
This is useful for promise chaining to example unzip an archive, etc.
💡 Progress gives you the following stats: percent, speed, file.
`callbackProgress(percent: number, speed: number, file: string)`
#### ⚙️ Options
| option | type | default | description |
| ----------- | ----------- | ---------------------------------- | ---------------------------------------------------------------------------------- |
| timeout | number | 3000 (ms) | Time before aborting request |
| maxRedirect | number | 3 | How many redirections to follow before aborting.
Use 0 to not follow redirects |
| maxRetry | number | 3 | How many retries on error before aborting.
Use 0 to not retry at all |
| retryDelay | number | 1000 (ms) | How long to wait before a retry.
Use 0 to instantly retry |
| headers | object | -> Chrome UA and UA Hint if https | Headers of your request |
| signal | AbortSignal | none | Abort signal |
| filename | string | null | Use this if you want to specify the filename (force rename) |
| hash | object | null | Verify checksum of downloaded file² |
²Checksum option
```ts
{
algo: string, //A Node.js supported crypto algo. eg: "sha1"
sum: string //Checksum
}
```
On error or mismatch it will trigger error/retry.
### `downloadAll(href: string[], destDir: string|string[], option?: object, callbackProgress?: fn): Promise`
Download all the files in the list one-by-one to destDir.
If `destDir` is an array, files[i] will be written to destDir[i] in a 1:1 relation.
In the same fashion you can force the filename of the files with option `{filename: [..,..,..]}`.
And again same thing for checksum: `{hash: [{algo: ..., sum: ...},..,..]}`.
Returns an array of `download()` response object.
## Torrent
### `download(torrent: string, dest: string, option?: object, callbackProgress?: fn): Promise`
⚠️ Requires the [webtorrent](https://www.npmjs.com/package/webtorrent) module.
Download files from a torrent url, torrent file, torrent magnet to `destDir`.
💡 Progress gives you the following stats: percent, speed.
`callbackProgress(percent: number, speed: number)`
💡 Torrent can be resumed.
Returns an objectect with torrent download location, torrent name, and for every files of the torrent its name, relative path and path.
```ts
{
path: string, //absolute
name: string, //torrent name
file: [
{
name: string, //filename
path: string, //relative
fullPath: string //absolute
}
]
}
```
#### ⚙️ Options
| option | type | default | description |
| ------------- | ----------| ------------ | ------------------------------------------ |
| timeout | number | 10 (sec) | Time to wait for peers before aborting |
| exclusion | string[] | none | Exclude files inside the torrent |
| downloadLimit | number | -1 (none) | Download speed limit |
| uploadLimit | number | 100 (kb/s) | Upload speed limit |