https://github.com/mcollina/http-deduplicate

deduplicate multiple http request to a single endpoint
https://github.com/mcollina/http-deduplicate

Last synced: 12 days ago
JSON representation

deduplicate multiple http request to a single endpoint

• Host: GitHub
• URL: https://github.com/mcollina/http-deduplicate
• Owner: mcollina
• License: mit
• Created: 2016-02-04T13:39:45.000Z (almost 9 years ago)
• Default Branch: master
• Last Pushed: 2016-02-06T11:57:46.000Z (almost 9 years ago)
• Last Synced: 2024-12-25T10:21:34.950Z (15 days ago)
• Language: JavaScript
• Size: 12.7 KB
• Stars: 5
• Watchers: 3
• Forks: 1
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# http-deduplicate

[![build status](https://secure.travis-ci.org/mcollina/http-deduplicate.svg)](http://travis-ci.org/mcollina/http-deduplicate)

Deduplicate multiple http request to a single endpoint.

You have an application composed of multiple HTTP servers, and you need

to issue HTTP requests from your frontend to other internal endpoints.

So, you call your endpoint for all requests. This module allows you to

reduce the traffic and response time by calling a given URL only once at

any given interval.

In my tests, http-deduplicate increases the throughput by a 2-5x factor, depending on

the application.

## Install

```

npm install http-deduplicate -g

```

## Example

```js

deduplicate(url, function (err, data, cb) {

  // do something async

  cb()

  // cb is optional and can be omitted

})

```

See [example.js](./example.js) for a full example

## API

### deduplicate(address, callback(err, data [, cb]))

Calls `address` (as defined in[http](https://nodejs.org/api/http.html#http_http_get_options_callback) and

[https](https://nodejs.org/api/https.html#https_https_get_options_callback) only once in a given period, all the other requests will receive the same data.

For extreme optimizations in high load scenarios, you can serve the

latest returned value up until all `callback` (if present) are

called.

## TODO

* [ ] HTTP headers

* [ ] HTTP caching (ETag)

* [ ] HTTP caching (Last-Modified)

## Acknowledgements

http-deduplicate is sponsored by [nearForm](http://nearform.com).

## License

MIT