{"id":15399913,"url":"https://github.com/mcollina/http-deduplicate","last_synced_at":"2025-04-16T02:20:31.564Z","repository":{"id":57268198,"uuid":"51078273","full_name":"mcollina/http-deduplicate","owner":"mcollina","description":"deduplicate multiple http request to a single endpoint","archived":false,"fork":false,"pushed_at":"2016-02-06T11:57:46.000Z","size":13,"stargazers_count":5,"open_issues_count":0,"forks_count":1,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-03-29T04:05:15.332Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mcollina.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2016-02-04T13:39:45.000Z","updated_at":"2019-07-11T16:33:41.000Z","dependencies_parsed_at":"2022-09-02T02:50:10.816Z","dependency_job_id":null,"html_url":"https://github.com/mcollina/http-deduplicate","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mcollina%2Fhttp-deduplicate","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mcollina%2Fhttp-deduplicate/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mcollina%2Fhttp-deduplicate/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mcollina%2Fhttp-deduplicate/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mcollina","download_url":"https://codeload.github.com/mcollina/http-deduplicate/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249183696,"owners_count":21226242,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-01T15:51:05.576Z","updated_at":"2025-04-16T02:20:31.535Z","avatar_url":"https://github.com/mcollina.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# http-deduplicate\n\n[![build status](https://secure.travis-ci.org/mcollina/http-deduplicate.svg)](http://travis-ci.org/mcollina/http-deduplicate)\n\nDeduplicate multiple http request to a single endpoint.\n\nYou have an application composed of multiple HTTP servers, and you need\nto issue HTTP requests from your frontend to other internal endpoints.\nSo, you call your endpoint for all requests. This module allows you to\nreduce the traffic and response time by calling a given URL only once at\nany given interval.\n\nIn my tests, http-deduplicate increases the throughput by a 2-5x factor, depending on\nthe application.\n\n## Install\n\n```\nnpm install http-deduplicate -g\n```\n\n## Example\n\n```js\ndeduplicate(url, function (err, data, cb) {\n  // do something async\n  cb()\n  // cb is optional and can be omitted\n})\n```\n\nSee [example.js](./example.js) for a full example\n\n## API\n\n### deduplicate(address, callback(err, data [, cb]))\n\nCalls `address` (as defined in[http](https://nodejs.org/api/http.html#http_http_get_options_callback) and\n[https](https://nodejs.org/api/https.html#https_https_get_options_callback) only once in a given period, all the other requests will receive the same data.\n\nFor extreme optimizations in high load scenarios, you can serve the\nlatest returned value up until all `callback` (if present) are\ncalled.\n\n## TODO\n\n* [ ] HTTP headers\n* [ ] HTTP caching (ETag)\n* [ ] HTTP caching (Last-Modified)\n\n## Acknowledgements\n\nhttp-deduplicate is sponsored by [nearForm](http://nearform.com).\n\n## License\n\nMIT\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmcollina%2Fhttp-deduplicate","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmcollina%2Fhttp-deduplicate","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmcollina%2Fhttp-deduplicate/lists"}