Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/simonw/dogproxy
Experimental HTTP proxy (using node.js) for avoiding the dog pile effect.
https://github.com/simonw/dogproxy
Last synced: 22 days ago
JSON representation
Experimental HTTP proxy (using node.js) for avoiding the dog pile effect.
- Host: GitHub
- URL: https://github.com/simonw/dogproxy
- Owner: simonw
- Created: 2010-02-03T11:43:38.000Z (almost 15 years ago)
- Default Branch: master
- Last Pushed: 2019-02-15T16:01:42.000Z (over 5 years ago)
- Last Synced: 2024-10-07T13:52:24.723Z (about 1 month ago)
- Language: JavaScript
- Homepage:
- Size: 3.91 KB
- Stars: 61
- Watchers: 3
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: readme.md
Awesome Lists containing this project
README
# dogproxy
A very simple dog pile preventing proxy.
The dog pile effect (also known as a thundering herd) affects caching systems
that use time-based expiry. In a heavy traffic system, a cached item expiring
could cause dozens of parallel "update" requests to be fired at once.
dogproxy assumes that these update requests go over HTTP. It joins together
simultaneous requests for the same URL, and only performs one actual fetch to
that resource.
STATUS: highly experimental. Known bugs (Content-Type is not correctly passed
through, for example). Expect to modify this code further before use.
Usage:
$ node dogproxy.js
Server running at http://127.0.0.1:8009/
Testing using ab:
$ ab -n 10 -c 5 'http://127.0.0.1:8009/?url=http://example.com/'
dogproxy.js says:
```
Request for http://example.com/
... kicking off backend HTTP fetch
Request for http://example.com/
... already in flight, adding to queue
Request for http://example.com/
... already in flight, adding to queue
Request for http://example.com/
... already in flight, adding to queue
Request for http://example.com/
... already in flight, adding to queue
Fetched http://example.com/, alerting 5 waiting clients
Request for http://example.com/
... kicking off backend HTTP fetch
Request for http://example.com/
... already in flight, adding to queue
Request for http://example.com/
... already in flight, adding to queue
Request for http://example.com/
... already in flight, adding to queue
Request for http://example.com/
... already in flight, adding to queue
Fetched http://example.com/, alerting 5 waiting clients
```