{"id":19494746,"url":"https://github.com/ceejbot/scurry","last_synced_at":"2025-08-10T04:08:51.916Z","repository":{"id":9490651,"uuid":"11380901","full_name":"ceejbot/scurry","owner":"ceejbot","description":"consistent hash ring of leveldbs","archived":false,"fork":false,"pushed_at":"2018-06-10T18:40:02.000Z","size":183,"stargazers_count":24,"open_issues_count":1,"forks_count":3,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-07-27T20:04:20.097Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ceejbot.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2013-07-13T00:56:22.000Z","updated_at":"2024-12-21T00:23:21.000Z","dependencies_parsed_at":"2022-09-14T05:21:32.500Z","dependency_job_id":null,"html_url":"https://github.com/ceejbot/scurry","commit_stats":null,"previous_names":[],"tags_count":10,"template":false,"template_full_name":null,"purl":"pkg:github/ceejbot/scurry","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ceejbot%2Fscurry","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ceejbot%2Fscurry/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ceejbot%2Fscurry/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ceejbot%2Fscurry/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ceejbot","download_url":"https://codeload.github.com/ceejbot/scurry/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ceejbot%2Fscurry/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":269598535,"owners_count":24444597,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-09T02:00:10.424Z","response_time":111,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-10T21:32:49.192Z","updated_at":"2025-08-10T04:08:51.743Z","avatar_url":"https://github.com/ceejbot.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"scurry\n======\n\nA leveldb-backed consistent hash ring, for your caching needs. I can see the day when you might want to put data in this \u0026 feel reasonably sort of confident you might get it back out again. If this scares you, it's supposed to.\n\n[![on npm](http://img.shields.io/npm/v/scurry.svg?style=flat)](https://www.npmjs.org/package/scurry)   [![Tests](http://img.shields.io/travis/ceejbot/scurry.svg?style=flat)](http://travis-ci.org/ceejbot/scurry)  [![Coverage Status](http://img.shields.io/coveralls/ceejbot/scurry.svg?style=flat)](https://coveralls.io/r/ceejbot/scurry)    [![Dependencies](http://img.shields.io/david/ceejbot/scurry.svg?style=flat)](https://david-dm.org/ceejbot/scurry)\n\n\n## One part each\n\nRod Vagg's [levelup](https://github.com/rvagg/node-levelup) leveldb bindings for node + [sublevel](https://github.com/dominictarr/level-sublevel) to create buckets.\n\nDominic Tarr's [crtd](https://github.com/dominictarr/crdt), which uses his [scuttlebutt implementation](https://github.com/dominictarr/scuttlebutt) to keep a document in sync.\n\n[light-cycle](https://github.com/ceejbot/light-cycle), a lightweight consistent hash ring structure that can be mixed into most anything.\n\n[restify](http://mcavage.me/node-restify/) to provide a simple RESTful api to data in the buckets.\n\n## Shake with ice\n\nRun a server:\n\n`node index.js --id=node-one -m -p 3333 -g 4114 -d ./db | ./node_modules/.bin/bunyan -o short`\n\nRun a client or five:\n\n```shell\nnode index.js --id=node-two -p 3334 -g 4114 -s 10.0.0.5 -d ./db2 | ./node_modules/.bin/bunyan -o short\nnode index.js --id=node-three -p 3335 -g 4114 -s 10.0.0.5 -d ./db3 | ./node_modules/.bin/bunyan -o short\nnode index.js --id=node-four -p 3336 -g 4114 -s 10.0.0.5 -d ./db4 | ./node_modules/.bin/bunyan -o short\n```\n\nReplace `10.0.0.5` with the IP address of your server.\n\n## Strain into a chilled glass\n\nThen stuff some data in:\n\n```shell\nhttp PUT 10.0.0.5:3334/vodkas/1 name=\"Sobieski\" rating=5\nhttp PUT 10.0.0.5:3335/vodkas/2 name=\"Tito's Handmade\" rating=5\nhttp PUT 10.0.0.5:3335/vodkas/3 name=\"Bimber\" rating=4\n```\n\nGet it back out: `http GET 10.0.0.5:3336/vodkas/2 | json`\n\n(Human-friendly shell commands courtesy of [httpie](https://github.com/jkbr/httpie)).\n\n## Garnish with a twist of orange peel\n\nAPI endpoints exposed:\n\n- `GET /:bucket`: stream sorted keys for a bucket (works!)\n- `POST /:bucket`: add an item to the cache; id is generated for you \u0026 returned\n- `PUT /:bucket/:id`: add/update an item in the cache; 204 response\n- `GET /:bucket/:id`: get an item from the cache\n- `HEAD /:bucket/:id`: headers for an item\n- `DEL /:bucket/:id`: remove an item\n\n### TTLs\n\nSend this header to specify a time-to-live for your cached data: `X-Scurry-TTL: [seconds]`\n\n### Conditional requests\n\nScurry sends an ETag header and a last-modified timestamp.\n\n### Storage format\n\nAs of version 0.0.4, the data stored in the LevelDB nodes is json structured as follows:\n\n```javascript\n{\n\tversion:        1,                       // storage version\n    ts:             Date.now(),              // timestamp of last set()\n    payload:        value,                   // base64-encoded string if buffer, JSON string if not\n    etag:           crc.digest('hex'),       // md5 hex digest of payload\n    base64:         valueIsB64String,        // true if the payload had to be base64 encoded\n    'content-type': metadata['content-type'] // content-type if passed in\n};\n\n```\n\n## TODO\n\nUpcoming releases:\n\n- Release 0.0.4 will probably make streaming keys work, maybe. Done!\n- Release 0.0.5 will finalize the storage format. Probably done!\n- Release 0.0.6 will contemplate eviction. Not yet!\n\nGeneral goals:\n\n- Implement key streaming from multiple nodes. See notes in endpoints.handleGetBucket().\n- The RESTful server needs error handling.\n- Reconnect on errors.\n- Error handling.\n- Better logging. Configurable, for one thing.\n- Back ends should be pluggable; the API is very small.\n- Stretch goal: replication?\n\n## License\n\nMIT.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fceejbot%2Fscurry","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fceejbot%2Fscurry","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fceejbot%2Fscurry/lists"}