Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/emre/redis-router
A redis sharding library/api for your sharding needs.
https://github.com/emre/redis-router
Last synced: 3 months ago
JSON representation
A redis sharding library/api for your sharding needs.
- Host: GitHub
- URL: https://github.com/emre/redis-router
- Owner: emre
- Created: 2013-04-04T19:03:59.000Z (over 11 years ago)
- Default Branch: master
- Last Pushed: 2021-04-20T17:07:57.000Z (over 3 years ago)
- Last Synced: 2024-06-05T09:23:52.798Z (5 months ago)
- Language: Python
- Homepage: http://emre.github.io/redis-router/
- Size: 170 KB
- Stars: 173
- Watchers: 9
- Forks: 2
- Open Issues: 2
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
redis-router
============
redis_router, a redis sharding library/api for your redis sharding needs.
how it works
==============
> Consistent hashing is a special kind of hashing. When a hash table is resized and consistent hashing is used,
> only K/n keys need to be remapped on average, where K is the number of keys, and n is the number of slots.
> In contrast, in most traditional hash tables, a change in the number of array slots causes
> nearly all keys to be remapped.
redis_router uses last.fm's
libketama in the back.
installation
==========
install libketama/ketama_python first.
After that;
```
pip install redis-router
```
or if you like 90s:
```
easy_install redis-router
```
or add redis_router directory to the your path.
quick start
============
servers.txt (server:ip weight)
```
127.0.0.1:6379 100
127.0.0.1:6380 100
```
your python code:
``` python
router = Router("servers.txt")
router.set("forge", 13)
router.set("spawning_pool", 18)
```
output with loglevel=DEBUG
```
DEBUG:key 'forge' hashed as 4113771093 and mapped to 127.0.0.1:6379
DEBUG:key 'spawning_pool' hashed as 1434709819 and mapped to 127.0.0.1:6380
DEBUG:key 'forge' hashed as 4113771093 and mapped to 127.0.0.1:6379
DEBUG:key 'spawning_pool' hashed as 1434709819 and mapped to 127.0.0.1:6380
13 6
```
redis_router as a server
========================================
If you have clients using X programming language other than python, you can use HTTP or TCP interface to connect
and send commands to redis_router.
running TCP interface
=======================
``` python
from redis_router.tcp_interface import RouterServer
r = RouterServer('0.0.0.0', 5000)
r.run()
```
playing with it
```
$ telnet localhost 5000
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
set selam timu
True
get selam
timu
dbsize
13
```
HTTP API
=============
``` python
from redis_router.http_interface import start_server
start_server('0.0.0.0', 5000)
```
example request:
* initialize a set with two members.
``` bash
$ curl -X POST --data "command=sadd&arguments=teams,galatasaray,fenerbahce" http://localhost:5000
```
``` json
{
"response": 2
}
```
* get members
``` bash
$ curl -X POST --data "command=smembers&arguments=teams" http://localhost:5000
```
``` json
{
"response": [
"fenerbahce",
"galatasaray"
]
}
```
running tests
=================
``` bash
$ py.test tests.py
=============================================== test session starts =========================
platform linux2 -- Python 2.7.3 -- pytest-2.3.4
collected 11 items
tests.py ...........
============================================ 11 passed in 0.33 seconds ======================
```
FAQ
=========
> Q: What about data invalidation if I move servers, change the config etc.
It's minimum. At least better than:
```
Node = Hash(key) MOD N
```
> Q: I want to see some stats about sharding efficiency.
Results for 100.000 random keys.
```
results: {
redis.client.Redis object at 0x8df75a4: 33558,
redis.client.Redis object at 0x8df7644: 31207,
redis.client.Redis object at 0x8df7504: 35235
}
```
> Q: Can I use this with PHP or [INSERT RANDOM LANGUAGE HERE]
Yes.
There are TCP server
and HTTP Server options.
You can always use libketama's implementations in your language though.
[](https://bitdeli.com/free "Bitdeli Badge")