Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/boramalper/pydis
A redis clone in Python 3 to disprove some falsehoods about performance.
https://github.com/boramalper/pydis
clone experiment optimization performance redis
Last synced: about 1 month ago
JSON representation
A redis clone in Python 3 to disprove some falsehoods about performance.
- Host: GitHub
- URL: https://github.com/boramalper/pydis
- Owner: boramalper
- License: isc
- Archived: true
- Created: 2019-02-27T17:55:03.000Z (almost 6 years ago)
- Default Branch: master
- Last Pushed: 2020-11-15T18:03:06.000Z (about 4 years ago)
- Last Synced: 2024-07-05T15:11:34.216Z (5 months ago)
- Topics: clone, experiment, optimization, performance, redis
- Language: Python
- Homepage:
- Size: 40 KB
- Stars: 635
- Watchers: 21
- Forks: 47
- Open Issues: 6
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# pydis
**pydis** is an experiment to disprove some of the falsehoods about performance
and optimisation regarding software and interpreted languages in particular.Below you will find a [Redis](https://github.com/antirez/redis/) clone,
**pydis**, written *in ~250 lines of idiomatic Python code*, providing a
subset of redis' functionality for which there are
[official benchmarks](https://redis.io/topics/benchmarks).Briefly, **pydis** is ~50% as fast as Redis measured in number operations per
second.P.S. This is not a criticism of Redis, which is a brilliant project and a
system-level software that powers thousands of infrastructures. It just happened
to be one of the fastest software I could imagine *and* clone the same day.## Disclaimer
I have used the following libraries written in C for performance:- [uvloop](https://github.com/MagicStack/uvloop)
> uvloop is a fast, drop-in replacement of the built-in asyncio event loop. uvloop is implemented in Cython and uses libuv under the hood.
>
- [hiredis](https://pypi.org/project/hiredis/)> Python extension that wraps protocol parsing code in hiredis.
## Discussion
- Hacker News
1. 2020-11-15 -- [Pydis – Redis clone in 250 lines of Python, for performance comparison ](https://news.ycombinator.com/item?id=25100218)
1. 2019-03-02 -- [Pydis: Redis clone in Python 3 to make points about performance](https://news.ycombinator.com/item?id=19287717)
1. 2019-03-01 -- [pydis - A redis clone in Python 3 to disprove some falsehoods about performance](https://www.reddit.com/r/Python/comments/awav6k/pydis_a_redis_clone_in_python_3_to_disprove_some/)
## Results
```bash
redis-benchmark -q -t set,get,incr,lpush,rpush,lpop,rpop,sadd,hset,spop,lrange,mset -n 100000 -P 5
```- 100,000 requests in total per command.
- Requests are pipelined in groups of 5.
Benchmark | pydis | redis | Ratio
--- | ---: | ---: | ---
SET | 271,947 | 467,361 | 0.582
GET | 274,283 | 467,237 | 0.587
INCR | 213,409 | 478,669 | 0.446
LPUSH | 216,082 | 381,033 | 0.567
RPUSH | 231,143 | 399,238 | 0.579
LPOP | 248,527 | 384,332 | 0.647
RPOP | 241,144 | 429,971 | 0.561
SADD | 219,475 | 434,257 | 0.505
HSET | 220,178 | 377,637 | 0.583
SPOP | 288,068 | 477,705 | 0.603
LRANGE (100) | 26,170 | 96,254 | 0.272
LRANGE (300) | 9,163 | 24,768 | 0.370
LRANGE (500) | 5,771 | 19,351 | 0.298
LRANGE (600) | 4,705 | 13,869 | 0.339
MSET | 125,215 | 195,121 | 0.642### Host System
- Ubuntu 20.04
- Python 3.8.5 (GCC 9.3.0)
- Redis 5.0.7 `malloc=jemalloc-5.2.1 bits=64 build=636cde3b5c7a3923`## Contributions
Contributions are very welcome, given that they fall into one of the following
categories:- Those that improve the performance.
- The aim of this exercise is to prove that interpreted languages can be just
as fast as C. So whilst using a faster parser in C with Python bindings is
okay, rewriting **pydis** in [Cython](https://cython.org/) is not.
- I will accept "minor" deviations from idioms only if the performance gains
are worth it; stick to idiomatic Python otherwise!
- Those to achieve feature parity with Redis *for which there are official
benchmarks*.
- We are not trying to develop a full-featured Redis clone here so please do
not implement commands for which there are no official benchmarks.
- Those that fix formatting etc.
- Please do not invent your own style, use PEP 8.