https://github.com/redislabs/redis_info_provider
A Python framework for querying, caching, and serving to consumers info for an arbitrary number of redis-server instances.
https://github.com/redislabs/redis_info_provider
Last synced: about 1 year ago
JSON representation
A Python framework for querying, caching, and serving to consumers info for an arbitrary number of redis-server instances.
- Host: GitHub
- URL: https://github.com/redislabs/redis_info_provider
- Owner: RedisLabs
- License: gpl-3.0
- Created: 2018-05-15T09:37:48.000Z (about 8 years ago)
- Default Branch: master
- Last Pushed: 2024-05-22T05:56:12.000Z (about 2 years ago)
- Last Synced: 2025-04-06T17:01:32.476Z (about 1 year ago)
- Language: Python
- Homepage:
- Size: 66.4 KB
- Stars: 0
- Watchers: 12
- Forks: 4
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Redis Info Provider
A Python framework for querying, caching, and serving to consumers info for an arbitrary number of redis-server
instances (result of executing `INFO all` command).
## Description
Got a bunch of Redis instances you want clients to be able to query `INFO` for, but don't want them running commands
directly on your instances? This is the module for you!
Feature highlights:
* Concurrent, single-threaded architecture leveraging [gevents](http://www.gevent.org).
* Dynamically-adjusted per-instance polling intervals based on instance activity level. (Timing range is user-tweakable.)
* Flexible API for info querying.
* Compatible with both Python 2 and Python 3.
## Getting Started
### Installing
> This module supports Python 2/3 and has been tested with Python >= 2.7.14 and Python >= 3.6.4.
Download or clone the library source, `cd` into the download location, and run:
```
python setup.py install
```
or:
```
pip install .
```
### Running the Tests
```
python setup.py test
```
### Packaging Source Distribution
```
python setup.py sdist
```
## Usage Examples
The framework comes with sample client/server applications to demonstrate its use. They use Python's included
[xmlrpclib](https://docs.python.org/2/library/xmlrpclib.html) module and can be found in the `examples` directory:
```
examples/
info_client.py
info_server.py
```
Experiment with the example programs and browse through their code to get an idea of how the system is designed to be
used and its capabilities. A `--help` command-line flag is available for both to list their supported arguments.
## Architecture
### Shard Publisher
The central orchestrator component of the system. The shard publisher tracks the currently running shards in the system
and serves as the repository for live shard objects. It also provides subscribable notification services for new shards
being added to the system or live shards being removed from it.
### Shard
A class (`RedisShard`), instances of which represent tracked live shards. The shard object provides its *identifier* and
a way to obtain a working *connection* to that shard. It also stores the latest available *INFO* for the shard (once one
is available), as well as tracks that info's *age*. Lastly, it implements logic for determining how often the shard
should be polled for new info, based on the shard's activity level.
### Shard Watcher
The shard watcher is responsible for watching for new shards being added to the system, as well as live shards being
removed from it, and notifying the shard publisher accordingly. A default implementation (`LocalShardWatcher`) is
provided with the framework, which attempts to track all running instances of `redis-server` on the local machine;
however, this is meant for demonstration purposes only, and is not designed for production code.
> **The system is designed for you to plug in your own shard watcher that implements your application-specific logic.**
### Poller
The poller uses gevent-based concurrency to periodically poll all live shards for up-to-date info. The polling interval
is determined independently for each shard, based on the logic implemented in the `RedisShard` class.
### Servicer
Provides the API to be exposed by an appropriate server for querying the system.
Currently the servicer interface is written for usage with RPC modules such as
[xmlrpclib](https://docs.python.org/2/library/xmlrpclib.html) in mind, and this is what's used in the included example
server/client. However, writing an adapter wrapper to support a different kind of server interface (e.g.: gRPC, HTTP
REST) should be a relatively straightforward task.