{"id":13471047,"url":"https://github.com/RedisAI/redis-inference-optimization","last_synced_at":"2025-03-26T13:30:42.780Z","repository":{"id":38239011,"uuid":"68528301","full_name":"RedisAI/redis-inference-optimization","owner":"RedisAI","description":"A Redis module for serving tensors and executing deep learning graphs","archived":false,"fork":false,"pushed_at":"2025-02-18T09:06:37.000Z","size":8091,"stargazers_count":831,"open_issues_count":102,"forks_count":105,"subscribers_count":28,"default_branch":"master","last_synced_at":"2025-03-19T10:06:12.958Z","etag":null,"topics":["machine-learning","onnxruntime","pytorch","redisai","serving-tensors","tensorflow"],"latest_commit_sha":null,"homepage":"https://redisai.io","language":"C","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/RedisAI.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2016-09-18T14:59:36.000Z","updated_at":"2025-03-05T12:50:15.000Z","dependencies_parsed_at":"2024-01-17T22:13:45.379Z","dependency_job_id":"7d2c13f9-f917-48c4-8ff6-f0251e2eb96d","html_url":"https://github.com/RedisAI/redis-inference-optimization","commit_stats":null,"previous_names":[],"tags_count":23,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RedisAI%2Fredis-inference-optimization","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RedisAI%2Fredis-inference-optimization/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RedisAI%2Fredis-inference-optimization/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RedisAI%2Fredis-inference-optimization/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/RedisAI","download_url":"https://codeload.github.com/RedisAI/redis-inference-optimization/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245662710,"owners_count":20652068,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["machine-learning","onnxruntime","pytorch","redisai","serving-tensors","tensorflow"],"created_at":"2024-07-31T16:00:38.895Z","updated_at":"2025-03-26T13:30:42.750Z","avatar_url":"https://github.com/RedisAI.png","language":"C","readme":"\u003e [!CAUTION]\n\u003e **Redis-inference-optimization is no longer actively maintained or supported.**\n\u003e\n\u003e We are grateful to the redis-inference-optimization community for their interest and support.\n\u003e Previously, redis-inference-optimization was named RedisAI, but was renamed in Jan 2025 to reduce confusion around Redis' other AI offerings. To learn more about Redis' current AI offerings, visit [the Redis website](https://redis.io/redis-for-ai).\n\n# Redis-inference-optimization\nRedis-inference-optimization is a Redis module for executing Deep Learning/Machine Learning models and managing their data. Its purpose is being a \"workhorse\" for model serving, by providing out-of-the-box support for popular DL/ML frameworks and unparalleled performance. **Redis-inference-optimization both maximizes computation throughput and reduces latency by adhering to the principle of data locality**, as well as simplifies the deployment and serving of graphs by leveraging on Redis' production-proven infrastructure.\n\n# Quickstart\nRedis-inference-optimization is a Redis module. To run it you'll need a Redis server (v6.0.0 or greater), the module's shared library, and its dependencies.\n\nThe following sections describe how to get started with redis-inference-optimization.\n\n## Docker\nThe quickest way to try redis-inference-optimization is by launching its official Docker container images.\n### On a CPU only machine\n```\ndocker run -p 6379:6379 redislabs/redisai:1.2.7-cpu-bionic\n```\n\n### On a GPU machine\nFor GPU support you will need a machine you'll need a machine that has Nvidia driver (CUDA 11.3 and cuDNN 8.1), nvidia-container-toolkit and Docker 19.03+ installed. For detailed information, checkout [nvidia-docker documentation](https://github.com/NVIDIA/nvidia-docker)\n\n```\ndocker run -p 6379:6379 --gpus all -it --rm redislabs/redisai:1.2.7-gpu-bionic\n```\n\n\n## Building\nYou can compile and build the module from its source code. \n\n### Prerequisites\n* Packages: git, python3, make, wget, g++/clang, \u0026 unzip\n* CMake 3.0 or higher needs to be installed.\n* CUDA 11.3 and cuDNN 8.1 or higher needs to be installed if GPU support is required.\n* Redis v6.0.0 or greater.\n\n### Get the Source Code\nYou can obtain the module's source code by cloning the project's repository using git like so:\n\n```sh\ngit clone --recursive https://github.com/RedisAI/redis-inference-optimization\n```\n\nSwitch to the project's directory with:\n\n```sh\ncd redis-inference-optimization\n```\n\n### Building the Dependencies\nUse the following script to download and build the libraries of the various redis-inference-optimization backends (TensorFlow, PyTorch, ONNXRuntime) for CPU only:\n\n```sh\nbash get_deps.sh\n```\n\nAlternatively, you can run the following to fetch the backends with GPU support.\n\n```sh\nbash get_deps.sh gpu\n```\n\n### Building the Module\nOnce the dependencies have been built, you can build the redis-inference-optimization module with:\n\n```sh\nmake -C opt clean ALL=1\nmake -C opt\n```\n\nAlternatively, run the following to build redis-inference-optimization with GPU support:\n\n```sh\nmake -C opt clean ALL=1\nmake -C opt GPU=1\n```\n\n### Backend Dependancy\n\nRedis-inference-optimization currently supports PyTorch (libtorch), Tensorflow (libtensorflow), TensorFlow Lite, and ONNXRuntime as backends. This section shows the version map between redis-inference-optimization and supported backends. This extremely important since the serialization mechanism of one version might not match with another. For making sure your model will work with a given redis-inference-optimization version, check with the backend documentation about incompatible features between the version of your backend and the version redis-inference-optimization is built with.\n\n\n| redis-inference-optimization | PyTorch  | TensorFlow | TFLite | ONNXRuntime |\n|:--------|:--------:|:----------:|:------:|:-----------:|\n| 1.0.3   |  1.5.0   |   1.15.0   | 2.0.0  |    1.2.0    |\n| 1.2.7   |  1.11.0  |   2.8.0    | 2.0.0  |   1.11.1    |\n| master  |  1.11.0  |   2.8.0    | 2.0.0  |   1.11.1    |\n\nNote: Keras and TensorFlow 2.x are supported through graph freezing. \n\n## Loading the Module\nTo load the module upon starting the Redis server, simply use the `--loadmodule` command line switch, the `loadmodule` configuration directive or the [Redis `MODULE LOAD` command](https://redis.io/commands/module-load) with the path to module's library.\n\nFor example, to load the module from the project's path with a server command line switch use the following:\n\n```sh\nredis-server --loadmodule ./install-cpu/redis-inference-optimization.so\n```\n\n### Give it a try\n\nOnce loaded, you can interact with redis-inference-optimization using redis-cli. \n\n### Client libraries\nSome languages already have client libraries that provide support for redis-inference-optimization's commands. The following table lists the known ones:\n\n| Project            | Language              | License      | Author                                           | URL                                                         |\n| -------            | --------              | -------      | ------                                           | ---                                                         |\n| JredisAI           | Java                  | BSD-3        | [RedisLabs](https://redislabs.com/)              | [Github](https://github.com/redis-inference-optimization/JRedisAI)               |\n| redisAI-py         | Python                | BSD-3        | [RedisLabs](https://redislabs.com/)              | [Github](https://github.com/redisAI/redisAI-py)             |\n| redisAI-go         | Go                    | BSD-3        | [RedisLabs](https://redislabs.com/)              | [Github](https://github.com/RedisAI/redisAI-go)             |\n| redisAI-js         | Typescript/Javascript | BSD-3        | [RedisLabs](https://redislabs.com/)              | [Github](https://github.com/redisAI/redisAI-js)             |\n| redis-modules-sdk  | TypeScript            | BSD-3-Clause | [Dani Tseitlin](https://github.com/danitseitlin) | [Github](https://github.com/danitseitlin/redis-modules-sdk) |\n| redis-modules-java | Java                  | Apache-2.0   | [dengliming](https://github.com/dengliming)      | [Github](https://github.com/dengliming/redis-modules-java)  |\n| smartredis         | C++                   | BSD-2-Clause | [Cray Labs](https://github.com/CrayLabs)         | [Github](https://github.com/CrayLabs/SmartRedis)            |\n| smartredis         | C                     | BSD-2-Clause | [Cray Labs](https://github.com/CrayLabs)         | [Github](https://github.com/CrayLabs/SmartRedis)            |\n| smartredis         | Fortran               | BSD-2-Clause | [Cray Labs](https://github.com/CrayLabs)         | [Github](https://github.com/CrayLabs/SmartRedis)            |\n| smartredis         | Python                | BSD-2-Clause | [Cray Labs](https://github.com/CrayLabs)         | [Github](https://github.com/CrayLabs/SmartRedis)            |\n\n## License\nRedis-inference-optimization is licensed under your choice of the Redis Source Available License 2.0 (RSALv2) or the Server Side Public License v1 (SSPLv1).\n","funding_links":[],"categories":["C","Model Serving and Monitoring","Model Deployment and Orchestration Frameworks","Deployment and Serving"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FRedisAI%2Fredis-inference-optimization","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FRedisAI%2Fredis-inference-optimization","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FRedisAI%2Fredis-inference-optimization/lists"}