{"id":27972690,"url":"https://github.com/lucit-systems-and-development/unicorn-depthcache-cluster-for-binance","last_synced_at":"2025-05-07T23:13:40.573Z","repository":{"id":259024482,"uuid":"852218125","full_name":"LUCIT-Systems-and-Development/unicorn-depthcache-cluster-for-binance","owner":"LUCIT-Systems-and-Development","description":"A highly scalable Kubernetes application from LUCIT to manage multiple and redundant UNICORN Binance Local Depth Cache  Instances on a Kubernetes Cluster for high-frequency access to Binance's DepthCache data (order books). ","archived":false,"fork":false,"pushed_at":"2024-10-28T14:33:58.000Z","size":1656,"stargazers_count":2,"open_issues_count":8,"forks_count":0,"subscribers_count":0,"default_branch":"master","last_synced_at":"2025-05-07T23:13:35.166Z","etag":null,"topics":["asyncio","binance","depthcache","orderbook","python","unicorn-binance-local-depth-cache"],"latest_commit_sha":null,"homepage":"https://unicorn-depthcache-cluster-for-binance.docs.lucit.tech/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/LUCIT-Systems-and-Development.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"custom":["https://shop.lucit.services"]}},"created_at":"2024-09-04T12:37:32.000Z","updated_at":"2024-11-02T18:57:28.000Z","dependencies_parsed_at":"2024-10-28T15:53:01.235Z","dependency_job_id":null,"html_url":"https://github.com/LUCIT-Systems-and-Development/unicorn-depthcache-cluster-for-binance","commit_stats":null,"previous_names":["lucit-systems-and-development/unicorn-binance-depth-cache-cluster","lucit-systems-and-development/unicorn-depthcache-cluster-for-binance","lucit-systems-and-development/unicorn-binance-depthcache-cluster"],"tags_count":4,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LUCIT-Systems-and-Development%2Funicorn-depthcache-cluster-for-binance","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LUCIT-Systems-and-Development%2Funicorn-depthcache-cluster-for-binance/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LUCIT-Systems-and-Development%2Funicorn-depthcache-cluster-for-binance/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LUCIT-Systems-and-Development%2Funicorn-depthcache-cluster-for-binance/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/LUCIT-Systems-and-Development","download_url":"https://codeload.github.com/LUCIT-Systems-and-Development/unicorn-depthcache-cluster-for-binance/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252968121,"owners_count":21833252,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["asyncio","binance","depthcache","orderbook","python","unicorn-binance-local-depth-cache"],"created_at":"2025-05-07T23:13:39.998Z","updated_at":"2025-05-07T23:13:40.567Z","avatar_url":"https://github.com/LUCIT-Systems-and-Development.png","language":"Python","funding_links":["https://shop.lucit.services"],"categories":[],"sub_categories":[],"readme":"[![Get a UNICORN DepthCache Cluster for Binance License](https://github.com/LUCIT-Systems-and-Development/unicorn-depthcache-cluster-for-binance/blob/master/images/logo/LUCIT-UBDCC-License-Offer.png)](https://shop.lucit.services/software/unicorn-depthcache-cluster-for-binance)\n\n[![License](https://img.shields.io/badge/license-LSOSL-blue)](https://unicorn-depthcache-cluster-for-binance.docs.lucit.tech/license.html)\n[![Build and Publish PyPi (lucit-ubdcc-dcn)](https://github.com/LUCIT-Systems-and-Development/unicorn-depthcache-cluster-for-binance/actions/workflows/build_wheels_lucit_ubdcc_dcn.yml/badge.svg)](https://github.com/LUCIT-Systems-and-Development/unicorn-depthcache-cluster-for-binance/actions/workflows/build_wheels_lucit_ubdcc_dcn.yml)\n[![Build and Publish PyPi (lucit-ubdcc-mgmt)](https://github.com/LUCIT-Systems-and-Development/unicorn-depthcache-cluster-for-binance/actions/workflows/build_wheels_lucit_ubdcc_mgmt.yml/badge.svg)](https://github.com/LUCIT-Systems-and-Development/unicorn-depthcache-cluster-for-binance/actions/workflows/build_wheels_lucit_ubdcc_mgmt.yml)\n[![Build and Publish PyPi (lucit-ubdcc-restapi)](https://github.com/LUCIT-Systems-and-Development/unicorn-depthcache-cluster-for-binance/actions/workflows/build_wheels_lucit_ubdcc_restapi.yml/badge.svg)](https://github.com/LUCIT-Systems-and-Development/unicorn-depthcache-cluster-for-binance/actions/workflows/build_wheels_lucit_ubdcc_restapi.yml)\n[![Build and Publish PyPi (lucit-ubdcc-shared-modules)](https://github.com/LUCIT-Systems-and-Development/unicorn-depthcache-cluster-for-binance/actions/workflows/build_wheels_lucit_ubdcc_shared_modules.yml/badge.svg)](https://github.com/LUCIT-Systems-and-Development/unicorn-depthcache-cluster-for-binance/actions/workflows/build_wheels_lucit_ubdcc_shared_modules.yml)\n[![Read the Docs](https://img.shields.io/badge/read-%20docs-yellow)](https://unicorn-depthcache-cluster-for-binance.docs.lucit.tech)\n[![Github](https://img.shields.io/badge/source-github-cbc2c8)](https://github.com/LUCIT-Systems-and-Development/unicorn-depthcache-cluster-for-binance)\n[![Telegram](https://img.shields.io/badge/community-telegram-41ab8c)](https://t.me/unicorndevs)\n[![Gitter](https://img.shields.io/badge/community-gitter-41ab8c)](https://gitter.im/unicorn-trading-suite/unicorn-depthcache-cluster-for-binance?utm_source=badge\u0026utm_medium=badge\u0026utm_campaign=pr-badge\u0026utm_content=badge)\n[![Get Free Professional Support](https://img.shields.io/badge/chat-lucit%20support-004166)](https://www.lucit.tech/get-support.html)\n\n# UNICORN DepthCache Cluster for Binance (UBDCC)\n\nA highly scalable Kubernetes application from LUCIT to manage multiple and redundant UNICORN Binance Local Depth Cache \nInstances on a Kubernetes Cluster for high-frequency access to Binance's DepthCache data (order books). \n\nThe cluster can be accessed from any programming language via a REST API, allowing Asks and Bids to be retrieved in \nJSON format.\n\n[Get help](https://www.lucit.tech/get-support.html)!\n\nIf you like the project, please \n[![star](https://raw.githubusercontent.com/lucit-systems-and-development/unicorn-binance-local-depth-cache/master/images/misc/star.png)](https://github.com/LUCIT-Systems-and-Development/unicorn-depthcache-cluster-for-binance/stargazers) it on \n[GitHub](https://github.com/LUCIT-Systems-and-Development/unicorn-depthcache-cluster-for-binance)! \n\n## Get a UNICORN DepthCache Cluster for Binance License\n\n***Licenses will only be publicly available in the store in a few days. If you are interested in a free trial license, \nplease [contact us via the chat](https://www.lucit.tech/get-support.html)!***\n\nTo run the *UNICORN DepthCache Cluster for Binance* you need a [valid license](https://shop.lucit.services/software/unicorn-depthcache-cluster-for-binance)!\n\n## What is UBDCC?\n\nThe main idea is to deploy the UBDCC on a Kubernetes cluster with, for example, 4 rented servers. After transferring a \nvalid license, you can create and manage DepthCaches within the cluster environment instead of on local servers and \naccess them from multiple clients.\n\nFor example, when you configure the system to create 200 DepthCaches with a `desired_quantity` of `2`, UBDCC will deploy\n2 DepthCaches for each symbol/market. These DepthCaches are evenly distributed across the nodes of the cluster and can \ndownload order book snapshots from the Binance Rest API using their own public IP addresses. On the first run, each \nserver starts 50 DepthCaches, synchronizing the full set of 200 as quickly as possible. Afterward, replicas are \ninitiated, with each node handling 100 DepthCaches.\n\n[![Visual overview](https://lucid.app/publicSegments/view/7ba7d734-4bb2-467f-b7b9-74ea0d1deec2/image.png)](https://lucid.app/publicSegments/view/7ba7d734-4bb2-467f-b7b9-74ea0d1deec2/image.png)\n\n## Key Features\n\n- **Asynchronous Operation**: The entire cluster code is built to run asynchronously.\n- **Load Balancing \u0026 Failover**: All requests for data (Asks/Bids) are handled via a load balancer with built-in \nfailover, ensuring high availability and quick response times.\n  - Local requests for Asks/Bids: ~0.01 seconds\n  - Requests via the Internet: ~0.06 seconds\n- **Flexible Data Retrieval**: You can trim the amount of transferred data at the cluster level, either by limiting to\na specific amount of top Asks/Bids or by setting a threshold.\n- **HTTP Access**: DepthCache values can be retrieved through HTTP using both synchronous and asynchronous methods \nprovided by \n[UBLDC](https://unicorn-binance-local-depth-cache.docs.lucit.tech/unicorn_binance_local_depth_cache.html#module-unicorn_binance_local_depth_cache.cluster).\n- **Top Performance**: The entire code base is deployed in the Kubernetes cluster as a compiled C-Extension!\n- **Supported Architectures**: CPython 3.12 on 64-bit (x86_64) and 32-bit (i686): musllinux (based on musl libc 1.1+),\n  manylinux (based on glibc 2.5+ and 2.17+) compatible with manylinux1 and manylinux2014\n- **Manages Binance Weight Costs**: If the weight costs become too high, the cluster throttles the initialization.\n- **Supported Exchanges**:\n\n| Exchange                                                           | Exchange string               | \n|--------------------------------------------------------------------|-------------------------------| \n| [Binance](https://www.binance.com)                                 | `binance.com`                 |\n| [Binance Testnet](https://testnet.binance.vision/)                 | `binance.com-testnet`         |\n| [Binance USD-M Futures](https://www.binance.com)                   | `binance.com-futures`         |\n| [Binance USD-M Futures Testnet](https://testnet.binancefuture.com) | `binance.com-futures-testnet` |\n| [Binance US](https://www.binance.us/)                              | `binance.us`                  |\n\n\n## Current State\n\nThe first MVP is stable and offers the most critical features for efficient DepthCache management. Future improvements \nmight include switching to websockets instead of REST queries, or implementing simultaneous queries for both Asks and \nBids. [Vote here for new features!](https://github.com/LUCIT-Systems-and-Development/unicorn-depthcache-cluster-for-binance/issues?q=is%3Aissue+is%3Aopen+label%3Aenhancement)\n\nFor more information, check out the \n[GitHub Repository](https://github.com/LUCIT-Systems-and-Development/unicorn-depthcache-cluster-for-binance) and the\n[Docs](https://unicorn-binance-depth-cache-cluster.docs.lucit.tech).\n\n## Watch a Demo Video\n[![Watch the demo video](https://img.youtube.com/vi/hq2iZPiFnvE/maxresdefault.jpg)](https://www.youtube.com/watch?v=hq2iZPiFnvE)\n\n## Installation\n\n- Get a Kubernetes cluster with powerful CPUs from a provider of your choice and connect `kubectl`\n- Install dependencies\n\n```\nkubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml\n```\n\n### Helm Chart\n- [Install Helm](https://helm.sh/docs/intro/install) \n- Prepare `helm`\n\n``` \nhelm repo add lucit-ubdcc https://unicorn-depthcache-cluster-for-binance.docs.lucit.tech/helm\nhelm repo update\n```\n\n- Install the UNICORN DepthCache Cluster for Binance\n\n``` \nhelm install lucit-ubdcc lucit-ubdcc/lucit-ubdcc\n```\n\n- Get the \"LoadBalancer Ingress\" IP, the default Port is TCP 80!\n\n```\nkubectl describe services lucit-ubdcc-restapi\n```\n\n#### Choose an explizit version\n- Find a version to choose\n\n``` \nhelm search repo lucit-ubdcc\n```\n\n- Then\n\n``` \nhelm install lucit-ubdcc lucit-ubdcc/lucit-ubdcc --version 0.1.4\n```\n\n#### Choose a namespace\n``` \nhelm install lucit-ubdcc lucit-ubdcc/lucit-ubdcc --namespace lucit-ubdcc\n```\n\n#### Choose an alternate public port\n``` \nhelm install lucit-ubdcc lucit-ubdcc/lucit-ubdcc --set publicPort.restapi=8080\n```\n  \n### Kubernetes Deployment\n- [Download the deployment files](https://github.com/LUCIT-Systems-and-Development/unicorn-depthcache-cluster-for-binance/tree/master/admin/k8s)\n- Apply the deployment files with `kubectl`\n\n``` \nkubectl apply -f ./setup/01_namespace_lucit-ubdcc.yaml\nkubectl apply -f ./setup/02_role_lucit-ubdcc.yaml\nkubectl apply -f ./setup/03_rolebinding_lucit-ubdcc.yaml\nkubectl apply -f ./lucit-ubdcc-dcn.yaml  \nkubectl apply -f ./lucit-ubdcc-mgmt.yaml\nkubectl apply -f ./lucit-ubdcc-mgmt_service.yaml\nkubectl apply -f ./lucit-ubdcc-restapi.yaml\nkubectl apply -f ./lucit-ubdcc-restapi_service.yaml\n```\n\n- Get the \"LoadBalancer Ingress\" IP, the default Port is TCP 80:\n\n```\nkubectl describe services lucit-ubdcc-restapi\n```\n\n## Security\nIn any case, you should set the firewall in the web interface of the Kubernetes provider so that only your systems \nhave access to UBDCC.\n\nIf you want to do this, you can add HTTPS to the LoadBalancer with most providers.\n  \n## Uninstallation\n```\nkubectl delete -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml\n```\n\n### Helm Chart\n```\nhelm uninstall lucit-ubdcc\n```\n\n### Kubernetes Deployment\n- Delete the deployment with `kubectl`\n\n``` \nkubectl delete -f ./setup/01_namespace_lucit-ubdcc.yaml\nkubectl delete -f ./setup/02_role_lucit-ubdcc.yaml\nkubectl delete -f ./setup/03_rolebinding_lucit-ubdcc.yaml\nkubectl delete -f ./lucit-ubdcc-dcn.yaml  \nkubectl delete -f ./lucit-ubdcc-mgmt.yaml\nkubectl delete -f ./lucit-ubdcc-mgmt_service.yaml\nkubectl delete -f ./lucit-ubdcc-restapi.yaml\nkubectl delete -f ./lucit-ubdcc-restapi_service.yaml\n```\n\n## Accessing the DepthCaches\n\nThe UNICORN DepthCache Cluster for Binance is accessed with the Python module [UNICORN Binance Local Depth Cache](https://github.com/LUCIT-Systems-and-Development/unicorn-binance-local-depth-cache?tab=readme-ov-file#connect-to-a-unicorn-depthcache-cluster-for-binance).\n\nJust try this [examples](https://github.com/LUCIT-Systems-and-Development/unicorn-binance-local-depth-cache/tree/master/examples/unicorn_depthcache_cluster_for_binance)!\n\n## How to report Bugs or suggest Improvements?\n[List of planned features](https://github.com/LUCIT-Systems-and-Development/unicorn-depthcache-cluster-for-binance/issues?q=is%3Aissue+is%3Aopen+label%3Aenhancement) - click ![thumbs-up](https://raw.githubusercontent.com/lucit-systems-and-development/unicorn-binance-suite/master/images/misc/thumbup.png) if you need one of them or suggest a new feature!\n\nBefore you report a bug, [try the latest release](https://github.com/LUCIT-Systems-and-Development/unicorn-depthcache-cluster-for-binance#installation-and-upgrade). If the issue still exists, provide the error trace, OS \nand Python version and explain how to reproduce the error. A demo script is appreciated.\n\nIf you don't find an issue related to your topic, please open a new [issue](https://github.com/LUCIT-Systems-and-Development/unicorn-depthcache-cluster-for-binance/issues)!\n\n[Report a security bug!](https://github.com/LUCIT-Systems-and-Development/unicorn-depthcache-cluster-for-binance/security/policy)\n\n## Contributing\n[UNICORN DepthCache Cluster for Binance](https://www.lucit.tech/unicorn-depthcache-cluster-for-binance.html) is an open \nsource project which welcomes contributions which can be anything from simple documentation fixes and reporting dead links to new features. To \ncontribute follow \n[this guide](https://github.com/LUCIT-Systems-and-Development/unicorn-depthcache-cluster-for-binance/blob/master/CONTRIBUTING.md).\n \n### Contributors\n[![Contributors](https://contributors-img.web.app/image?repo=LUCIT-Systems-and-Development/unicorn-depthcache-cluster-for-binance)](https://github.com/LUCIT-Systems-and-Development/unicorn-depthcache-cluster-for-binance/graphs/contributors)\n\nWe ![love](https://raw.githubusercontent.com/lucit-systems-and-development/unicorn-binance-suite/master/images/misc/heart.png) open source!\n\n## Disclaimer\nThis project is for informational purposes only. You should not construe this information or any other material as \nlegal, tax, investment, financial or other advice. Nothing contained herein constitutes a solicitation, recommendation, \nendorsement or offer by us or any third party provider to buy or sell any securities or other financial instruments in \nthis or any other jurisdiction in which such solicitation or offer would be unlawful under the securities laws of such \njurisdiction.\n\n### If you intend to use real money, use it at your own risk!\n\nUnder no circumstances will we be responsible or liable for any claims, damages, losses, expenses, costs or liabilities \nof any kind, including but not limited to direct or indirect damages for loss of profits.\n\n## Commercial Support\n\n[![Get professional and fast support](https://raw.githubusercontent.com/LUCIT-Systems-and-Development/unicorn-trading-suite/master/images/support/LUCIT-get-professional-and-fast-support.png)](https://www.lucit.tech/get-support.html)\n\n***Do you need a developer, operator or consultant?*** [Contact us](https://www.lucit.tech/contact.html) for a non-binding initial consultation!\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flucit-systems-and-development%2Funicorn-depthcache-cluster-for-binance","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flucit-systems-and-development%2Funicorn-depthcache-cluster-for-binance","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flucit-systems-and-development%2Funicorn-depthcache-cluster-for-binance/lists"}