{"id":26130618,"url":"https://github.com/emeraldpay/dshackle","last_synced_at":"2025-10-19T14:16:17.264Z","repository":{"id":35099445,"uuid":"191297773","full_name":"emeraldpay/dshackle","owner":"emeraldpay","description":"Fault Tolerant Load Balancer for Ethereum and Bitcoin APIs","archived":false,"fork":false,"pushed_at":"2024-11-27T18:50:07.000Z","size":6836,"stargazers_count":320,"open_issues_count":77,"forks_count":56,"subscribers_count":7,"default_branch":"master","last_synced_at":"2025-04-09T04:00:58.525Z","etag":null,"topics":["bitcoin","ethereum","java","kotlin","l7","load-balancer"],"latest_commit_sha":null,"homepage":"","language":"Kotlin","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/emeraldpay.png","metadata":{"files":{"readme":"README.adoc","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-06-11T05:08:08.000Z","updated_at":"2025-04-07T10:31:13.000Z","dependencies_parsed_at":"2023-01-15T14:00:48.096Z","dependency_job_id":"3d7e2d03-72ef-485b-b5d3-118eaef016a1","html_url":"https://github.com/emeraldpay/dshackle","commit_stats":null,"previous_names":[],"tags_count":134,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/emeraldpay%2Fdshackle","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/emeraldpay%2Fdshackle/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/emeraldpay%2Fdshackle/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/emeraldpay%2Fdshackle/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/emeraldpay","download_url":"https://codeload.github.com/emeraldpay/dshackle/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254471059,"owners_count":22076585,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bitcoin","ethereum","java","kotlin","l7","load-balancer"],"created_at":"2025-03-10T20:52:07.064Z","updated_at":"2025-10-19T14:16:17.254Z","avatar_url":"https://github.com/emeraldpay.png","language":"Kotlin","readme":"= Emerald Dshackle\n:imagesdir: docs/assets\nifdef::env-github[]\n:imagesdir: https://raw.githubusercontent.com/emeraldpay/dshackle/master/docs/assets\nendif::[]\n:version: 0.16.0\n:version-short: 0.16\n:link-docs: https://github.com/emeraldpay/dshackle/tree/release/v{version-short}/docs\n\nimage:https://github.com/emeraldpay/dshackle/workflows/Tests/badge.svg[\"Unit Tests\"]\nimage:https://codecov.io/gh/emeraldpay/dshackle/branch/master/graph/badge.svg[\"Coverage\",link=\"https://codecov.io/gh/emeraldpay/dshackle\"]\nimage:https://img.shields.io/docker/pulls/emeraldpay/dshackle?style=flat-square[\"Docker\",link=\"https://hub.docker.com/r/emeraldpay/dshackle\"]\nimage:https://img.shields.io/github/license/emeraldpay/dshackle.svg?style=flat-square\u0026maxAge=2592000[\"License\",link=\"https://github.com/emeraldpay/dshackle/blob/master/LICENSE\"]\nimage:https://img.shields.io/discord/1107840420240707704?style=flat-square[Discord,link=\"https://discord.gg/k9HpF9Jqee\"]\n\n\n[.lead]\n_Emerald Dshackle is a Fault Tolerant Load Balancer for Blockchain API._\n\nIts primary goal is to ensure reliable routing to multiple nodes, executing each request on a suitable provider.\nThis is achieved by evaluating various node characteristics including location, state, current height, and the RPC methods it can offer.\n\nIt tries to recover from connection errors, faulty nodes, invalid responses, etc.\nIf an upstream node lags behind, loses peers below the minimum requirement, initiates a resync, or goes offline, Dshackle temporarily removes it from the request pool.\nDshackle reinstates the connection to the node once the upstream issue is resolved.\n\nThe upstreams may be blockchain nodes such as Bitcoind, Geth, Parity, or public providers like Infura, QuickNode, etc.\nIt continuously checks their availability and the network's current status, executing commands while ensuring the response is consistent and data is successfully broadcast to the network.\n\nProvides:\n\n- Standard Bitcoin and Ethereum JSON RPC API over both HTTP and WebSocket\n- An advanced gRPC-based API, featuring upstream selection, asynchronous execution, and more.\n- **Secure** TLS with optional client authentication\n- Blockchain-aware edge **caching**, in memory and Redis\n- Intelligent routing based on **data availability** (peers, height, sync status)\n- **Data consistency**, always providing the most up-to-date state\n- Automatic **failover** and retry\n- Separate public blockchain nodes from your internal servers\n\nBlockchains support:\n\n- Ethereum and Ethereum Classic\n- Various Ethereum testnets\n- Bitcoin\n- Bitcoin testnets\n\nimage::dshackle-intro.png[alt=\"\",width=80%,align=\"center\"]\n\nWARNING: The project is still under development, please use with caution.\n\n== Quick Start\n\n=== Configuration\n\nCreate file `dshackle.yaml` with the following content:\n\n[source,yaml]\n----\nversion: v1\nport: 2449\ntls:\n  enabled: false\n\nproxy:\n  host: 0.0.0.0\n  port: 8545\n  routes:\n    - id: eth\n      blockchain: ethereum\n    - id: kovan\n      blockchain: kovan\n    - id: btc\n      blockchain: bitcoin\n\ncluster:\n  upstreams:\n    - id: infura-eth\n      blockchain: ethereum\n      connection:\n        ethereum:\n          rpc:\n            url: \"https://mainnet.infura.io/v3/${INFURA_USER}\"\n          ws:\n            url: \"wss://mainnet.infura.io/ws/v3/${INFURA_USER}\"\n    - id: infura-kovan\n      blockchain: kovan\n      connection:\n        ethereum:\n          rpc:\n            url: \"https://kovan.infura.io/v3/${INFURA_USER}\"\n          ws:\n            url: \"wss://kovan.infura.io/ws/v3/${INFURA_USER}\"\n    - id: bitcoin-main\n      blockchain: bitcoin\n      connection:\n        bitcoin:\n          rpc:\n            url: \"http://localhost:8332\"\n            basic-auth:\n              username: bitcoin\n              password: mypassword\n----\n\nWhich sets the following:\n\n- gRPC access through 0.0.0.0:2449\n** TLS security is disabled (_please don't use in production!_)\n- JSON RPC access through 0.0.0.0:8545 (both HTTP and WebsScket)\n** proxy requests to Ethereum and Kovan upstreams\n** request path for Ethereum Mainnet is `/eth`, `/kovan` for Kovan Testnet, and `/btc` for bitcoin\n** i.e. call Ethereum Mainnet by `POST http://127.0.0.0:8545/eth` with JSON RPC payload\n- two upstreams, one for Ethereum Mainnet and another for Kovan Testnet (both upstreams are configured to use Infura endpoint)\n- for Ethereum Mainnet it connects using JSON RPC and WebSocket connections,\n- for Bitcoin Mainet only JSON RPC is used\n- `${INFURA_USER}` will be provided through environment variable\n\nPlease note that you can configure many upstreams for a single blockchains.\nIf there is more than one upstream, then Dshackle routes requests to them as Round Robin.\nIf one of them becomes unavailable, Dshackle continues to use only active nodes.\n\nI.e., you can set up a node in the local network, plus Infura with `role: fallback`.\nIf anything happened to your local node, you still have access to a consistent state of the Ethereum blockchain via Infura.\n\n{link-docs}[See full documentations].\n\n==== Run docker image\n\nOfficial Docker image you can find at: https://hub.docker.com/r/emeraldpay/dshackle[emeraldpay/dshackle]\n\n.Setup Infura username\n[source,bash]\n----\nexport INFURA_USER=...\n----\n\n.Run Dshackle\n[source,bash,subs=\"attributes\"]\n----\ndocker run -p 2449:2449 -p 8545:8545 -v $(pwd):/etc/dshackle -e \"INFURA_USER=$INFURA_USER\" emeraldpay/dshackle:{version-short}\n----\n\nNow it listens on port 2449 at the localhost and can be connected from any gRPC compatible client.\nTools such as https://github.com/fullstorydev/grpcurl[gRPCurl] can automatically parse protobuf definitions and connect to it (actual Protobuf sources are located in a separate repository which you can find at https://github.com/emeraldpay/proto)\n\nAlternatively you can connect to port 8545 with traditional JSON RPC requests\n\n==== Access using JSON RPC over HTTP\n\nDshackle implements standard JSON RPC interface, providing additional caching layer, upstream readiness/liveness checks, retry and other features for building Fault Tolerant services.\n\n.Request using Curl\n[source,bash]\n----\ncurl --request POST \\\n  --url http://localhost:8545/eth \\\n  --header 'content-type: application/json' \\\n  --data '{\"jsonrpc\":\"2.0\", \"method\":\"eth_getBalance\", \"id\":1, \"params\":[\"0x690b2bdf41f33f9f251ae0459e5898b856ed96be\", \"latest\"]}'\n----\n\n.Output\n[source,bash]\n----\n{\"jsonrpc\":\"2.0\",\"id\":1,\"result\":\"0x72fa5e0181\"}\n----\n\n==== Access using JSON RPC over WebSocket\n\nOr the same Proxy URL can be accessed through WebSocket\n\n[source,bash]\n----\nwebsocat ws://localhost:8545/eth\n----\n\nThen make RPC calls or subscriptions:\n\n----\n\u003e | {\"jsonrpc\":\"2.0\", \"id\": 1, \"method\": \"eth_subscribe\", \"params\": [\"newHeads\"]}\n\n\u003c | {\"jsonrpc\":\"2.0\",\"id\":1,\"result\":\"1f8\"}\n\u003c | {\"jsonrpc\":\"2.0\",\"method\":\"eth_subscription\",\"params\":{\"result\":{....},\"subscription\":\"1f8\"}}\n----\n\n==== Access using gRPC\n\nNOTE: It's not necessary to use gRPC, as Dshackle can provide standard JSON RPC proxy, but Dshackle gRPC interface improves performance and provides additional features.\n\nDshackle provides a custom gRPC based API, which provides additional methods and other features such as streaming responses.\nPlease refer to the documentation: {link-docs}/07-methods.adoc[gRPC Methods]\nThe Protobuf definitions could be found in link:proto/[./proto].\n\n.Connect and listen for new blocks on Ethereum Mainnet\n[source,bash]\n----\ngrpcurl -import-path ./proto/ -proto blockchain.proto -d '{\\\"type\\\": 100}' -plaintext 127.0.0.1:2449 emerald.Blockchain/SubscribeHead\n----\n\n`type: 100` specifies the blockchain id, and 100 means Ethereum Mainnet. `1` is for Bitcoin Mainnet.\nThere we use Ethereum because it creates new blocks every 14 seconds, which works better for demo purposes, but the same request applied to Bitcoin as well.\n\n.Output would be like\n----\n{\n  \"chain\": \"CHAIN_ETHEREUM\",\n  \"height\": 8396159,\n  \"blockId\": \"fc58a258adccc94466ae967b1178eea721349b0667f59d5fe1b0b436460bce75\",\n  \"timestamp\": 1566423564000,\n  \"weight\": \"AnMcf2VJB5kOSQ==\"\n}\n{\n  \"chain\": \"CHAIN_ETHEREUM\",\n  \"height\": 8396160,\n  \"blockId\": \"787899711b862b77df8d2faa69de664048598265a9f96abf178d341076e200e0\",\n  \"timestamp\": 1566423574000,\n  \"weight\": \"AnMch35tO6hSGg==\"\n}\n...\n...\n----\n\nThe output above is for a _streaming subscription_ to all new blocks on the Ethereum Mainnet.\n\nIt's one of the services provided by Dshackle, in addition to standard methods provided by RPC JSON of underlying nodes.\n\n.You can also subscribe to balances changes of the balance on an address:\n[source,bash]\n----\ngrpcurl -import-path ./proto/ -proto blockchain.proto -d '{\\\"asset\\\": {\\\"chain\\\": \\\"100\\\", \\\"code\\\": \\\"ether\\\"}, \\\"address\\\": {\\\"address_single\\\": {\\\"address\\\": \\\"0xc02aaa39b223fe8d0a0e5c4f27ead9083c756cc2\\\"}}}' -plaintext 127.0.0.1:2449 emerald.Blockchain/SubscribeBalance\n----\n\n.and see how balance of the contract responsible for Wrapped Ether is changing:\n----\n{\n  \"asset\": {\n    \"chain\": \"CHAIN_ETHEREUM\",\n    \"code\": \"ETHER\"\n  },\n  \"address\": {\n    \"address\": \"0xc02aaa39b223fe8d0a0e5c4f27ead9083c756cc2\"\n  },\n  \"balance\": \"2410941696896999943701015\"\n}\n{\n  \"asset\": {\n    \"chain\": \"CHAIN_ETHEREUM\",\n    \"code\": \"ETHER\"\n  },\n  \"address\": {\n    \"address\": \"0xc02aaa39b223fe8d0a0e5c4f27ead9083c756cc2\"\n  },\n  \"balance\": \"2410930748488073834320430\"\n}\n...\n----\n\nThe balance subscription works with main coin (_ether_, _bitcoin_), or with tokens like ERC-20 if configured additionally.\nSee link:{link-docs}/reference-configuration.adoc[Configuration Reference].\n\nSee other enhanced methods in the link:{link-docs}/07-methods.adoc[Documentation for Enhanced Methods].\n\n== Documentation\n\nFor detailed documentation see:\n\n- Release {version}: {link-docs}[{link-docs}]\n- Development version: link:docs/[] directory.\n\n== Client Libraries\n\n=== JSON RPC\n\nDshackle should be compatible with all standard libraries that use Ethereum JSON RPC over HTTP.\n\n=== Java gRPC Client\n\nhttps://github.com/emeraldpay/emerald-java-client\n\n[source,groovy]\n----\nrepositories {\n    maven { url  \"https://maven.emrld.io\" }\n}\n\ndependencies {\n    implementation 'io.emeraldpay:emerald-api:0.11.0'\n}\n----\n\n=== Javascript gRPC Client\n\nimage:https://img.shields.io/npm/v/@emeraldpay/api-node.svg[\"npm (scoped)\",link=\"https://www.npmjs.com/package/@emeraldpay/api-node\"]\n\nhttps://github.com/emeraldpay/emerald-api-js\n\n[source,json]\n----\n\"dependencies\": {\n    \"@emeraldpay/api-node\": \"0.3.2\",\n}\n----\n\nSee more in the documentation for {link-docs}/11-client-libraries.adoc[Client Libraries].\n\n== Development\n\nWARNING: The code in `master` branch is considered a development version, which may lack proper testing and should not be used in production.\n\n=== Setting up environment\n\nDshackle is JVM based project written in Kotlin.\nTo build and run it from sources you'll need to install https://openjdk.java.net/projects/jdk/13/[Java JDK] and https://gradle.org/[Gradle]\n\n=== Build Dshackle\n\n==== Build everything\n\n[source,bash]\n----\ngradle build\n----\n\n==== Make a Zip distribution\n\n[source,bash]\n----\ngradle distZip\n----\n\nYou can find a redistributable zip in `build/distributions`\n\n==== Make a Docker distribution\n\n[source, bash]\n----\ngradle jib -Pdocker=gcr.io/myproject\n----\n\nGradle will prepare a Docker image and upload it to your custom Docker Registry at `gcr.io/myproject` (please change to address of your actual registry)\n\n==== Architecture\n\nDshackle is built using:\n\n- Kotlin\n- Spring Framework + Spring Boot\n- Spring Reactor\n- Netty\n- Etherjar\n- gRPC and HTTP2 protocol\n- Groovy and Spock for testing\n\n== Community\n\n=== Development Chat\n\nJoin our Discord chat to discuss development and ask questions:\n\nimage:https://img.shields.io/discord/1107840420240707704?style=flat-square[Discord,link=\"https://discord.gg/k9HpF9Jqee\"]\n\n\n== Commercial Support\n\nWant to support the project, prioritize a specific feature, or get commercial help with using Dshackle in your project?\nPlease contact splix@emerald.cash to discuss the possibility.\n\n== License\n\nCopyright 2023 EmeraldPay, Inc\n\nLicensed under the Apache License, Version 2.0 (the \"License\"); you may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\nhttp://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software distributed under the License is distributed on an \"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and limitations under the License.\n","funding_links":[],"categories":["By Language","Infrastructure"],"sub_categories":["Kotlin","Load Balancers"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Femeraldpay%2Fdshackle","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Femeraldpay%2Fdshackle","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Femeraldpay%2Fdshackle/lists"}