{"id":19901890,"url":"https://github.com/hyperledger/firefly-ethconnect","last_synced_at":"2025-04-07T05:08:32.737Z","repository":{"id":38296069,"uuid":"137403357","full_name":"hyperledger/firefly-ethconnect","owner":"hyperledger","description":"Ethereum connectivity bridge for Web Services \u0026 async Messaging APIs - familiar to Enterprise Apps","archived":false,"fork":false,"pushed_at":"2025-03-13T10:25:34.000Z","size":37027,"stargazers_count":75,"open_issues_count":28,"forks_count":32,"subscribers_count":18,"default_branch":"main","last_synced_at":"2025-03-31T02:13:01.544Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/hyperledger.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":"CODEOWNERS","security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-06-14T20:00:38.000Z","updated_at":"2024-10-17T07:11:05.000Z","dependencies_parsed_at":"2023-11-07T17:26:21.240Z","dependency_job_id":"a99e47f6-ce0f-4b31-af7a-468acf28e99b","html_url":"https://github.com/hyperledger/firefly-ethconnect","commit_stats":null,"previous_names":["kaleido-io/ethconnect","hyperledger-labs/firefly-ethconnect"],"tags_count":30,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hyperledger%2Ffirefly-ethconnect","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hyperledger%2Ffirefly-ethconnect/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hyperledger%2Ffirefly-ethconnect/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hyperledger%2Ffirefly-ethconnect/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/hyperledger","download_url":"https://codeload.github.com/hyperledger/firefly-ethconnect/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247595334,"owners_count":20963943,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-12T20:16:31.804Z","updated_at":"2025-04-07T05:08:32.718Z","avatar_url":"https://github.com/hyperledger.png","language":"Go","readme":"[![codecov](https://codecov.io/gh/hyperledger/firefly-ethconnect/branch/main/graph/badge.svg?token=nO6ihSAzpV)](https://codecov.io/gh/hyperledger/firefly-ethconnect) [![Go Report Card](https://goreportcard.com/badge/github.com/hyperledger/firefly-ethconnect)](https://goreportcard.com/report/github.com/hyperledger/firefly-ethconnect)\n\n## EthConnect - Hyperledger FireFly connector for Ethereum networks\n\nThis repo contains the original implementation of the FireFly Connector\nfor Ethereum networks.\n\n\u003e For new projects, you are recommended to use the full Hyperledger FireFly\n\u003e project stack - and consider using the new\\\n\u003e [EVMConnect](https://github.com/hyperledger/firefly-evmconnect) connector\n\u003e for connectivity to your EVM based chains (whether public, or private).\n\nSince creation in 2018 a large amount of function has evolved through\nuse in production projects. Primarily this function has been driven by\nprivate sidechains, using Quorum, Hyperledger Besu and Go-Ethereum with\nfinality friendly high-throughput consensus algorithms such as\nIBFT and QBFT (and RAFT for latency optimized projects).\n\nA Web and Messaging API, taking the hassle out of submitting Ethereum transactions:\n- REST API generation for any ABI\n- Reliable event streaming - using Apache Kafka\n- High throughput tx submission\n- Concurrency management\n- Nonce management\n- Solidity compilation\n- ABI type mapping\n- Transaction receipt polling\n\nProvides an integration bridge into Ethereum permissioned chains, from simple\nWeb Service and Messaging interfaces that are friendly to existing Enterprise\napplications \u0026 middleware.\nFor example to allow connectivity from an Enterprise Service Bus (ESB) or other\nEnterprise Application Integration (EAI) tier, or applications running in a\nJava EE Application Server.\n\n[![hyperledger/firefly-ethconnect](ethconnect.png)](ethconnect.pdf)\n\n## License\n\nThis code is distributed under the [Apache 2 license](LICENSE).\n\nA dynamically linked plugin project is used to provide low level utilities like RLP encoding:\nhttps://github.com/kaleido-io/ethbinding\n\nAdditional code and network plugin extension points are provided for extensions.\n\n## Example payloads\n\nThe HTTP Webhooks and Kafka message payloads share a common schema.\nFor HTTP you can specify `Content-type: application/x-yaml` and send\nin a YAML payload, which will be converted to JSON.\n\nWhen sending messages to Kafka directly, JSON must be sent.\n\n### YAML to submit a transaction\n\nSend a transaction with parameters to an existing deployed contract.\n\n```yaml\nheaders:\n  type: SendTransaction\nfrom: 0xb480F96c0a3d6E9e9a263e4665a39bFa6c4d01E8\nto: 0xe1a078b9e2b145d0a7387f09277c6ae1d9470771\nparams:\n  - value: 4276993775\n    type: uint256\ngas: 1000000\nmethodName: set\n```\n\n### YAML to deploy a contract\n\nIdeal for deployment of simple contracts that can be specified inline (see #18).\n\n```yaml\nheaders:\n  type: DeployContract\nfrom: '0xb480F96c0a3d6E9e9a263e4665a39bFa6c4d01E8'\nparams:\n  - 12345\ngas: 1000000\nsolidity: |-\n  pragma solidity \u003e=0.4.22 \u003c0.6.0;\n\n  contract simplestorage {\n     uint public storedData;\n\n     function simplestorage(uint initVal) public {\n        storedData = initVal;\n     }\n\n     function set(uint x) public {\n        storedData = x;\n     }\n\n     function get() public view returns (uint retVal) {\n        return storedData;\n     }\n  }\n```\n\n## Why put a Web / Messaging API in front of an Ethereum node?\n\nThe JSON/RPC specification exposed natively by Go-ethereum and other Ethereum\nprotocol implementations, provides a rich low-level API for interfacing with\nthe node. It is usually exposed over HTTP (as well as IPC) so can be connected\nover a network, and have security layered in front of it.\n\nHowever, applications seldom code directly to the JSON/RPC API when deploying\ncontracts and sending transactions, because it is:\n- Asynchronous in nature, needing polling to obtain a transaction receipt\n- Based on Recursive Length Prefix (RLP) encoding of payloads\n\nInstead thick client libraries such as [web3.js](https://github.com/ethereum/web3.js/), [web3j](https://github.com/web3j/web3j), [web3.py](https://github.com/ethereum/web3.py), [Nethereum](https://github.com/Nethereum/Nethereum) and [ethjs](https://github.com/ethjs/ethjs) are used to submit transactions.\n\nThese thick client libraries perform many of the same functions as hyperledger/firefly-ethconnect, simplifying submission of transactions, receipt checking,\nnonce management etc.\n\nIn the modern world of Microservice architectures, having a simple, efficient\nand stateless REST API to submit transactions is a desirable alternative. A small self-contained, optimized, and independently scalable layer that can exposing a simple API that can be consumed by any application, programming language, or Integration tool.\n\ne.g. making it trivial to submit transactions. No coding required, just open up [Postman](https://www.getpostman.com/) and send in some YAML or JSON copied from a README like this one.\n\nIn an Enterprise context, the availability of standardized interfaces like HTTP and Kafka means applications and integration tools (a common ESB or other EAI tooling) can be hooked up easily. No need to insert complex code libraries into applications, many of which are LGPL licensed.\n\n## Why Messaging?\n\nSo you ask, if the goal is simplicity, why not just put the HTTP API in front of JSON/RPC and be done with it?\n\nThere are some challenges in Enterprise grade Blockchain solutions (particularly in high throughput permissioned/private chains) that cannot be solved by a stateless HTTP bridging layer alone.\n\nSo for Firefly, we started with a robust Messaging tier and layered the HTTP interface on top.\n\n## The asynchronous nature of Ethereum transactions\n\nEthereum transactions can take many seconds or minutes (depending on the backlog and block period) from submission until they make it into a block, and a receipt is available. Each individual node in Ethereum provides a built-in pool where transactions can be buffered while waiting to enter a block.\n\nConnecting a synchronous blocking HTTP interface directly to an inherently asynchronous system like this can cause problems. The HTTP requester times out waiting for a response, but has no way to know if it should retry or if the transaction will eventually succeed. It cannot cancel the request.\n\nProviding a Messaging layer with at-least-once delivery and message ordering, allows the asynchronous nature of Ethereum to be reflected back to the remote application.\n\nApplications that have their own state stores are able to communicate over Messaging / Kafka with simple JSON payloads to stream transactions into a scalable set of Ethereum nodes, and process the replies as they occur. The application can scale horizontally. Applications can also be decoupled from the Ethereum network with an integration technology like an Enterprise Service Bus (ESB).\n\nWhen spikes in workload occur that create a large queue of transactions that need to be fed into the Ethereum network at a lower rate, the hyperledger/firefly-ethconnect bridge feeds them in at an optimal rate.\n\n### Ethereum Webhooks and the REST Receipt Store (MongoDB)\n\nAnother key goal of having a robust Messaging layer under the covers is that any application can send messages into Ethereum reliably.\n\n- `POST` a [trivial YAML/JSON payload](#yaml-to-submit-a-transaction)\n  - to `/hook` to complete once the message is confirmed by Kafka (0.5s - tunable)\n  - to `/fasthook` to complete immediately when the message is sent to Kafka\n- Receive back an `id` for the request straight away\n   - Let the bridge do the retry polling to get the Ethereum receipt once a block is cut\n\nThis is ideal for Applications, Integration-as-a-Service (IaaS) platforms, and built-in SaaS integrations that support Webhooks for event submission.\n\nHowever, SaaS administrators and end-users still need to be able to track down what happened for a particular transaction, to diagnose problems, and correlate details of the Ethereum transaction with the business event. See [Messages](#messages) below for details of the responses stored in the Receipt store.\n\nSo kaleido-io/ethereum comes with a built-in receipt store, backed by MongoDB. It listens reliably for the replies over Kafka, and inserts each of them into the Database.\n\nIt provides a trivially simple REST API:\n- `GET` `/reply/a789940d-710b-489f-477f-dc9aaa0aef77` to look for an individual reply\n- `GET` `/replies` to list the replies\n  - Ordered by time _received_ (not the order submitted) - listing the newest first\n  - `limit` and `skip` query parameters can be used to paginate the results\n\nA capped collection can be used in MongoDB to limit the storage. For example to store only the last 1000 replies received.\n\n### Nonce management for Scale and Message Ordering\n\nThe transaction pooling/execution logic within an Ethereum node is based upon the concept of a `nonce`, which must be incremented exactly once each time a transaction is submitted from the same Ethereum address. There can be no gaps in the nonce values, or messages build up in the `queued transaction` pool waiting for the gap to be filled (which is the responsibility of the\nsender to fill). This allows for deterministic ordering of transactions sent by the same sender.\n\nThe management of this `nonce` pushes complexity back to the application tier - especially for horizontally scaled Enterprise applications sending many transactions using the same sender. By using an ordered Messaging stream to submit messages into the Ethereum network, many applications are able to delegate this complexity to hyperledger/firefly-ethconnect.\n\nThe hyperledger/firefly-ethconnect bridge contains all the logic necessary to communicate with the node to determine the next nonce, and also to cope with multiple requests being in flight within the same block, for the same sender (including [with IBFT](https://github.com/ethereum/EIPs/issues/650#issuecomment-360085474)).\n\nWhen using the bridge, an application can submit simple YAML/JSON formatted transactions\nin a highly parallel way across many instances using the same sender address to the bridge\nover HTTPS/Kafka _without_ a nonce. Then through ordered message delivery and nonce management\ncode within the hyperledger/firefly-ethconnect bridge it will be assigned a nonce and submitted\ninto the Ethereum node. The nonce assigned is returned by the bridge in the reply.\n\nIf a sender needs to achieve exactly-once delivery of transactions (vs. at-least-once) it is still necessary to allocate the nonce within the application and pass it into hyperledger/firefly-ethconnect in the payload.  This allows the sender to control allocation of nonces using its internal state store / locking.\n\n\u003e There's a good summary of at-least-once vs. exactly-once semantics in the [Akka documentation](https://doc.akka.io/docs/akka/current/general/message-delivery-reliability.html?language=scala#discussion-what-does-at-most-once-mean-)\n\nGiven many Enterprise scenarios involve writing hashed proofs of completion of an off-chain transaction to a shared ledger (vs. performing the actual transaction on the chain), at-least-once delivery is sufficient in a wide range of cases. Smart Contracts can be implemented in an idempotent way, to re-store or discard the proof if submitted twice.\n\nAn `idempotent` interface is one that can be called multiple times with the same input,\nand achieve the same result. By example, if you were using an interface on an online\nshopping system and clicked the `Complete order` button twice, you could tell if their\ninterface was idempotent by whether you received one parcel at your house (idempotent)\nor two (not idempotent).\n\nIn the case of storing a hashed proof of an external event, this can be achieved quite\ntrivially. Some examples as follows:\n- The hash itself is a key into a map, and if on submission it is already\n  found in the map the duplicate submission is discarded (if the other parts of the\n  proof such as the originating address or metadata differ, this should be flagged\n  as an error).\n- There is a transaction identifier known to all external parties to the off-chain\n  transaction, and this is used as a key into a map of transactions.\n- In a voting scenario with multiple signed copies of a proof being submitted,\n  the public address of each sender (whether an Ethereum address, or\n  other off-chain cryptography) is used as a key into a map of votes.\n\n## Why Kafka?\n\nWe selected Kafka as the first Messaging platform, because Kafka has message ordering and scale characteristics that are ideally suited to the Ethereum transaction model:\n- Transactions can be sprayed across partitions, while retaining order of the transactions for a particular sender. Allowing independent and dynamic scaling of the application, hyperledger/firefly-ethconnect bridge and Go-ethereum node components.\n- The modern replication based cloud-native and continuously available architecture is ideal for Hyperledger projects, and is likely to be a good fit for the modern Microservice architectures that are common in Blockchain projects.\n\n## Topics\n\n[![hyperledger/firefly-ethconnect - Topics](ethconnect_topics.png)](ethconnect.pdf)\n\nThe topic usage is very simple:\n- One topic delivering messages into the bridge\n  - The Kafka-\u003eEthereum bridge listens with a consumer group\n  - If configured, the Webhook-\u003eKafka bridge sends to this topic\n  - The Kafka-\u003eEthereum bridge updates the offset only after a message is delivered into Ethereum, and a success/failure message is sent back (at-least-once delivery)\n  - The Kafka-\u003eEthereum bridge keeps a configurable number of messages in-flight - received from the input topic, but not yet replied to\n- One topic delivering replies\n  - All replies go to single topic, with a header that correlates replies\n  - If configured, the Webhook-\u003eKafka bridge listens to this topic with a consumer group\n  - The Webhook-\u003eKafka bridge marks the offset of each message after inserting into MongoDB (if the receipt store is configured)\n\n## Messages\n\n### Example transaction receipt\n\nAn example successful `TransactionSuccess` reply message, which indicates a transaction receipt was obtained, and the status of that transaction was `1` (a similar payload with `header.type` set to `TransactionFailure` is returned if EVM execution is unsuccessful):\n```json\n{\n  \"headers\": {\n    \"id\": \"3eca1f95-d43a-4884-525d-8d7efa7f8c9c\",\n    \"requestId\": \"a789940d-710b-489f-477f-dc9aaa0aef77\",\n    \"requestOffset\": \"zzyly4jg5f-zze37213zm-requests:0:35479\",\n    \"timeElapsed\": 23.160396176,\n    \"timeReceived\": \"2018-07-25T12:15:19Z\",\n    \"type\": \"TransactionSuccess\"\n  },\n  \"blockHash\": \"0x7c6d389b27c57a0f5e4792712f1dd61a733230e3188511a6f59e0559c2c06352\",\n  \"blockNumber\": \"6329\",\n  \"blockNumberHex\": \"0x18b9\",\n  \"cumulativeGasUsed\": \"26817\",\n  \"cumulativeGasUsedHex\": \"0x68c1\",\n  \"from\": \"0xb480f96c0a3d6e9e9a263e4665a39bfa6c4d01e8\",\n  \"gasUsed\": \"26817\",\n  \"gasUsedHex\": \"0x68c1\",\n  \"nonce\": \"458\",\n  \"nonceHex\": \"0x1cA\",\n  \"status\": \"1\",\n  \"statusHex\": \"0x1\",\n  \"to\": \"0x6287111c39df2ff2aaa367f0b062f2dd86e3bcaa\",\n  \"transactionHash\": \"0xdb7578e473767105c314dd73d34b630a193227826bc6ddf6261b45a27405e7e7\",\n  \"transactionIndex\": \"0\",\n  \"transactionIndexHex\": \"0x0\"\n}\n```\n\nNumeric values are supplied in two formats for convenience of different receiving applications:\n- Simple numeric values, wrapped in strings to handle the potential of big integers\n- Hex values encoded identically to the native JSON/RPC interface\n\nThe MongoDB receipt store adds two additional fields, used to retrieve the entries efficient on the REST interface:\n```json\n{\n  \"_id\": \"a789940d-710b-489f-477f-dc9aaa0aef77\",\n  \"receivedAt\": 1532520942884\n}\n```\n\n### Example error\n\nIn the case that the Kafka-\u003eEthereum is unable to submit a transaction and obtain an\nEthereum receipt, an `Error` reply is sent back. This includes details of the error,\nas well as the full original payload sent (will always be JSON).\n\nThe request payload is provided, because in cases where a transaction receipt is not\nobtained, it could be impossible to obtain the original transaction details from the\nblockchain.\n\n```json\n{\n        \"errorMessage\": \"unknown account\",\n        \"headers\": {\n            \"id\": \"8d94a12e-ec63-4463-6c41-348e050e9044\",\n            \"requestId\": \"f53c73e9-2512-4e91-6e2c-faec0e138716\",\n            \"requestOffset\": \"u0d7zazjno-u0aopxc5lf-requests:0:4\",\n            \"timeElapsed\": 0.001892497,\n            \"timeReceived\": \"2018-07-27T12:20:50Z\",\n            \"type\": \"Error\"\n        },\n        \"requestPayload\": \"{\\\"from\\\":\\\"0x2942a3BE3599FbE25939e60eE3a137f10E09FD1e\\\",\\\"gas\\\":1000000,\\\"headers\\\":{\\\"id\\\":\\\"f53c73e9-2512-4e91-6e2c-faec0e138716\\\",\\\"type\\\":\\\"SendTransaction\\\"},\\\"methodName\\\":\\\"set\\\",\\\"params\\\":[{\\\"type\\\":\\\"uint256\\\",\\\"value\\\":4276993775}],\\\"to\\\":\\\"0xe1a078b9e2b145d0a7387f09277c6ae1d9470771\\\"}\"\n    }\n```\n\n## Running the Bridge\n\n### Installation\n\nRequires [Go 1.11](https://golang.org/dl/) or later to install with `go get`\n\n```sh\ngo get github.com/hyperledger/firefly-ethconnect\n```\n\n## Development environment\n\n### Pre-requisites\n\nBefore building, the `ethbinding` plugin project must be present in an `ethbinding` directory\nat the same level as the ethconnect source. This project provides dynamic bindings to\ngo-ethereum.\n\nUnit tests also require a version of solc 0.5 - 0.7, installed with `brew install solidity@7`\n[or similar](https://docs.soliditylang.org/en/v0.8.7/installing-solidity.html).\n\n### Building\n\nWith Go 1.11 or later simply use\n```sh\nmake\n```\n\nCan be built from source using Go 1.10 using `vgo`.\n```sh\ngo get -u golang.org/x/vgo\nVGO=vgo make -e\n```\n\n### Ways to run\n\nYou can run a single bridge using simple commandline options, which is ideal for exploring the configuration options. Or you can use a YAML server definition to run multiple modes in single process.\n\n### Running the Kafka-\u003eEthereum bridge via cmdline params\n\n```sh\n$ ./ethconnect kafka --help\nCopyright (C) 2018,2021 Kaleido\nLicensed under the Apache License, Version 2.0\nVersion:  (Build Date: )\n\nKafka-\u003eEthereum (JSON/RPC) Bridge\n\nUsage:\n  ethconnect kafka [flags]\n\nFlags:\n  -b, --brokers stringArray      Comma-separated list of bootstrap brokers\n  -i, --clientid string          Client ID (or generated UUID)\n  -g, --consumer-group string    Client ID (or generated UUID)\n  -h, --help                     help for kafka\n  -m, --maxinflight int          Maximum messages to hold in-flight\n  -P, --predict-nonces           Predict the next nonce before sending txns (default=false for node-signed txns)\n  -r, --rpc-url string           JSON/RPC URL for Ethereum node\n  -p, --sasl-password string     Password for SASL authentication\n  -u, --sasl-username string     Username for SASL authentication\n  -C, --tls-cacerts string       CA certificates file (or host CAs will be used)\n  -c, --tls-clientcerts string   A client certificate file, for mutual TLS auth\n  -k, --tls-clientkey string     A client private key file, for mutual TLS auth\n  -e, --tls-enabled              Encrypt network connection with TLS (SSL)\n  -z, --tls-insecure             Disable verification of TLS certificate chain\n  -t, --topic-in string          Topic to listen to\n  -T, --topic-out string         Topic to send events to\n  -x, --tx-timeout int           Maximum wait time for an individual transaction (seconds)\n\nGlobal Flags:\n  -d, --debug int   0=error, 1=info, 2=debug (default 1)\n  -Y, --print-yaml-confg   Print YAML config snippet and exit\n```\n\n### Running the Webhooks-\u003eKafka bridge via cmdline params\n\n```\n$ethconnect webhooks --help\nCopyright (C) 2018,2021 Kaleido\nLicensed under the Apache License, Version 2.0\n\nVersion:  (Build Date: )\n\nWebhooks-\u003eKafka Bridge\n\nUsage:\n  ethconnect webhooks [flags]\n\nFlags:\n  -b, --brokers stringArray                 Comma-separated list of bootstrap brokers\n  -i, --clientid string                     Client ID (or generated UUID)\n  -g, --consumer-group string               Client ID (or generated UUID)\n  -h, --help                                help for webhooks\n  -L, --listen-addr string                  Local address to listen on\n  -l, --listen-port int                     Port to listen on (default 8080)\n  -D, --mongodb-database string             MongoDB receipt store database\n  -q, --mongodb-query-limit int             Maximum docs to return on a rest call (cap on limit)\n  -r, --mongodb-receipt-collection string   MongoDB receipt store collection\n  -x, --mongodb-receipt-maxdocs int         Receipt store capped size (new collections only)\n  -m, --mongodb-url string                  MongoDB URL for a receipt store\n  -p, --sasl-password string                Password for SASL authentication\n  -u, --sasl-username string                Username for SASL authentication\n  -C, --tls-cacerts string                  CA certificates file (or host CAs will be used)\n  -c, --tls-clientcerts string              A client certificate file, for mutual TLS auth\n  -k, --tls-clientkey string                A client private key file, for mutual TLS auth\n  -e, --tls-enabled                         Encrypt network connection with TLS (SSL)\n  -z, --tls-insecure                        Disable verification of TLS certificate chain\n  -t, --topic-in string                     Topic to listen to\n  -T, --topic-out string                    Topic to send events to\n\nGlobal Flags:\n  -d, --debug int   0=error, 1=info, 2=debug (default 1)\n  -Y, --print-yaml-confg   Print YAML config snippet and exit\n```\n\n### Example server YAML definition\n\nThe below example shows how to run both a Webhooks-\u003eKafka and Kafka-\u003eEthereum bridge\nin a single process.\n\nThe server will exit if either bridge stops.\n\nYou can use the `-Y, --print-yaml-config` option on the `kafka` and `webhooks` command\nlines to print out a YAML snippet with detailed configuration - such as TLS mutual auth settings, not included in the example below.\n\n```yaml\nkafka:\n  example-kafka-to-eth:\n    maxTXWaitTime: 60\n    maxInFlight: 25\n    kafka:\n      brokers:\n      - broker-url-1.example.com:9092\n      - broker-url-2.example.com:9092\n      - broker-url-3.example.com:9092\n      topicIn: \"example-requests\"\n      topicOut: \"example-replies\"\n      sasl:\n        username: \"example-sasl-user\"\n        password: \"example-sasl-pass\"\n      tls:\n        enabled: true\n      consumerGroup: \"example-kafka-to-eth-cg\"\n    rpc:\n      url: \"http://localhost:8545\"\nwebhooks:\n  example-webhoooksto-kafka:\n    http:\n      port: 8001\n    mongodb:\n      url: \"localhost:27017/?replicaSet=repl1\"\n      database: \"ethconnect\"\n      collection: \"ethconnect-replies\"\n      maxDocs: 1000\n      queryLimit: 100\n    kafka:\n      brokers:\n      - broker-url-1.example.com:9092\n      - broker-url-2.example.com:9092\n      - broker-url-3.example.com:9092\n      topicIn: \"example-replies\"\n      topicOut: \"example-requests\"\n      sasl:\n        username: \"example-sasl-user\"\n        password: \"example-sasl-pass\"\n      tls:\n        enabled: true\n      consumerGroup: \"example-webhoooksto-kafka-cg\"\n```\n\nNote that 3 separate stores are utilizing LevelDB: OpenAPI configuration, event stream management, and transaction receipts. A sample configuration is provided below.\n\n```yaml\nkafka:\n  kafka-tx-streaming:\n    ...\nrest:\n  rest-gateway:\n    ...\n    openapi:\n      # for OpenAPI configurations\n      storagePath: \"/data/ethconnect/openapi\"\n      # for event stream management (subscriptions, etc.)\n      eventsDB: \"/data/ethconnect/eventsdb\"\n      ...\n    # for transaction receipts when using async endpoints\n    leveldb:\n      path: \"/data/ethconnect/receiptsdb\"\n      queryLimit: 100\n    ...\nplugins:\n  securityModule: \"\"\n```\n\n## Tuning\n\nThe following tuning parameters are currently exposed on the Kafka-\u003eEthereum bridge:\n\n### Maximum messages to hold in-flight (maxinflight)\n\nThis is the maximum number of messages that will be re-submitted if a bridge is\nterminated while processing messages.\n\nIn order for the bridge to get high performance by packing many transactions into a\nsingle block (blocks might only be cut every few seconds), the bridge needs to hold a\nnumber of 'in-flight' messages from Kafka that have been submitted into the Ethereum\nnode, but not yet made it into a block.\n\nAn individual instance of the Kafka-Ethereum bridge will receive messages in order from\none or more Kafka topic partitions. Each message needs to be checked, then submitted\ninto Ethereum. Some might immediate error, and others will be ordered into blocks.\nThe Ethereum node does not order necessarily order transactions into blocks in the order\nthey were sent (apart from for a single sender, where the nonce is used as discussed above).\nHowever, Kafka maintains a single `offset` for each partition that is the point at which\nmessages will be redelivered to the bridge if it fails.\n\nSo the bridge does not `mark` the Kafka `offset` complete until **all** replies up to\nthat offset have been successfully written to the reply topic (with either a transaction\nreceipt or an error).\n\n### Maximum wait time for an individual transaction (tx-timeout)\n\nThis is the maximum amount of time to wait for an _individual_ transaction to enter a block\nafter submission before sending a Kafka error reply back to the sender.\n\nIt is possible to submit transactions into Ethereum that will be accepted, but will never\nmake it into a block:\n- Transactions that require funding, where the funds are not available (pending transactions)\n- Transactions with explicitly set nonce values, that are not the next to be executed (queued transactions)\n\nWhen transactions are being submitted from multiple nodes into a chain, it is also possible\nfor a build-up of transactions to occur such that it can take a significant amount of time\nto get a transaction into a block.\n\nIn the case of a timeout, the transaction hash will be sent back in the `Error` reply\nso that an administrator can later check the state of the transaction in the node.\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhyperledger%2Ffirefly-ethconnect","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhyperledger%2Ffirefly-ethconnect","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhyperledger%2Ffirefly-ethconnect/lists"}