{"id":25158444,"url":"https://github.com/zt-9/go-kafka-example","last_synced_at":"2025-04-03T12:42:28.697Z","repository":{"id":205387078,"uuid":"714108393","full_name":"zt-9/go-kafka-example","owner":"zt-9","description":"Data processing with Apache Kafka, REST API and Redis","archived":false,"fork":false,"pushed_at":"2023-11-09T23:34:31.000Z","size":50,"stargazers_count":3,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-02-09T01:51:41.525Z","etag":null,"topics":["apache-kafka","blockchain","echo-framework","golang","redis","rest-api"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/zt-9.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-11-04T00:03:23.000Z","updated_at":"2024-12-04T04:03:21.000Z","dependencies_parsed_at":"2024-07-25T07:43:15.928Z","dependency_job_id":null,"html_url":"https://github.com/zt-9/go-kafka-example","commit_stats":null,"previous_names":["zoey-t/go-kafka-example","zt-9/go-kafka-example"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zt-9%2Fgo-kafka-example","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zt-9%2Fgo-kafka-example/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zt-9%2Fgo-kafka-example/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zt-9%2Fgo-kafka-example/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/zt-9","download_url":"https://codeload.github.com/zt-9/go-kafka-example/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247005379,"owners_count":20868019,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["apache-kafka","blockchain","echo-framework","golang","redis","rest-api"],"created_at":"2025-02-09T01:51:44.770Z","updated_at":"2025-04-03T12:42:28.677Z","avatar_url":"https://github.com/zt-9.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# go-kafka-example\nData processing with Apache Kafka, REST API and Redis.\n\nThis project involves processing on-chain blockchain data using Apache Kafka, REST API, and Redis.\nThere are three topics: \"address\", \"label\" and \"transaction\". These topics are handled by a single producer and consumer.\n\nThe producer, operating on port 8080, receives REST API requests from users, sending the respective messages to the consumer (port 8081). Redis is used as the in-memory database.\n\n\n## Prerequisites\n- docker ([install](https://docs.docker.com/engine/install/))\n## stack\n- [bitnami/kafka](https://hub.docker.com/r/bitnami/kafka): kafka docker image\n- [ibm/sarama](https://github.com/IBM/sarama): Go library for Apache Kafka\n- [echo](https://echo.labstack.com/): Go web framework used to build REST APIs\n- [redis](https://redis.uptrace.dev/guide/go-redis.html): for in-memory database\n## Run the application\n1. Start docker: \n    ```bash\n    make run/docker\n    ```\n2. Start the producer in a new terminal\n    ```bash\n    make run/producer\n    ```\n3. Start the consumer in a new terminal\n    ```bash\n    make run/consumer\n    ```\n4. Send API requests to `localhost:8080`\n   \n    For example:\n   \n    - create an address:\n        send POST request to `localhost:8080/addresses/0x87631B45877794f9cdd50a70c827403e3C36d072`\n        with body\n        ```json\n        {\n        \"address\": \"0x87631B45877794f9cdd50a70c827403e3C36d072\",\n        \"labels\": [\"eoa\"]\n        \n        }\n        ```\n\n    - get an address:\n        send GET request to `localhost:8080/addresses/0x87631B45877794f9cdd50a70c827403e3C36d072`\n     - create a transaction: \n       send POST request to `localhost:8080/transactions/0x6ffa912cc7da2b5ec51a2cc1152ab39a54f0c72f1b3f32072c9bba154b585780` with body\n       ```json\n        {\n        \"hash\":\"0x6ffa912cc7da2b5ec51a2cc1152ab39a54f0c72f1b3f32072c9bba154b585780\",\n        \"chainid\": 2,\n        \"from\": \"0x4838B106FCe9647Bdf1E7877BF73cE8B0BAD5f97\",\n        \"to\":\"0x388C818CA8B9251b393131C08a736A67ccB19297\",\n        \"status\":\"success\"\n        }\n       ```\n## REST API endpoints\n- port: `localhost:8080`\n- `/addresses`: `GET`\n- `/addresses/:address`: `GET`, `POST`, `PUT`, `DELETE`\n- `/labels`: `GET`\n- `/labels/:label`: `GET`, `POST`, `PUT`, `DELETE`\n- `/transactions`: `GET`\n- `/transactions/:hash`: `GET`, `POST`, `PUT`, `DELETE`\n\n## Project structure\n- `/cmd`\nmain applications for this project\n- `/cmd/producer`\nkafka producer. it sends messages to kafka.\n- `/cmd/consumer`\nkafka consumer. It process the received \n- `/config`\nconfiguration\n- `/controllers`\ncontrollers that handles api endpoints request\n- `/models`\ndata structures and models used in REST API\n- `/utils`\nutil/helper functions\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzt-9%2Fgo-kafka-example","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fzt-9%2Fgo-kafka-example","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzt-9%2Fgo-kafka-example/lists"}