{"id":23519784,"url":"https://github.com/ramonbecker/springkafka","last_synced_at":"2026-05-02T20:39:11.692Z","repository":{"id":246042544,"uuid":"462455204","full_name":"RamonBecker/SpringKafka","owner":"RamonBecker","description":null,"archived":false,"fork":false,"pushed_at":"2022-02-26T18:46:44.000Z","size":115,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-10-26T04:38:03.971Z","etag":null,"topics":["kafka","kafka-consumer","kafka-producer","landoop","rest-proxy","spring","spring-boot","springboot","zookeeper"],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/RamonBecker.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-02-22T20:01:53.000Z","updated_at":"2022-02-26T18:47:45.000Z","dependencies_parsed_at":"2024-06-25T14:44:01.312Z","dependency_job_id":"2629e4f4-1127-4266-8e19-184ca78e9c1d","html_url":"https://github.com/RamonBecker/SpringKafka","commit_stats":null,"previous_names":["ramonbecker/springkafka"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/RamonBecker/SpringKafka","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RamonBecker%2FSpringKafka","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RamonBecker%2FSpringKafka/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RamonBecker%2FSpringKafka/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RamonBecker%2FSpringKafka/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/RamonBecker","download_url":"https://codeload.github.com/RamonBecker/SpringKafka/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RamonBecker%2FSpringKafka/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32549385,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-02T19:18:06.202Z","status":"ssl_error","status_checked_at":"2026-05-02T19:16:21.335Z","response_time":132,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["kafka","kafka-consumer","kafka-producer","landoop","rest-proxy","spring","spring-boot","springboot","zookeeper"],"created_at":"2024-12-25T16:10:31.624Z","updated_at":"2026-05-02T20:39:06.684Z","avatar_url":"https://github.com/RamonBecker.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"# SpringKafka\n\n## :information_source: Information \n\nProject developed with Spring, Kafka, Zookeeper, Schema-Registry, Rest-Proxy and Docker. Only the backend part was developed. Consumers and producers were created. In v1, only examples of consumers and products were developed. In v2, three applications were developed: consumers, producers and an application that is both producer and consumer. In these applications avro schemes were created for the consumption and sending of messages. A complete service environment was also created via docker-compose. For docker-compose there are two versions, which can be found in v1 and v2. It is recommended to use v2 with landoop.\n\nSome explanations of the technologies that were used:\n\n- **Schema Registry:** Provides a service layer for message metadata. It provides a RESTful interface to store and retrieve the Avro schemas that were created and it uses Apache Kafka itself as a storage layer.\n\n- **REST Proxy**: Provides a RESTful interface to an Apache Kafka cluster, which facilitates the production and consumption of messages.\n\n- **Apache Zookeeper**: Kafka uses Zookeeper to synchronize settings between different clusters.\n\n- **Consumer:** Read data from any partition, allowing consumption to be scaled according to message production. They can also be organized into Consumer Groups for a given topic. Using Consumer Groups, each Consumer in the group reads from a single partition, so the group as a whole consumes messages from the entire topic.\n\n- **Producer:** They are conceptually simpler than Consumers, as there is no need for group coordination (Consumer Groups). Its main function is to map each message to a topic partition and send a production request to the partition leader.\n\nExample of how Kafka works:\n\n![exemplo](https://user-images.githubusercontent.com/44611131/155854371-5fb86373-5ae8-4839-9897-d446a6f68df0.jpg)\n\n\n\n## ⚠️ Prerequisite\n[![Java Badge](https://img.shields.io/badge/Java-ED8B00?style=for-the-badge\u0026logo=java\u0026logoColor=white)](https://www.oracle.com/br/java/technologies/javase-downloads.html) \u003e= 17\n\n\n![Spring Badge](https://img.shields.io/badge/Spring-6DB33F?style=for-the-badge\u0026logo=spring\u0026logoColor=white)\n\n![Kafka](https://img.shields.io/badge/Apache_Kafka-231F20?style=for-the-badge\u0026logo=apache-kafka\u0026logoColor=white)\n\n![Spring Boot Badge](https://img.shields.io/badge/Spring_Boot-F2F4F9?style=for-the-badge\u0026logo=spring-boot)\n\n![JSON Badge](https://img.shields.io/badge/json-5E5C5C?style=for-the-badge\u0026logo=json\u0026logoColor=white)\n\n![h2-logo-2 (1)](https://user-images.githubusercontent.com/44611131/136869740-c514d30e-d529-4167-a459-4fcd647cce19.png)\n\n![JPA-hibernate (1)](https://img.shields.io/badge/Hibernate-59666C?style=for-the-badge\u0026logo=Hibernate\u0026logoColor=white)\n\n![Docker](https://img.shields.io/badge/Docker-2CA5E0?style=for-the-badge\u0026logo=docker\u0026logoColor=white)\n\n![Postman Badge](https://img.shields.io/badge/Postman-FF6C37?style=for-the-badge\u0026logo=Postman\u0026logoColor=white) OR ![Insomnia Badge](https://img.shields.io/badge/Insomnia-5849be?style=for-the-badge\u0026logo=Insomnia\u0026logoColor=white)\n\n\n\n##  🔧 Install \n\n\n![](https://img.shields.io/badge/Linux-FCC624?style=for-the-badge\u0026logo=linux\u0026logoColor=black)\n\n\n```\ngit clone https://github.com/RamonBecker/SpringKafka.git\n```\n\n![](https://img.shields.io/badge/Windows-0078D6?style=for-the-badge\u0026logo=windows\u0026logoColor=white)\n```\ngit clone https://github.com/RamonBecker/SpringKafka.git\nor install github https://desktop.github.com/ \n```\n\n## 🔨 Docker\n\nBefore cloning the project, you will need to install docker on your operating system.\n\nFor windows, enter the following from the link:\n\n```\nhttps://docs.docker.com/desktop/windows/install/\n```\n\nFor linux, follow the procedure below:\n- Update your existing list of packages:\n\n```\nsudo apt update\n```\n\n- Install some prerequisite packages that let apt use packages over HTTPS:\n\n```\nsudo apt install apt-transport-https ca-certificates curl software-properties-common\n\n```\n\n- Add the GPG key to the official Docker repository on your system:\n\n```\ncurl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -\n```\n- Add the Docker repository to the APT sources:\n\n```\nsudo add-apt-repository \"deb [arch=amd64] https://download.docker.com/linux/ubuntu focal stable\"\n\n```\n\n- Update the package database with Docker packages from the newly added repository:\n\n```\nsudo apt update\n```\n\n- Make sure you are about to install from the Docker repository instead of the default Ubuntu repository:\n\n```\napt-cache policy docker-ce\n```\n\n- Install Docker:\n\n```\nsudo apt install docker-ce\n```\n\n- Check if it is working:\n\n```\nsudo systemctl status docker\n```\n\n\nAfter cloning the project, perform the following\nEnter the services folder, then enter the v2 folder and run the following command:\n\n```\ndocker-compose up   \n```\n\n## Access to Landoop\nBefore accessing the landoop you must run the docker container that is in the service folder in v2\nTo access landoop, access your browser of choice:\n\n```\nhttp://localhost:3030/\n```\n\nWith landoop you will be able to have an overview of your topics, brokers, messages, as well as verify that the following services are working:\n```\n9092 : Kafka Broker\n8081 : Schema Registry\n8082 : Kafka REST Proxy\n2181 : ZooKeeper\n```\n\n![landoop](https://user-images.githubusercontent.com/44611131/155855329-90041f16-1da5-460c-bcf5-76181e57b04b.PNG)\n\n\n## 🔨 POSTMAN\n\nTo test the end points you will need to install POSTMAN or Insomnia. I recommend installing POSTMAN. I'll leave the collections on the DRIVE to facilitate testing.\n\nDownload and import collections into your POSTMAN: \n```\nhttps://drive.google.com/drive/folders/1fJjv9oOSvx3ZhJShdG34Ok3hf_EjV5ZP?usp=sharing\n```\n\n## ⚙️ Testing the project\n\n\nTo test the project you will need to clone the repository on your machine. You can choose both tools: postman or insomnia.\n```\nTo download postman go to the following link: https://www.postman.com/downloads/\n```\n```\nTo download insomnia rentre from the following link: https://insomnia.rest/download\n```\nRemembering that you must choose the operating system corresponding to your machine.\nAfter you have made your registration in these tools, run them and you must type the following in the address: \n\n\nProducer endpoints:\n```\nlocalhost:8181/peoples\n\nlocalhost:8181/cars\n```\nRest-Proxy endpoints:\n\n- Check broker ID \n```\nlocalhost:8082/v3/clusters/\n```\n- Topic creation\n- To create a topic, the broker ID must be consulted\n```\nhttp://localhost:8082/v3/clusters/{idBroker}/topics\n```\n- Check the topics\n- To verify the topic, the broker ID must be queried\n```\nhttp://localhost:8082/v3/clusters/{idBroker}/topics\n```\n\n- Send message to topic\n\n```\nhttp://localhost:8082/topics/topic-01\n```\n- To create a group and a consumer\n```\nhttp://localhost:8082/consumers/consumer-group-01\n```\n\n- To register a consumer\n```\nhttp://localhost:8082/consumers/consumer-group-01/instances/consumidor1/subscription\n```\n\nTo consume some message\n```\nhttp://localhost:8082/consumers/consumer-group-01/instances/consumidor1/records\n```\n## Accessing the H2 database\n\n\nIf you want to modify the user to access the database, modify the application.properties file.\nChange the following:\n```\nspring.datasource.username=user\nspring.datasource.password=password\n```\n\nAccess in browser\n```\nhttp://localhost:8080/h2-console\n```\nIf the JDBC URL is not configured, replace it with:\n```\njdbc:h2:mem:testdb\n```\n![h2](https://user-images.githubusercontent.com/44611131/137225410-277de0f7-cf7c-48b1-b9e4-9be2d5b4ff43.PNG)\n\n\n## :zap: Technologies\t\n\n- Java\n- JPA\n- Kafka\n- Hibernate\n- Spring Boot\n- H2 Database\n- Design Pattern MVC\n- Design Pattern DTO\n- Lombok\n\n## :memo: Developed features\n\n- [x] Consumer creation;\n- [x] Producer creation;\n- [x] Consume message;\n- [x] Produce message;\n- [x] Consult the topics\n\n\n## :technologist:\t Author\n\nBy Ramon Becker 👋🏽 Get in touch!\n\n\n\n[\u003cimg src='https://cdn.jsdelivr.net/npm/simple-icons@3.0.1/icons/github.svg' alt='github' height='40'\u003e](https://github.com/RamonBecker)  [\u003cimg src='https://cdn.jsdelivr.net/npm/simple-icons@3.0.1/icons/linkedin.svg' alt='linkedin' height='40'\u003e](https://www.linkedin.com/in/https://www.linkedin.com/in/ramon-becker-da-silva-96b81b141//)\n![Gmail Badge](https://img.shields.io/badge/-ramonbecker68@gmail.com-c14438?style=flat-square\u0026logo=Gmail\u0026logoColor=white\u0026link=mailto:ramonbecker68@gmail.com)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Framonbecker%2Fspringkafka","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Framonbecker%2Fspringkafka","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Framonbecker%2Fspringkafka/lists"}