{"id":16179181,"url":"https://github.com/jonashackt/spring-rabbitmq-messaging-microservices","last_synced_at":"2025-03-16T10:31:37.220Z","repository":{"id":38419060,"uuid":"156265202","full_name":"jonashackt/spring-rabbitmq-messaging-microservices","owner":"jonashackt","description":"Example project showing how to build a scalable microservice architecture using Spring Boot \u0026 RabbitMQ","archived":false,"fork":false,"pushed_at":"2025-03-05T02:35:27.000Z","size":734,"stargazers_count":37,"open_issues_count":9,"forks_count":16,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-03-16T01:23:44.869Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jonashackt.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-11-05T18:40:22.000Z","updated_at":"2025-03-05T02:35:30.000Z","dependencies_parsed_at":"2024-02-08T22:40:27.586Z","dependency_job_id":"11c96cb2-75f0-40d9-95cd-926db36ee6f8","html_url":"https://github.com/jonashackt/spring-rabbitmq-messaging-microservices","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jonashackt%2Fspring-rabbitmq-messaging-microservices","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jonashackt%2Fspring-rabbitmq-messaging-microservices/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jonashackt%2Fspring-rabbitmq-messaging-microservices/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jonashackt%2Fspring-rabbitmq-messaging-microservices/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jonashackt","download_url":"https://codeload.github.com/jonashackt/spring-rabbitmq-messaging-microservices/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243858811,"owners_count":20359257,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-10T05:25:47.945Z","updated_at":"2025-03-16T10:31:36.760Z","avatar_url":"https://github.com/jonashackt.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"spring-rabbitmq-messaging-microservices\n======================================================================================\n[![Build Status](https://github.com/jonashackt/spring-rabbitmq-messaging-microservices/workflows/build/badge.svg)](https://github.com/jonashackt/spring-rabbitmq-messaging-microservices/actions)\n[![renovateenabled](https://img.shields.io/badge/renovate-enabled-yellow)](https://renovatebot.com)\n\nExample project showing how to build a scalable microservice architecture using Spring Boot \u0026amp; RabbitMQ\n\n![spring-rabbitmq-messaging-diagram](https://yuml.me/diagram/scruffy/class/[weatherservice]-\u003e[RabbitMQ],[weatherservice]^-.-[RabbitMQ],[RabbitMQ]-\u003e[weatherbackend03],[RabbitMQ]^-.-[weatherbackend03],[RabbitMQ]-\u003e[weatherbackend02],[RabbitMQ]^-.-[weatherbackend02],[RabbitMQ]-\u003e[weatherbackend01],[RabbitMQ]^-.-[weatherbackend01])\n\nWe´re using [RabbitMQ Docker image](https://hub.docker.com/_/rabbitmq/) here. So if you fire it up with `docker-compose up -d`, you can easily login to the management gui at http://localhost:15672 using `guest` \u0026 `guest` as credentials.\n\n### Testcontainers only inside weatherbackend\n\n![spring-rabbitmq-messaging-diagram](https://yuml.me/diagram/scruffy/class/[weatherbackend]-\u0026gt;[RabbitMQ],[weatherbackend]^-.-[RabbitMQ])\n\nAlthough we could also use [docker-compose.yml](docker-compose.yml) right here in the weatherbackend test classes, this could lead to errors - because [testcontainers](https://www.testcontainers.org/) would also try to spin up a `weatherbackend` Docker container, which we don´t have at build time of the weatherbackend itself (cause the Spring Boot jar isn´t ready right then).\n\nBut there´s something like the `org.testcontainers.containers.GenericContainer` we can use to spin up a RabbitMQ without a docker-compose.yml. Just have a look into the test class [SendAndReceiveTest](weatherbackend/src/test/java/de/jonashackt/SendAndReceiveTest.java):\n\n```\n...\nimport org.junit.ClassRule;\nimport org.junit.Rule;\n...\nimport org.junit.runner.RunWith;\nimport org.springframework.beans.factory.annotation.Autowired;\nimport org.springframework.boot.test.context.SpringBootTest;\nimport org.springframework.boot.test.util.TestPropertyValues;\nimport org.springframework.context.ApplicationContextInitializer;\nimport org.springframework.context.ConfigurableApplicationContext;\nimport org.springframework.test.context.ContextConfiguration;\nimport org.springframework.test.context.junit4.SpringRunner;\nimport org.testcontainers.containers.GenericContainer;\nimport org.testcontainers.containers.wait.strategy.Wait;\n...\n\n@RunWith(SpringRunner.class)\n@SpringBootTest(classes = WeatherBackendApplication.class)\n@ContextConfiguration(initializers = {SendAndReceiveTest.Initializer.class})\npublic class SendAndReceiveTest {\n\n    static class Initializer implements ApplicationContextInitializer\u003cConfigurableApplicationContext\u003e {\n\n        @Override\n        public void initialize(ConfigurableApplicationContext configurableApplicationContext) {\n\n            TestPropertyValues.of(\n                    \"spring.rabbitmq.host=\" + rabbitMq.getContainerIpAddress(),\n                    \"spring.rabbitmq.port=\" + rabbitMq.getMappedPort(5672))\n                    .applyTo(configurableApplicationContext.getEnvironment());\n        }\n    }\n\n    @ClassRule\n    public static GenericContainer rabbitMq = new GenericContainer(\"rabbitmq:management\")\n            .withExposedPorts(5672)\n            .waitingFor(Wait.forListeningPort());\n```\n\nAs Testcontainers doesn´t guarantee that the RabbitMQ container is reachable under the host name `localhost` (see https://github.com/testcontainers/testcontainers-java/issues/669#issuecomment-385873331) and therefore the test execution leads to `ConnectionRefused` Exceptions from the Spring Boot Autoconfiguraiton trying to reach RabbitMQ on this host, we need to go a different path.\n\nBut since we need to configure the RabbitMQ host url and port in the early stage of SpringBootTest initialization, we need the help of a `org.springframework.context.ConfigurableApplicationContext` to dynamically set our `spring.rabbitmq.host` property. Togehter with [TestPropertyValues](https://dzone.com/articles/testcontainers-and-spring-boot) this could be done easily as seen in the code.\n\n\n### Testcontainers, the 'real' docker-compose.yml and the weatherservice\n\nAt the weatherservice we´re also using [testcontainers](https://www.testcontainers.org/) to fully instanciate every microservice needed to test the whole interaction with RabbitMQ:\n\n![spring-rabbitmq-messaging-diagram](https://yuml.me/diagram/scruffy/class/[weatherservice]-\u003e[RabbitMQ],[weatherservice]^-.-[RabbitMQ],[RabbitMQ]-\u003e[weatherbackend],[RabbitMQ]^-.-[weatherbackend])\n\nTherefore the sequence of module build inside our [pom.xml](pom.xml) here is crucial:\n\n```\n\t\u003cmodules\u003e\n\t\t\u003cmodule\u003eweathermodel\u003c/module\u003e\n\t\t\u003cmodule\u003eweatherbackend\u003c/module\u003e\n\t\t\u003cmodule\u003eweatherservice\u003c/module\u003e\n\t\u003c/modules\u003e\n```\nFirst the shared domain \u0026 event classes are packaged into a .jar file, so that every service is able to use it.\n\nThen the weatherbackend is build and tested - which does everything in the context of one microservice.\n\nThe final `weatherservice` build then uses the successful build output of the `weatherbackend` inside the corresponding [Dockerfile](weatherbackend/Dockerfile):\n\n```\n...\n# Add Spring Boot app.jar to Container\nADD \"target/weatherbackend-0.0.1-SNAPSHOT.jar\" app.jar\n...\n```\n\nNow the service definition inside the [docker-compose.yml](docker-compose.yml) again uses that Dockerfile to spin up a microservice Docker container containing the `weatherbackend`:\n\n```\nversion: '3.7'\n\nservices:\n\n rabbitmq:\n  image: rabbitmq:management\n  ports:\n    - \"5672:5672\"\n    - \"15672:15672\"\n  tty:\n    true\n\n weatherbackend:\n  build: ./weatherbackend\n  ports:\n    - \"8090\"\n  environment:\n    - \"SPRING.RABBITMQ.HOST=rabbitmq\"\n  tty:\n    true\n  restart:\n    unless-stopped\n```\n\nNote the definition of the environment variable `spring.rabbitmq.host`, since the RabbitMQ containers´ host inside the weatherbackend´s Docker Container isn´t `localhost` but instead Docker DNS style `rabbitmq`!\n\nThe Test class [WeatherServiceSendAndReceiveTest](weatherservice/src/test/java/de/jonashackt/WeatherServiceSendAndReceiveTest.java) uses `org.testcontainers.containers.DockerComposeContainer` to leverage to `real` docker-compose.yml:\n\n```\npackage de.jonashackt;\n\nimport com.fasterxml.jackson.core.JsonProcessingException;\nimport de.jonashackt.messaging.MessageSender;\nimport org.junit.ClassRule;\nimport org.junit.Rule;\nimport org.junit.Test;\nimport org.junit.contrib.java.lang.system.SystemOutRule;\nimport org.junit.runner.RunWith;\nimport org.springframework.beans.factory.annotation.Autowired;\nimport org.springframework.boot.test.context.SpringBootTest;\nimport org.springframework.test.context.junit4.SpringRunner;\nimport org.testcontainers.containers.DockerComposeContainer;\nimport org.testcontainers.containers.wait.strategy.Wait;\n\nimport java.io.File;\n\nimport static de.jonashackt.common.ModelUtil.exampleEventGetOutlook;\nimport static de.jonashackt.messaging.Queues.QUEUE_WEATHER_BACKEND;\nimport static org.hamcrest.Matchers.containsString;\nimport static org.junit.Assert.assertThat;\n\n@RunWith(SpringRunner.class)\n@SpringBootTest(classes = WeatherServiceApplication.class)\npublic class WeatherServiceSendAndReceiveTest {\n\n    @ClassRule\n    public static DockerComposeContainer services =\n            new DockerComposeContainer(new File(\"../docker-compose.yml\"))\n                    .withExposedService(\"rabbitmq\", 5672, Wait.forListeningPort())\n                    .withExposedService(\"weatherbackend\", 8090, Wait.forListeningPort());\n\n    @Rule\n    public final SystemOutRule systemOutRule = new SystemOutRule().enableLog();\n\n    @Autowired\n    private MessageSender messageSender;\n\n    @Test\n    public void is_EventGetOutlook_send_and_EventGeneralOutlook_received() throws JsonProcessingException, InterruptedException {\n\n        messageSender.sendMessage(QUEUE_WEATHER_BACKEND, exampleEventGetOutlook());\n\n        Thread.sleep(5000); // We have to wait a bit here, since our Backend needs 3+ seconds to calculate the outlook\n\n        assertThat(systemOutRule.getLog(), containsString(\"EventGeneralOutlook received in weatherservice.\"));\n    }\n}\n```\n\n\n### Scale weatherbackend \u0026 observe, which retrieves the Events with Elastic stack\n\nTo scale the weatherbackend Docker Containers, we can easily facilitate Docker Compose services scaling:\n\n```\ndocker-compose up -d --scale weatherbackend=3\n```\n\nNow we have 3 weatherbackends, as the original architecture diagram suggests:\n\n![spring-rabbitmq-messaging-diagram](https://yuml.me/diagram/scruffy/class/[RabbitMQ]-\u003e[weatherbackend03],[RabbitMQ]^-.-[weatherbackend03],[RabbitMQ]-\u003e[weatherbackend02],[RabbitMQ]^-.-[weatherbackend02],[RabbitMQ]-\u003e[weatherbackend01],[RabbitMQ]^-.-[weatherbackend01])\n\nIf we fire up our `weatherservice` now, we can send events that one of the weatherbackends will retrieve. But which is retrieving which event? We need to use log correlation like with the [Elastic stack](https://www.elastic.co/) for that. The easiest way to do so, is to use https://github.com/jonashackt/docker-elk. Just clone this repo and do another `docker-compose up -d`.\n\nTo connect our microservices to the Elastic stack, there are multiple possibilties. An easy way is to use the [logstash-logback-encoder](https://github.com/logstash/logstash-logback-encoder) and configure it via a `logback-spring.xml` inside the `resources` directory in each app:\n\n```\n\u003c?xml version=\"1.0\" encoding=\"UTF-8\"?\u003e\n\u003cconfiguration\u003e\n    \u003cinclude resource=\"org/springframework/boot/logging/logback/base.xml\"/\u003e\n    \u003clogger name=\"org.springframework\" level=\"WARN\"/\u003e\n\t\u003clogger name=\"de.jonashackt\" level=\"DEBUG\"/\u003e\n\n    \u003c!-- Logstash-Configuration --\u003e\n\t\u003c!-- For details see https://github.com/logstash/logstash-logback-encoder --\u003e\n\t\u003cappender name=\"logstash\" class=\"net.logstash.logback.appender.LogstashTcpSocketAppender\"\u003e\n\t\t\u003cdestination\u003elocalhost:5000\u003c/destination\u003e\n\t\t\u003c!-- encoder is required --\u003e\n\t   \u003cencoder class=\"net.logstash.logback.encoder.LogstashEncoder\"\u003e\n\t   \t\u003cincludeCallerData\u003etrue\u003c/includeCallerData\u003e\n\t   \t\u003ccustomFields\u003e{\"service_name\":\"weatherservice\"}\u003c/customFields\u003e\n\t   \t\u003cfieldNames\u003e\n\t   \t\t\u003cmessage\u003elog-msg\u003c/message\u003e\n\t   \t\u003c/fieldNames\u003e\n\t   \u003c/encoder\u003e\n\t   \u003ckeepAliveDuration\u003e5 minutes\u003c/keepAliveDuration\u003e\n\t\u003c/appender\u003e\n\t\n\t\u003croot level=\"INFO\"\u003e\n\t    \u003cappender-ref ref=\"logstash\" /\u003e\n\t\u003c/root\u003e\n\t\n\u003c/configuration\u003e\n```\n\nNow with everything in place, fire a request to `weatherservice` with ``.\n\nOpen up Kibana after successful startup at http://localhost:5601/app/kibana and first create an index pattern after in __Management/Index Patterns__ called: `logstash-*`. Then click __Next step__ and choose `@timestamp` from the dropdown. Finally click __Create index pattern__. Then head over to __Discover__. Now fire up some events after starting `weatherservice`:\n\n```\ncurl -v localhost:8095/event\n# or 100 events like this\ncurl -v localhost:8095/events/100\n```\n\nNow we should see our services working:\n\n![events-in-kibana](screenshots/kibana-logs.png)\n\n\n### Scale Containers automatically depending on workload\n\nInitialize local Swarm mode:\n\n```\ndocker swarm init\n```\n\nNow deploy our application as Docker Stack\n\n```\ndocker stack deploy --compose-file docker-stack.yml {{ application_stack_name }}\n```\n\n\n### Architects heaven: GitHub + Diagram + Markdown-Code\n\nShould be easy right?! I tried: https://yuml.me/diagram/scruffy/class/samples and there´s also a nice editor:\n\n[https://yuml.me/diagram/scruffy/class/edit/](https://yuml.me/diagram/scruffy/class/edit/[weatherservice]-\u003e[RabbitMQ],[weatherservice]^-.-[RabbitMQ],[RabbitMQ]-\u003e[weatherbackend03],[RabbitMQ]^-.-[weatherbackend03],[RabbitMQ]-\u003e[weatherbackend02],[RabbitMQ]^-.-[weatherbackend02],[RabbitMQ]-\u003e[weatherbackend01],[RabbitMQ]^-.-[weatherbackend01])","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjonashackt%2Fspring-rabbitmq-messaging-microservices","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjonashackt%2Fspring-rabbitmq-messaging-microservices","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjonashackt%2Fspring-rabbitmq-messaging-microservices/lists"}