{"id":21835215,"url":"https://github.com/dhi13man/spring-multi-data-source","last_synced_at":"2025-04-14T08:52:01.049Z","repository":{"id":176283212,"uuid":"655263067","full_name":"Dhi13man/spring-multi-data-source","owner":"Dhi13man","description":"To mitigate Spring's limitations with multiple data sources in a single service, this library provides two custom Java annotations that automatically generate all the required bean definition configurations, and package-segregated repositories, for each data source.","archived":false,"fork":false,"pushed_at":"2024-10-08T17:36:12.000Z","size":129,"stargazers_count":12,"open_issues_count":0,"forks_count":1,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-03-27T22:22:51.817Z","etag":null,"topics":["annotation-processor","annotations","database","java","maven","spring","spring-boot"],"latest_commit_sha":null,"homepage":"https://central.sonatype.com/artifact/io.github.dhi13man/spring-multi-data-source","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Dhi13man.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.MD","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE.MD","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-06-18T11:37:48.000Z","updated_at":"2025-02-04T11:12:20.000Z","dependencies_parsed_at":"2024-08-05T17:22:08.306Z","dependency_job_id":null,"html_url":"https://github.com/Dhi13man/spring-multi-data-source","commit_stats":null,"previous_names":["dhi13man/spring-multi-data-source"],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Dhi13man%2Fspring-multi-data-source","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Dhi13man%2Fspring-multi-data-source/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Dhi13man%2Fspring-multi-data-source/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Dhi13man%2Fspring-multi-data-source/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Dhi13man","download_url":"https://codeload.github.com/Dhi13man/spring-multi-data-source/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248852085,"owners_count":21171837,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["annotation-processor","annotations","database","java","maven","spring","spring-boot"],"created_at":"2024-11-27T20:18:34.694Z","updated_at":"2025-04-14T08:52:01.024Z","avatar_url":"https://github.com/Dhi13man.png","language":"Java","funding_links":["https://img.buymeacoffee.com/button-api/?text=Buy%20me%20an%20Ego%20boost\u0026emoji=%F0%9F%98%B3\u0026slug=dhi13man\u0026button_colour=FF5F5F\u0026font_colour=ffffff\u0026font_family=Lato\u0026outline_colour=000000\u0026coffee_colour=FFDD00****","https://www.buymeacoffee.com/dhi13man"],"categories":[],"sub_categories":[],"readme":"# spring-multi-data-source\n\n[![License](https://img.shields.io/github/license/dhi13man/spring-multi-data-source)](https://github.com/Dhi13man/spring-multi-data-source/blob/main/LICENSE)\n[![Contributors](https://img.shields.io/github/contributors-anon/dhi13man/spring-multi-data-source?style=flat)](https://github.com/Dhi13man/spring-multi-data-source/graphs/contributors)\n[![GitHub forks](https://img.shields.io/github/forks/dhi13man/spring-multi-data-source?style=social)](https://github.com/Dhi13man/spring-multi-data-source/network/members)\n[![GitHub Repo stars](https://img.shields.io/github/stars/dhi13man/spring-multi-data-source?style=social)](https://github.com/Dhi13man/spring-multi-data-source/stargazers)\n[![Last Commit](https://img.shields.io/github/last-commit/dhi13man/spring-multi-data-source)](https://github.com/Dhi13man/spring-multi-data-source/commits/main)\n[![GitHub issues](https://img.shields.io/github/issues/dhi13man/spring-multi-data-source)](https://github.com/Dhi13man/spring-multi-data-source/issues)\n[![Build, Format, Test](https://github.com/dhi13man/spring-multi-data-source/actions/workflows/maven.yml/badge.svg)](https://github.com/Dhi13man/spring-multi-data-source/actions)\n\n[![Apache Maven](https://img.shields.io/badge/apache_maven-C71A36?style=for-the-badge\u0026logo=apachemaven\u0026logoColor=white)](https://maven.apache.org/)\n[![Maven Central](https://img.shields.io/maven-central/v/io.github.dhi13man/spring-multi-data-source?style=for-the-badge\u0026link=https%3A%2F%2Fmvnrepository.com%2Fartifact%2Fio.github.dhi13man%2Fspring-multi-data-source)](https://mvnrepository.com/artifact/io.github.dhi13man/spring-multi-data-source)\n\n[![\"Buy Me A Coffee\"](https://img.buymeacoffee.com/button-api/?text=Buy%20me%20an%20Ego%20boost\u0026emoji=%F0%9F%98%B3\u0026slug=dhi13man\u0026button_colour=FF5F5F\u0026font_colour=ffffff\u0026font_family=Lato\u0026outline_colour=000000\u0026coffee_colour=FFDD00****)](https://www.buymeacoffee.com/dhi13man)\n\n[![Medium Article](https://img.shields.io/badge/Medium-12100E?style=for-the-badge\u0026logo=medium\u0026logoColor=white)](https://medium.com/@dhi13man/simplify-multiple-data-source-integration-for-spring-boot-services-c465ce1dcdb6)\n\nSpring Boot has multiple limitations when using multiple data sources in a single service. This\nproject aims to solve those limitations by providing custom annotations that can be used to generate\nthe required Bean-providing configuration classes and repositories during the build process itself,\nwhich the service can then use.\n\nThe best part is that the entirety of the generated code is clean, human-readable, and can be\ndirectly carried over to the relevant packages of the main code if you no longer wish to be tied\ndown to this library in the future.\n\n## Table of Contents\n\n\u003c!-- TOC --\u003e\n* [spring-multi-data-source](#spring-multi-data-source)\n  * [Table of Contents](#table-of-contents)\n  * [Introduction](#introduction)\n  * [Annotations Provided](#annotations-provided)\n    * [@EnableMultiDataSourceConfig](#enablemultidatasourceconfig)\n      * [@EnableMultiDataSourceConfig.DataSourceConfig](#enablemultidatasourceconfigdatasourceconfig)\n    * [@TargetSecondaryDataSource](#targetsecondarydatasource)\n  * [Usage](#usage)\n  * [Building from Source (Maven)](#building-from-source-maven)\n  * [Removing Dependency on spring-multi-data-source without Losing Functionality](#removing-dependency-on-spring-multi-data-source-without-losing-functionality)\n  * [Contributing](#contributing)\n  * [License](#license)\n  * [Resources](#resources)\n\u003c!-- TOC --\u003e\n\n## Introduction\n\nThe limitations of using multiple data sources in a single service in Spring are:\n\n1. We need to split the packages of repositories to allow one `@EnableJpaRepositories` mapped to one\n   package for each data source.\n\n2. There is a lot of boilerplate config generation involved to create beans of data sources, entity\n   managers, transaction managers etc. for each data source.\n\n3. To get `EntityManagerFactoryBuilder` injected, we need to declare one of the data sources and all\n   its beans as `@Primary`. Otherwise, the service won't even start up.\n\nTo mitigate the above limitations, I have created two custom annotations in Java that can be used\nfor configuring multi-data source configurations for a service. Let's break down each annotation:\n\n## Annotations Provided\n\n### @EnableMultiDataSourceConfig\n\n- This annotation is used to enable multi-data source configuration for the service. This will\n  replace the `@EnableJpaRepositories` and `@EntityScan` annotations used by Spring.\n\n- It can be applied to a class (target: `ElementType.TYPE`).\n\n- It has the following attributes:\n    - `exactEntityPackages`: An array of exact packages to scan for entities. These packages are\n      scanned to find the entities related to the data sources.\n    - `repositoryPackages`: An array of packages to scan for repositories. These packages are\n      scanned to find the repositories related to the data sources.\n    - `datasourcePropertiesPrefix`: The prefix of the data source properties in the\n      application properties file. The properties for each data source will be placed under this\n      prefix followed by the kebab case of the data source name. Eg. When set as `spring.datasource`\n      for master and readReplica data sources, the properties will be placed under\n      `spring.datasource.master` and `spring.datasource.read-replica` respectively.\n    - `generatedConfigPackage`: The package where the generated data source configs will\n      be placed. The generated config class with relevant beans will follow a specific naming\n      format. If this is not specified, the generated config will be placed in the same package as\n      the class where this annotation is applied, followed by `.generated.config`.\n    - `generatedRepositoryPackagePrefix`: The prefix of the package where the generated copies\n      of the repositories will be placed. The generated repositories will follow a specific\n      naming format. If this is not specified, the generated repositories will be placed in the\n      same package as the class where this annotation is applied, followed by\n      `.generated.repositories` and then `.\u003cdata_source_name\u003e`.\n    - `primaryDataSourceConfig`: A `@DataSourceConfig` annotation. This annotation represents\n      the primary data source and its configuration. The primary data source will be able\n      to access every repository other than the repositories generated for the secondary data\n      sources.\n    - `secondaryDataSourceConfigs`: An array of `@DataSourceConfig` annotations. Each annotation\n      represents a data source and its configuration. The secondary data sources will only be able\n      to access the repositories generated for them.\n\n#### @EnableMultiDataSourceConfig.DataSourceConfig\n\n- This sub-annotation is used to configure a data source and its properties for\n  `@EnableMultiDataSourceConfig`. It can not be applied directly anywhere other than in\n  the `dataSourceConfigs` attribute of `@EnableMultiDataSourceConfig`.\n\n- It has the following attributes:\n    - `dataSourceName`: The name of the data source. It is used to generate the data source\n      beans and to name the generated classes, packages, and property paths for the data\n      source properties.\n    - `dataSourceClassPropertiesPath`:The application properties key/path of the data source class'\n      properties. Eg. `spring.datasource.hikari` for Hikari data sources.\n    - `overridingPropertiesPath`:  The application properties key/path under which the JPA\n      properties to override for this data source are located. This allows overriding of the JPA\n      properties for each data source. By default, it will take the default `spring.jpa.properties`\n      path.\n\n### @TargetSecondaryDataSource\n\n- This annotation is used to create copies of repositories in relevant packages and\n  autoconfigure them to use the relevant data sources.\n\n- It can be applied to a method (target: `ElementType.METHOD`).\n\n- It has the following attributes:\n    - `dataSourceName` (or `value`): The name of the data source to use for the repository.\n\nBoth annotations are available at the source level and are not retained at runtime. They are\nintended to be used for generating code for configuring data sources during the build process.\n\n## Usage\n\n1. Add `spring-multi-data-source` as a dependency in your service with a scope of `provided`. Eg.\n   for Maven:\n\n   ```xml\n   \u003cdependency\u003e\n     \u003cgroupId\u003ecom.dhi13man.spring\u003c/groupId\u003e\n     \u003cartifactId\u003espring-multi-data-source\u003c/artifactId\u003e\n     \u003cversion\u003e${desired.version}\u003c/version\u003e\n     \u003cscope\u003eprovided\u003c/scope\u003e\n   \u003c/dependency\u003e\n   ```\n\n2. Add the `@EnableMultiDataSourceConfig` annotation to a configuration class in your service, and\n   specify the relevant attributes. At a bare minimum the `exactEntityPackages`\n   and `repositoryPackages` attributes need to be specified. Ensure that you are no longer using\n   `@EnableJpaRepositories` and `@EntityScan` annotations.\n\n   ```java\n   @Configuration\n   @EnableMultiDataSourceConfig(\n      repositoryPackages = {\n        \"com.sample\"\n      },\n      primaryDataSourceConfig = @DataSourceConfig(\n          dataSourceName = \"master\",\n          exactEntityPackages = {\n              \"com.sample.project.sample_service.entities.mysql\",\n              // Assuming master wants access to read entities as well. If not, above package is fine\n              \"com.sample.project.sample_service.read_entities.mysql\",\n              \"com.sample.project.sample_service.read_entities_v2.mysql\"\n          },\n          // In example application properties below (Usage Step 7), extra JPA Properties specific to this data source are provided under this key\n          overridingPropertiesPath = \"spring.datasource.master.extra-properties\"\n      ),\n      secondaryDataSourceConfigs = {\n          @DataSourceConfig(\n              dataSourceName = \"read-replica\",\n              exactEntityPackages = \"com.sample.project.sample_service.read_entities.mysql\"\n          ),\n          @DataSourceConfig(\n              dataSourceName = \"replica-2\",\n              exactEntityPackages = {\n                  \"com.sample.project.sample_service.read_entities.mysql\",\n                  // Assuming replica-2 wants access to read entities as well as read entities v2\n                  \"com.sample.project.sample_service.read_entities_v2.mysql\"\n              }\n          ),\n      }\n   )\n   public class ServiceConfig {\n   }\n   ```\n\n3. Add the `@TargetSecondaryDataSource` annotation to the repository methods that need to be\n   configured for a specific data source, and specify the data source name.\n\n    ```java\n    @Repository\n    public interface ServiceRepository extends JpaRepository\u003cServiceEntity, Long\u003e {\n    \n       @TargetSecondaryDataSource(\"read-replica\")\n       ServiceEntity findByCustomIdAndDate(String id, Date date);\n    \n       // To override the default JpaRepository methods in the generated repository\n       // All base methods that have not been overridden along with this annotation will throw an \n       // UnsupportedOperationException.\n       @TargetSecondaryDataSource(\"read-replica\")\n       @Override\n       ServiceEntity getById(Long id);\n    }\n   ```\n\n4. Build the service and the generated classes will become available in\n   the `target/generated-sources/annotations` directory of the service. Add that folder as a\n   generated sources root in your IDE.\n\n5. The configuration classes generated by the annotation processor will be named\n   `\u003cDataSourceName\u003eDataSourceConfig` and will be placed in the package specified by the\n   `generatedConfigPackage` attribute. These classes will provide the beans for the data\n   source, transaction manager, entity manager factory, etc. for each data source which can be\n   easily autowired with the given name constants.\n\n   For example, if the data source name is `read-replica`, the generated configuration class will be\n   named `ReadReplicaDataSourceConfig` and will be placed in the package given by the\n   `generatedConfigPackage` attribute.\n\n6. The repositories generated by the annotation processor will be named\n   `\u003cDataSourceName\u003e\u003cRepositoryName\u003e` and will be placed in the package specified by the\n   `generatedRepositoryPackagePrefix` attribute followed by the snake case of the data source name.\n   These repositories will be configured to use the relevant data source and can be autowired with\n   the given name constants.\n\n   For example, if the data source name is `read-replica` and the repository name\n   is `ServiceRepository`, the generated repository will be named `ReadReplicaServiceRepository` and\n   will be placed in the package given by the `generatedRepositoryPackagePrefix` attribute followed\n   by `read_replica`.\n\n7. The application data source properties will need to be provided under the key `spring.datasource`\n   followed by the kebab case of the data source name.\n\n   ```yaml\n   spring:\n    datasource:\n      master: # This will become the master data source property as opposed to the usual direct spring.datasource property\n        driver-class-name: com.mysql.cj.jdbc.Driver\n        url: jdbc:mysql://${DB_IP}:${DB_PORT}/${MASTER_DB_NAME}\n        username: ${DB_USERNAME}\n        password: ${DB_PASSWORD}\n        type: com.zaxxer.hikari.HikariDataSource\n        extra-properties:  # Made possible by overridingPropertiesPath in Step 2 for master data source.\n          hibernate.generate_statistics: true  # Generate hibernate statistics only for master data source.\n      read-replica:  # This will become the property for the kebab case of the secondary data source name\n        driver-class-name: com.mysql.cj.jdbc.Driver\n        url: jdbc:mysql://${READ_REPLICA_DB_IP}:${DB_PORT}/${READ_REPLICA_DB_NAME}\n        username: ${DB_USERNAME}\n        password: ${DB_PASSWORD}\n        type: com.zaxxer.hikari.HikariDataSource\n    jpa:\n      # Global JPA Properties\n      properties: \n        # Hibernate properties can only be picked from here when using multiple data sources.\n        hibernate.generate_statistics: false\n        hibernate.physical_naming_strategy: org.springframework.boot.orm.jpa.hibernate.SpringPhysicalNamingStrategy\n        hibernate.implicit_naming_strategy: org.springframework.boot.orm.jpa.hibernate.SpringImplicitNamingStrategy\n   ```\n\n8. Please always go through the generated code to learn more about what configs to give and what\n   beans to use for each data source.\n\n## Building from Source (Maven)\n\n1. Clone the repository.\n2. Run `mvn clean install` to build the project and install it in your local maven repository.\n3. Add the dependency in your project as mentioned above.\n4. Run `mvn clean package` to build the project and generate the jar file.\n5. The jar file will be available in the `target` directory.\n6. Add the jar file as a dependency in your project.\n7. Run `mvn clean compile` or `mvn clean install` in your project to generate the code.\n8. The generated code will be available in the `target/generated-sources/annotations` directory.\n9. Add that directory as a generated sources root in your IDE.\n10. Use the generated code as mentioned above.\n\n## Removing Dependency on spring-multi-data-source without Losing Functionality\n\nA big selling point of this library is that it is not a black box. The generated code is clean,\nhuman-readable, and can be directly carried over to the relevant packages of the main code if you no\nlonger wish to be tied down to this library.\n\n1. Move the generated configuration classes and repositories to the relevant packages in your\n   project from the `target/generated-sources/annotations` directory.\n2. Remove `implements IMultiDataSourceConfig` from the generated `@Configuration` classes.\n3. Remove the `@EnableMultiDataSourceConfig` annotation from your configuration class.\n4. Remove the `@TargetSecondaryDataSource` annotation from your repository methods.\n5. Remove the `spring-multi-data-source` dependency from your project pom.\n\nAnd that's all you have to do! You are no longer tied down to this library and have the freedom to\nuse and modify the generated code to your liking.\n\n## Contributing\n\nPlease feel free to raise issues and submit pull requests. Please check\nout [CONTRIBUTING.md](CONTRIBUTING.md) for more details.\n\n## License\n\nThis project is licensed under the GNU Lesser General Public License v3.0. Please check\nout [LICENSE](LICENSE) for more details.\n\n## Resources\n\n1. [Spring Boot Official Documentation on configuring multiple data sources](https://docs.spring.io/spring-boot/docs/current/reference/htmlsingle/#howto.data-access.configure-two-datasources)\n\n2. [Configuring and Using Multiple DataSources in Spring Boot](https://www.baeldung.com/spring-boot-configure-multiple-datasources)\n\n3. [javapoet (for generating code in Java)](https://github.com/square/javapoet)\n\n4. [Annotation Processing in Java](https://www.baeldung.com/java-annotation-processing-builder)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdhi13man%2Fspring-multi-data-source","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdhi13man%2Fspring-multi-data-source","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdhi13man%2Fspring-multi-data-source/lists"}