{"id":15508354,"url":"https://github.com/tinesoft/spring-esdata-loader","last_synced_at":"2025-04-23T02:20:41.289Z","repository":{"id":57723648,"uuid":"189472414","full_name":"tinesoft/spring-esdata-loader","owner":"tinesoft","description":"Set of JUnit Rules/Extensions to easily load data to test your spring-data elasticsearch-based projects","archived":false,"fork":false,"pushed_at":"2019-10-29T19:47:41.000Z","size":658,"stargazers_count":7,"open_issues_count":1,"forks_count":2,"subscribers_count":5,"default_branch":"develop","last_synced_at":"2024-10-19T21:19:12.647Z","etag":null,"topics":["elasticsearch","hacktoberfest","integration-tests","junit-jupiter","junit4","spring","spring-data-elasticsearch","spring-test"],"latest_commit_sha":null,"homepage":"https://tinesoft.github.io/spring-esdata-loader/","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tinesoft.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":".github/CONTRIBUTING.md","funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":".github/CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null},"funding":{"github":"tinesoft","patreon":null,"open_collective":null,"ko_fi":null,"tidelift":null,"custom":null}},"created_at":"2019-05-30T19:42:55.000Z","updated_at":"2024-09-21T15:01:54.000Z","dependencies_parsed_at":"2022-09-14T01:34:56.586Z","dependency_job_id":null,"html_url":"https://github.com/tinesoft/spring-esdata-loader","commit_stats":null,"previous_names":[],"tags_count":5,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tinesoft%2Fspring-esdata-loader","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tinesoft%2Fspring-esdata-loader/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tinesoft%2Fspring-esdata-loader/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tinesoft%2Fspring-esdata-loader/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tinesoft","download_url":"https://codeload.github.com/tinesoft/spring-esdata-loader/tar.gz/refs/heads/develop","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250354788,"owners_count":21416786,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["elasticsearch","hacktoberfest","integration-tests","junit-jupiter","junit4","spring","spring-data-elasticsearch","spring-test"],"created_at":"2024-10-02T09:37:48.635Z","updated_at":"2025-04-23T02:20:41.255Z","avatar_url":"https://github.com/tinesoft.png","language":"Java","funding_links":["https://github.com/sponsors/tinesoft"],"categories":[],"sub_categories":[],"readme":"\n# spring-esdata-loader\n[![Build  Status](https://travis-ci.org/tinesoft/spring-esdata-loader.svg?branch=master)](https://travis-ci.org/tinesoft/spring-esdata-loader)\n[![codebeat badge](https://codebeat.co/badges/3c8b3893-a79e-4e3d-b769-a9285c771be2)](https://codebeat.co/projects/github-com-tinesoft-spring-esdata-loader-master)\n![GitHub](https://img.shields.io/github/license/tinesoft/spring-esdata-loader)\n\n`spring-esdata-loader` is a Java 8+ testing library to help you write integration tests for your [spring-data elasticsearch](https://spring.io/projects/spring-data-elasticsearch)-based projects, by allowing you to easily load data into Elasticsearch, using entity mappings (i.e domain classes annotated with `@Document`, `@Field`, etc) and via specific **Junit 4**'s Rules or **JUnit Jupiter**'s Extensions.\n\nThe library reads all the metadata it needs from the entity classes (index name, index type, etc) , uses them to create/refresh the index on the ES server and feeds it with the data using the `ElasticsearchOperations` present in your test application context.\n\n## Features\n\n* **Simple API** and no configuration required\n* Support  for  **JUnit 4** via `@LoadEsDataRule`, `@DeleteEsDataRule`\n* Support  for  **JUnit  Jupiter** via \n  * `@LoadEsDataConfig` / `@LoadEsDataExtension` \n  * `@DeleteEsDataConfig` / `@DeleteEsDataExtension`\n* Built-in support for **gzipped data**\n* **Multiple data formats**(dump, manual)\n* Written  in  **Java  8**\n* Based on **Spring (Data, Test)**\n\n## Dependencies\n\n`spring-esdata-loader` is based on dependencies that you already have in your Spring (Boot) project, if you are doing Elasticsearch with Spring :\n\n* [Spring  Data  Elasticsearch](https://mvnrepository.com/artifact/org.springframework.data/spring-data-elasticsearch) (*requires* Spring Data Elasticsearch 3+, tested with [v3.1.8.RELEASE](https://mvnrepository.com/artifact/org.springframework.data/spring-data-elasticsearch/3.1.8.RELEASE )\n* [Spring  Test](https://mvnrepository.com/artifact/org.springframework/spring-test) (*requires* Spring Test 5+, tested with [5.1.7.RELEASE](https://mvnrepository.com/artifact/org.springframework/spring-test/5.1.7.RELEASE)\n\n## Installation \u0026 Usage\n\nThe library is split into 2 independent sub-modules, both are available on [JCenter](https://bintray.com/bintray/jcenter?filterByPkgName=spring-esdata-loader) and [Maven Central](https://search.maven.org/search?q=spring-esdata-loader):\n\n* `spring-esdata-loader-junit4` for testing with **JUnit 4**\n* `spring-esdata-loader-junit-jupiter` for testing with **JUnit Jupiter**\n\nTo get started,\n\n1. add the appropriate dependency to your gradle or maven project\n\n\u003ctable\u003e\n    \u003ctr\u003e\n        \u003cth\u003e\u003c/th\u003e\n        \u003cth\u003eGradle\u003c/th\u003e\n        \u003cth\u003eMaven\u003c/th\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n        \u003ctd\u003eJUnit 4\u003c/td\u003e\n        \u003ctd\u003e\n\u003cpre lang=\"groovy\"\u003edependencies {\n    testImplementation 'com.github.spring-esdata-loader:spring-esdata-loader-junit4:1.1.0'\n}\u003c/pre\u003e\n        \u003c/td\u003e\n        \u003ctd\u003e\n\u003cpre lang=\"xml\"\u003e\u0026lt;dependency\u0026gt;\n    \u0026lt;groupId\u0026gt;com.github.spring-esdata-loader\u0026lt;/groupId\u0026gt;\n    \u0026lt;artifactId\u003espring-esdata-loader-junit4\u0026lt;/artifactId\u0026gt;\n    \u0026lt;version\u003e1.1.0\u0026lt;/version\u0026gt;\n    \u0026lt;scope\u003etest\u0026lt;/scope\u0026gt;\n\u0026lt;/dependency\u0026gt;\u003c/pre\u003e\n        \u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n        \u003ctd\u003eJUnit Jupiter\u003c/td\u003e\n        \u003ctd\u003e\n\u003cpre lang=\"groovy\"\u003edependencies {\n    testImplementation 'com.github.spring-esdata-loader:spring-esdata-loader-junit-jupiter:1.1.0'\n}\u003c/pre\u003e\n        \u003c/td\u003e\n        \u003ctd\u003e\n\u003cpre lang=\"xml\"\u003e\u0026lt;dependency\u0026gt;\n    \u0026lt;groupId\u0026gt;com.github.spring-esdata-loader\u0026lt;/groupId\u0026gt;\n    \u0026lt;artifactId\u003espring-esdata-loader-junit-jupiter\u0026lt;/artifactId\u0026gt;\n    \u0026lt;version\u003e1.1.0\u0026lt;/version\u0026gt;\n    \u0026lt;scope\u003etest\u0026lt;/scope\u0026gt;\n\u0026lt;/dependency\u0026gt;\u003c/pre\u003e\n        \u003c/td\u003e\n    \u003c/tr\u003e\n\u003c/table\u003e\n\n2. write your test class. You can have a look at:\n\n* [junit4](/junit4) - if you are using **JUnit 4**\n* [junit-jupiter](/junit-jupiter) - if you are using **JUnit Jupiter**\n\n## Supported Data Formats\n\n`spring-esdata-loader` currently supports 2 formats to load data into Elasticsearch: **DUMP** and **MANUAL**.\n\n### Dump data format\n\nHere is an example:\n```json\n{\"_index\":\"author\",\"_type\":\"Author\",\"_id\":\"1\",\"_score\":1,\"_source\":{\"id\":\"1\",\"firstName\":\"firstName1\",\"lastName\":\"lastName1\"}}\n{\"_index\":\"author\",\"_type\":\"Author\",\"_id\":\"2\",\"_score\":1,\"_source\":{\"id\":\"2\",\"firstName\":\"firstName2\",\"lastName\":\"lastName2\"}}\n{\"_index\":\"author\",\"_type\":\"Author\",\"_id\":\"3\",\"_score\":1,\"_source\":{\"id\":\"3\",\"firstName\":\"firstName3\",\"lastName\":\"lastName3\"}}\n{\"_index\":\"author\",\"_type\":\"Author\",\"_id\":\"4\",\"_score\":1,\"_source\":{\"id\":\"4\",\"firstName\":\"firstName4\",\"lastName\":\"lastName4\"}}\n{\"_index\":\"author\",\"_type\":\"Author\",\"_id\":\"5\",\"_score\":1,\"_source\":{\"id\":\"5\",\"firstName\":\"firstName5\",\"lastName\":\"lastName5\"}}\n{\"_index\":\"author\",\"_type\":\"Author\",\"_id\":\"6\",\"_score\":1,\"_source\":{\"id\":\"6\",\"firstName\":\"firstName6\",\"lastName\":\"lastName6\"}}\n{\"_index\":\"author\",\"_type\":\"Author\",\"_id\":\"7\",\"_score\":1,\"_source\":{\"id\":\"7\",\"firstName\":\"firstName7\",\"lastName\":\"lastName7\"}}\n{\"_index\":\"author\",\"_type\":\"Author\",\"_id\":\"8\",\"_score\":1,\"_source\":{\"id\":\"8\",\"firstName\":\"firstName8\",\"lastName\":\"lastName8\"}}\n{\"_index\":\"author\",\"_type\":\"Author\",\"_id\":\"9\",\"_score\":1,\"_source\":{\"id\":\"9\",\"firstName\":\"firstName9\",\"lastName\":\"lastName9\"}}\n{\"_index\":\"author\",\"_type\":\"Author\",\"_id\":\"10\",\"_score\":1,\"_source\":{\"id\":\"10\",\"firstName\":\"firstName10\",\"lastName\":\"lastName10\"}}\n\n```\nYou can use a tool like [elasticdump](https://npmjs.com/package/elasticdump) (*requires* [NodeJS](https://nodejs.org/)) to extract existing data\nfrom your Elasticsearch server, and them dump them into a JSON file.\n\n```\n$ npx elasticdump --input=http://localhost:9200/my_index --output=my_index_data.json\n```\n\n\u003e The above command will run `elasticdump` to extract data from an index named `my_index` on a ES server located at http://localhost:9200 and then save the result into a file named `my_index_data.json`\n\n\u003e If you change the `--output` part above into `--output=$ | gzip my_data.json.gz` the data will be automatically gzipped\n\n### Manual data format\n\nIn this format, you specify your target data directly (no metadata like `_index`, `_source`, ...), as an Array of JSON objects.\n\nThis is more suitable when you create test data from scratch (as opposed to dumping existing ones from a ES server) because it is easier to tweak later on to accommodate future modifications in tests.\n\nHere is an example:\n```json\n[\n    {\"id\":\"1\",\"firstName\":\"firstName1\",\"lastName\":\"lastName1\"},\n    {\"id\":\"2\",\"firstName\":\"firstName2\",\"lastName\":\"lastName2\"},\n    {\"id\":\"3\",\"firstName\":\"firstName3\",\"lastName\":\"lastName3\"},\n    {\"id\":\"4\",\"firstName\":\"firstName4\",\"lastName\":\"lastName4\"},\n    {\"id\":\"5\",\"firstName\":\"firstName5\",\"lastName\":\"lastName5\"},\n    {\"id\":\"6\",\"firstName\":\"firstName6\",\"lastName\":\"lastName6\"},\n    {\"id\":\"7\",\"firstName\":\"firstName7\",\"lastName\":\"lastName7\"},\n    {\"id\":\"8\",\"firstName\":\"firstName8\",\"lastName\":\"lastName8\"},\n    {\"id\":\"9\",\"firstName\":\"firstName9\",\"lastName\":\"lastName9\"},\n    {\"id\":\"10\",\"firstName\":\"firstName10\",\"lastName\":\"lastName10\"}\n]\n```\n## Contributing\n\nContributions are always welcome! Just fork the project, work on your feature/bug fix, and submit it.\nYou can also contribute by creating issues. Please read the [contribution guidelines](.github/CONTRIBUTING.md) for more information.\n\n## License\n\nCopyright (c) 2019  Tine Kondo. Licensed under the MIT License (MIT)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftinesoft%2Fspring-esdata-loader","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftinesoft%2Fspring-esdata-loader","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftinesoft%2Fspring-esdata-loader/lists"}