{"id":37071467,"url":"https://github.com/borys25ol/elasticsearch-reindex","last_synced_at":"2026-01-14T08:23:08.711Z","repository":{"id":42504748,"uuid":"392319207","full_name":"borys25ol/elasticsearch-reindex","owner":"borys25ol","description":"Tool for migration Elasticsearch indexes between different nodes.","archived":false,"fork":false,"pushed_at":"2024-11-19T11:31:50.000Z","size":108,"stargazers_count":7,"open_issues_count":0,"forks_count":2,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-09-01T01:15:26.698Z","etag":null,"topics":["cli","elasticsearch","migration-tool","python","reindex"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/borys25ol.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-08-03T13:00:54.000Z","updated_at":"2024-12-13T08:30:30.000Z","dependencies_parsed_at":"2022-09-15T14:42:04.870Z","dependency_job_id":null,"html_url":"https://github.com/borys25ol/elasticsearch-reindex","commit_stats":null,"previous_names":[],"tags_count":4,"template":false,"template_full_name":null,"purl":"pkg:github/borys25ol/elasticsearch-reindex","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/borys25ol%2Felasticsearch-reindex","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/borys25ol%2Felasticsearch-reindex/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/borys25ol%2Felasticsearch-reindex/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/borys25ol%2Felasticsearch-reindex/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/borys25ol","download_url":"https://codeload.github.com/borys25ol/elasticsearch-reindex/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/borys25ol%2Felasticsearch-reindex/sbom","scorecard":{"id":248593,"data":{"date":"2025-08-11","repo":{"name":"github.com/borys25ol/elasticsearch-reindex","commit":"5478a2b493973cc3e96551a7386f6a96292acc97"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":3.4,"checks":[{"name":"Packaging","score":-1,"reason":"packaging workflow not detected","details":["Warn: no GitHub/GitLab publishing workflow detected."],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"Dangerous-Workflow","score":10,"reason":"no dangerous workflow patterns detected","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"Code-Review","score":1,"reason":"Found 1/6 approved changesets -- score normalized to 1","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"Maintained","score":0,"reason":"0 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"Token-Permissions","score":0,"reason":"detected GitHub workflow tokens with excessive permissions","details":["Warn: no topLevel permission defined: .github/workflows/style.yaml:1","Info: no jobLevel write permissions found"],"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"Pinned-Dependencies","score":0,"reason":"dependency not pinned by hash detected -- score normalized to 0","details":["Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/style.yaml:19: update your workflow using https://app.stepsecurity.io/secureworkflow/borys25ol/elasticsearch-reindex/style.yaml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/style.yaml:22: update your workflow using https://app.stepsecurity.io/secureworkflow/borys25ol/elasticsearch-reindex/style.yaml/main?enable=pin","Warn: pipCommand not pinned by hash: .github/workflows/style.yaml:28","Warn: pipCommand not pinned by hash: .github/workflows/style.yaml:29","Info:   0 out of   2 GitHub-owned GitHubAction dependencies pinned","Info:   0 out of   2 pipCommand dependencies pinned"],"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Security-Policy","score":0,"reason":"security policy file not detected","details":["Warn: no security policy file detected","Warn: no security file to analyze","Warn: no security file to analyze","Warn: no security file to analyze"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: project has a license file: LICENSE:0","Info: FSF or OSI recognized license: MIT License: LICENSE:0"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Signed-Releases","score":-1,"reason":"no releases found","details":null,"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"Branch-Protection","score":0,"reason":"branch protection not enabled on development/release branches","details":["Warn: branch protection not enabled for branch 'main'"],"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}},{"name":"Vulnerabilities","score":9,"reason":"1 existing vulnerabilities detected","details":["Warn: Project is vulnerable to: GHSA-9hjg-9r4m-mvj7"],"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}},{"name":"SAST","score":0,"reason":"SAST tool is not run on all commits -- score normalized to 0","details":["Warn: 0 commits out of 27 are checked with a SAST tool"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}}]},"last_synced_at":"2025-08-17T08:04:01.568Z","repository_id":42504748,"created_at":"2025-08-17T08:04:01.568Z","updated_at":"2025-08-17T08:04:01.568Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28413833,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-14T08:16:59.381Z","status":"ssl_error","status_checked_at":"2026-01-14T08:13:45.490Z","response_time":107,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cli","elasticsearch","migration-tool","python","reindex"],"created_at":"2026-01-14T08:23:08.199Z","updated_at":"2026-01-14T08:23:08.704Z","avatar_url":"https://github.com/borys25ol.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"Elasticsearch Reindex\n====================\n\n[![forthebadge made-with-python](http://ForTheBadge.com/images/badges/made-with-python.svg)](https://www.python.org/)\n\n[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)\n[![Checked with mypy](http://www.mypy-lang.org/static/mypy_badge.svg)](http://mypy-lang.org/)\n[![Imports: isort](https://img.shields.io/badge/%20imports-isort-%231674b1?style=flat\u0026labelColor=ef8336)](https://pycqa.github.io/isort/)\n[![Pre-commit: enabled](https://img.shields.io/badge/pre--commit-enabled-brightgreen?logo=pre-commit\u0026logoColor=white\u0026style=flat)](https://github.com/pre-commit/pre-commit)\n\n## Description\n`elasticsearch-reindex` is a CLI tool for transferring Elasticsearch indexes between different servers.\n\n## Installing\n\nInstall the package using pip:\n\n```bash\npip install elasticsearch-reindex\n```\n\nUsage\n-----\n\n## Configuration\n\nEnsure the source Elasticsearch host is whitelisted in the destination host.\nEdit the elasticsearch.yml configuration file on the destination Elasticsearch server.\n\nYou should edit Elasticsearch YML config:\n\n#### Path to config file:\n\n```shell\n/etc/elasticsearch/elasticsearch.yml\n```\n\nAdd the following line to the file:\n\n```shell\nreindex.remote.whitelist: \u003ces-source-host\u003e:\u003ces-source-port\u003e\n```\n\n\n## Running the Tool\n\nUse the CLI to migrate data between Elasticsearch instances:\n```shell\nelasticsearch_reindex \\\n        --source_host http(s)://es-source-host:es-source-port \\\n        --source_http_auth username:password \\\n        --dest_host http(s)://es-dest-host:es-dest-port \\\n        --dest_http_auth username:password \\\n        --check_interval 5 \\\n        --concurrent_tasks 3 \\\n        -i test_index_1 -i test_index_2\n```\n\nAlso, there is a command alias `elasticsearch-reindex`:\n```shell\nelasticsearch-reindex ...\n```\n\n### CLI Parameters\n\n\nRequired fields:\n\n* `source_host` - Elasticsearch endpoint where data will be extracted.\n\n* `dest_host` - Elasticsearch endpoint where data will be transfered.\n\nOptional fields:\n\n* `source_http_auth` - HTTP Basic authentication, username and password.\n\n* `dest_http_auth` - HTTP Basic authentication, username and password.\n\n* `check_interval` - Time period (in second) to check task success status.\n\n    `Default value` - `10` (seconds)\n\n* `concurrent_tasks` - How many parallel task Elasticsearch will process.\n\n    `Default value` - `1` (sync mode)\n\n* `indexes` - List of user ES indexes to migrate instead of all source indexes.\n\n\n### Run library from Python script:\n\n```python\nfrom elasticsearch_reindex import ReindexManager\n\n\ndef main() -\u003e None:\n  \"\"\"\n  Example reindex function.\n  \"\"\"\n  dict_config = {\n    \"source_host\": \"http://localhost:9201\",\n    \"dest_host\": \"http://localhost:9202\",\n    \"check_interval\": 20,\n    \"concurrent_tasks\": 5,\n  }\n  reindex_manager = ReindexManager.from_dict(data=dict_config)\n  reindex_manager.start_reindex()\n\n\nif __name__ == \"__main__\":\n  main()\n\n```\n\nWith custom user indexes:\n\n```python\nfrom elasticsearch_reindex import ReindexManager\n\n\ndef main() -\u003e None:\n  \"\"\"\n  Example reindex function with HTTP Basic authentication.\n  \"\"\"\n  dict_config = {\n    \"source_host\": \"http://localhost:9201\",\n    # If the source host requires authentication\n    # \"source_http_auth\": \"tmp-source-user:tmp-source-PASSWD.220718\",\n    \"dest_host\": \"http://localhost:9202\",\n    # If the destination host requires authentication\n    # \"dest_http_auth\": \"tmp-reindex-user:tmp--PASSWD.220718\",\n    \"check_interval\": 20,\n    \"concurrent_tasks\": 5,\n    \"indexes\": [\"es-index-1\", \"es-index-2\", \"es-index-n\"],\n  }\n  reindex_manager = ReindexManager.from_dict(data=dict_config)\n  reindex_manager.start_reindex()\n\n\nif __name__ == \"__main__\":\n  main()\n\n```\n\nLocal install\n-------------\n\nSet up and activate a Python 3 virtual environment:\n\n```shell\nmake ve\n```\n\nTo install Git hooks:\n\n```shell\nmake install_hooks\n```\n\nCreate .env file and fill the data:\n```shell\ncp .env.example .env\n```\n\nExport env variables:\n```shell\nexport $(xargs \u003c .env)\n```\n\n### Key Environment Variables::\n\nVariable for enable testing:\n\n* `ENV` - variable for enable testing mode.\nFor activate test mode set to value - `test`.\n\nElasticsearch docker settings:\n\n* `ES_SOURCE_PORT` - Source Elasticsearch port\n\n* `ES_DEST_PORT` - Destination Elasticsearch port\n\n* `ES_VERSION` - Elasticsearch version\n\n* `LOCAL_IP` - Address of you local host machine in LAN like `192.168.4.106`.\n\n### How to find your Local IP?\n\n* MacOS (find it in response):\n```shell\nifconfig\n```\n\n* Linux (find it in response):\n```shell\nip r\n```\n\nTesting\n-------\n\nStart Elasticsearch nodes using Docker Compose:\n```shell\ndocker-compose up -d\n```\n\nVerify Elasticsearch nodes are running:\n\n* Source Elasticsearch:\n\n```shell\ncurl -X GET $LOCAL_IP:$ES_SOURCE_PORT\n```\n\n* Destination Elasticsearch:\n\n```shell\ncurl -X GET $LOCAL_IP:$ES_DEST_PORT\n```\n\n\nExport to `PYTHONPATH` env variable:\n```shell\nexport PYTHONPATH=\".\"\n```\n\nFor run tests with `pytest` use:\n```shell\nmake test\n```\n\nFor run tests with `pytest` and `coverage` report use:\n```shell\nmake test-cov\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fborys25ol%2Felasticsearch-reindex","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fborys25ol%2Felasticsearch-reindex","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fborys25ol%2Felasticsearch-reindex/lists"}