{"id":14981081,"url":"https://github.com/flant/elasticsearch-extractor","last_synced_at":"2025-09-05T11:20:39.989Z","repository":{"id":42003372,"uuid":"313571252","full_name":"flant/elasticsearch-extractor","owner":"flant","description":"Simple web UI to extract any index from Elasticsearch snapshot into repository.","archived":false,"fork":false,"pushed_at":"2025-02-11T09:12:30.000Z","size":1038,"stargazers_count":46,"open_issues_count":1,"forks_count":4,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-05-08T21:44:01.270Z","etag":null,"topics":["elasticsearch"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/flant.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-11-17T09:37:07.000Z","updated_at":"2025-02-11T09:11:26.000Z","dependencies_parsed_at":"2024-06-18T22:35:27.731Z","dependency_job_id":"5a49c751-415c-47dd-8d3b-2954ee71654d","html_url":"https://github.com/flant/elasticsearch-extractor","commit_stats":{"total_commits":74,"total_committers":4,"mean_commits":18.5,"dds":0.08108108108108103,"last_synced_commit":"655104ce4e075ed14f49d7150e60a3637e8eb7af"},"previous_names":[],"tags_count":33,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/flant%2Felasticsearch-extractor","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/flant%2Felasticsearch-extractor/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/flant%2Felasticsearch-extractor/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/flant%2Felasticsearch-extractor/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/flant","download_url":"https://codeload.github.com/flant/elasticsearch-extractor/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253154026,"owners_count":21862448,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["elasticsearch"],"created_at":"2024-09-24T14:02:53.570Z","updated_at":"2025-05-08T21:44:14.140Z","avatar_url":"https://github.com/flant.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"**elasticsearch-extractor** is a simple web UI for end users to extract any index from the desired Elasticsearch snapshot within repository (S3-compatible or any other registered in your cluster).\n\nIt requires Elasticsearch v7.0 or greater.\n\n# Motivation / idea\n\n*[This announcement](https://blog.flant.com/announcing-elasticsearch-extractor-open-source-tool/) of the project (Jan'21) sheds some light on why elasticsearch-extractor has emerged.*\n\nWe deal with lots of logs stored in Elasticsearch clusters. They are regularly archived into snapshots and stored in S3. Being available in snapshots for a long period, these logs are quite often needed by engineers (to be examined for different reasons). That's why we want to have a simple interface for getting these logs from snapshots.\n\nWhile we really like [Cerebro](https://github.com/lmenezes/cerebro/), it gives you *too much* power over Elasticsearch clusters and doesn't allow you to control permissions for its users. Here's how we've ended up with creating a much simpler tool that implements the only function: to extract an index from Elasticsearch snapshot.\n\nelasticsearch-extractor operates as:\n* a simple web UI for end users;\n* a server proxying requests to Elasticsearch.\n\nSince there's no authentication layer implemented in elasticsearch-extractor, you have to use your relevant infrastructure components for that, e.g. Ingress controller (if Elasticsearch is in Kubernetes) or nginx/Apache built-in capabilities.\n\n# Using\n\n## Installing \u0026 running in Linux with systemd\n\nTo build \u0026 install elasticsearch-extractor on Linux (with systemd), please use the following commands:\n\n```\n$ go get -u github.com/jteeuwen/go-bindata/...\n$ git clone https://github.com/flant/elasticsearch-extractor.git\n$ cd elasticsearch-extractor\n$ make\n$ sudo cp ./build/elasticsearch-extractor /usr/local/sbin/extractor\n$ sudo cp ./examples/main.yml /usr/local/etc/extractor.yml\n$ sudo cp ./examples/extractor.service /etc/systemd/system/\n$ vim /usr/local/etc/extractor.yml # config is small and self-descriptive\n$ sudo systemctl daemon-reload \u0026\u0026 systemctl start extractor\n$ sudo systemctl enable extractor\n```\n\n## Hot reload\nUse `air`:\n```bash\n$ go install github.com/air-verse/air@latest\n$ air\n\n```\n\n## Running in Docker\n\n```\ndocker-compose up --build --force-recreate -d\n```\n\n# Further information\n\nPlease feel free to use [issues](https://github.com/flant/elasticsearch-extractor/issues) and [discussions](https://github.com/flant/elasticsearch-extractor/discussions) to get help from maintainers \u0026 community.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fflant%2Felasticsearch-extractor","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fflant%2Felasticsearch-extractor","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fflant%2Felasticsearch-extractor/lists"}