{"id":21994503,"url":"https://github.com/codelibs/elasticsearch-indexing-proxy","last_synced_at":"2026-03-15T08:53:57.226Z","repository":{"id":24048315,"uuid":"100449794","full_name":"codelibs/elasticsearch-indexing-proxy","owner":"codelibs","description":"This plugin intercepts indexing requests and writes them to files","archived":false,"fork":false,"pushed_at":"2021-12-14T20:32:22.000Z","size":140,"stargazers_count":6,"open_issues_count":4,"forks_count":3,"subscribers_count":7,"default_branch":"master","last_synced_at":"2025-03-30T18:04:33.819Z","etag":null,"topics":["elasticsearch-plugin"],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/codelibs.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-08-16T05:09:41.000Z","updated_at":"2023-09-02T12:40:14.000Z","dependencies_parsed_at":"2022-07-15T13:47:12.137Z","dependency_job_id":null,"html_url":"https://github.com/codelibs/elasticsearch-indexing-proxy","commit_stats":null,"previous_names":[],"tags_count":7,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/codelibs%2Felasticsearch-indexing-proxy","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/codelibs%2Felasticsearch-indexing-proxy/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/codelibs%2Felasticsearch-indexing-proxy/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/codelibs%2Felasticsearch-indexing-proxy/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/codelibs","download_url":"https://codeload.github.com/codelibs/elasticsearch-indexing-proxy/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251748929,"owners_count":21637413,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["elasticsearch-plugin"],"created_at":"2024-11-29T21:09:31.984Z","updated_at":"2026-03-15T08:53:52.204Z","avatar_url":"https://github.com/codelibs.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"Elasticsearch Indexing Proxy Plugin\n===================================\n\n## Overview\n\nThis plugin provides a proxy feature for target indices.\nFor target indices, this plugin intercepts indexing requests and writes them to files.\nMoreover, Document Sender provided by this plugin reads the files and sends them to any indices.\n\n## Version\n\n[Versions in Maven Repository](http://central.maven.org/maven2/org/codelibs/elasticsearch-indexing-proxy/)\n\n### Issues/Questions\n\nPlease file an [issue](https://github.com/codelibs/elasticsearch-indexing-proxy/issues \"issue\").\n\n## Installation\n\nNot released yet.\n\n    $ $ES_HOME/bin/elasticsearch-plugin install org.codelibs:elasticsearch-indexing-proxy:5.5.0\n\n## Getting Started\n\n### Configuration\n\nThis plugin has the following settings.\n\n| Setting                              | Type    | Default      | Description                   |\n|:-------------------------------------|:-------:|:------------:|:------------------------------|\n| idxproxy.data.path                   | string  | es.data.path | directory to store data file  |\n| idxproxy.target.indices              | string  | none         | target indices                |\n| idxproxy.data.file\\_size             | size    | 100m         | file size                     |\n| idxproxy.sender.interval             | time    | 30s          | interval to check file        |\n| idxproxy.sender.retry\\_count         | int     | 10           | retry count for error file    |\n| idxproxy.sender.request.retry\\_count | int     | 3            | retry count for error request |\n| idxproxy.sender.skip.error\\_file     | boolean | true         | skip error file               |\n\nFor example, put settings as below:\n\n```\nidxproxy.data.path: /opt/elasticsearch/idxproxy-data\nidxproxy.target.indices: sample\n```\n\nThe above settings intercepts indexing(index,update,delete) requests and stores them to /opt/elasticsearch/idxproxy-data/[0-9]+.dat.\n\n### Create Index and Alias\n\nCreate sample1 index and sample alias:\n\n```\ncurl -XPUT 'localhost:9200/sample1?pretty' -H 'Content-Type: application/json' -d'\n{\n    \"settings\" : {\n        \"index\" : {\n            \"number_of_shards\" : 3,\n            \"number_of_replicas\" : 0\n        }\n    }\n}\n'\ncurl -XPOST 'localhost:9200/_aliases?pretty' -H 'Content-Type: application/json' -d'\n{\n    \"actions\" : [\n        { \"add\" : { \"index\" : \"sample1\", \"alias\" : \"sample\" } }\n    ]\n}\n'\n```\n\n### Send Document\n\nSend 1 document to sample alias:\n\n```\ncurl -XPOST 'localhost:9200/sample/doc/1?pretty' -H 'Content-Type: application/json' -d'\n{\n    \"msg\" : \"test 1\"\n}\n'\n```\n\nCheck sample index:\n\n```\ncurl 'localhost:9200/sample/_search?q=*:*\u0026pretty'\n```\n\nIf this plugin works correctly, hits.total is 0.\n.tmp file is a working files in idxproxy.data.path directory.\n\n```\n$ ls idxproxy-data/\n0000000000000000001.tmp\n```\n\n### Flush Data file\n\nSend refresh request:\n\n```\ncurl -XPOST 'localhost:9200/_refresh?pretty'\n```\n\n.dat file and new .tmp file are created.\n\n```\nls idxproxy-data/\n0000000000000000001.dat\t\t0000000000000000002.tmp\n```\n\n### Start Document Sender\n\nStart Document Sender for sample1 index:\n\n```\ncurl -XPOST 'localhost:9200/sample1/_idxproxy/process?pretty'\n```\n\nCheck sample index:\n\n```\ncurl 'localhost:9200/sample/_search?q=*:*\u0026pretty'\n```\n\nYou will find a sent document.\n\n### Stop Document Sender\n\nStop Document Sender for sample1 index:\n\n```\ncurl -XDELETE 'localhost:9200/sample1/_idxproxy/process?pretty'\n```\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcodelibs%2Felasticsearch-indexing-proxy","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcodelibs%2Felasticsearch-indexing-proxy","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcodelibs%2Felasticsearch-indexing-proxy/lists"}