{"id":18136162,"url":"https://github.com/albogdano/elasticsearch-river-amazonsqs","last_synced_at":"2025-04-19T14:40:43.507Z","repository":{"id":2282716,"uuid":"3240021","full_name":"albogdano/elasticsearch-river-amazonsqs","owner":"albogdano","description":"[DEPRECATED] Amazon SQS river plugin for ElasticSearch 1.4","archived":false,"fork":false,"pushed_at":"2017-11-27T10:56:26.000Z","size":65,"stargazers_count":20,"open_issues_count":0,"forks_count":6,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-04-13T09:51:38.205Z","etag":null,"topics":["deprecated","elasticsearch","elasticsearch-river","java","plugin"],"latest_commit_sha":null,"homepage":"https://paraio.org","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/albogdano.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2012-01-22T14:40:12.000Z","updated_at":"2023-06-19T00:28:54.000Z","dependencies_parsed_at":"2022-09-07T07:10:44.770Z","dependency_job_id":null,"html_url":"https://github.com/albogdano/elasticsearch-river-amazonsqs","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/albogdano%2Felasticsearch-river-amazonsqs","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/albogdano%2Felasticsearch-river-amazonsqs/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/albogdano%2Felasticsearch-river-amazonsqs/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/albogdano%2Felasticsearch-river-amazonsqs/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/albogdano","download_url":"https://codeload.github.com/albogdano/elasticsearch-river-amazonsqs/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249716354,"owners_count":21315066,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deprecated","elasticsearch","elasticsearch-river","java","plugin"],"created_at":"2024-11-01T14:11:02.516Z","updated_at":"2025-04-19T14:40:43.485Z","avatar_url":"https://github.com/albogdano.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"# AWS SQS River Plugin for Elasticsearch 1.4 [![Build Status](https://travis-ci.org/albogdano/elasticsearch-river-amazonsqs.svg?branch=master)](https://travis-ci.org/albogdano/elasticsearch-river-amazonsqs)\n\n\u003e :information_source: This plugin is now built into [Para](https://github.com/Erudika/para) and works with Elasticsearch 6+\n\n\u003e :warning: Rivers are now deprecated since Elasticsearch 1.5 - [learn more](https://www.elastic.co/blog/deprecating_rivers)\n\nThe AWS SQS plugin uses Amazon's SQS as a river by long polling for messages from a given queue.\nRight after a message is indexed it gets deleted from the queue. This plugin works with Elasticsearch 1.4.\n\n## Installation\n\n````\n$ cd elasticsearch-dir\n$ bin/plugin -i river-amazonsqs -u https://s3-eu-west-1.amazonaws.com/albogdano/river-amazonsqs.zip\n````\n## Configuration\n\nTo configure put this in your `elasticsearch.yml`:\n\n    cloud.aws.region: AWS REGION\n    cloud.aws.access_key: AWS ACCESS KEY\n    cloud.aws.secret_key: AWS SECRET KEY\n    cloud.aws.sqs.queue_url: AWS QUEUE URL\n    cloud.aws.sqs.debug: (false by default)\n    cloud.aws.sqs.sleep: (seconds)\n    cloud.aws.sqs.longpolling_interval: (seconds)\n\n**OR** use a river configuration like this:\n\n    curl -XPUT 'localhost:9200/_river/my_sqs_river/_meta' -d '{\n      \"type\": \"amazonsqs\",\n      \"amazonsqs\": {\n          \"region\": \"AWS REGION\",\n          \"access_key\": \"AWS ACCESS KEY\",\n          \"secret_key\": \"AWS SECRET KEY\",\n          \"queue_url\": \"AWS QUEUE URL\",\n          \"debug\": false,\n          \"sleep\": 60,\n          \"longpolling_interval\": 20\n        },\n        \"index\": {\n          \"max_messages\": 10,\n          \"index\": \"es_index_name\"\n        }\n    }'\n\n## Details\n\nMessages are in the following JSON format:\n\n    {\n      \"_id\": \"123\",\n      \"_index\": \"es_index_name\",\n      \"_type\": \"es_data_type\",\n      \"_data\": { \"key1\": \"value1\" ...}\n    }\n\n- The fields `_id` and `_type` are required.\n- If `_data` is missing the data with this id will be deleted from the index.\n- If `_data` is anything other than JSON object we discard it and treat the messages as a delete request.\n- If `_index` is missing it will fallback to the index that was initially configured,\notherwise the `_index` property overrides the default configuration and allows you to dynamically switch between indexes.\n- If `_id` is an integer it will be converted to `String` because SQS doesn't convert integers to strings automatically.\n- When the queue is empty the river will sleep for `sleep` seconds before sending a new request for messages to the queue.\nLong polling is done by the Amazon SQS client using the `waitTimeSeconds` attribute which is set to `longpolling_interval` _(must be between 0 and 20)_.\n\n## License\n[Apache 2.0](LICENSE)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Falbogdano%2Felasticsearch-river-amazonsqs","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Falbogdano%2Felasticsearch-river-amazonsqs","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Falbogdano%2Felasticsearch-river-amazonsqs/lists"}