https://github.com/mantis-software-company/rabbitmq2solr

Simple utility that consumes messages from a RabbitMQ queue and index to Solr
https://github.com/mantis-software-company/rabbitmq2solr

Last synced: about 1 month ago
JSON representation

Simple utility that consumes messages from a RabbitMQ queue and index to Solr

• Host: GitHub
• URL: https://github.com/mantis-software-company/rabbitmq2solr
• Owner: mantis-software-company
• License: apache-2.0
• Created: 2022-03-11T12:22:56.000Z (almost 3 years ago)
• Default Branch: main
• Last Pushed: 2023-07-25T19:20:48.000Z (over 1 year ago)
• Last Synced: 2024-10-04T02:17:34.676Z (3 months ago)
• Language: Python
• Size: 11.7 KB
• Stars: 0
• Watchers: 4
• Forks: 0
• Open Issues: 2
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# rabbitmq2solr

rabbitmq2solr is a simple utility that consumes messages from a RabbitMQ queue and index to Solr.

Messages are expected to be JSON documents that send to Solr update endpoint. If indexing fails, the message is sent to a dead letter queue for later use.

The messages that sent to DLQ, delivered as persistent messages. This job will exit when the queue is empty.

# Environment Variables

- `MQ_HOST`: The hostname of the RabbitMQ server.

- `MQ_PORT`: The port of the RabbitMQ server. Optional. Default: 5672.

- `MQ_USER`: The username to use when connecting to the RabbitMQ server.

- `MQ_PASS`: The password to use when connecting to the RabbitMQ server.

- `MQ_VHOST`: The virtual host to use when connecting to the RabbitMQ server.

- `MQ_QUEUE`: The name of the queue to consume messages from.

- `MQ_QUEUE_DURABLE`: Whether or not the queue should be durable. Optional. Default: True.

- `MQ_DLQ_EXCHANGE`: The exchange of the dead letter queue to use. If index errors occur, messages will be moved to this queue.

- `MQ_DLQ_ROUTING_KEY`: The routing key of the dead letter queue to use.

- `SOLR_BASE_URL`: The base url of the Solr server. Example: http://localhost:8983/solr (without trailing slash).

- `SOLR_COLLECTION`: The name of the Solr collection to index to. Optional. Default: Routing key of message.

- `LOG_LEVEL`: The log level to use. Optional. Default: DEBUG.

- `CONSUMER_POOL_SIZE`: The number of concurrent consumers to use.