https://github.com/Senzing/rabbitmq-performance

Tweeks to RabbitMQ
https://github.com/Senzing/rabbitmq-performance

documentation senzing-gdev

Last synced: about 23 hours ago
JSON representation

Tweeks to RabbitMQ

• Host: GitHub
• URL: https://github.com/Senzing/rabbitmq-performance
• Owner: Senzing
• License: apache-2.0
• Created: 2022-01-20T13:57:25.000Z (over 2 years ago)
• Default Branch: main
• Last Pushed: 2024-05-29T17:03:08.000Z (24 days ago)
• Last Synced: 2024-05-30T07:07:53.356Z (24 days ago)
• Topics: documentation, senzing-gdev
• Homepage:
• Size: 142 KB
• Stars: 1
• Watchers: 3
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • Changelog: CHANGELOG.md
  • Contributing: CONTRIBUTING.md
  • License: LICENSE
  • Code of conduct: CODE_OF_CONDUCT.md
  • Codeowners: .github/CODEOWNERS

Lists

• awesome - rabbitmq-performance - Tweeks to RabbitMQ (Documentation)

README

        
# rabbitmq-performance

A couple of things to keep in mind with RabbitMQ is that it performs better when it is smaller.  I've let Qs grow to 40M+ records in it and RabbitMQ ends up being a severe bottleneck as it tries to manage it.  Also, RabbitMQ has a consumer_timeout which will cause connections to get closed if a record takes too long to process (like might happen in huge entities or PostgreSQL XID pauses).

## Setup the dead letter exchange

This is a place for rejected messages to go during processing.  Messages responded to with pika.basic_reject() go here.  You don't want to size limit this.

```

Queue Name: senzing-dlx-queue

durable: yes

Exchange Name: senzing-dlx-exchange

Routing Key: senzing.deadletter

```

## Setup the main input data exchange

```

Queue Name: senzing-rabbitmq-queue

max-length: 50000

overflow: reject-publish

DLX Exchange Name: senzing-dlx-exchange

DLX Routing Key: senzing.deadletter

Exchange Name: senzing-rabbitmq-exchange

Routing Key: senzing.records

```