Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/kak-tus/corrie
Reliable (with RabbitMQ) Clickhouse writer.
https://github.com/kak-tus/corrie
Last synced: about 1 month ago
JSON representation
Reliable (with RabbitMQ) Clickhouse writer.
- Host: GitHub
- URL: https://github.com/kak-tus/corrie
- Owner: kak-tus
- License: mit
- Created: 2018-07-19T19:23:32.000Z (over 6 years ago)
- Default Branch: master
- Last Pushed: 2020-07-27T18:58:17.000Z (over 4 years ago)
- Last Synced: 2024-06-20T11:06:56.021Z (6 months ago)
- Language: Go
- Homepage:
- Size: 419 KB
- Stars: 6
- Watchers: 2
- Forks: 1
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Changelog: Changes
- License: LICENSE
Awesome Lists containing this project
README
# Corrie
Reliable (with RabbitMQ) Clickhouse writer.
Notice: if you need performance, it is better idea to use [Ruthie](https://github.com/kak-tus/ruthie) - based on Redis Cluster it has much more better performance, then Corrie, based on RabbitMQ.
## Configuration
### CORRIE_RABBITMQ_ADDR
RabbitMQ address in host:port format
```
CORRIE_RABBITMQ_ADDR=rabbitmq.example.com:5672
```
### CORRIE_RABBITMQ_VHOST
RabbitMQ virtual host to store queue
```
CORRIE_RABBITMQ_VHOST=corrie
```
### CORRIE_RABBITMQ_USER, CORRIE_RABBITMQ_PASSWORD
RabbitMQ user and password
```
CORRIE_RABBITMQ_USER=corrie
CORRIE_RABBITMQ_PASSWORD=somepassword
```
### CORRIE_CLICKHOUSE_ADDR
Primary ClickHouse address in host:port form
```
CORRIE_CLICKHOUSE_ADDR=clickhouse1.example.com:9000
```
### CORRIE_CLICKHOUSE_ALTADDRS
Comma separated list of alternative ClickHouse addresses to loadbalancing. Can be empty
```
CORRIE_CLICKHOUSE_ALTADDRS=clickhouse2.example.com:9000
```
### CORRIE_BATCH
Set batch size of ClickHouse writes.
```
CORRIE_BATCH=10000
```
## Run
```
docker run --rm -it kaktuss/corrie
```
## Write data
To write data use [message](https://godoc.org/github.com/kak-tus/corrie/message) package.
You can write data with nanachi RabbitMQ client (see example) or with any other client.
Pay attention, that Corrie uses sharded queue (with nanachi) hardcoded to use 3 shards. Shards count will be configurable later.