https://github.com/uplol/bristle
🌲 bristle is a clickhouse ingestion service for processing arbitrary protobufs via grpc
https://github.com/uplol/bristle
clickhouse grpc metrics protobuf
Last synced: 11 months ago
JSON representation
🌲 bristle is a clickhouse ingestion service for processing arbitrary protobufs via grpc
- Host: GitHub
- URL: https://github.com/uplol/bristle
- Owner: uplol
- Created: 2021-02-12T09:14:35.000Z (about 5 years ago)
- Default Branch: master
- Last Pushed: 2022-01-02T22:56:25.000Z (about 4 years ago)
- Last Synced: 2025-04-06T09:12:34.286Z (12 months ago)
- Topics: clickhouse, grpc, metrics, protobuf
- Language: Go
- Homepage:
- Size: 93.8 KB
- Stars: 6
- Watchers: 1
- Forks: 2
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# bristle
bristle is a grpc service which can process and store arbitrary protobuf messages within Clickhouse.
- loads protobuf descriptors at runtime, linking and validating them with Clickhouse tables
- batches writes to Clickhouse to avoid overwhelming it with merges
- buffers data during instability or outages of Clickhouse
bristle is intended to be a generic Clickhouse data consumer that allows you to safely and efficiently iterate on your full data pipeline, just by using compiled protobuf descriptors.
## Why?
UPLOL built bristle to handle a few main use cases:
- Having a reliable and low-configuration option for experimenting and iterating with data in Clickhouse. We have good experience and usage of protobufs internally, so building our data infrastructure around them is a logical extension.
- Ingesting and processing large amounts of multi-tenant data in a high-availability and high-integrity environment.
- Simple top-level observability ingestion-service for metrics and logs. Clickhouse is a fantastic data-store for observability data, and using bristle we can trivially spin up new metric, log, or event collector services that require very little logic.