https://github.com/awslabs/flowgger

A fast data collector in Rust
https://github.com/awslabs/flowgger

graylog kafka logging logging-server ltsv rust syslog

Last synced: 3 months ago
JSON representation

A fast data collector in Rust

• Host: GitHub
• URL: https://github.com/awslabs/flowgger
• Owner: awslabs
• License: bsd-2-clause
• Created: 2015-08-09T17:08:53.000Z (over 9 years ago)
• Default Branch: master
• Last Pushed: 2024-05-23T15:15:09.000Z (8 months ago)
• Last Synced: 2024-08-05T19:25:08.202Z (5 months ago)
• Topics: graylog, kafka, logging, logging-server, ltsv, rust, syslog
• Language: Rust
• Size: 726 KB
• Stars: 821
• Watchers: 32
• Forks: 54
• Open Issues: 13
• Metadata Files:
  • Readme: README.md
  • Changelog: CHANGELOG.md
  • License: LICENSE

Awesome Lists containing this project

• awesome-rust - jedisct1/flowgger
• awesome-rust-cn - jedisct1/flowgger
• awesome-repositories - awslabs/flowgger - A fast data collector in Rust (Rust)
• awesome-rust-zh - jedisct1/flowgger - 快速，简单和轻量级的数据收集器 (应用 / 贡献)
• awesome-rust - jedisct1/flowgger - A fast, simple and lightweight data collector (Applications)
• awesome-rust-cn - jedisct1/flowgger
• fucking-awesome-rust - jedisct1/flowgger - A fast, simple and lightweight data collector (Applications)

README

        



   



[![CI Build](https://github.com/awslabs/flowgger/actions/workflows/ci.yml/badge.svg)](https://github.com/awslabs/flowgger/actions/workflows/ci.yml) [![License: BSD2](https://img.shields.io/badge/License-BSD2-brightgreen.svg)](https://github.com/awslabs/flowgger/blob/master/LICENSE)



### New major version: 0.3.0 (2022-03-14)

#### Breaking Changes

*   Migrate from chrono to [time](https://docs.rs/time/latest/time/) as per https://rustsec.org/advisories/RUSTSEC-2020-0071

    * String formatting changed from strftime to [time](https://docs.rs/time/latest/time/format_description/index.html) custom formatting - see ```flowgger.toml``` for examples on change

---

Flowgger is a fast, simple and lightweight data collector written in Rust.

It reads log entries over a given protocol, extracts them, decodes them using a

given format, re-encodes them into a different format, and asynchronously pushes

the result into a remote data store.

Flowgger is designed to be:

- Paranoid: it carefully validates input data to prevent injection of

malformed/incomplete records down the chain.

- Safe: written in Rust, without any `unsafe` code.

- Fast: even though messages are systematically parsed and validated, Flowgger

is orders of magnitude faster than Logstash and Fluentd.

- Standalone: it comes as a single executable file, and doesn't require a JVM.

Flowgger supports common input types: stdin, UDP, TCP, TLS and Redis,

as well as multiple input formats: JSON (GELF), LTSV, Cap'n Proto and

RFC5424. Normalized messages can be sent to Kafka, Graylog, to downstream

Flowgger servers, or to other log collectors for further processing.

# [Jump to the Flowgger documentation](https://github.com/awslabs/flowgger/wiki)