https://github.com/hainenber/hetman

Log shipping agent to various downstream services
https://github.com/hainenber/hetman

Last synced: 20 days ago
JSON representation

Log shipping agent to various downstream services

• Host: GitHub
• URL: https://github.com/hainenber/hetman
• Owner: hainenber
• Created: 2023-03-12T11:34:56.000Z (almost 2 years ago)
• Default Branch: main
• Last Pushed: 2024-01-19T03:45:43.000Z (about 1 year ago)
• Last Synced: 2024-11-21T19:13:56.749Z (3 months ago)
• Language: Go
• Homepage:
• Size: 478 KB
• Stars: 0
• Watchers: 1
• Forks: 0
• Open Issues: 9
• Metadata Files:
  • Readme: README.md
  • Changelog: CHANGELOG.md

Awesome Lists containing this project

README

        
# Hetman

Hetman is an agent fetching logs from target files and push to various downstream services

At the moment, the agent only supports Loki as downstream log consumer

## Features

* Resume to correct read position when restarted after sudden interruption.

* Persist failed-to-deliver logs to disk.

* Apply exponential backoff to failed delivery request(s) if downstream is having intermittent issue.

* Correctly follow post-renaming both old and new files, similar to `logrotate`'s `create` directive.

* Correctly follow file when rotated by truncating old files, similar to `logrotate`'s `copytruncate` directive.

* Reset read position of a tailed file during truncation.

* Gracefully reload configurations changes when sending SIGHUP to running agent.

* Deliver logs in batches.

* Built-in backpressure engine to control input/output bandwidth.

* High percentage of code coverage (>= 80% for the majority of components)

* Capable to parse multi-line logs.

* Support multiple downstreams:

    * Loki

    * Kafka

## Roadmap

- [ ] Fleet management.