{"id":19670207,"url":"https://github.com/mcpt/karby","last_synced_at":"2026-05-15T02:42:50.639Z","repository":{"id":40526534,"uuid":"277418489","full_name":"mcpt/karby","owner":"mcpt","description":"A simple HTTP sink for vector.dev!","archived":false,"fork":false,"pushed_at":"2023-03-16T06:42:07.000Z","size":27,"stargazers_count":1,"open_issues_count":2,"forks_count":1,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-01-10T03:26:48.750Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mcpt.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-07-06T01:54:06.000Z","updated_at":"2023-06-20T10:25:12.000Z","dependencies_parsed_at":"2025-01-10T03:26:05.464Z","dependency_job_id":"130d46c1-3ae7-4266-ab25-e5bd1f1d989d","html_url":"https://github.com/mcpt/karby","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mcpt%2Fkarby","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mcpt%2Fkarby/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mcpt%2Fkarby/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mcpt%2Fkarby/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mcpt","download_url":"https://codeload.github.com/mcpt/karby/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":240980798,"owners_count":19888343,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-11T17:05:15.666Z","updated_at":"2026-05-15T02:42:50.594Z","avatar_url":"https://github.com/mcpt.png","language":"Ruby","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Karby\n\u003e It sucks!\n\nKarby is a basic log aggregator that works with [Vector](https://github.com/timberio/vector). Every batch is saved as a different file, and an aggregator script run via `cron` aggregates logs everyday.\n\n## Installation\nFirst, clone this git repository into the home directory of a new user (`karby`).\n\nNext, install dependencies by running `bundle install` in the root directory of the repository.\n\n### Karby Server\nTo deploy the server copy the included service file `karby.service` to `/etc/systemd/system/`. Then run the following commands as root:\n```sh\nsystemctl enable karby.service\nsystemctl start karby.service\n```\n\nYou can check if the service is running with `systemctl status karby.service`.\n\n#### Zerotier\nIf you want to run this over zerotier only, first change the `After` line in your service file to the following:\n```\nAfter=network-online.target zerotier-one.service\n```\nThen change the `BIND_ADDRESS` in your service file to your local zerotier ip.\n\n### Karby Aggregator\nAdd the following line to the `karby` user's crontab to have each days logs aggregated:\n```\n0 6 * * * env PRE_AGGREGATION_DIR=/home/karby/temp POST_AGGREGATION_DIR=/home/karby/logs DESTROY_LOG_PARTS=FALSE ruby /home/karby/karby-aggregator.rb \"$(date -d \"yesterday 6:00\" '+%Y-%m-%d')\" \u003e/dev/null 2\u003e\u00261\n```\n\nIf you want to destroy the part logs after the aggregation change `DESTROY_LOG_PARTS=TRUE` in the line above.\n\n**Note:** This will aggregate the previous day's logs at 06:00 the next day.\n\n## RVM\nIf using RVM to manage ruby versions, you will likely need to generate a wrapper script for the gems to be viewed properly (see [this page](https://rvm.io/integration/init-d)).\n\n## Sample Vector Sink\n```\n[sinks.karby]\n  type = \"http\"\n  inputs = []\n  compression = \"none\"\n  healthcheck = true\n  uri = \"http://127.0.0.1:4567/\"\n\n  batch.max_size = 1049000\n  batch.timeout_secs = 0\n\n  buffer.type = \"memory\"\n  buffer.max_events = 500\n  buffer.when_full = \"block\"\n\n  encoding.codec = \"json\"\n  encoding.only_fields = [\"timestamp\",\"message\"]\n\n  request.in_flight_limit = 10\n  request.rate_limit_duration_secs = 1\n  request.rate_limit_num = 1000\n  request.timeout_secs = 30\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmcpt%2Fkarby","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmcpt%2Fkarby","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmcpt%2Fkarby/lists"}