{"id":16848117,"url":"https://github.com/araddon/loges","last_synced_at":"2025-03-22T05:31:29.420Z","repository":{"id":5623786,"uuid":"6831652","full_name":"araddon/loges","owner":"araddon","description":"Logging/Metrics Daemon: Logstash Formatted write to Elasticsearch with Fluentd, Kafka, Monit inputs and Graphite, Elasticsearch outputs","archived":true,"fork":false,"pushed_at":"2018-02-13T17:52:21.000Z","size":80,"stargazers_count":81,"open_issues_count":1,"forks_count":6,"subscribers_count":10,"default_branch":"master","last_synced_at":"2024-12-21T16:22:34.861Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/araddon.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2012-11-23T18:36:38.000Z","updated_at":"2024-10-26T17:09:28.000Z","dependencies_parsed_at":"2022-09-18T17:00:52.728Z","dependency_job_id":null,"html_url":"https://github.com/araddon/loges","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/araddon%2Floges","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/araddon%2Floges/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/araddon%2Floges/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/araddon%2Floges/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/araddon","download_url":"https://codeload.github.com/araddon/loges/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244912800,"owners_count":20530764,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-13T13:10:08.676Z","updated_at":"2025-03-22T05:31:28.827Z","avatar_url":"https://github.com/araddon.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"Logging Utility \u0026 Daemon\n---------------------------------\n\nRead log data from Inputs [Tail Files, Stdin, Monit], perform transforms \n[Combine lines(multi-line-errors)] and output to [ElasticSearch, Stdout]\nRecognize lines with metrics and send to Monitoring [Graphite, InfluxDB, ..]\n\n\n![Drawing](https://docs.google.com/drawings/d/1nGVabfy3PB0Zq-gsghKRkGU3eGz4zrcmpIrB0e2cs9M/pub?w=695\u0026h=401)\n\nWhy?  \n---------\nWe had 2 needs:  1) to send data to Elasticsearch for viewing in http://kibana.org/ \nand 2) if possible, unify the Logging/Metrics systems data-collection-forwarding.  \n\nThere are probably better tools out there for this but putting together the \nspecific combination of: (LogStash format in Elasticsearch, Tail files, \nRead Kafka, Metrics read from log files) didn't happen, see alternates below.\n\n\nFeatures\n-----------------\n\n* **Inputs**\n  * Stdin \n  * Tail Logs (multiple files)\n  * Monit (metrics via http)\n  * Http  (custom)\n* **Transforms**:\n   * Logstash http://logstash.net/ \n   * Colorizer for console\n   * Concat into single line when needed (e.g. error stack trace)\n   * Separate Metrics Log Lines from regular log lines\n   * Custom plugins\n* **Log Line Outputs**\n   * Stdout (optional colorized)\n   * Elasticsearch\n* **Metric Outputs**\n   * Graphite\n\nAlternatives\n-----------------\n\n* Go https://github.com/gliderlabs/logspout\n* Go https://github.com/onemorecloud/dendrite\n* Go https://github.com/ryandotsmith/l2met\n* Go http://blog.mozilla.org/services/2013/04/30/introducing-heka/\n* Ruby, http://fluentd.org/\n* JVM, http://logstash.org \n* Go https://github.com/cloudfoundry/loggregator\n\nUsage\n----------------------\n\n```sh\nloges --source=monit,tail --filter=stdfiles --out=elasticsearch --metrics=graphite \\\n   /path/to/my/file \\\n   /path/to/another/file\n\n```\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faraddon%2Floges","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Faraddon%2Floges","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faraddon%2Floges/lists"}