{"id":16931616,"url":"https://github.com/leodido/logs2metrics-talk","last_synced_at":"2025-04-11T18:33:16.817Z","repository":{"id":149104176,"uuid":"151316908","full_name":"leodido/logs2metrics-talk","owner":"leodido","description":"Companion code of the talk about extracting metrics from logs with the TICK stack","archived":false,"fork":false,"pushed_at":"2018-11-16T22:26:49.000Z","size":5916,"stargazers_count":5,"open_issues_count":0,"forks_count":4,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-03-25T14:38:13.349Z","etag":null,"topics":["chronograf-log-viewer","influxdb","logging","metrics","observability","syslog","talk","telegraf","tick-stack"],"latest_commit_sha":null,"homepage":"https://speakerdeck.com/leodido/from-logs-to-metrics","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/leodido.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":"leodido"}},"created_at":"2018-10-02T20:12:37.000Z","updated_at":"2023-07-02T01:33:20.000Z","dependencies_parsed_at":null,"dependency_job_id":"33c7e413-2698-4083-9403-688203f368ca","html_url":"https://github.com/leodido/logs2metrics-talk","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/leodido%2Flogs2metrics-talk","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/leodido%2Flogs2metrics-talk/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/leodido%2Flogs2metrics-talk/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/leodido%2Flogs2metrics-talk/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/leodido","download_url":"https://codeload.github.com/leodido/logs2metrics-talk/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248458968,"owners_count":21107190,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["chronograf-log-viewer","influxdb","logging","metrics","observability","syslog","talk","telegraf","tick-stack"],"created_at":"2024-10-13T20:44:22.888Z","updated_at":"2025-04-11T18:33:16.807Z","avatar_url":"https://github.com/leodido.png","language":"Go","funding_links":["https://github.com/sponsors/leodido"],"categories":[],"sub_categories":[],"readme":"# Talk: From logs to metric with the TICK stack\n\n[**Slides**](http://bit.ly/from-logs-to-metrics-tick).\n\nThis repository contains the PoC associated with the talk \"From logs to metric with the TICK stack\".\n\nIts main goal is to show how to extract (structured) value from the huge amount of (unstructured) information that logs contain.\n\nIn brief, the steps are as follows: parsing of syslog messages into structured data, ingesting/collecting them via Telegraf syslog input plugin, visualizing and plot them via Chronograf's log viewer, and eliciting new meaningful metrics (eg. number of process OOM killed) to plot processing them via a Kapacitor [UDF](https://docs.influxdata.com/kapacitor/v1.5/guides/socket_udf/).\n\nThe stack used to achieve this is:\n\n- [Telegraf](https://github.com/influxdata/telegraf) with [syslog input plugin](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/syslog), which uses this blazing fast [go-syslog](https://github.com/influxdata/go-syslog) parser\n- Chronograf\n- InfluxDB\n- [Kapacitor](https://github.com/influxdata/kapacitor)\n\n![Chronograf Log Viewer](images/logviewer-chronograf.png \"Chronograf Log Viewer\")\n\n![Exploring RFC5425 syslog messages with Chronograf](images/exploring-syslog-chronograf.png \"Exploring RFC5425 syslog messages with Chronograf\")\n\n![Couting OOMs](images/ooms-num.png \"Couting OOMs\")\n\n![Counting OOMs of stress pod](images/ooms-stress.png \"Counting OOMs of stress pod\")\n\n## Setup\n\nFirst of all we need a local k8s environment.\n\nLet's proceed with minikube.\n\n```bash\nminikube start --docker-opt log-driver=journald\n```\n\nNote that we need the **journald log driver** for the inner docker since the rsyslog's mmkubernetes module [only works with it](https://www.rsyslog.com/doc/master/configuration/modules/mmkubernetes.html) (or with **json-file docker log driver**).\n\nThe following step is to become a YAML developer :hear_no_evil: :speak_no_evil:, applying all the YAML files describing our setup.\n\n| ![YAML meme](images/yaml-dev.jpg) | ![The life of a YAML developer](images/yaml-dev-life.jpg) |\n|:---:|:---:|\n\nAssuming your minikube setup is capable of provisioning volumes, execute the following commands.\n\n```bash\nkubectl apply -f namespace.yaml\nkubectl apply -f roles.yaml\nkubectl apply -f influxdb.yaml\nkubectl apply -f telelog.yaml\nkubectl apply -f chronograf.yaml\nkubectl apply -f kapacitor.yaml\nkubectl apply -f stress.yaml\n```\n\nFinally to access Chronograf from within our local browser we need the following port forward.\n\n```bash\nkubectl port-forward svc/chronograf -n logging 8888:80\n```\n\nGo to [localhost:8888](http://localhost:8888) now!\n\n## Run with local up cluster\n\n_TBD_.\n\n## Developing the Kapacitor UDF\n\nFile `docker-compose.yaml` is useful during the development and debugging of the Kapacitor UDF.\n\nTo make it working do not forget to forward the port of the influxdb within minikube.\n\n```bash\nkubectl port-forward svc/influxdb -n logging 8686:8686\n```\n\nThen run\n\n```bash\ndocker-compose up -d\n```\n\n## Other suitable docker log drivers\n\nIt is possible to use this with **[syslog docker log driver](https://docs.docker.com/config/containers/logging/syslog/#options)** with following log options:\n\n- `syslog-format=rfc5424micro`\n- `syslog-address=udp://1.2.3.4:1111` (telegraf syslog plugin)\n\nIn such case:\n\n- there is not need for rsyslog\n- telegraf syslog plugin in UDF mode (at the moment in TCP/TLS mode there is not way to disable octet framing requirement - ie., RFC5425)\n- syslog facility will be fixed (depending on the `syslog-facility` option)\n\n_TBD_: create an alternative setup for this setup.\n\n---\n\n[![Analytics](https://ga-beacon.appspot.com/UA-49657176-1/logs2metrics-talk?flat)](https://github.com/igrigorik/ga-beacon)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fleodido%2Flogs2metrics-talk","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fleodido%2Flogs2metrics-talk","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fleodido%2Flogs2metrics-talk/lists"}