{"id":13463795,"url":"https://github.com/fluent/fluent-bit-kubernetes-logging","last_synced_at":"2025-03-25T09:31:05.315Z","repository":{"id":37431813,"uuid":"82602426","full_name":"fluent/fluent-bit-kubernetes-logging","owner":"fluent","description":"Fluent Bit Kubernetes Daemonset","archived":true,"fork":false,"pushed_at":"2022-09-26T17:05:39.000Z","size":114,"stargazers_count":467,"open_issues_count":46,"forks_count":249,"subscribers_count":21,"default_branch":"master","last_synced_at":"2024-10-29T16:21:06.943Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/fluent.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-02-20T21:03:41.000Z","updated_at":"2024-10-18T16:21:12.000Z","dependencies_parsed_at":"2022-08-19T20:50:07.955Z","dependency_job_id":null,"html_url":"https://github.com/fluent/fluent-bit-kubernetes-logging","commit_stats":null,"previous_names":["fluent/fluent-bit-kubernetes-daemonset"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fluent%2Ffluent-bit-kubernetes-logging","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fluent%2Ffluent-bit-kubernetes-logging/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fluent%2Ffluent-bit-kubernetes-logging/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fluent%2Ffluent-bit-kubernetes-logging/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/fluent","download_url":"https://codeload.github.com/fluent/fluent-bit-kubernetes-logging/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245435086,"owners_count":20614826,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-07-31T14:00:28.663Z","updated_at":"2025-03-25T09:31:04.984Z","avatar_url":"https://github.com/fluent.png","language":null,"readme":"# Kubernetes Logging with Fluent Bit\n\n\u003e :warning: This repository is no longer maintained. Please use the [charts](https://github.com/fluent/helm-charts/tree/main/charts/fluent-bit) from the [Fluent Bit Helm Chart](https://github.com/fluent/helm-charts) project.\n\u003e If you need any further assistance, reach out to the community on the [available channels](https://fluentbit.io/community/)\n\n\n## Overview\n\n[Fluent Bit](http://fluentbit.io) is a lightweight and extensible __Log and Metrics Processor__ that comes with full support for Kubernetes:\n\n- Read Kubernetes/Docker log files from the file system or through systemd Journal\n- Enrich logs with Kubernetes metadata\n- Deliver logs to third party services like Elasticsearch, Splunk, Datadog, InfluxDB, HTTP, etc.\n\nThis repository contains a set of Yaml files to deploy Fluent Bit which consider namespace, RBAC, Service Account, etc.\n\n## Getting started\n\n[Fluent Bit](http://fluentbit.io) must be deployed as a DaemonSet so that it will be available on every node of your Kubernetes cluster. To get started run the following commands to create the namespace, service account and role setup:\n\n\nFor Kubernetes v1.21 and below\n\n```\n$ kubectl create namespace logging\n$ kubectl create -f https://raw.githubusercontent.com/fluent/fluent-bit-kubernetes-logging/master/fluent-bit-service-account.yaml\n$ kubectl create -f https://raw.githubusercontent.com/fluent/fluent-bit-kubernetes-logging/master/fluent-bit-role.yaml\n$ kubectl create -f https://raw.githubusercontent.com/fluent/fluent-bit-kubernetes-logging/master/fluent-bit-role-binding.yaml\n```\n\nFor Kubernetes v1.22 and above\n\n```\n$ kubectl create namespace logging\n$ kubectl create -f https://raw.githubusercontent.com/fluent/fluent-bit-kubernetes-logging/master/fluent-bit-service-account.yaml\n$ kubectl create -f https://raw.githubusercontent.com/fluent/fluent-bit-kubernetes-logging/master/fluent-bit-role-1.22.yaml\n$ kubectl create -f https://raw.githubusercontent.com/fluent/fluent-bit-kubernetes-logging/master/fluent-bit-role-binding-1.22.yaml\n```\n\n\nIf you are deploying fluent-bit on openshift, you additionally need to run:\n\n```\n$ kubectl create -f https://raw.githubusercontent.com/fluent/fluent-bit-kubernetes-logging/master/fluent-bit-openshift-security-context-constraints.yaml\n```\n\n\n#### Fluent Bit to Elasticsearch\n\nThe next step is to create a ConfigMap that will be used by our Fluent Bit DaemonSet:\n\n```\n$ kubectl create -f https://raw.githubusercontent.com/fluent/fluent-bit-kubernetes-logging/master/output/elasticsearch/fluent-bit-configmap.yaml\n```\n\nIf the cluster uses a CRI runtime, like containerd or CRI-O, change the `Parser` described in `input-kubernetes.conf` from docker to cri.\n\nFluent Bit DaemonSet ready to be used with Elasticsearch on a normal Kubernetes Cluster:\n\n```\n$ kubectl create -f https://raw.githubusercontent.com/fluent/fluent-bit-kubernetes-logging/master/output/elasticsearch/fluent-bit-ds.yaml\n```\n\n#### Fluent Bit to Elasticsearch on Minikube\n\nIf you are using Minikube for testing purposes, use the following alternative DaemonSet manifest:\n\n```\n$ kubectl create -f https://raw.githubusercontent.com/fluent/fluent-bit-kubernetes-logging/master/output/elasticsearch/fluent-bit-ds-minikube.yaml\n```\n\n#### Fluent Bit to Kafka\n\nCreate a ConfigMap that will be used by our Fluent Bit DaemonSet:\n\n```\n$ kubectl create -f https://raw.githubusercontent.com/fluent/fluent-bit-kubernetes-logging/master/output/kafka/fluent-bit-configmap.yaml\n```\n\nFluent Bit DaemonSet ready to be used with Kafka on a normal Kubernetes Cluster:\n\n```\n$ kubectl create -f https://raw.githubusercontent.com/fluent/fluent-bit-kubernetes-logging/master/output/kafka/fluent-bit-ds.yaml\n```\n\n#### Fluent Bit to Elasticsearch on Minikube\n\nIf you are using Minikube for testing purposes, use the following alternative DaemonSet manifest:\n\n```\n$ kubectl create -f https://raw.githubusercontent.com/fluent/fluent-bit-kubernetes-logging/master/output/elasticsearch/fluent-bit-ds-minikube.yaml\n```\n\n## Details\n\nThe default configuration of Fluent Bit makes sure of the following:\n\n- Consume all containers logs from the running Node.\n- The [Tail input plugin](http://fluentbit.io/documentation/0.12/input/tail.html) will not append more than __5MB__  into the engine until they are flushed to the Elasticsearch backend. This limit aims to provide a workaround for [backpressure](http://fluentbit.io/documentation/0.13/configuration/backpressure.html) scenarios.\n- The Kubernetes filter will enrich the logs with Kubernetes metadata, specifically _labels_ and _annotations_. The filter only goes to the API Server when it cannot find the cached info, otherwise it uses the cache.\n- The default backend in the configuration is Elasticsearch set by the [Elasticsearch Output Plugin](http://fluentbit.io/documentation/0.13/output/elasticsearch.html). It uses the Logstash format to ingest the logs. If you need a different Index and Type, please refer to the plugin option and do your own adjustments.\n- There is an option called __Retry_Limit__ set to False that means if Fluent Bit cannot flush the records to Elasticsearch it will re-try indefinitely until it succeeds.\n\n## Get in touch with us!\n\nYour contribution to testing is highly appreciated. We aim to make logging cheaper for everybody so your feedback is fundamental. Please get in touch on:\n\n- [Mailing List / Google Group](https://groups.google.com/forum/#!forum/fluent-bit)\n- [Slack Channel #fluent-bit](http://slack.fluentd.org)\n","funding_links":[],"categories":["Others","Tools"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffluent%2Ffluent-bit-kubernetes-logging","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffluent%2Ffluent-bit-kubernetes-logging","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffluent%2Ffluent-bit-kubernetes-logging/lists"}