https://github.com/vinayaks439/fluentd-k8s
A repo containing manifests inorder to install fluentd on k8s cluster. Fluentd is used for logging and the logs can be shipped anywhere for eg: loki, gcs is concerated more in the config.
https://github.com/vinayaks439/fluentd-k8s
fluentd gcs k8s-deployment kubernetes logging loki
Last synced: about 2 months ago
JSON representation
A repo containing manifests inorder to install fluentd on k8s cluster. Fluentd is used for logging and the logs can be shipped anywhere for eg: loki, gcs is concerated more in the config.
- Host: GitHub
- URL: https://github.com/vinayaks439/fluentd-k8s
- Owner: Vinayaks439
- Created: 2021-01-13T14:29:13.000Z (over 5 years ago)
- Default Branch: main
- Last Pushed: 2021-01-22T22:27:32.000Z (over 5 years ago)
- Last Synced: 2025-03-29T13:17:30.497Z (about 1 year ago)
- Topics: fluentd, gcs, k8s-deployment, kubernetes, logging, loki
- Language: Ruby
- Homepage:
- Size: 32.2 KB
- Stars: 1
- Watchers: 2
- Forks: 1
- Open Issues: 0
-
Metadata Files:
- Readme: README.MD
Awesome Lists containing this project
README
This is a repo which contains all necessary yaml files and custom built docker file for install fluentd on your gke cluster.
Note: The configMap used here for fluentd basically focuses on shipping logs to loki and gcs.
For more plugins and configuration setup refer to https://docs.fluentd.org/
Step 1: Build the Dockerfile,
Note: You can add your custom plugins inside Gemfile
Step 2: You can push the obtained image to either any container registry or DockerHub
Note: You will be using this image in your daemonSet.
Step 3: Edit your fluentd configMap according to your preference and apply it using:
kubectl apply -f fluentd-configmap.yaml
Note: I have used loki-grafana as the namespace, please change it according to your preference.
Step 4: Create your Secrets file for gcs service account ( Hoping that your already have the credentials.json file )
kubectl create secret generic -n=loki-grafana fluentd-secret --from-file=./cred.json
Note: This is for shipping logs to GCS bucket ( You can check the env in daemonSet yaml file )
Step 5: Apply RBAC yaml file for basically giving access to fluentd to watch, list and get pods and namespaces.
kubectl apply -f fluentd-rbac.yaml
Step 6: Check the fluentd-daemon.yaml and edit it according to your preference and deploy the fluentd daemonSet.
kubeectl apply -f fluentd-daemon.yaml
Step 7: Check if your pods are running by doing:
kubectl get po -n=
Step 8: There you go if your pods are up and running you can check in Loki if your labels are showing up or not.
If they are showing up, then check url for loki which you have given in configMap.
This url can be either a service running for loki pod or an external endpoint.