{"id":15348190,"url":"https://github.com/ironcore864/efk-in-k8s","last_synced_at":"2025-03-27T14:45:50.109Z","repository":{"id":73865837,"uuid":"145404549","full_name":"IronCore864/efk-in-k8s","owner":"IronCore864","description":null,"archived":false,"fork":false,"pushed_at":"2018-08-20T11:00:49.000Z","size":9,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-02-01T18:44:32.157Z","etag":null,"topics":["devops","efk","kubernetes","logging"],"latest_commit_sha":null,"homepage":null,"language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/IronCore864.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-08-20T10:46:09.000Z","updated_at":"2018-12-06T10:54:17.000Z","dependencies_parsed_at":null,"dependency_job_id":"557d6283-7743-441f-8b66-fd5b0e0e76da","html_url":"https://github.com/IronCore864/efk-in-k8s","commit_stats":{"total_commits":2,"total_committers":1,"mean_commits":2.0,"dds":0.0,"last_synced_commit":"9efa9311e7a3cae2661de799eedd51bc731366b8"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/IronCore864%2Fefk-in-k8s","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/IronCore864%2Fefk-in-k8s/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/IronCore864%2Fefk-in-k8s/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/IronCore864%2Fefk-in-k8s/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/IronCore864","download_url":"https://codeload.github.com/IronCore864/efk-in-k8s/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245864736,"owners_count":20685093,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["devops","efk","kubernetes","logging"],"created_at":"2024-10-01T11:45:17.981Z","updated_at":"2025-03-27T14:45:50.087Z","avatar_url":"https://github.com/IronCore864.png","language":null,"funding_links":[],"categories":[],"sub_categories":[],"readme":"\n# 0 Prerequisites\n\nIf using google cloud for kubernetes as a service:\n\nInstall gcloud CLI on mac: https://cloud.google.com/sdk/docs/quickstart-macos\n\nThen run:\n\n```\ngcloud container clusters get-credentials test-cluster-1\n```\n\nto get access to the k8s cluster\n\nAfter that, you can try to run:\n\n```\nkubectl config get-contexts\n```\n\nto confirm that the cluster is in the output.\n\nIn google cloud kubernetes service, RBAC (role based access control) is enabled by default. In order to deploy the EFK stack into k8s, you will need to grant user permission to create roles in k8s.\n\nRead more about it here: https://cloud.google.com/kubernetes-engine/docs/how-to/role-based-access-control\n\nIn short, you need to run:\n\n```\nkubectl create clusterrolebinding cluster-admin-binding --clusterrole=cluster-admin --user=guotiexin@gmail.com\n```\n\n# 1 Get the Repository for Deploying EFK\n\n```\ngit clone https://github.com/kubernetes/kubernetes.git\n```\n\nActually here only `cluster/addons/fluentd-elasticsearch` folder is needed, contents of which are already in this repo.\n\nThis contains the yaml file for deploying elastic search, fluentd, and kibana.\n\n# 2 Deploy Fluentd\n\n## 2.1 Prerequisites\n\nIn order for Fluentd to work, every Kubernetes node must be labeled with beta.kubernetes.io/fluentd-ds-ready=true, as otherwise the Fluentd DaemonSet will ignore them.\n\n## 2.2 Deploy Fluentd\n\nAt the root directory of the kubernetes repository, do:\n\n```\ncd cluster/addons/fluentd-elasticsearch\nkubectl create -f fluentd-es-configmap.yaml\nkubectl create -f fluentd-es-ds.yaml\n```\n\nThis creates the config map for fluentd, and deploys fluentd as a daemon set.\n\n# 3 Deploy Kibana\n\nUnder the same directory (`repo_root/cluster/addons/fluentd-elasticsearch`), do:\n\n```\nkubectl create -f kibana-deployment.yaml\nkubectl create -f kibana-service.yaml\n```\n\nThis deploys Kibana using \"deployment\", and exposes it as an internal service.\n\n# 4 Deploy Elastic Search\n\nUnder the same directory (`repo_root/cluster/addons/fluentd-elasticsearch`), do:\n\n```\nkubectl create -f es-statefulset.yaml\nkubectl create -f es-service.yaml\n```\n\nThis deploys Kibana using a stateful set, and exposes it as an internal service.\n\n# 5 Test and Verify\n\n## 5.1 Verify services/deployment/pods\n\n```\nkubectl get services --namespace kube-system\nkubectl get deployments --namespace kube-system\nkubectl get daemonset --namespace kube-system\nkubectl get statefulset --namespace kube-system\nkubectl get pods --namespace kube-system\n```\n\nYou should be able to see\n\n- \"elasticsearch-logging\" and \"kibana-logging\" in the service output\n- \"kibana-logging\" in the deployment\n- \"fluentd-es-v2.2.0\" in the daemonset\n- elasticsearch-logging in the statefulset\n\nAnd the pods should be in READY state.\n\n## 5.2 Using Kibana\n\nIn this setup, Kibana is only exposed as an internal service, no internal/external load balancer is created by default, so you need kubernetes proxy in order to access Kibana:\n\n```\nkubectl proxy\n```\n\nThen access:\n\nhttp://127.0.0.1:8001/api/v1/namespaces/kube-system/services/kibana-logging/proxy/\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fironcore864%2Fefk-in-k8s","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fironcore864%2Fefk-in-k8s","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fironcore864%2Fefk-in-k8s/lists"}