{"id":16812088,"url":"https://github.com/zwwhdls/kubeadmissionwebhookdemo","last_synced_at":"2025-04-11T01:42:54.399Z","repository":{"id":97390779,"uuid":"214677567","full_name":"zwwhdls/KubeAdmissionWebhookDemo","owner":"zwwhdls","description":"🧸A demo about Kubernetes AdmissionWebhook by kubebuilder.","archived":false,"fork":false,"pushed_at":"2024-05-07T13:15:01.000Z","size":89,"stargazers_count":21,"open_issues_count":2,"forks_count":8,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-03-24T22:51:20.793Z","etag":null,"topics":["admission-webhook","demo-app","golang","kubebuilder","kubernetes"],"latest_commit_sha":null,"homepage":"https://blog.hdls.me/15708754600835.html","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/zwwhdls.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-10-12T16:18:46.000Z","updated_at":"2025-01-20T13:29:09.000Z","dependencies_parsed_at":null,"dependency_job_id":"39cf9ef0-4d08-4288-b7de-78c48b79971f","html_url":"https://github.com/zwwhdls/KubeAdmissionWebhookDemo","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zwwhdls%2FKubeAdmissionWebhookDemo","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zwwhdls%2FKubeAdmissionWebhookDemo/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zwwhdls%2FKubeAdmissionWebhookDemo/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zwwhdls%2FKubeAdmissionWebhookDemo/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/zwwhdls","download_url":"https://codeload.github.com/zwwhdls/KubeAdmissionWebhookDemo/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248327644,"owners_count":21085255,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["admission-webhook","demo-app","golang","kubebuilder","kubernetes"],"created_at":"2024-10-13T10:20:39.120Z","updated_at":"2025-04-11T01:42:54.370Z","avatar_url":"https://github.com/zwwhdls.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# KubeAdmissionWebhookDemo\n\n这里使用一个简单的场景做一个演示，我们自定义一个名为 App 资源，当用户创建一个 App 实例时，我们根据用户的描述创建出一个 Deployment。\n\n然后我们添加一个 `MutatingAdmissionWebhook`，当用户通过 App 创建 Deployment 时，自动添加一个 sidecar 容器到 Pod 中（这里使用 nginx 作为 sidecar）。\n\n## 初始化 API 及 Controller\n\n第一步是创建出 CRD 及其 Controller，几行命令就能搞定：\n\n```bash\n$ export GO111MODULE=on\n\n$ mkdir $GOPATH/src/zww-app\n$ cd $GOPATH/src/zww-app\n$ kubebuilder init --domain o0w0o.cn --owner \"zwwhdls\"\n\n$ kubebuilder create api --group app --version v1 --kind App\n```\n\n我这里做的比较简单，`AppSpec` 只定义了一个 deploy 属性（就是 `appsv1.DeploymentSpec`），Controller 中会根据 deploy 属性生成对应的 Deployment：\n\n```go\ntype AppSpec struct {\n\t// INSERT ADDITIONAL SPEC FIELDS - desired state of cluster\n\t// Important: Run \"make\" to regenerate code after modifying this file\n\tDeploy appsv1.DeploymentSpec `json:\"deploy,omitempty\"`\n}\n```\n\n在完善了 `AppSpec` 和 Controller 的 `Reconcile` 函数后，使 Kubebuilder 重新生成代码，并将 `config/crd` 下的 CRD yaml 应用到当前集群：\n\n```bash\nmake\nmake install\n```\n\n## 创建 Webhook Server\n\n接下来就是用 Kubebuilder 来生成 Webhooks 了：\n\n```bash\nkubebuilder create webhook --group app --version v1 --kind App \n```\n\n在路径 `api/v1` 下生成了一个名为 `app_webhook.go` 的文件。可以看到 Kubebuilder 已经帮你定义了两个变量：\n\n```go\nvar _ webhook.Defaulter = \u0026App{}\nvar _ webhook.Validator = \u0026App{}\n```\n\n这两个变量分别表示 MutatingWebhookServer 和 ValidatingWebhookServer，在程序启动的时候，这两个 Server 会 run 起来。\n\n对于 MutatingWebhookServer，Kubebuilder 预留了 `Default()` 函数，让用户来填写自己的逻辑：\n\n```go\n// Default implements webhook.Defaulter so a webhook will be registered for the type\nfunc (r *App) Default() {\n\tapplog.Info(\"default\", \"name\", r.Name)\n\n\t// TODO(user): fill in your defaulting logic.\n}\n```\n\n对于我们希望 Webhook 在资源发生什么样的变化时触发，可以通过这条注释修改：\n\n```go\n// +kubebuilder:webhook:path=/mutate-app-o0w0o-cn-v1-app,mutating=true,failurePolicy=fail,groups=app.o0w0o.cn,resources=apps,verbs=create;update,versions=v1,name=mapp.kb.io\n```\n\n对应的参数为：\n\n- failurePolicy：表示 ApiServer 无法与 webhook server 通信时的失败策略，取值为 \"ignore\" 或 \"fail\"；\n- groups：表示这个 webhook 在哪个 Api Group 下会收到请求；\n- mutating：这个参数是个 bool 型，表示是否是 mutating 类型；\n- name：webhook 的名字，需要与 configuration 中对应；\n- path：webhook 的 path；\n- resources：表示这个 webhook 在哪个资源发生变化时会收到请求；\n- verbs：表示这个 webhook 在资源发生哪种变化时会收到请求，取值为 “create“, \"update\", \"delete\", \"connect\", 或 \"*\" (即所有)；\n- versions：表示这个 webhook 在资源的哪个 version 发生变化时会收到请求；\n\n对于 ValidatingWebhookServer，Kubebuilder 的处理与 MutatingWebhookServer 一致，这里不再赘述。\n\n方便起见，我只定义了 MutatingWebhookServer 的 `Default` 函数，为每个 App 类型资源的 pod 注入一个 nginx sidecar 容器：\n\n```go\nfunc (r *App) Default() {\n\tapplog.Info(\"default\", \"name\", r.Name)\n\tvar cns []core.Container\n\tcns = r.Spec.Deploy.Template.Spec.Containers\n\n\tcontainer := core.Container{\n\t\tName:  \"sidecar-nginx\",\n\t\tImage: \"nginx:1.12.2\",\n\t}\n\n\tcns = append(cns, container)\n\tr.Spec.Deploy.Template.Spec.Containers = cns\n}\n```\n\n## 运行 Webhook Server\n\n首先需要将 MutatingWebhookConfiguration 稍作修改，使得 ApiServer 能够与 Webhook Server 通信。具体方法如下：\n\n### 配置 Server Path\n\n本文仅分享本地开发测试的调试方案，线上部署方案请参考[官方文档](https://book.kubebuilder.io/cronjob-tutorial/running.html)。\n\n第一步，配置 Server Path；将 service 去掉，换成 `url: https://\u003cserver_ip\u003e:9443/mutate-app-o0w0o-cn-v1-app` ，其中 `server_ip` 是 Webhook Server 的 ip，如果运行在本地，就是本地的 ip。需要注意的是 url 中的 path 要与 `app_webhook.go` 中定义的保持一致。\n\n### 配置证书\n\n第二步，配置 caBundle；由于在 Kube 里，所有与 ApiServer 交互的组件都需要与 ApiServer 进行双向 TLS 认证，我们这里需要先手动签发自签名 CA 证书：\n\n```bash\n$ openssl genrsa -out ca.key 2048\n$ openssl req -x509 -new -nodes -key ca.key -subj \"/CN=\u003cserver_ip\u003e\" -days 10000 -out ca.crt\n$ openssl genrsa -out server.key 2048\n$ cat \u003c\u003c EOF \u003ecsr.conf\n\u003e [ req ]\n\u003e default_bits = 2048\n\u003e prompt = no\n\u003e default_md = sha256\n\u003e req_extensions = req_ext\n\u003e distinguished_name = dn\n\u003e \n\u003e [ dn ]\n\u003e C = \u003ccountry\u003e\n\u003e ST = \u003cstate\u003e\n\u003e L = \u003ccity\u003e\n\u003e O = \u003corganization\u003e\n\u003e OU = \u003corganization unit\u003e\n\u003e CN = \u003cserver_ip\u003e\n\u003e \n\u003e [ req_ext ]\n\u003e subjectAltName = @alt_names\n\u003e \n\u003e [ alt_names ]\n\u003e IP.1 = \u003cserver_ip\u003e\n\u003e \n\u003e [ v3_ext ]\n\u003e authorityKeyIdentifier=keyid,issuer:always\n\u003e basicConstraints=CA:FALSE\n\u003e keyUsage=keyEncipherment,dataEncipherment\n\u003e extendedKeyUsage=serverAuth,clientAuth\n\u003e subjectAltName=@alt_names\n\u003e EOF\n$ openssl req -new -key server.key -out server.csr -config csr.conf\n$ openssl x509 -req -in server.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out server.crt -days 10000 -extensions v3_ext -extfile csr.conf\n```\n\n证书生成后将 `server.key` 和 `server.crt` 拷贝到 Kubebuilder 设置的 webhook server 的私钥和证书路径下：\n\nwebhook server 的私钥路径：`$(TMPDIR)/k8s-webhook-server/serving-certs/tls.key`\nwebhook server 的证书路径：`$(TMPDIR)/k8s-webhook-server/serving-certs/tls.crt`\n\n*注：如果 $(TMPDIR) 为空，则默认路径为 \"/tmp/k8s-webhook-server/...\"，但 android 系统默认路径为 \"/data/local/tmp/k8s-webhook-server/...\"*\n\n而 MutatingWebhookConfiguration 中的 caBundle 为 ca.crt 的 base64 编码结果。最终 yaml 结果为：\n\n```yaml\napiVersion: admissionregistration.k8s.io/v1beta1\nkind: MutatingWebhookConfiguration\nmetadata:\n  creationTimestamp: null\n  name: mutating-webhook-configuration\nwebhooks:\n- clientConfig:\n  caBundle: LS0tLS1CRUdJTiBDRVJ...FLS0tLS0=\n  url: https://\u003cserver_ip\u003e:9443/mutate-app-o0w0o-cn-v1-app\n  failurePolicy: Fail\n  name: mapp.kb.io\n  rules:\n    ...\n```\n\nValidatingWebhookConfiguration 的修改与 MutatingWebhookConfiguration 类似，只需要注意 server path 与 `app_webhook.go` 中一致即可。两个配置文件都修改好之后在集群中 apply 一下即可。\n\n### 运行\n\n最后直接在本地运行 CRD Controller 及 Webhook Server：\n\n```bash\nmake run\n```\n\n### 验证\n\n简单运行一个 app 试试：\n\n```yaml\napiVersion: app.o0w0o.cn/v1\nkind: App\nmetadata:\n  name: app-sample\nspec:\n  deploy:\n    selector:\n      matchLabels:\n        app: app-sample\n    template:\n      metadata:\n        name: sample\n        labels:\n          app: app-sample\n      spec:\n        containers:\n          - name: cn\n            image: daocloud.io/library/redis:4.0.14-alpine\n```\n\n查看是否已经注入了 sidecar 容器：\n\n```bash\n$ kubectl apply -f config/samples/app_v1_app.yaml\n$ kubectl get app\nNAME         AGE\napp-sample   43s\n$ kubectl get deploy\nNAME                READY   UP-TO-DATE   AVAILABLE   AGE\napp-sample-deploy   0/1     1            0           43s\n$ kubectl get po\nNAME                                 READY   STATUS              RESTARTS   AGE\napp-sample-deploy-5b5cfb9c9b-z8jk5   0/2     ContainerCreating   0          43s\n```\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzwwhdls%2Fkubeadmissionwebhookdemo","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fzwwhdls%2Fkubeadmissionwebhookdemo","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzwwhdls%2Fkubeadmissionwebhookdemo/lists"}