{"id":16542448,"url":"https://github.com/estesp/buildkit-cluster-example","last_synced_at":"2026-03-05T22:30:54.373Z","repository":{"id":43185268,"uuid":"177018121","full_name":"estesp/buildkit-cluster-example","owner":"estesp","description":"Simple example for using an in-cluster BuildKit instance for container builds","archived":false,"fork":false,"pushed_at":"2019-03-22T21:19:29.000Z","size":12,"stargazers_count":19,"open_issues_count":0,"forks_count":3,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-03-11T06:54:16.014Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Dockerfile","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/estesp.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-03-21T20:21:24.000Z","updated_at":"2024-09-21T22:16:50.000Z","dependencies_parsed_at":"2022-08-29T08:30:28.002Z","dependency_job_id":null,"html_url":"https://github.com/estesp/buildkit-cluster-example","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/estesp/buildkit-cluster-example","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/estesp%2Fbuildkit-cluster-example","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/estesp%2Fbuildkit-cluster-example/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/estesp%2Fbuildkit-cluster-example/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/estesp%2Fbuildkit-cluster-example/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/estesp","download_url":"https://codeload.github.com/estesp/buildkit-cluster-example/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/estesp%2Fbuildkit-cluster-example/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":30152836,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-05T21:15:50.531Z","status":"ssl_error","status_checked_at":"2026-03-05T21:15:11.173Z","response_time":93,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-11T18:57:33.409Z","updated_at":"2026-03-05T22:30:54.335Z","avatar_url":"https://github.com/estesp.png","language":"Dockerfile","funding_links":[],"categories":["Dockerfile"],"sub_categories":[],"readme":"# BuildKit example for in-cluster build\n\nThis project provides a simple example, using [BuildKit](https://github.com/moby/buildkit) to provide in-cluster\nbuild capabilities without the use of a host-provided Docker runtime (or\nhaving to rely on a Docker-in-Docker configuration).\n\n## Background\n\nOne recurring pattern in the Kubernetes world involves a potential need\nto build containers within the cluster (versus using an out-of-cluster service).\nWhen 99.9% of Kubernetes clusters were built with the Docker\nengine backing each node of the cluster, one common \"hack\" of the wall\nbetween Kubernetes and the host was to use a privileged pod with the Docker API\nsocket mounted, allowing a container to perform `docker build` commands\nagainst the host-installed Docker engine. Of course once a\ncontainer has access to the Docker socket, there are other worrying security\nconcerns given the capability for that special container to now interfere and\ninteract with all other containers on the host. One potential mitigation is\nextra isolation, like a Docker-in-Docker (DinD)-based solution.\n\nFast forward to today and Docker is not necessarily the default container\nruntime underneath many Kubernetes clusters and is potentially not even\ninstalled on cluster worker nodes given the advancement of CRI-compliant runtimes\nlike containerd and cri-o, among others. This anti-pattern of mounting the\nDocker socket in a pod can be replaced with tools like BuildKit, which require\nno Docker API socket and can depend on simply the OCI-compliant `runc` executor,\nor containerd.\n\nThis repository provides a small skeleton example, providing an in-cluster\nbuildkitd pod listening on a TCP port so that in-cluster builds can\nuse a rootless, unprivileged BuildKit to perform container builds.\n\n## Other Options\n\nWhile I wanted to learn more about BuildKit internals and usage specifically, don't\nignore that there are other more polished frameworks that provide this at a higher\nlevel already. Many of them build on BuildKit, or at least provide BuildKit as an\noption. Here is a short list of other projects to consider if you are looking for\nmore complete solutions:\n\n - [img](https://github.com/genuinetools/img) from [jessfraz](https://github.com/jessfraz). Related [blog post](https://blog.jessfraz.com/post/building-container-images-securely-on-kubernetes/)\n - [Kaniko](https://github.com/GoogleContainerTools/kaniko) from [Google's ContainerTools](https://github.com/GoogleContainerTools) project.\n - [Knative build](https://github.com/knative/build), part of the [Knative](https://cloud.google.com/knative/) project. [Nice blog](https://starkandwayne.com/blog/build-docker-images-inside-kubernetes-with-knative-build/) on using it from [DrNic](https://github.com/drnic).\n - More details on different in-cluster build options within Knative in this [blog post](https://www.morethanseven.net/2018/07/29/knative-build-with-docker-buildkit-img/) from Gareth Rushgrove\n\n## Using this example repository\n\n### Expectations\n\nYou have a Kubernetes cluster for which you have administrative access to\ndeploy a pod. The underlying runtime backing your worker nodes is immaterial as\nBuildKit will not use that runtime in this example.\n\nThis example will have you deploy the BuildKit daemon, relying on `runc` as the\nbuild worker, using the pre-built DockerHub image for rootless BuildKit [v0.4.0](https://github.com/moby/buildkit/releases/tag/v0.4.0),\nreleased in March 2019.\n\nYou will then build an image based off the existing v0.4.0 BuildKit image to\nhave the `buildctl` command and a simple `Dockerfile.example` in a single image\nfor use within a Kubernetes batch job to demonstrate calling the BuildKit daemon\npod over TCP to build a container. You can tag and push this image to a registry\nas long as you provide the proper secrets to log into your registry, using\nBuildKit's support for registry authentication and push. The example will use a\nmanually created secret to demonstrate this with DockerHub, but can be used with\nany registry.\n\n### Demonstration steps\n\nSteps below assume you have cloned this repository and your current working directory is in this repository.\n\n1. Deploy the buildkitd pod in your cluster. No modifications should be necessary to [`buildkitd.yaml`](https://github.com/estesp/buildkit-cluster-example/blob/master/buildkitd.yaml):\n\n```\n$ kubectl apply -f buildkitd.yaml\n```\n\n**NOTE:** *For a discussion on why this pod is still privileged, please [read this document](https://github.com/moby/buildkit/blob/v0.4.0/docs/rootless.md#security-consideration) from the BuildKit maintainers. TL;DR work is underway to remove this requirement.*\n\n2. Create a secret with any required Docker credentials from a local Docker installation where you already are logged in to desired registries:\n\n```\n  $ kubectl create secret generic dockerconfig --from-file=/home/\u003cuser\u003e/.docker/config.json\n```\n\n**NOTE:** *This will not work if done from a Mac OS \"Docker Desktop\" installation where system credential helpers are used for storing your credentials.*\n\n3. Modify the [`build-example.yaml`](https://github.com/estesp/buildkit-cluster-example/blob/master/build-example.yaml) batch job spec for your desired settings for (a) the image you want to use, and (b) the push target in the `args` section, after the `--exporter-opt` field. You can either build the [`Dockerfile`](https://github.com/estesp/buildkit-cluster-example/blob/master/Dockerfile) in the root of this\nrepo with your own `Dockerfile.example` (or any other changes you wish to make) or simply use the `estesp/buildexample:1.0` already listed, which has been built from that `Dockerfile` already.\n\n4. When you are satisfied with your copy of the `build-example.yaml` you can apply it to your cluster to run a build against the BuildKit daemon already running, and push an image to your desired target:\n\n```\n$ kubectl apply -f build-example.yaml\n```\n\n5. You can `kubectl get po` to see that your batch job ran properly, and use `kubectl logs` to show the output from your BuildKit run.\n\nIn my case:\n\n```\n$ kubectl get po\nNAME                             READY   STATUS      RESTARTS   AGE\nbuilder-d44n9                    0/1     Completed   0          88m\nbuildkitd-765fcc4854-htxbt       1/1     Running     0          3h41m\n```\n\nAnd my successful image build and push:\n\n```\n$ kubectl logs builder-d44n9\ntime=\"2019-03-22T19:46:01Z\" level=warning msg=\"--exporter \u003cexporter\u003e is deprecated. Please use --output type\n=\u003cexporter\u003e[,\u003copt\u003e=\u003coptval\u003e] instead.\"\ntime=\"2019-03-22T19:46:01Z\" level=warning msg=\"--frontend-opt \u003copt\u003e=\u003coptval\u003e is deprecated. Please use --opt\n \u003copt\u003e=\u003coptval\u003e instead.\"\n\n#2 [internal] load build definition from Dockerfile.example\n#2       digest: sha256:e323acd1c1ceb435985588acb4550e981ed92f1a820e72beea0350fe50ffa621\n#2         name: \"[internal] load build definition from Dockerfile.example\"\n#2      started: 2019-03-22 19:46:01.065994531 +0000 UTC\n#2    completed: 2019-03-22 19:46:01.066102728 +0000 UTC\n#2     duration: 108.197µs\n#2      started: 2019-03-22 19:46:01.066222014 +0000 UTC\n#2    completed: 2019-03-22 19:46:01.079980927 +0000 UTC\n#2     duration: 13.758913ms\n#2 transferring dockerfile: 39B done\n\n\n#1 [internal] load .dockerignore\n#1       digest: sha256:acce6bf27717d36e9a4ea09664e3d0b1342bce019e65d1b679f46203bbcf43b4\n#1         name: \"[internal] load .dockerignore\"\n#1      started: 2019-03-22 19:46:01.065710708 +0000 UTC\n#1    completed: 2019-03-22 19:46:01.065791408 +0000 UTC\n#1     duration: 80.7µs\n#1      started: 2019-03-22 19:46:01.06590812 +0000 UTC\n#1    completed: 2019-03-22 19:46:01.076959812 +0000 UTC\n#1     duration: 11.051692ms\n#1 transferring context: 2B done\n\n\n#3 [internal] load metadata for docker.io/library/alpine:latest\n#3       digest: sha256:d4fb25f5b5c00defc20ce26f2efc4e288de8834ed5aa59dff877b495ba88fda6\n#3         name: \"[internal] load metadata for docker.io/library/alpine:latest\"\n#3      started: 2019-03-22 19:46:01.101981096 +0000 UTC\n#3    completed: 2019-03-22 19:46:01.538928609 +0000 UTC\n#3     duration: 436.947513ms\n\n\n#4 [1/2] FROM docker.io/library/alpine:latest@sha256:644fcb1a676b5165371437...\n#4       digest: sha256:d170cf83c5743380c000554f8ece996d64f35677d36744df8b470ff41cfded6a\n#4         name: \"[1/2] FROM docker.io/library/alpine:latest@sha256:644fcb1a676b5165371437feaa922943aaf7afcf\na8bfee4472f6860aad1ef2a0\"\n#4      started: 2019-03-22 19:46:01.539572137 +0000 UTC\n#4    completed: 2019-03-22 19:46:01.539856953 +0000 UTC\n#4     duration: 284.816µs\n#4 resolve docker.io/library/alpine:latest@sha256:644fcb1a676b5165371437feaa922943aaf7afcfa8bfee4472f6860aad\n1ef2a0 done\n#5 [2/2] RUN touch .i-was-here\n#5       digest: sha256:711798196f4307d0c80e356059f32e62ac46653449125c1c65f467b64f19e658\n#5         name: \"[2/2] RUN touch .i-was-here\"\n#5      started: 2019-03-22 19:46:01.540170107 +0000 UTC\n#5    completed: 2019-03-22 19:46:01.540209588 +0000 UTC\n#5     duration: 39.481µs\n#5       cached: true\n\n\n#6 exporting to image\n#6       digest: sha256:b77b4e1dd2dfbd75d86a8ab4d3a2911fab70ec665fb2a05cb3a5de4f8a048e42\n#6         name: \"exporting to image\"\n#6      started: 2019-03-22 19:46:01.540282029 +0000 UTC\n#6 exporting layers done\n#6 exporting manifest sha256:d2138b9afd64b4f11e89469538ebb158d63087b8b8a42aab9a22d680aceffb9e done\n#6 exporting config sha256:846f932de93756f788053508a6e162c026f2fddbdd22cfe37646a85c07c5ba1d 0.0s done\n#6 pushing layers\n#6 pushing layers 0.4s done\n#6 pushing manifest for docker.io/estesp/mybkimg:latest\n#6    completed: 2019-03-22 19:46:02.070280786 +0000 UTC\n#6     duration: 529.998757ms\n#6 pushing manifest for docker.io/estesp/mybkimg:latest 0.1s done\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Festesp%2Fbuildkit-cluster-example","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Festesp%2Fbuildkit-cluster-example","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Festesp%2Fbuildkit-cluster-example/lists"}