{"id":18451795,"url":"https://github.com/norbjd/kueueleuleu","last_synced_at":"2026-01-06T06:13:07.991Z","repository":{"id":214790180,"uuid":"737079391","full_name":"norbjd/kueueleuleu","owner":"norbjd","description":"Run containers sequentially inside Kubernetes Pods/Jobs/CronJobs.","archived":false,"fork":false,"pushed_at":"2024-05-31T18:16:38.000Z","size":64,"stargazers_count":2,"open_issues_count":1,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-01-31T07:43:22.528Z","etag":null,"topics":["containers","cronjobs","go","jobs","kubernetes","pods","sequential"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/norbjd.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"COPYING","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-12-29T18:48:47.000Z","updated_at":"2024-05-31T18:16:41.000Z","dependencies_parsed_at":null,"dependency_job_id":"ffbcad46-141e-4b41-a15c-7a79282c1a86","html_url":"https://github.com/norbjd/kueueleuleu","commit_stats":null,"previous_names":["norbjd/kueueleuleu"],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/norbjd%2Fkueueleuleu","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/norbjd%2Fkueueleuleu/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/norbjd%2Fkueueleuleu/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/norbjd%2Fkueueleuleu/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/norbjd","download_url":"https://codeload.github.com/norbjd/kueueleuleu/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245598314,"owners_count":20641884,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["containers","cronjobs","go","jobs","kubernetes","pods","sequential"],"created_at":"2024-11-06T07:29:38.473Z","updated_at":"2026-01-06T06:13:07.985Z","avatar_url":"https://github.com/norbjd.png","language":"Go","funding_links":[],"categories":["Go"],"sub_categories":[],"readme":"# kueueleuleu\n\nRun containers sequentially inside Kubernetes `Pod`s, `Job`s, `CronJob`s.\n\nThe name `kueueleuleu` (`\\kø lø.lø\\`) comes from the French [`queue leu-leu`](https://fr.wiktionary.org/wiki/%C3%A0_la_queue_leu-leu), meaning \"one after the other\" or \"[in a single file](https://youtu.be/eIRbCW3vH-Y?t=106)\". The initial `k` is obviously for `kubernetes`.\n\n## Motivation\n\nAs of now, Kubernetes does not support running containers inside a `Pod` sequentially, despite this being a common thing asked by users:\n\n- [How to run containers sequentially as a Kubernetes job?](https://stackoverflow.com/questions/40713573/how-to-run-containers-sequentially-as-a-kubernetes-job)\n- [How to run multiple Kubernetes jobs in sequence?](https://stackoverflow.com/questions/48029943/how-to-run-multiple-kubernetes-jobs-in-sequence)\n- [Kubernetes — Sequencing Container Startup, by Paul Dally](https://pauldally.medium.com/sequencing-container-startup-ab30965c067d)\n- [How to start one container before staring another in the same pod?](https://groups.google.com/g/kubernetes-users/c/JqvIuUmt5fk)\n\nWithout an external way to control the containers run order, all containers inside a `Pod` start and run at the same time.\n\n### Existing known solutions\n\n\u003cdetails\u003e\n\u003csummary\u003eTL;DR: Convert your containers to init containers OR use external workflow solutions.\u003c/summary\u003e\n\n#### Use init containers\n\n[Init containers](https://kubernetes.io/docs/concepts/workloads/pods/init-containers/#differences-from-regular-containers) always run sequentially:\n\n\u003e If you specify multiple init containers for a Pod, kubelet runs each init container sequentially. Each init container must succeed before the next can run.\n\nThus, a common workaround to allow running containers sequentially is to convert all containers to init containers, but:\n\n- this requires to change the pod spec ourselves, which is not always possible or might feel \"hacky\"\n- init containers have some restrictions (e.g. does not support probes, handles resource requests/limits differently than regular containers)\n\n#### Use external tools designed for workflows\n\nThere are some solutions to define complex containers workflows (and among them, running containers sequentially):\n\n- [Argo Workflows](https://argoproj.github.io/workflows)\n- [Tekton](https://tekton.dev)\n\nBut, for such a simple use-case (run containers one after the other), they might seem overkill:\n\n- need to install (and maintain) a fully-fledged tool\n- need to use specific CRDs (Argo `Workflow` or Tekton `Pipeline`)\n\u003c/details\u003e\n\n### Kueueleuleu!\n\nUnlike the known solutions mentioned above, this simple library aims to:\n\n- hide the sequential containers orchestration logic from the user\n- let users manipulate raw kubernetes objects (`Pod`s, `Job`s, `CronJob`s), so they can use everything natively supported by Kubernetes (volumes, runtime classes, etc.)\n- provide an easy way to convert any of these kubernetes objects to `kueueleuleu` objects\n\n## How to use?\n\nThere are two ways to use `kueueleuleu`:\n\n- using the CLI\n- using the Go library\n\n### Using the CLI\n\n#### Install the CLI\n\n##### With `go install`\n\n\u003e [!WARNING]  \n\u003e This requires Go \u003e= 1.21.\n\n```shell\nVERSION=v0.1.1\ngo install github.com/norbjd/kueueleuleu/cmd/kueueleuleu@$VERSION\n```\n\nThe binary will be located in `$GOPATH/bin` by default.\n\n##### Download (and optionally verify) the binary\n\n```shell\nVERSION=v0.1.1\n\n# change accordingly to your OS/arch\nOS=linux # or darwin\nARCH=amd64 # or arm64\n\ncurl -Lo kueueleuleu \"https://github.com/norbjd/kueueleuleu/releases/download/$VERSION/kueueleuleu-$OS-$ARCH\"\nchmod u+x kueueleuleu\n\n# optional: verify the provenance with slsa-verifier (https://github.com/slsa-framework/slsa-verifier)\n# this ensures authenticity and trustworthiness of the binary\ncurl -Lo kueueleuleu.intoto.jsonl \"https://github.com/norbjd/kueueleuleu/releases/download/$VERSION/kueueleuleu-$OS-$ARCH.intoto.jsonl\"\n\n# should display \"PASSED: Verified SLSA provenance\"\nslsa-verifier verify-artifact kueueleuleu \\\n  --provenance-path kueueleuleu.intoto.jsonl \\\n  --source-uri github.com/norbjd/kueueleuleu \\\n  --source-tag $VERSION\n```\n\nOnce downloaded and verified, you can move it to `/usr/local/bin` (or somewhere else in your `$PATH`).\n\n#### Use the CLI\n\n```shell\n# create a simple pod with two containers: by default, both will run at the same time\ncat \u003e simplepod.yaml \u003c\u003c'EOF'\napiVersion: v1\nkind: Pod\nmetadata:\n  name: two-steps-pod\nspec:\n  containers:\n    - name: step1\n      image: alpine\n      command:\n        - sh\n        - -c\n      args:\n        - echo \"start step1\" \u0026\u0026 sleep 5 \u0026\u0026 echo \"end step1\"\n    - name: step2\n      image: alpine\n      command:\n        - sh\n        - -c\n      args:\n        - echo \"start step2\" \u0026\u0026 sleep 2 \u0026\u0026 echo \"end step2\"\n  restartPolicy: Never\nEOF\n\n# convert this pod to run containers sequentially and apply\n# if you like chaining commands, this is similar to: cat simplepod.yaml | kueueleuleu -f - | kubectl apply -f -\n# if kueueleuleu is not in your PATH, replace with /path/to/kueueleuleu\nkueueleuleu -f simplepod.yaml | kubectl apply -f -\n```\n\nOnce the pod is finished, check the logs (`kubectl logs two-steps-pod --all-containers --timestamps | sort`) to see containers have been executed sequentially (first, `step1`, and then `step2`):\n\n```\n2023-12-29T13:32:50.488543841Z 2023/12/29 13:32:50 Entrypoint initialization\n2023-12-29T13:32:52.139027305Z start step1\n2023-12-29T13:32:57.139823758Z end step1\n2023-12-29T13:32:57.392700813Z start step2\n2023-12-29T13:32:59.393583491Z end step2\n```\n\n\u003e [!NOTE]\n\u003e The first log is related to internal logic: the `entrypoint` mentioned is here to order the containers' execution. You can ignore this, unless you are interested in the internals (see \"Internals\" section).\n\nWithout `kueueleuleu` (`kubectl apply -f simplepod.yaml`), containers logs are intertwined because both containers are running at the same time:\n\n```\n2023-12-29T13:33:43.942542903Z start step1\n2023-12-29T13:33:44.848147388Z start step2\n2023-12-29T13:33:46.848812173Z end step2\n2023-12-29T13:33:48.943165982Z end step1\n```\n\nConversion also work with `Job`s and `CronJob`s, and even with YAML files containing multiple resources (see `cmd/testdata/*_input.yaml` for examples, and `cmd/testdata/*_output.yaml` for results after using `kueueleuleu`).\n\n### Using the library\n\nFirst, run `go get github.com/norbjd/kueueleuleu` to download the dependency.\n\n```go\npackage main\n\nimport (\n    \"context\"\n    \"fmt\"\n    \"os\"\n\n    \"github.com/norbjd/kueueleuleu\"\n    corev1 \"k8s.io/api/core/v1\"\n    metav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n    \"k8s.io/client-go/kubernetes\"\n    \"k8s.io/client-go/tools/clientcmd\"\n)\n\nfunc main() {\n    // original pod: containers will run at the same time\n    twoStepsPod := corev1.Pod{\n        ObjectMeta: metav1.ObjectMeta{\n            Name: \"two-steps-pod\",\n        },\n        Spec: corev1.PodSpec{\n            Containers: []corev1.Container{\n                {\n                    Name:    \"step1\",\n                    Image:   \"alpine\",\n                    Command: []string{\"sh\", \"-c\"},\n                    Args:    []string{`echo \"start step1\" \u0026\u0026 sleep 5 \u0026\u0026 echo \"end step1\"`},\n                },\n                {\n                    Name:    \"step2\",\n                    Image:   \"alpine\",\n                    Command: []string{\"sh\", \"-c\"},\n                    Args:    []string{`echo \"start step2\" \u0026\u0026 sleep 2 \u0026\u0026 echo \"end step2\"`},\n                },\n            },\n            RestartPolicy: \"Never\",\n        },\n    }\n\n    // convert the pod to run containers sequentially\n    twoSequentialStepsPod, err := kueueleuleu.ConvertPod(twoStepsPod)\n    if err != nil {\n        panic(err)\n    }\n\n    // create the pod\n    config, _ := clientcmd.BuildConfigFromFlags(\"\", os.Getenv(\"KUBECONFIG\"))\n    kubeClient, _ := kubernetes.NewForConfig(config)\n\n    createdPod, err := kubeClient.CoreV1().Pods(\"default\").Create(\n        context.Background(), \u0026twoSequentialStepsPod, metav1.CreateOptions{},\n    )\n    if err != nil {\n        panic(err)\n    }\n\n    fmt.Printf(\"Created: %s\\n\", createdPod.Name)\n}\n```\n\nThe main advantage of `kueueleuleu` is that you can continue to manipulate standard kubernetes resources (like `Pod`s) in your code. There is only a single helper if you want to know which container inside the pod is really running (as now they are running sequentially):\n\n```go\ncurrentlyRunningContainerName, _ := kueueleuleu.GetRunningContainerName(*createdPod)\n\nfmt.Printf(\"Running container: %s\\n\", currentlyRunningContainerName)\n```\n\nAs for the CLI, the conversion also work with `Job`s and `CronJob`s: just use `kueueleuleu.ConvertJob` or `kueueleuleu.ConvertCronJob`.\n\n## Internals\n\nUnder the hood, containers sequential orchestration is managed using [Tekton entrypoint](https://github.com/tektoncd/pipeline/blob/v0.55.0/cmd/entrypoint/README.md). I have just \"reverse-engineered\" the way Tekton generates `Pod`s from `PipelineRun`. But, unlike Tekton, there is no need to install a separate controller in the cluster or using `CRD`s, which makes `kueueleuleu` lighter to use. In return, `kueueleuleu` cannot be used for complex workflows, and I don't consider supporting these: its **only** job is to run containers **sequentially**.\n\nI have used Tekton entrypoint because it is already doing the job, and I didn't want to rewrite another wrapper to do more or less the same thing.\n\nAnother solution I have considered had been to convert containers to init containers (see \"Use init containers\" section above). But, considering init containers limitations, I thought this solution was not viable.\n\n## Limitations\n\nContainers passed in objects (`Pod`, `Job`, `CronJob`) **MUST** have their `command` field set. Otherwise, `kueueleuleu` will return an error (`container does not have a command`). This is because Tekton entrypoint used under the hood requires a command. Tekton Pipelines (who also uses this entrypoint) manages to \"guess\" the entrypoint if a command is not provided (`entrypoint hack` refered [here](https://github.com/tektoncd/pipeline/issues/6877#issuecomment-1618082473)). But, I'd rather not implement this here today as it does not always work (e.g. if image is not pushed into a registry, like in `kind` environments, we will fail to guess the entrypoint).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnorbjd%2Fkueueleuleu","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnorbjd%2Fkueueleuleu","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnorbjd%2Fkueueleuleu/lists"}