{"id":13581283,"url":"https://github.com/iximiuz/cdebug","last_synced_at":"2025-04-13T13:12:45.905Z","repository":{"id":62012821,"uuid":"555891400","full_name":"iximiuz/cdebug","owner":"iximiuz","description":"cdebug - a swiss army knife of container debugging","archived":false,"fork":false,"pushed_at":"2025-03-12T10:07:59.000Z","size":555,"stargazers_count":1479,"open_issues_count":14,"forks_count":55,"subscribers_count":20,"default_branch":"main","last_synced_at":"2025-04-04T14:41:08.692Z","etag":null,"topics":["containerd","containers","debug","distroless","docker","kubernetes"],"latest_commit_sha":null,"homepage":"https://iximiuz.com/en/posts/docker-debug-slim-containers/","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/iximiuz.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-10-22T15:22:44.000Z","updated_at":"2025-04-03T15:25:07.000Z","dependencies_parsed_at":"2024-01-14T04:40:55.853Z","dependency_job_id":"322f3bea-a268-4d48-bf2e-45b49043b44f","html_url":"https://github.com/iximiuz/cdebug","commit_stats":{"total_commits":91,"total_committers":7,"mean_commits":13.0,"dds":0.1428571428571429,"last_synced_commit":"3638de2c208ad5bfba29a7303be8a40fedf38689"},"previous_names":[],"tags_count":18,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/iximiuz%2Fcdebug","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/iximiuz%2Fcdebug/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/iximiuz%2Fcdebug/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/iximiuz%2Fcdebug/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/iximiuz","download_url":"https://codeload.github.com/iximiuz/cdebug/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248717237,"owners_count":21150389,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["containerd","containers","debug","distroless","docker","kubernetes"],"created_at":"2024-08-01T15:01:59.960Z","updated_at":"2025-04-13T13:12:45.883Z","avatar_url":"https://github.com/iximiuz.png","language":"Go","readme":"# cdebug - a swiss army knife of container debugging\n\n```diff\n! Support development of this project \u003e patreon.com/iximiuz\n```\n\nWith this tool you can:\n\n- Troubleshoot containers and pods lacking shell and/or debugging tools.\n- Forward unpublished or even localhost ports to your host system.\n- Expose endpoints from the host system to containers \u0026 Kubernetes networks.\n- Handily export image's and/or container's filesystem to local folders.\n- and more :)\n\nThe following _commands_ x _runtimes_ are supported:\n\n|                       | Docker | Podman | containerd | OCI (runc, crun) | Kubernetes | CRI    |\n| :---                  | :---:  | :---:  | :---:      | :---:            | :---:      | :---:  |\n| `exec`                | ✅     | -      | ✅         | -                | ✅          | -      |\n| `port-forward` local  | ✅     | -      | -          | -                | -          | -      |\n| `port-forward` remote | 🛠️      | -      | -          | -                | -          | -      |\n| `export`              | -      | -      | -          | -                | -          | -      |\n\n## Installation\n\nIt's a statically linked Go binary, so you know the drill:\n\n```sh\nGOOS=linux\nGOARCH=amd64\n\ncurl -Ls https://github.com/iximiuz/cdebug/releases/latest/download/cdebug_${GOOS}_${GOARCH}.tar.gz | tar xvz\n\nsudo mv cdebug /usr/local/bin\n```\n\n### Homebrew\n\nIf you're a [Homebrew](https://brew.sh/) user, you can install the tool via brew on macOS or Linux:\n\n```sh\n$ brew install cdebug\n```\n\nAt the moment, the following systems are (kinda sorta) supported:\n\n- linux/amd64\n- darwin/amd64\n- darwin/arm64\n\n## Commands\n\n### cdebug exec\n\nExecute commands or start interactive shells in scratch, slim, or distroless containers, with ease:\n\n```sh\n# Start a busybox:musl shell in the Docker container:\ncdebug exec -it mycontainer\ncdebug exec -it docker://mycontainer\n\n# Execute a command in the Docker container:\ncdebug exec mycontainer cat /etc/os-release\n\n# Use a different debugging toolkit image:\ncdebug exec -it --image=alpine mycontainer\n\n# Use a nixery.dev image (https://nixery.dev/):\ncdebug exec -it --image=nixery.dev/shell/vim/ps/tshark mycontainer\n\n# Exec into a containerd container:\ncdebug exec -it containerd://mycontainer ...\ncdebug exec --namespace myns -it containerd://mycontainer ...\n\n# Exec into a nerdctl container:\ncdebug exec -it nerdctl://mycontainer ...\n\n# Start a shell in a Kubernetes pod:\ncdebug exec -it pod/mypod\ncdebug exec -it k8s://mypod\ncdebug exec --namespace=myns -it pod/mypod\n\n# Start a shell in a Kubernetes pod's container:\ncdebug exec -it pod/mypod/mycontainer\n```\n\nThe `cdebug exec` command is a crossbreeding of `docker exec` and `kubectl debug` commands.\nYou point the tool at a running container, say what toolkit image to use, and it starts\na debugging \"sidecar\" container that _feels_ like a `docker exec` session to the target container:\n\n- The root filesystem of the debugger **_is_** the root filesystem of the target container.\n- The target container isn't recreated and/or restarted.\n- No extra volumes or copying of debugging tools is needed.\n- The debugging tools **_are_** available in the target container.\n\nBy default, the `busybox:musl` (statically compiled) image is used for the debugger sidecar, but you can override it\nwith the `--image` flag. Combining this with the superpower of Nix and [Nixery](https://nixery.dev/),\nyou can get all your favorite tools by simply listing them in the image name:\n\n```\ncdebug exec -it --image nixery.dev/shell/ps/vim/tshark \u003ctarget-container\u003e\n```\n\n\u003cdetails\u003e\n\u003csummary\u003eHow it works\u003c/summary\u003e\n\nThe technique is based on the ideas from this [blog post](https://iximiuz.com/en/posts/docker-debug-slim-containers).\n\n![How: cdebug exec](assets/images/cdebug-exec.png)\n\nOversimplifying, the debugger container is started like:\n\n```sh\ndocker run [-it] \\\n  --network container:\u003ctarget\u003e \\\n  --pid container:\u003ctarget\u003e \\\n  --uts container:\u003ctarget\u003e \\\n  \u003ctoolkit-image\u003e\n  sh -c \u003c\u003cEOF\nln -s /proc/$$/root/bin/ /proc/1/root/.cdebug\n\nexport PATH=$PATH:/.cdebug\nchroot /proc/1/root sh\nEOF\n```\n\nThe secret sauce is the symlink + PATH modification + chroot-ing.\n\n\u003c/details\u003e\n\n### cdebug port-forward\n\nForward local ports to containers and vice versa. This command is another crossbreeding -\nthis time it's `kubectl port-forward` and `ssh -L|-R`.\n\nCurrently, only local port forwarding (`cdebug port-forward -L`) is supported,\nbut remote port forwarding is under active development.\n\nLocal port forwarding use cases (works for Docker Desktop too!):\n\n- Publish \"unpublished\" port 80 to a random port on the host: `cdebug port-forward \u003ctarget\u003e -L 80`\n- Expose container's localhost to the host system: `cdebug port-forward \u003ctarget\u003e -L 127.0.0.1:5432`\n- Proxy local traffic to a remote host via the target: `cdebug port-forward \u003ctarget\u003e -L \u003cLOCAL_HOST\u003e:\u003cLOCAL_PORT\u003e:\u003cREMOTE_HOST\u003e:\u003cREMOTE_PORT\u003e`\n- 🛠️ Expose a Kubernetes service to the host system: `cdebug port-forward \u003ctarget\u003e -L 8888:my.svc.cluster.local:443`\n\nRemote port forwarding use cases:\n\n- Start a container/Pod forwarding traffic destined to its `\u003cIP\u003e:\u003cport\u003e` to a non-cluster endpoint reachable from the host system.\n- ...\n\n\u003cdetails\u003e\n\u003csummary\u003eHow it works\u003c/summary\u003e\n\n**Local port forwarding** is implemented by starting an extra forwarder container in the\ntarget's network and publishing its ports to the host using the standard means (e.g.,\n`docker run --publish`). The forwarder container itself runs something like:\n\n`socat TCP-LISTEN:\u003cREMOTE_PORT\u003e,fork TCP-CONNECT:\u003cREMOTE_HOST\u003e:\u003cREMOTE_PORT\u003e`\n\n![How: cdebug port-forward -L (direct)](assets/images/cdebug-port-forward-local-direct.png)\n\nIf the _REMOTE_HOST_ doesn't belong to the target or it's the target's localhost,\nan extra sidecar container is started in the target's network namespace with another\nsocat forwarding traffic from the target public interface to `REMOTE_HOST:REMOTE_PORT`.\n\n![How: cdebug port-forward -L (sidecar)](assets/images/cdebug-port-forward-local-sidecar.png)\n\n**Remote port forwarding** will use similar tricks but combined with more advanced\nreverse tunneling.\n\n\u003c/details\u003e\n\n## Examples\n\nBelow are a few popular scenarios formatted as reproducible demos.\n\n### A simple interactive shell to a distroless container\n\nFirst, a target container is needed. Let's use a distroless nodejs image for that:\n\n```sh\n$ docker run -d --rm \\\n  --name my-distroless gcr.io/distroless/nodejs \\\n  -e 'setTimeout(() =\u003e console.log(\"Done\"), 99999999)'\n```\n\nNow, let's start an interactive shell (using busybox) into the above container:\n\n```sh\n$ cdebug exec -it my-distroless\n```\n\nExploring the filesystem shows that it's a rootfs of the nodejs container:\n\n```sh\n/ $# ls -lah\ntotal 60K\ndrwxr-xr-x    1 root     root        4.0K Oct 17 23:49 .\ndrwxr-xr-x    1 root     root        4.0K Oct 17 23:49 ..\n👉 lrwxrwxrwx 1 root     root          18 Oct 17 23:49 .cdebug-c153d669 -\u003e /proc/55/root/bin/\n-rwxr-xr-x    1 root     root           0 Oct 17 19:49 .dockerenv\ndrwxr-xr-x    2 root     root        4.0K Jan  1  1970 bin\ndrwxr-xr-x    2 root     root        4.0K Jan  1  1970 boot\ndrwxr-xr-x    5 root     root         340 Oct 17 19:49 dev\ndrwxr-xr-x    1 root     root        4.0K Oct 17 19:49 etc\ndrwxr-xr-x    3 nonroot  nonroot     4.0K Jan  1  1970 home\ndrwxr-xr-x    1 root     root        4.0K Jan  1  1970 lib\ndrwxr-xr-x    2 root     root        4.0K Jan  1  1970 lib64\ndrwxr-xr-x    5 root     root        4.0K Jan  1  1970 nodejs\n...\n```\n\nNotice 👉  above - that's where the debugging tools live:\n\n```sh\n/ $# echo $PATH\n/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/.cdebug-c153d669\n```\n\nThe process tree of the debugger container is the process tree of the target:\n\n```sh\n/ $# ps auxf\nPID   USER     TIME  COMMAND\n    1 root      0:00 /nodejs/bin/node -e setTimeout(() =\u003e console.log(\"Done\"),\n   13 root      0:00 sh -c  set -euo pipefail  sleep 999999999 \u0026 SANDBOX_PID=$!\n   19 root      0:00 sleep 999999999\n   21 root      0:00 sh\n   28 root      0:00 [sleep]\n   39 root      0:00 [sleep]\n   45 root      0:00 ps auxf\n```\n\n### An interactive shell with code editor (vim)\n\nIf the tools provided by busybox aren't enough, you can bring your own tools with\na ~~little~~ huge help of the [nixery](https://nixery.dev/) project:\n\n```sh\n$ cdebug exec -it --image nixery.dev/shell/vim my-distroless\n```\n\n### An interactive shell with tshark and other advanced tools\n\nEven more powerful exammple:\n\n```sh\n$ cdebug exec -it --image nixery.dev/shell/ps/findutils/tshark my-distroless\n```\n\n### Debugging containerd containers (no Docker required)\n\nFirst, start the target container:\n\n```sh\n$ sudo ctr image pull docker.io/library/nginx:latest\n$ sudo ctr run -d docker.io/library/nginx:latest nginx-1\n```\n\nRun an interactive shell in the target container using simple `cdebug exec`:\n\n```\n$ sudo cdebug exec -it containerd://nginx-1\n/ $# wget -O- 127.0.0.1\n```\n\nRun VIM in the target container using `cdebug exec --image nixery.dev/shell/vim`:\n\n```sh\n$ sudo cdebug exec -it --rm --image nixery.dev/shell/vim containerd://nginx-1\n```\n\n### Debugging nerdctl containers (no Docker required)\n\nStart a container using nerdctl:\n\n```sh\n$ sudo $(which nerdctl) run -d --name nginx-1 nginx\n9f8763d82259a6e3e747df83d0ce8b7ee3d33d94269a72cd04e0e3862a3abc5f\n```\n\nRun the debugger using the `nerdctl://` schema and the target's name:\n\n```sh\n$ sudo cdebug exec -it --rm nerdctl://nginx-1\n```\n\nOr run a debugging session in the above container using the `containerd://` schema:\n\n```sh\n$ sudo cdebug exec -it --rm containerd://9f876\n```\n\n### Debugging Kubernetes Pods (without node access)\n\nStart the target Pod:\n\n```sh\n$ kubectl run --image nginx:alpine nginx-1\n$ kubectl run --image=nginx:alpine nginx-1 \\\n  --overrides='{ \"apiVersion\": \"v1\", \"spec\": { \"containers\": [{ \"name\": \"app\", \"image\": \"nginx:alpine\" }] } }'\npod/nginx-1 created\n\n$ kubectl get pods\nNAME    READY   STATUS    RESTARTS   AGE\nnginx-1   1/1     Running   0         5s\n```\n\nRun the debugger in the Pod (it'll start a new ephemeral container):\n\n```sh\n$ cdebug exec -it pod/nginx-1\n```\n\nExpected output:\n\n```text\nDebugger container name: cdebug-3023d11d\nStarting debugger container...\nWaiting for debugger container...\nAttaching to debugger container...\nIf you don't see a command prompt, try pressing enter.\n/ # ps auxf\nPID   USER     TIME  COMMAND\n    1 root      0:00 sh /.cdebug-entrypoint.sh\n   10 root      0:00 /bin/sh -i\n   11 root      0:00 ps auxf\n```\n\nRun the debugger in the Nginx container (`app`):\n\n```sh\n$ cdebug exec -it pod/nginx-1/app\n```\n\n```text\ncdebug exec -it pod/nginx-1/app\nDebugger container name: cdebug-b44ca485\nStarting debugger container...\nWaiting for debugger container...\nAttaching to debugger container...\nIf you don't see a command prompt, try pressing enter.\n/ # ps auxf\nPID   USER     TIME  COMMAND\n    1 root      0:00 nginx: master process nginx -g daemon off;\n   30 nginx     0:00 nginx: worker process\n   ...\n   41 nginx     0:00 nginx: worker process\n   42 root      0:00 sh /.cdebug-entrypoint.sh\n   51 root      0:00 /bin/ash -i\n   52 root      0:00 ps auxf\n```\n\n### Debugging Kubernetes Pods (with node access)\n\nCurrently, only containerd CRI is supported. First, you'll need to list the running\ncontainers:\n\n```sh\n$ ctr -n k8s.io container ls\nCONTAINER       IMAGE                                       RUNTIME\n155227c0e9aa8   k8s.gcr.io/pause:3.5                        io.containerd.runc.v2\n2220eacd9cb26   registry.k8s.io/kube-apiserver:v1.25.3      io.containerd.runc.v2\n22efcb35a651a   registry.k8s.io/etcd:3.5.4-0                io.containerd.runc.v2\n28e06cc63b822   docker.io/calico/cni:v3.24.1                io.containerd.runc.v2\n30754c8492f18   docker.io/calico/node:v3.24.1               io.containerd.runc.v2\n61acdb0231516   docker.io/calico/kube-controllers:v3.24.1   io.containerd.runc.v2\n...\n```\n\nNow you can exec into a Pod's container bringing your own debugging tools:\n\n```sh\n$ cdebug exec -n k8s.io -it --rm containerd://2220ea\n```\n\n### Publish \"forgotten\" port\n\nStart an nginx container but don't expose its port 80:\n\n```sh\n$ docker run -d --name nginx-1 nginx:1.23\n```\n\nForward local port 8080 to the nginx's 80:\n\n```sh\n$ cdebug port-forward nginx-1 -L 8080:80\n$ curl localhost:8080\n```\n\n### Expose localhost's ports\n\nStart a containerized service that listens only on its localhost:\n\n```sh\n$ docker run -d --name svc-1 python:3-alpine python3 -m 'http.server' -b 127.0.0.1 8888\n```\n\nTap into the above service:\n\n```sh\n$ cdebug port-forward svc-1 -L 127.0.0.1:8888\nPulling forwarder image...\nlatest: Pulling from shell/socat\nDigest: sha256:b43b6cf8d22615616b13c744b8ff525f5f6c0ca6c11b37fa3832a951ebb3c20c\nStatus: Image is up to date for nixery.dev/shell/socat:latest\nForwarding 127.0.0.1:49176 to 127.0.0.1:8888 through 172.17.0.4:34128\n\n$ curl localhost:49176\n\u003c!DOCTYPE HTML\u003e\n\u003chtml lang=\"en\"\u003e\n\u003chead\u003e\n...\n```\n\n## F.A.Q\n\n**Q:** `cdebug exec` fails with `ln: /proc/1/root/...: Permission denied` or the like error?\n\nNon-privileged targets can cause this error because by default, `cdebug` tries to start the debugger container with the same privileges as the target container. Try `cdebug exec --privileged` instead.\n\n## Similar tools\n\n- [`slim debug`](https://github.com/slimtoolkit/slim) - a `debug` command for Slim(toolkit) (originally contributed by [D4N](https://github.com/D4N))\n- [`debug-ctr`](https://github.com/felipecruz91/debug-ctr) - a debugger that creates a new container out of the original container with the toolkit mounted in a volume (by [Felipe Cruz Martinez](https://github.com/felipecruz91))\n- [`docker-debug`](https://github.com/zeromake/docker-debug) - much like `cdebug exec` but without the chroot trick.\n- [`docker-opener`](https://github.com/artemkaxboy/docker-opener) - a multi-purpose tool that in particular can run a shell session into your container (and if there is no shell inside, it'll bring its own busybox).\n- [`cntr`](https://github.com/Mic92/cntr) - is \"a replacement for `docker exec` that brings all your developers tools with you\" by mounting the file system from one container (or the host) into the target container and creating a nested container with the help of a FUSE filesystem. Supports a huge range of runtimes (docker, podman, LXC/LXD, rkt, systemd-nspawn, containerd) because it operates on the OS level.\n- [`kdiag`](https://github.com/solo-io/kdiag) - a kubectl plugin to get shell access to scratch containers, stream logs from multiple pods simultaneously, and do reverse port forwarding to Kubernetes clusters.\n\n- [`kpexec`](https://github.com/ssup2/kpexec) - a CLI that runs commands in a Kubernetes container with high privileges (via a privileged Pod with Node access).\n\n## TODO:\n\n- More `exec` flags (like in `docker run`): `--cap-add`, `--cap-drop`, `--env`, `--volume`, etc.\n- Helper command(s) suggesting nix(ery) packages\n- Non-docker runtimes (Podman, CRI, OCI)\n- More E2E Tests\n\n## Contributions\n\nIt's a pre-alpha with no sound design yet, so I may not be accepting all PRs. Sorry about that :)\n","funding_links":[],"categories":["Go","Containers","kubernetes"],"sub_categories":["Shell into containers"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fiximiuz%2Fcdebug","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fiximiuz%2Fcdebug","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fiximiuz%2Fcdebug/lists"}