{"id":13582453,"url":"https://github.com/containerd/stargz-snapshotter","last_synced_at":"2025-05-14T22:05:46.234Z","repository":{"id":36990319,"uuid":"213618352","full_name":"containerd/stargz-snapshotter","owner":"containerd","description":"Fast container image distribution plugin with lazy pulling","archived":false,"fork":false,"pushed_at":"2025-05-07T20:22:02.000Z","size":15059,"stargazers_count":1294,"open_issues_count":73,"forks_count":123,"subscribers_count":21,"default_branch":"main","last_synced_at":"2025-05-07T21:58:19.870Z","etag":null,"topics":["containerd","containers","docker","docker-image","filesystem"],"latest_commit_sha":null,"homepage":"https://github.com/containerd/containerd/issues/3731","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/containerd.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2019-10-08T10:55:40.000Z","updated_at":"2025-05-07T14:35:55.000Z","dependencies_parsed_at":"2023-01-17T12:01:34.773Z","dependency_job_id":"445c0f46-fe23-4e10-83c9-98d9a06484ac","html_url":"https://github.com/containerd/stargz-snapshotter","commit_stats":{"total_commits":880,"total_committers":36,"mean_commits":"24.444444444444443","dds":0.2534090909090909,"last_synced_commit":"dc0ff532c19d2681955a2c09c9a934e462efa24b"},"previous_names":["ktock/remote-snapshotter"],"tags_count":110,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/containerd%2Fstargz-snapshotter","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/containerd%2Fstargz-snapshotter/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/containerd%2Fstargz-snapshotter/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/containerd%2Fstargz-snapshotter/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/containerd","download_url":"https://codeload.github.com/containerd/stargz-snapshotter/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254235687,"owners_count":22036962,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["containerd","containers","docker","docker-image","filesystem"],"created_at":"2024-08-01T15:02:43.839Z","updated_at":"2025-05-14T22:05:46.158Z","avatar_url":"https://github.com/containerd.png","language":"Go","funding_links":[],"categories":["Go","docker","Go (531)","🖼️ Container Images"],"sub_categories":["Image Distribution \u0026 Caching"],"readme":"[[⬇️ **Download**]](https://github.com/containerd/stargz-snapshotter/releases)\n[[📔**Browse images**]](./docs/pre-converted-images.md)\n[[☸**Quick Start (Kubernetes)**]](#quick-start-with-kubernetes)\n[[🤓**Quick Start (nerdctl)**]](https://github.com/containerd/nerdctl/blob/master/docs/stargz.md)\n[[🔆**Install**]](./docs/INSTALL.md)\n\n# Stargz Snapshotter\n\n[![Tests Status](https://github.com/containerd/stargz-snapshotter/workflows/Tests/badge.svg)](https://github.com/containerd/stargz-snapshotter/actions?query=workflow%3ATests+branch%3Amain)\n[![Benchmarking](https://github.com/containerd/stargz-snapshotter/workflows/Benchmark/badge.svg)](https://github.com/containerd/stargz-snapshotter/actions?query=workflow%3ABenchmark+branch%3Amain)\n[![Nightly](https://github.com/containerd/stargz-snapshotter/workflows/Nightly/badge.svg)](https://github.com/containerd/stargz-snapshotter/actions?query=workflow%3ANightly+branch%3Amain)\n\nRead also introductory blog: [Startup Containers in Lightning Speed with Lazy Image Distribution on Containerd](https://medium.com/nttlabs/startup-containers-in-lightning-speed-with-lazy-image-distribution-on-containerd-243d94522361)\n\nPulling image is one of the time-consuming steps in the container lifecycle.\nResearch shows that time to take for pull operation accounts for 76% of container startup time[[FAST '16]](https://www.usenix.org/node/194431).\n*Stargz Snapshotter* is an implementation of snapshotter which aims to solve this problem by *lazy pulling*.\n*Lazy pulling* here means a container can run without waiting for the pull completion of the image and necessary chunks of the image are fetched *on-demand*.\n\n[*eStargz*](/docs/stargz-estargz.md) is a lazily-pullable image format proposed by this project.\nThis is compatible to [OCI](https://github.com/opencontainers/image-spec/)/[Docker](https://github.com/moby/moby/blob/master/image/spec/v1.2.md) images so this can be pushed to standard container registries (e.g. ghcr.io) as well as this is *still runnable* even on eStargz-agnostic runtimes including Docker.\neStargz format is based on [stargz image format by CRFS](https://github.com/google/crfs) but comes with additional features like runtime optimization and content verification.\n\nThe following histogram is the benchmarking result for startup time of several containers measured on Github Actions, using GitHub Container Registry.\n\n\u003cimg src=\"docs/images/benchmarking-result-ecdb227.png\" width=\"600\" alt=\"The benchmarking result on ecdb227\"\u003e\n\n`legacy` shows the startup performance when we use containerd's default snapshotter (`overlayfs`) with images copied from `docker.io/library` without optimization.\nFor this configuration, containerd pulls entire image contents and `pull` operation takes accordingly.\nWhen we use stargz snapshotter with eStargz-converted images but without any optimization (`estargz-noopt`) we are seeing performance improvement on the `pull` operation because containerd can start the container without waiting for the `pull` completion and fetch necessary chunks of the image on-demand.\nBut at the same time, we see the performance drawback for `run` operation because each access to files takes extra time for fetching them from the registry.\nWhen we use [eStargz with optimization](/docs/ctr-remote.md) (`estargz`), we can mitigate the performance drawback observed in `estargz-noopt` images.\nThis is because [stargz snapshotter prefetches and caches *likely accessed files* during running the container](/docs/stargz-estargz.md).\nOn the first container creation, stargz snapshotter waits for the prefetch completion so `create` sometimes takes longer than other types of image.\nBut it's still shorter than waiting for downloading all files of all layers.\n\nThe above histogram is [the benchmarking result on the commit `ecdb227`](https://github.com/containerd/stargz-snapshotter/actions/runs/398606060).\nWe are constantly measuring the performance of this snapshotter so you can get the latest one through the badge shown top of this doc.\nPlease note that we sometimes see dispersion among the results because of the NW condition on the internet and the location of the instance in the Github Actions, etc.\nOur benchmarking method is based on [HelloBench](https://github.com/Tintri/hello-bench).\n\n:nerd_face: You can also run containers on IPFS with lazy pulling. This is an experimental feature. See [`./docs/ipfs.md`](./docs/ipfs.md) for more details.\n\nStargz Snapshotter is a **non-core** sub-project of containerd.\n\n## Quick Start with Kubernetes\n\n- For more details about stargz snapshotter plugin and its configuration, refer to [Containerd Stargz Snapshotter Plugin Overview](/docs/overview.md).\n- For more details about setup lazy pulling of eStargz with containerd, CRI-O, Podman, systemd, etc., refer to [Install Stargz Snapshotter and Stargz Store](./docs/INSTALL.md).\n- For more details about integration status of eStargz with tools in commuinty, refer to [Integration of eStargz with other tools](./docs/integration.md)\n\nFor using stargz snapshotter on kubernetes nodes, you need the following configuration to containerd as well as run stargz snapshotter daemon on the node.\nWe assume that you are using containerd (\u003e v1.4.2) as a CRI runtime.\n\n```toml\nversion = 2\n\n# Plug stargz snapshotter into containerd\n# Containerd recognizes stargz snapshotter through specified socket address.\n# The specified address below is the default which stargz snapshotter listen to.\n[proxy_plugins]\n  [proxy_plugins.stargz]\n    type = \"snapshot\"\n    address = \"/run/containerd-stargz-grpc/containerd-stargz-grpc.sock\"\n\n# Use stargz snapshotter through CRI\n[plugins.\"io.containerd.grpc.v1.cri\".containerd]\n  snapshotter = \"stargz\"\n  disable_snapshot_annotations = false\n```\n\n**Note that `disable_snapshot_annotations = false` is required since containerd \u003e v1.4.2**\n\nYou can try our [prebuilt](/Dockerfile) [KinD](https://github.com/kubernetes-sigs/kind) node image that contains the above configuration.\n\n```console\n$ kind create cluster --name stargz-demo --image ghcr.io/containerd/stargz-snapshotter:0.12.1-kind\n```\n\n:information_source: kind binary v0.16.x or newer is recommended for `ghcr.io/containerd/stargz-snapshotter:0.12.1-kind`.\n\n:information_source: You can get latest node images from [`ghcr.io/containerd/stargz-snapshotter:${VERSION}-kind`](https://github.com/orgs/containerd/packages/container/package/stargz-snapshotter) namespace.\n\nThen you can create eStargz pods on the cluster.\nIn this example, we create a stargz-converted Node.js pod (`ghcr.io/stargz-containers/node:17.8.0-esgz`) as a demo.\n\n```yaml\napiVersion: v1\nkind: Pod\nmetadata:\n  name: nodejs\nspec:\n  containers:\n  - name: nodejs-stargz\n    image: ghcr.io/stargz-containers/node:17.8.0-esgz\n    command: [\"node\"]\n    args:\n    - -e\n    - var http = require('http');\n      http.createServer(function(req, res) {\n        res.writeHead(200);\n        res.end('Hello World!\\n');\n      }).listen(80);\n    ports:\n    - containerPort: 80\n```\n\nThe following command lazily pulls `ghcr.io/stargz-containers/node:17.8.0-esgz` from Github Container Registry and creates the pod so the time to take for it is shorter than the original image `library/node:13.13`.\n\n```console\n$ kubectl --context kind-stargz-demo apply -f stargz-pod.yaml \u0026\u0026 kubectl --context kind-stargz-demo get po nodejs -w\n$ kubectl --context kind-stargz-demo port-forward nodejs 8080:80 \u0026\n$ curl 127.0.0.1:8080\nHello World!\n```\n\nStargz snapshotter also supports [further configuration](/docs/overview.md) including private registry authentication, mirror registries, etc.\n\n## Getting eStargz images\n\n- For more examples and details about the image converter `ctr-remote`, refer to [Optimize Images with `ctr-remote image optimize`](/docs/ctr-remote.md).\n- For more details about eStargz format, refer to [eStargz: Standard-Compatible Extensions to Tar.gz Layers for Lazy Pulling Container Images](/docs/stargz-estargz.md)\n\nFor lazy pulling images, you need to prepare eStargz images first.\nThere are several ways to achieve that.\nThis section describes some of them.\n\n### Trying pre-built eStargz images\n\nYou can try our pre-converted eStargz images on ghcr.io listed in [Trying pre-converted images](/docs/pre-converted-images.md).\n\n### Building eStargz images using BuildKit\n\nBuildKit supports building eStargz image since v0.10.\n\nYou can try it using [Docker Buildx](https://docs.docker.com/buildx/working-with-buildx/).\nThe following command builds an eStargz image and push it to `ghcr.io/ktock/hello:esgz`.\nFlags `oci-mediatypes=true,compression=estargz` enable to build eStargz.\n\n```\n$ docker buildx build -t ghcr.io/ktock/hello:esgz \\\n    -o type=registry,oci-mediatypes=true,compression=estargz,force-compression=true \\\n    /tmp/buildctx/\n```\n\n\u003e NOTE1: `force-compression=true` isn't needed if the base image is already eStargz.\n\n\u003e NOTE2: Docker still does not support lazy pulling of eStargz.\n\neStargz-enabled BuildKit (v0.10) will be [included to Docker v22.XX](https://github.com/moby/moby/blob/v22.06.0-beta.0/vendor.mod#L51) however you can build eStargz images with the prior version using Buildx [driver](https://github.com/docker/buildx/blob/master/docs/reference/buildx_create.md#-set-the-builder-driver-to-use---driver) feature.\nYou can enable the specific version of BuildKit using [`docker buildx create`](https://docs.docker.com/engine/reference/commandline/buildx_create/) (this example specifies `v0.10.3`).\n\n```\n$ docker buildx create --use --name v0.10.3 --driver docker-container --driver-opt image=moby/buildkit:v0.10.3\n$ docker buildx inspect --bootstrap v0.10.3\n```\n\n### Building eStargz images using Kaniko\n\n[Kaniko](https://github.com/GoogleContainerTools/kaniko) is an image builder runnable in containers and Kubernetes.\nSince v1.5.0, it experimentally supports building eStargz.\n`GGCR_EXPERIMENT_ESTARGZ=1` is needed.\n\n```console\n$ docker run --rm -e GGCR_EXPERIMENT_ESTARGZ=1 \\\n    -v /tmp/buildctx:/workspace -v ~/.docker/config.json:/kaniko/.docker/config.json:ro \\\n    gcr.io/kaniko-project/executor:v1.8.1 --destination ghcr.io/ktock/hello:esgz\n```\n\n### Building eStargz images using nerdctl\n\n[nerdctl](https://github.com/containerd/nerdctl), Docker-compatible CLI of containerd, supports building eStargz images.\n\n```console\n$ nerdctl build -t ghcr.io/ktock/hello:1 /tmp/buildctx\n$ nerdctl image convert --estargz --oci ghcr.io/ktock/hello:1 ghcr.io/ktock/hello:esgz\n$ nerdctl push ghcr.io/ktock/hello:esgz\n```\n\n\u003e NOTE: `--estargz` should be specified in conjunction with `--oci`\n\nPlease refer to nerdctl document for details for further information (e.g. lazy pulling): https://github.com/containerd/nerdctl/blob/master/docs/stargz.md\n\n### Creating eStargz images using `ctr-remote`\n\n[`ctr-remote`](/docs/ctr-remote.md) allows converting an image into eStargz with optimizing it.\nAs shown in the above benchmarking result, on-demand lazy pulling improves the performance of pull but causes runtime performance penalty because reading files induce remotely downloading contents.\nFor solving this, `ctr-remote` has *workload-based* optimization for images.\n\nFor trying the examples described in this section, you can also use the docker-compose-based demo environment.\nYou can setup this environment as the following commands (put this repo on `${GOPATH}/src/github.com/containerd/stargz-snapshotter`).\n*Note that this runs privileged containers on your host.*\n\n```console\n$ cd ${GOPATH}/src/github.com/containerd/stargz-snapshotter/script/demo\n$ docker-compose build containerd_demo\n$ docker-compose up -d\n$ docker exec -it containerd_demo /bin/bash\n(inside container) # ./script/demo/run.sh\n```\n\nGenerally, container images are built with purpose and the *workloads* are defined in the Dockerfile with some parameters (e.g. entrypoint, envvars and user).\nBy default, `ctr-remote` optimizes the performance of reading files that are most likely accessed in the workload defined in the Dockerfile.\n[You can also specify the custom workload using options if needed](/docs/ctr-remote.md).\n\nThe following example converts the legacy `library/ubuntu:20.04` image into eStargz.\nThe command also optimizes the image for the workload of executing `ls` on `/bin/bash`.\nThe thing actually done is it runs the specified workload in a temporary container and profiles all file accesses with marking them as *likely accessed* also during runtime.\nThe converted image is still **docker-compatible** so you can run it with eStargz-agnostic runtimes (e.g. Docker).\n\n```console\n# ctr-remote image pull docker.io/library/ubuntu:20.04\n# ctr-remote image optimize --oci --entrypoint='[ \"/bin/bash\", \"-c\" ]' --args='[ \"ls\" ]' docker.io/library/ubuntu:20.04 registry2:5000/ubuntu:20.04\n# ctr-remote image push --plain-http registry2:5000/ubuntu:20.04\n```\n\nFinally, the following commands clear the local cache then pull the eStargz image lazily.\nStargz snapshotter prefetches files that are most likely accessed in the optimized workload, which hopefully increases the cache hit rate for that workload and mitigates runtime overheads as shown in the benchmarking result shown top of this doc.\n\n```console\n# ctr-remote image rm --sync registry2:5000/ubuntu:20.04\n# ctr-remote images rpull --plain-http registry2:5000/ubuntu:20.04\nfetching sha256:610399d1... application/vnd.oci.image.index.v1+json\nfetching sha256:0b4a26b4... application/vnd.oci.image.manifest.v1+json\nfetching sha256:8d8d9dbe... application/vnd.oci.image.config.v1+json\n# ctr-remote run --rm -t --snapshotter=stargz registry2:5000/ubuntu:20.04 test /bin/bash\nroot@8eabb871a9bd:/# ls\nbin  boot  dev  etc  home  lib  lib32  lib64  libx32  media  mnt  opt  proc  root  run  sbin  srv  sys  tmp  usr  var\n```\n\n\u003e NOTE: You can perform lazy pulling from any OCI-compatible registries (e.g. docker.io, ghcr.io, etc) as long as the image is formatted as eStargz.\n\n## Importing Stargz Snapshotter as go module\n\nCurrently, Stargz Snapshotter repository contains two Go modules as the following and both of them need to be imported.\n\n- `github.com/containerd/stargz-snapshotter`\n- `github.com/containerd/stargz-snapshotter/estargz`\n\nPlease make sure you import the both of them and they point to *the same commit version*.\n\n## Project details\n\nStargz Snapshotter is a containerd **non-core** sub-project, licensed under the [Apache 2.0 license](./LICENSE).\nAs a containerd non-core sub-project, you will find the:\n * [Project governance](https://github.com/containerd/project/blob/main/GOVERNANCE.md),\n * [Maintainers](./MAINTAINERS),\n * and [Contributing guidelines](https://github.com/containerd/project/blob/main/CONTRIBUTING.md)\n\ninformation in our [`containerd/project`](https://github.com/containerd/project) repository.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcontainerd%2Fstargz-snapshotter","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcontainerd%2Fstargz-snapshotter","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcontainerd%2Fstargz-snapshotter/lists"}