{"id":17548996,"url":"https://github.com/malthe/appendlayer","last_synced_at":"2025-04-24T02:20:36.718Z","repository":{"id":57411058,"uuid":"368989638","full_name":"malthe/appendlayer","owner":"malthe","description":"Append a tarball to an existing image in a container registry – without having to pull down the image locally.","archived":false,"fork":false,"pushed_at":"2022-09-30T08:54:08.000Z","size":29,"stargazers_count":8,"open_issues_count":0,"forks_count":2,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-04-18T10:23:10.383Z","etag":null,"topics":["container","docker","python","registry"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/malthe.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGES.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-05-19T20:10:30.000Z","updated_at":"2024-12-09T12:53:35.000Z","dependencies_parsed_at":"2022-09-09T16:02:06.642Z","dependency_job_id":null,"html_url":"https://github.com/malthe/appendlayer","commit_stats":null,"previous_names":[],"tags_count":3,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/malthe%2Fappendlayer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/malthe%2Fappendlayer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/malthe%2Fappendlayer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/malthe%2Fappendlayer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/malthe","download_url":"https://codeload.github.com/malthe/appendlayer/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250546363,"owners_count":21448313,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["container","docker","python","registry"],"created_at":"2024-10-21T02:48:25.939Z","updated_at":"2025-04-24T02:20:36.698Z","avatar_url":"https://github.com/malthe.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"Append layer\n------------\n\n[![PyPI version](https://badge.fury.io/py/appendlayer.svg)](https://badge.fury.io/py/appendlayer)\n\nThis standalone utility appends a tarball to an existing image in a\ncontainer registry – without having to pull down the image locally.\n\nIt supports any registry that implements the [OCI Distribution\nSpec](https://github.com/opencontainers/distribution-spec).\n\n\n### Why\n\nThe basic use-case for this utility is when you have a base image that\nis already available in a container registry, and you simply need to\nadd one or more files, then push the result back to the same registry.\n\nIn this case, you can do no better in terms of network transfer than\nthis utility. It does the minimum amount of work in order to get the\njob done.\n\nWith the [Docker\nADD](https://docs.docker.com/engine/reference/builder/#add) command,\nyou'd have to download the existing image and start up a build process\nin order to run the `ADD` command.\n\n```dockerfile\nFROM apache/airflow:2.3.2\nADD your_tar_file.tar.gz /opt/airflow/dag\n```\n\nThe resulting image would be exactly the same, but there is no special\noptimization in Docker that would avoid downloading the base image\n(although theoretically, it could be done but it would require bigger\nchanges in the data model in order to support lazy referencing of\nlayer data).\n\nIncidentally, the script was designed exactly with [Apache\nAirflow](https://airflow.apache.org/) in mind.\n\n\nNote that Buildkit [will have support for\nthis](https://github.com/moby/buildkit/issues/2414) from v0.10 but\nit's not clear how this will be supported in a Dockerfile.\n\n### Installation\n\nInstall the tool using pip:\n\n```bash\n$ pip install appendlayer\n```\n\nThis makes available \"appendlayer\" as a script in your environment.\n\nAlternatively, download the [appendlayer.py](./appendlayer.py) script\nand run it using Python directly:\n\n```bash\n$ python appendlayer.py\n```\n\nThe script has no external dependencies, using only what's included already with Python.\n\n\n### Usage\n\nPipe in the layer contents using a tarball and provide the repository (or _image_) name and the old and new tags:\n\n```bash\n$ echo \"Hello world\" \u003e test.txt\n$ tar cvf - test.txt | appendlayer \u003chost\u003e \u003crepository\u003e \u003cold-tag\u003e \u003cnew-tag\u003e\n```\n\nAlternatively, qualify source and destination using image syntax:\n```bash\n$ tar cvf - test.txt | appendlayer \u003chost\u003e \u003cold-repository\u003e:\u003cold-tag\u003e \u003cnew-repository\u003e:\u003cnew-tag\u003e\n```\n\nOr even across different hosts:\n```bash\n$ tar cvf - test.txt | appendlayer \u003cold-host\u003e/\u003cold-repository\u003e:\u003cold-tag\u003e \u003cnew-host\u003e/\u003cnew-repository\u003e:\u003cnew-tag\u003e\n```\n\nFor Azure Container Registry (ACR) for example, the _host_ is\ntypically `\u003cregistry-name\u003e.azurecr.io`.\n\n\n### Authentication\n\nThe script uses OAuth2 to authorize requests to the container\nregistry.\n\nThis is configured using either the `ACCESS_TOKEN` or `REFRESH_TOKEN`\nenvironment variable, or by extracting authentication details from the Docker\nconfiguration file (located based on the `DOCKER_CONFIG` environment\nvariable).\n\nFor example, for [Azure Container\nRegistry](https://azure.microsoft.com/en-us/services/container-registry/),\nto authorize to a specific container registry.\n\n```bash\n$ export REFRESH_TOKEN=$( \\\n      az acr login -t --name \u003cregistry-name\u003e.azurecr.io \\\n      --expose-token --output tsv --query accessToken)\n```\n\nTo authorize across multiple registries, use an access token:\n\n```bash\n$ export ACCESS_TOKEN=$( \\\n       az account get-access-token --query accessToken --output tsv)\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmalthe%2Fappendlayer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmalthe%2Fappendlayer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmalthe%2Fappendlayer/lists"}