{"id":16333117,"url":"https://github.com/fzakaria/shrinkwrap","last_synced_at":"2025-03-16T14:31:11.882Z","repository":{"id":42035651,"uuid":"441012430","full_name":"fzakaria/shrinkwrap","owner":"fzakaria","description":"A tool that embosses the needed dependencies on the top level executable","archived":false,"fork":false,"pushed_at":"2023-07-16T03:22:16.000Z","size":32,"stargazers_count":170,"open_issues_count":0,"forks_count":12,"subscribers_count":8,"default_branch":"main","last_synced_at":"2025-02-27T10:39:57.307Z","etag":null,"topics":["elf","glib","linking","nix"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/fzakaria.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-12-22T23:51:11.000Z","updated_at":"2025-02-25T18:14:27.000Z","dependencies_parsed_at":"2024-10-27T10:55:53.627Z","dependency_job_id":"177518a3-b952-4aca-8de9-d207f58949c7","html_url":"https://github.com/fzakaria/shrinkwrap","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fzakaria%2Fshrinkwrap","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fzakaria%2Fshrinkwrap/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fzakaria%2Fshrinkwrap/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fzakaria%2Fshrinkwrap/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/fzakaria","download_url":"https://codeload.github.com/fzakaria/shrinkwrap/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243818195,"owners_count":20352629,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["elf","glib","linking","nix"],"created_at":"2024-10-10T23:34:23.842Z","updated_at":"2025-03-16T14:31:11.561Z","avatar_url":"https://github.com/fzakaria.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Shrinkwrap\n\n![main](https://github.com/fzakaria/shrinkwrap/actions/workflows/main.yml/badge.svg)\n[![built with nix](https://builtwithnix.org/badge.svg)](https://builtwithnix.org)\n\n\u003e  A tool that embosses the needed dependencies on the top level executable\n\n# Introduction\n\nIt can be useful to _freeze_ all the dynamic shared objects needed by an application.\n\n_shrinkwrap_ is a tool which will discover all transitive dynamic shared objects, and lift them up to the executable referenced by absolute path.\n\nHere is an example where we will apply this to _ruby_. \n\nLets take a look at all the _dynamic shared objects_ needed by the Ruby interpreter.\n\n```console\n❯ ldd $(which ruby)\n\tlinux-vdso.so.1 (0x00007ffeed386000)\n\t/lib/x86_64-linux-gnu/libnss_cache.so.2 (0x00007f638ddf8000)\n\tlibruby-2.7.so.2.7 =\u003e /lib/x86_64-linux-gnu/libruby-2.7.so.2.7 (0x00007f638da79000)\n\tlibc.so.6 =\u003e /lib/x86_64-linux-gnu/libc.so.6 (0x00007f638d8b4000)\n\tlibpthread.so.0 =\u003e /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f638d893000)\n\tlibrt.so.1 =\u003e /lib/x86_64-linux-gnu/librt.so.1 (0x00007f638d888000)\n\tlibgmp.so.10 =\u003e /lib/x86_64-linux-gnu/libgmp.so.10 (0x00007f638d807000)\n\tlibdl.so.2 =\u003e /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f638d7ff000)\n\tlibcrypt.so.1 =\u003e /lib/x86_64-linux-gnu/libcrypt.so.1 (0x00007f638d7c4000)\n\tlibm.so.6 =\u003e /lib/x86_64-linux-gnu/libm.so.6 (0x00007f638d67f000)\n\t/lib64/ld-linux-x86-64.so.2 (0x00007f638de06000)\n```\n\nWe can see also that the _ruby_ application only lists a few needed shared objects itself.\n\n```console\n❯ patchelf --print-needed $(which ruby)\nlibruby-2.7.so.2.7\nlibc.so.6\n```\n\nLet's now apply _shrinkwrap_ and see the results.\n\n```console\n❯ nix run github:fzakaria/shrinkwrap $(which ruby)\n```\n\nIt automatically creates a `_stamped` copy of the filename if none provided and sets all the _NEEDED_ sections.\n\n```console\n❯ patchelf --print-needed ruby_stamped\n/lib/x86_64-linux-gnu/libm.so.6\n/lib/x86_64-linux-gnu/libcrypt.so.1\n/lib/x86_64-linux-gnu/libdl.so.2\n/lib/x86_64-linux-gnu/libgmp.so.10\n/lib/x86_64-linux-gnu/librt.so.1\n/lib/x86_64-linux-gnu/libpthread.so.0\n/lib/x86_64-linux-gnu/libruby-2.7.so.2.7\n/lib/x86_64-linux-gnu/libc.so.6\n\n❯ ldd ruby_stamped\n\tlinux-vdso.so.1 (0x00007ffe641f3000)\n\t/lib/x86_64-linux-gnu/libnss_cache.so.2 (0x00007f9cd4320000)\n\t/lib/x86_64-linux-gnu/libm.so.6 (0x00007f9cd41db000)\n\t/lib/x86_64-linux-gnu/libcrypt.so.1 (0x00007f9cd41a0000)\n\t/lib/x86_64-linux-gnu/libdl.so.2 (0x00007f9cd419a000)\n\t/lib/x86_64-linux-gnu/libgmp.so.10 (0x00007f9cd4119000)\n\t/lib/x86_64-linux-gnu/librt.so.1 (0x00007f9cd410e000)\n\t/lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f9cd40eb000)\n\t/lib/x86_64-linux-gnu/libruby-2.7.so.2.7 (0x00007f9cd3d8c000)\n\t/lib/x86_64-linux-gnu/libc.so.6 (0x00007f9cd3bc7000)\n\t/lib64/ld-linux-x86-64.so.2 (0x00007f9cd4336000)\n```\n\n## Motivation\n\nCertain _store_ based build tools such as [Guix](https://guix.gnu.org/), [Nix](https://nixos.org) or [Spack](https://spack.io/) make heavy use of _RUNPATH_ to help create reproducible and hermetic binaries.\n\nOne problem with the heavy use of _RUNPATH_, is that the search space could effect startup as it's `O(n)` on the number of entries (potentially worse if using _RPATH_). This can alo be expensive in _stat syscalls_, that has been well documented by in [this blog post](https://guix.gnu.org/blog/2021/taming-the-stat-storm-with-a-loader-cache/).\n\nSecondly, shared dynamic objects may be found due to the fact that they are cached during the linking stage. Meaning, if another shared object requires the same dependency but failed to specify where to find it, it may still properly resolved if discovered earlier in the linking process. This is extremely error prone and changing any of the executable's dependencies can change the link order and potentially cause the binary to no longer work.\n\nLifting up the needed shared objects to the top executable makes the dependency discovery _simple_, _quick_ and _hermetic_ since it can no longer change based on the order of visited dependencies.\n\n## Pitfalls\n\nAt the moment this only works with _glibc_ and not other _Standard C Libraries_ such as _musl_. The reason is that other linkers seem to resolve duplicate shared object files differently when they appear in the traversal. Consider the following example:\n\n```\n              +------------+\n              |            |\n              | Executable |\n              |            |\n      +-------+------------+----+\n      |                         |\n      |                         |\n+-----v-----+            +------v----+\n|           |            |           |\n| libbar.so |            | libfoo.so |\n|           |            |           |\n+-----+-----+            +-----------+\n      |               /some-fixed-path/libfoo.so\n      |\n+-----v------+\n|            |\n| libfoo.so  |\n|            |\n+------------+\n```\n\nIn _glibc_ the cache is keyed by the _soname_ value on the shared object. That allows the first found _libfoo.so_ at _/some-fixed-path/libfoo.so_ to be used for the one which _libbar.so_ depends on.\n\nUnfortunately, _musl_ does not support this functionality and ongoing discussions of inclusing it can be followed on the [mailing list](https://www.openwall.com/lists/musl/2021/12/21/1).\n\n## Development\n\nYou must have [Nix](https://nixos.org) installed for development.\n\nThis package uses [poetry2nix](https://github.com/nix-community/poetry2nix) to easily setup a development environment.\n\n```console\n\u003e nix develop\n```\n\nA helping `Makefile` is provided to run all the _linters_ and _formatters_.\n\n```console\n\u003e make lint\n```\n\nNote: I publish artifacts to [cachix](https://cachix.org/) that you can use to develop faster.\n```console\n\u003e cachix use fzakaria\n```\n\n## Experiments\n\nIncluded in the flake are different experiments for evaluating Shrinkwrap.\nIn most cases they provide a Docker image (tar.gz) which can be loaded.\n\n### emacs\n\nCreates a stamped version of the popular emacs editor similarly to the Guix experiment outlined in the [blog post](https://guix.gnu.org/blog/2021/taming-the-stat-storm-with-a-loader-cache/).\n\nYou can build the Docker image and inside will be `emacs-wrapped` as well as `emacs` and `strace` to recreate the experiment.\n```console\n\u003e nix build .#experiments.emacs\n\u003e docker load \u003c result\n643ace721190: Loading layer [==================================================\u003e]  786.9MB/786.9MB\nLoaded image: shrinkwrap-emacs-experiment:7jjlknqq660x1crrw7gm4m2qzalp71qj\n\u003e docker run -it emacs-experiment:7jjlknqq660x1crrw7gm4m2qzalp71qj /bin/bash\n\u003e patchelf --print-needed /bin/emacs-stamped\n/nix/store/m756011mkf1i0ki78i8y6ac3gf8qphvi-gcc-10.3.0-lib/lib/libstdc++.so.6\n/nix/store/xif6gg595hgmqawrcarapa8j2r7fbz9w-icu4c-70.1/lib/libicudata.so.70\n/nix/store/i6cmh2d4hbyp00rnh5rpf48pc7xfzx6j-libgpg-error-1.42/lib/libgpg-error.so.0\n/nix/store/q39ykk5fnhlbnl119iqjbgaw44kd65fy-util-linux-2.37.2-lib/lib/libblkid.so.1\n/nix/store/b1k5z0fdj0pnfz89k440al7ww4a263bf-libglvnd-1.3.4/lib/libGLX.so.0\n\n```\n\nIf you'd like you can pull the image directly from DockerHub via [fmzakari/shrinkwrap-emacs-experiment:7jjlknqq660x1crrw7gm4m2qzalp71qj](https://hub.docker.com/layers/shrinkwrap-emacs-experiment/fmzakari/shrinkwrap-emacs-experiment/7jjlknqq660x1crrw7gm4m2qzalp71qj/images/sha256-4633059bdf6c7ddbe23a4c6da11eba7ff58029eb870af01c98f10ada03324ee0?context=explore).\n\n```console\n\u003e docker pull fmzakari/shrinkwrap-emacs-experiment:7jjlknqq660x1crrw7gm4m2qzalp71qj\n\u003e docker run -it fmzakari/shrinkwrap-emacs-experiment:7jjlknqq660x1crrw7gm4m2qzalp71qj /bin/bash\n\u003e patchelf --print-needed /bin/emacs-stamped\n/nix/store/m756011mkf1i0ki78i8y6ac3gf8qphvi-gcc-10.3.0-lib/lib/libstdc++.so.6\n/nix/store/xif6gg595hgmqawrcarapa8j2r7fbz9w-icu4c-70.1/lib/libicudata.so.70\n/nix/store/i6cmh2d4hbyp00rnh5rpf48pc7xfzx6j-libgpg-error-1.42/lib/libgpg-error.so.0\n/nix/store/q39ykk5fnhlbnl119iqjbgaw44kd65fy-util-linux-2.37.2-lib/lib/libblkid.so.1\n/nix/store/b1k5z0fdj0pnfz89k440al7ww4a263bf-libglvnd-1.3.4/lib/libGLX.so.0\n```\n## Contributions\n\nThanks to [@trws](https://github.com/trws) for the inspiration and original version of this Python script.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffzakaria%2Fshrinkwrap","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffzakaria%2Fshrinkwrap","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffzakaria%2Fshrinkwrap/lists"}