{"id":13567400,"url":"https://github.com/coord-e/magicpak","last_synced_at":"2025-05-16T03:03:11.158Z","repository":{"id":38013550,"uuid":"252624042","full_name":"coord-e/magicpak","owner":"coord-e","description":":hammer: Build minimal docker images without static linking","archived":false,"fork":false,"pushed_at":"2023-08-07T03:10:44.000Z","size":370,"stargazers_count":610,"open_issues_count":8,"forks_count":14,"subscribers_count":6,"default_branch":"develop","last_synced_at":"2025-05-13T20:08:03.666Z","etag":null,"topics":["dockerfile","dynamic-linking"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/coord-e.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE-APACHE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-04-03T03:31:20.000Z","updated_at":"2025-02-06T08:22:59.000Z","dependencies_parsed_at":"2024-08-01T13:24:47.673Z","dependency_job_id":"2c0ee36a-99a8-4828-98dc-5be7dd2738c1","html_url":"https://github.com/coord-e/magicpak","commit_stats":{"total_commits":227,"total_committers":4,"mean_commits":56.75,"dds":"0.11894273127753308","last_synced_commit":"8ee7d87be2cad966f4f7f5d030bad020a9f6a35e"},"previous_names":[],"tags_count":11,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/coord-e%2Fmagicpak","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/coord-e%2Fmagicpak/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/coord-e%2Fmagicpak/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/coord-e%2Fmagicpak/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/coord-e","download_url":"https://codeload.github.com/coord-e/magicpak/tar.gz/refs/heads/develop","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254425906,"owners_count":22069253,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["dockerfile","dynamic-linking"],"created_at":"2024-08-01T13:02:30.535Z","updated_at":"2025-05-16T03:03:06.144Z","avatar_url":"https://github.com/coord-e.png","language":"Rust","readme":"# magicpak\n\n[![Actions Status](https://github.com/coord-e/magicpak/workflows/Test%20and%20Lint/badge.svg)](https://github.com/coord-e/magicpak/actions?workflow=Test+and+Lint)\n[![Actions Status](https://github.com/coord-e/magicpak/workflows/Release/badge.svg)](https://github.com/coord-e/magicpak/actions?workflow=Release)\n[![License](https://img.shields.io/crates/l/magicpak)](https://crates.io/crates/magicpak)\n\n`magicpak` enables you to build minimal docker images without any bothersome preparation such as static linking.\n\n```dockerfile\n# You prepare /bin/your_executable here...\n\nADD https://github.com/coord-e/magicpak/releases/download/v1.4.0/magicpak-x86_64-unknown-linux-musl /usr/bin/magicpak\nRUN chmod +x /usr/bin/magicpak\n\nRUN /usr/bin/magicpak -v /bin/your_executable /bundle\n\nFROM scratch\nCOPY --from=0 /bundle /.\n\nCMD [\"/bin/your_executable\"]\n```\n\nThat's it! The resulting image shall only contain what your executable requires at runtime. You can find more useful examples of `magicpak` under [example/](/example).\n\n## Feature\n\n`magicpak` is a command-line utility that analyzes and bundles runtime dependencies of the executable.  `magicpak` basically collects all shared object dependencies that are required by a dynamic linker at runtime. Additionally, `magicpak`'s contributions are summarized as follows:\n\n- **Simple**. You can build a minimal image just by adding a few lines to your `Dockerfile`.\n- **Full-featured**. You can bundle, test, and compress your executable at once. You can focus on your business because `magicpak` handles all `Dockerfile`-specific matters to decrease image size.\n- **Dynamic analysis**. `--dynamic` flag enables a dynamic analysis that can discover dependencies other than dynamically linked libraries.\n- **Flexible**. We expose a full control of resulting bundle with a family of options like `--include` and  `--exclude`. You can deal with dependencies that cannot be detected automatically.\n- **Stable**. We don't parse undocumented and sometimes inaccurate ldd(1) outputs. Instead, we use dlopen(3) and dlinfo(3) in glibc to query shared library locations to ld.so(8).\n\n`magicpak` is especially useful when you find it difficult to produce a statically linked executable. Also, `magicpak` is powerful when building from source is bothering or the source code is not public, because `magicpak` only requires the executable to build a minimal docker image.\n\n## Usage\n\nYou can start with `magicpak path/to/executable path/to/output`. This simply analyzes runtime dependencies of your executable statically and put everything your executable needs in runtime to the specified output directory. Once they've bundled, we can simply copy them to the `scratch` image in the second stage as follows.\n\n```dockerfile\nRUN magicpak path/to/executable /bundle\n\nFROM scratch\nCOPY --from=0 /bundle /.\n```\n\nSome executables work well in this way. However, others fail to run properly because `magicpak`'s static analysis isn't enough to detect all files needed by them at runtime. For this case, `magicpak` has `--include \u003cGLOB\u003e` option to specify the missing requirements manually. Moreover, you can use `--dynamic` to automatically include files that are accessed by the executable during execution.\n\nDespite our careful implementation, our analysis is unreliable in a way because we can't completely determine the runtime behavior before its execution. To ensure that `magicpak` collected all dependencies to perform a specific task, `--test` option is implemented. `--test` enables testing of the resulting bundle using chroot(2).\n\nThe size of the resulting image is our main concern. `magicpak` supports executable compression using `upx`. You can enable it with `--compress`.\n\n### Supported options\n\n```\nUsage: magicpak [OPTIONS] \u003cINPUT\u003e... \u003cOUTPUT\u003e\n\nArguments:\n  \u003cINPUT\u003e...  Input executable\n  \u003cOUTPUT\u003e    Output destination\n\nOptions:\n  -i, --include \u003cGLOB\u003e                Additionally include files/directories with glob patterns\n  -e, --exclude \u003cGLOB\u003e                Exclude files/directories from the resulting bundle with glob patterns\n      --mkdir \u003cPATH\u003e                  Make directories in the resulting bundle\n  -r, --install-to \u003cPATH\u003e             Specify the installation path of the executable in the bundle\n      --log-level \u003cLEVEL\u003e             Specify the log level [default: Warn] [possible values: Off, Error, Warn, Info, Debug]\n  -v, --verbose                       Verbose mode, same as --log-level Info\n  -t, --test                          Enable testing\n      --test-command \u003cCOMMAND\u003e        Specify the test command to use in --test\n      --test-stdin \u003cCONTENT\u003e          Specify stdin content supplied to the test command in --test\n      --test-stdout \u003cCONTENT\u003e         Test stdout of the test command\n  -d, --dynamic                       Enable dynamic analysis\n      --dynamic-arg \u003cARG\u003e             Specify arguments passed to the executable in --dynamic\n      --dynamic-stdin \u003cCONTENT\u003e       Specify stdin content supplied to the executable in --dynamic\n  -c, --compress                      Compress the executable with npx\n      --upx-arg \u003cARG\u003e                 Specify arguments passed to upx in --compress\n      --busybox \u003cPATH or NAME\u003e        Specify the path or name of busybox that would be used in testing [default: busybox]\n      --upx \u003cPATH or NAME\u003e            Specify the path or name of upx that would be used in compression [default: upx]\n      --cc \u003cPATH or NAME\u003e             Specify the path or name of c compiler that would be used in the name resolution of shared library dependencies [env: CC=] [default: cc]\n      --experimental-noload-resolver  [EXPERIMENTAL] Resolve dynamic library paths without loading in dlopen(3)\n  -h, --help                          Print help information\n```\n\n### Docker images\n\nWe provide some base images that contain `magicpak` and its optional dependencies to get started.\n\n| name                                                         | description                                                  |\n| ------------------------------------------------------------ | ------------------------------------------------------------ |\n| [magicpak/debian ![magicpak/debian](https://img.shields.io/docker/pulls/magicpak/debian)](https://hub.docker.com/r/magicpak/debian) | [library/debian](http://hub.docker.com/_/debian) with `magicpak` |\n| [magicpak/cc ![magicpak/cc](https://img.shields.io/docker/pulls/magicpak/cc)](https://hub.docker.com/r/magicpak/cc) | [library/debian](http://hub.docker.com/_/debian) with `build-essential`, `clang`, and `magicpak` |\n| [magicpak/haskell ![magicpak/haskell](https://img.shields.io/docker/pulls/magicpak/haskell)](https://hub.docker.com/r/magicpak/haskell) | [library/haskell](http://hub.docker.com/_/haskell) with `magicpak` |\n| [magicpak/rust ![magicpak/rust](https://img.shields.io/docker/pulls/magicpak/rust)](https://hub.docker.com/r/magicpak/rust) | [library/rust](http://hub.docker.com/_/rust) with `magicpak` |\n\n### Example\n\nThe following is a dockerfile using `magicpak` for a docker image of [`clang-format`](https://clang.llvm.org/docs/ClangFormat.html), a formatter for C/C++/etc. ([example/clang-format](/example/clang-format))\n\n```dockerfile\nFROM magicpak/debian:buster-magicpak1.4.0\n\nRUN apt-get -y update\nRUN apt-get -y --no-install-recommends install clang-format\n\nRUN magicpak $(which clang-format) /bundle -v  \\\n      --compress                               \\\n      --upx-arg --best                         \\\n      --test                                   \\\n      --test-stdin \"int main(  ){ }\"           \\\n      --test-stdout \"int main() {}\"            \\\n      --install-to /bin/\n\nFROM scratch\nCOPY --from=0 /bundle /.\n\nWORKDIR /workdir\n\nCMD [\"/bin/clang-format\"]\n```\n\n### Note on name resolution and glibc\n\nIf your program uses glibc for name resolution (most likely it does), the call to getaddrinfo(3) will result in an error after bundled by magicpak.\nThis can be resolved by manually including the NSS-related shared libraries as shown below.\n\n```dockerfile\n# example on x86_64 Debian-based image:\nRUN magicpak path/to/executable /bundle --include '/lib/x86_64-linux-gnu/libnss_*'\n```\n\n### Note on jemalloc\n\nIf your program depends on libjemalloc, magicpak may fail with the following message.\n\n```\nerror: Unable to lookup shared library: /lib/aarch64-linux-gnu/libjemalloc.so.2: cannot allocate memory in static TLS block\n```\n\nYou can use `--experimental-noload-resolver` flag to workaround this. See [#19](https://github.com/coord-e/magicpak/issues/19) for details.\n\n## Disclaimer\n\n`magicpak` comes with absolutely no warranty. There's no guarantee that the processed bundle works properly and identically to the original executable. Although I had no problem using `magicpak` for building various kinds of images, it is recommended to use this with caution and make a careful examination of the resulting bundle.\n\n## License\n\nLicensed under either of\n\n * Apache License, Version 2.0\n   ([LICENSE-APACHE](LICENSE-APACHE) or http://www.apache.org/licenses/LICENSE-2.0)\n * MIT license\n   ([LICENSE-MIT](LICENSE-MIT) or http://opensource.org/licenses/MIT)\n\nat your option.\n\n## Contribution\n\nUnless you explicitly state otherwise, any contribution intentionally submitted\nfor inclusion in the work by you, as defined in the Apache-2.0 license, shall be\ndual licensed as above, without any additional terms or conditions.\n","funding_links":[],"categories":["Rust"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcoord-e%2Fmagicpak","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcoord-e%2Fmagicpak","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcoord-e%2Fmagicpak/lists"}