{"id":17914183,"url":"https://github.com/knadh/stuffbin","last_synced_at":"2025-04-05T05:07:55.899Z","repository":{"id":43043365,"uuid":"164001166","full_name":"knadh/stuffbin","owner":"knadh","description":"Compress and embed static files and assets into Go binaries and access them with a virtual file system in production","archived":false,"fork":false,"pushed_at":"2023-12-05T16:16:38.000Z","size":41,"stargazers_count":182,"open_issues_count":2,"forks_count":27,"subscribers_count":6,"default_branch":"master","last_synced_at":"2025-03-29T04:12:11.324Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/knadh.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-01-03T17:08:13.000Z","updated_at":"2025-03-22T01:59:10.000Z","dependencies_parsed_at":"2023-12-05T17:48:35.636Z","dependency_job_id":null,"html_url":"https://github.com/knadh/stuffbin","commit_stats":{"total_commits":20,"total_committers":5,"mean_commits":4.0,"dds":"0.30000000000000004","last_synced_commit":"aa6399d254a7d2fccb74bc19b3f55927d4b462da"},"previous_names":[],"tags_count":5,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/knadh%2Fstuffbin","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/knadh%2Fstuffbin/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/knadh%2Fstuffbin/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/knadh%2Fstuffbin/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/knadh","download_url":"https://codeload.github.com/knadh/stuffbin/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247289428,"owners_count":20914464,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-28T19:56:40.276Z","updated_at":"2025-04-05T05:07:55.879Z","avatar_url":"https://github.com/knadh.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003ca href=\"https://zerodha.tech\"\u003e\u003cimg src=\"https://zerodha.tech/static/images/github-badge.svg\" align=\"right\" /\u003e\u003c/a\u003e\n\n# stuffbin\n\nstuffbin is a utility + package to compress and embed static files and assets into Go binaries for distribution. It supports falling back to the local file system when no embedded assets are available, for instance, in development mode. stuffbin is inspired by [zgok](https://github.com/srtkkou/zgok) but is simpler and has better abstractions.\n\n## stuffbin vs. Go 1.16 embed\nGo 1.16 introduced the [`//go:embed`](https://golang.org/pkg/embed/) directive that allows embedding of files into Go binaries without any external utilities. stuffbin offers a few key advantages over native embedding in its current form.\n\n- All files are ZIP compressed.\n- Custom path aliases (eg: embed `/home/local/path/file.txt` as `/app/file.txt`).\n- Dynamically embed files instead of static `//go:embed` directives.\n- Embed parent directories, sub-directories, and arbitrary paths (eg: `../../path`).\n  Go embed does not permit embedding of files outside of a .go file's directory. This makes it difficult for programs structured in a `cmd` directory to embed files outside it.\n- Better [filesystem abstraction](https://github.com/knadh/stuffbin/blob/e80f23deca3fbc201ae7fb5f59a9e4ea6f17878e/fs.go#L17) for virtual filesystem manipulation, including merging.\n\n## How does it work?\n\n![stuffbin](https://user-images.githubusercontent.com/547147/50650557-caa04680-0fa6-11e9-9f8e-4d76cf331dc6.png)\n\nstuffbin compresses and embeds arbitrary files to the end of Go binaries. This does not affect the normal execution of the binary as the compressed data that is appended beyond the binary's original size is simply ignored by the operating system. When a stuffed application is executed, stuffbin reads the compressed bytes from self (the executable), uncompresses the files on the fly into an in-memory filesystem, and provides a FileSystem interface to access them. This enables Go applications that have external file dependencies to be shipped a single _fat_ binary, commonly, web applications that have static file and template dependencies.\n\n- Built in ZIP compression\n- A virtual filesystem abstraction to access embedded files\n- Add static assets from nested directories recursively\n- Re-path files and whole directories with the :suffix format, eg: ../my/original/file.txt:/my/virtual/file.txt and /my/nested/dir:/virtual/dir\n- Template parsing helper similar to template.ParseGlob() to parse templates from the virtual filesystem\n- Launch an http.FileServer for serving static files\n- Gracefully failover to the local file system in the absence of embedded assets\n- CLI to stuff, unstuff and extract, and list stuffed files in binaries\n\n## Installation\n\n```shell\ngo install github.com/knadh/stuffbin/...@latest\n```\n\n### Homebrew\n\nFor macOS/Linux users, you can install via [brew](https://brew.sh/)\n\n```sh\n$ brew install stuffbin\n```\n\n## Usage\n\n#### Stuffing and embedding\n\n```shell\n# -a, -in, and -out params followed by the paths of files to embed.\n# To normalize paths, aliases can be suffixed with a colon.\nstuffbin -a stuff -in /path/to/exe -out /path/to/new.exe \\\n    static/file1.css static/file2.pdf /somewhere/else/file3.txt:/static/file3.txt\n```\n\n#### List files in a stuffed binary\n\n```shell\nstuffbin -a id -in /path/to/new/exe\n```\n\n#### Extract stuffed files from a binary\n\n```shell\nstuffbin -a unstuff -in /path/to/new/exe -out assets.zip\n```\n\n## In the application\n\nTo test this, `cd` into `./mock` and run `go run mock.go`\n\n```go\npackage main\n\nimport (\n\t\"fmt\"\n\t\"log\"\n\t\"net/http\"\n\t\"os\"\n\n\t\"github.com/knadh/stuffbin\"\n)\n\nfunc main() {\n\t// Get self executable path.\n\tpath, err := os.Executable()\n\tif err != nil {\n\t\tlog.Fatalf(\"error getting executable path: %v\", err)\n\t}\n\t// Read stuffed data from self.\n\tfs, err := stuffbin.UnStuff(path)\n\tif err != nil {\n\t\t// Binary is unstuffed or is running in dev mode.\n\t\t// Can halt here or fall back to the local filesystem.\n\t\tif err == stuffbin.ErrNoID {\n\t\t\t// First argument is to the root to mount the files in the FileSystem\n\t\t\t// and the rest of the arguments are paths to embed.\n\t\t\tfs, err = stuffbin.NewLocalFS(\"/\",\n\t\t\t\t\"./\", \"bar.txt:/virtual/path/bar.txt\")\n\t\t\tif err != nil {\n\t\t\t\tlog.Fatalf(\"error falling back to local filesystem: %v\", err)\n\t\t\t}\n\t\t} else {\n\t\t\tlog.Fatalf(\"error reading stuffed binary: %v\", err)\n\t\t}\n\t}\n\n\tfmt.Println(\"loaded files\", fs.List())\n\t// Read the file 'foo'.\n\tf, err := fs.Get(\"foo.txt\")\n\tif err != nil {\n\t\tlog.Fatalf(\"error reading foo.txt: %v\", err)\n\t}\n\tlog.Println(\"foo.txt =\", string(f.ReadBytes()))\n\n\t// Read the file 'bar'.\n\tf, err = fs.Get(\"/virtual/path/bar.txt\")\n\tif err != nil {\n\t\tlog.Fatalf(\"error reading /virtual/path/bar.txt: %v\", err)\n\t}\n\tlog.Println(\"/virtual/path/bar.txt =\", string(f.ReadBytes()))\n\n\tfmt.Println(\"stuffed files:\")\n\tfor _, f := range fs.List() {\n\t\tfmt.Println(\"\\t\", f)\n\t}\n\n\t// Compile templates with the helpers:\n\t// err, tpl := stuffbin.ParseTemplatesGlob(nil, fs, \"/templates/*.html\")\n\t//\n\t// Template func map.\n\t// mp := map[string]interface{}{\n\t// \t\"Foo\": func() string {\n\t// \t\treturn \"func\"\n\t// \t},\n\t// }\n\t// err, tpl := stuffbin.ParseTemplates(mp, fs, \"/templates/index.html\", \"/templates/hello.html\")\n\n\t// Expose an HTTP file server.\n\t// Try http://localhost:8000/static/foo.txt\n\t// Try http://localhost:8000/static/virtual/path/bar.txt\n\t// Try http://localhost:8000/static/subdir/baz.txt\n\thttp.Handle(\"/static/\", http.StripPrefix(\"/static/\", fs.FileServer()))\n\tlog.Println(\"listening on :8000\")\n\thttp.ListenAndServe(\":8000\", nil)\n}\n```\n\n### License\n\nLicensed under the MIT License.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fknadh%2Fstuffbin","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fknadh%2Fstuffbin","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fknadh%2Fstuffbin/lists"}