{"id":18784013,"url":"https://github.com/u-root/gobusybox","last_synced_at":"2025-04-12T17:45:23.086Z","repository":{"id":38020194,"uuid":"292972020","full_name":"u-root/gobusybox","owner":"u-root","description":"Tools for compiling many Go commands into one binary to save space. Builds are supported for vendor-based Go and module-based Go","archived":false,"fork":false,"pushed_at":"2025-01-01T17:01:34.000Z","size":1645,"stargazers_count":151,"open_issues_count":7,"forks_count":24,"subscribers_count":47,"default_branch":"main","last_synced_at":"2025-04-03T20:12:14.228Z","etag":null,"topics":["bazel","busybox","embedded","golang","starlark"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/u-root.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-09-05T00:26:32.000Z","updated_at":"2025-02-10T23:28:19.000Z","dependencies_parsed_at":"2024-02-18T01:24:18.574Z","dependency_job_id":"add18ed0-ebb2-426f-8d2f-0ce9caa05d51","html_url":"https://github.com/u-root/gobusybox","commit_stats":{"total_commits":258,"total_committers":9,"mean_commits":"28.666666666666668","dds":0.03875968992248058,"last_synced_commit":"d8fbaca23e26beab648c86c8a67335ad65d0d15c"},"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/u-root%2Fgobusybox","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/u-root%2Fgobusybox/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/u-root%2Fgobusybox/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/u-root%2Fgobusybox/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/u-root","download_url":"https://codeload.github.com/u-root/gobusybox/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248609541,"owners_count":21132915,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bazel","busybox","embedded","golang","starlark"],"created_at":"2024-11-07T20:41:37.951Z","updated_at":"2025-04-12T17:45:23.061Z","avatar_url":"https://github.com/u-root.png","language":"Go","readme":"# Go Busybox\n\n[![PkgGoDev](https://pkg.go.dev/badge/github.com/u-root/gobusybox/src)](https://pkg.go.dev/github.com/u-root/gobusybox/src)\n[![Build Status](https://circleci.com/gh/u-root/gobusybox.svg?style=svg)](https://circleci.com/gh/u-root/gobusybox/tree/main)\n[![Slack](https://slack.osfw.dev/badge.svg)](https://slack.osfw.dev)\n\n**Contact**: best bet to reach us is the\n[#u-root-dev](https://osfw.slack.com/messages/u-root-dev) channel on the [Open\nSource Firmware Slack](https://slack.osfw.dev) ([Sign Up\nLink](https://slack.osfw.dev)).\n\nGo Busybox is a set of Go tools that allow you to compile many Go commands into\none binary. The resulting binary uses its invocation arguments (`os.Args`) to\ndetermine which command is being called.\n\n| Feature    | Support status                                        |\n| ---------- | ----------------------------------------------------- |\n| Go version | Tested are 1.20-1.22                                  |\n| Packaging  | Go workspaces, Go modules, Go vendoring               |\n| `GOOS`     | any (linux is tested)                                 |\n| `GOARCH`   | any (amd64, arm, arm64, riscv64 are tested)           |\n| CGO        | *Not supported*                                       |\n\nAn example:\n\n```bash\ngo install github.com/u-root/gobusybox/src/cmd/makebb@latest\n\ngit clone github.com/u-root/u-root\ncd u-root\nmakebb ./cmds/core/dmesg ./cmds/core/strace\n```\n\nA binary named `bb` should appear. It can be invoked in one of two ways --\neither with a symlink or using a second argument.\n\n```bash\n./bb dmesg\n./bb strace echo \"hi\"\n```\n\nIt is meant to be used with symlinks for convenience:\n\n```bash\n# Make a symlink dmesg -\u003e bb\nln -s bb dmesg\n# Symlink means that argv[0] is the command name.\n./dmesg\n\n# Make a symlink strace -\u003e bb\nln -s bb strace\n./strace echo \"hi\"\n```\n\nGo Busybox does this by copying all the source for these Go commands and\nrewriting it [in a temporary directory](#how-it-works).\n\nGo Busybox can be used with **any Go commands** across multiple Go modules:\n\n```sh\nmkdir workspace\ncd workspace\n\ngit clone https://github.com/hugelgupf/p9\ngit clone https://github.com/u-root/cpu\n\ngo work init ./p9\ngo work use ./cpu\n\nmakebb ./cpu/cmds/* ./p9/cmd/*\n```\n\n\u003e [!IMPORTANT]\n\u003e `makebb` works any time `go build` or `go list` also work.\n\u003e\n\u003e For multi-module compilation, use Go workspaces or read below.\n\n## Path resolution \u0026 multi-module builds\n\n`makebb` and the APIs mentioned below all accept commands from file system\npaths, Go package paths, and globs matching path.Match thereof.\n\n### makebb with globs and exclusions\n\nIn addition to the standard syntaxes supported by `go list` and `go build`,\n`makebb` also accepts globs and exclusions.\n\n```sh\ngit clone https://github.com/u-root/u-root\ncd u-root\n\nmakebb ./cmds/core/ip ./cmds/core/init\n\n# Escaping the * to show that the Go APIs know how to resolve this as well\nmakebb ./cmds/core/\\*\nmakebb ./cmds/core/i\\* ./cmds/boot/pxeboot\n\n# All core commands except ip.\nmakebb ./cmds/core/\\* -./cmds/core/ip\n```\n\n### makebb with Go workspaces \u0026 `GBB_PATH`.\n\nTo compile commands from multiple modules, you may use workspaces.\n\n```shell\nmkdir workspace\ncd workspace\n\ngit clone https://github.com/u-root/u-root\ngit clone https://github.com/u-root/cpu\n\ngo work init ./u-root\ngo work use ./cpu\n\nmakebb \\\n    ./u-root/cmds/core/init \\\n    ./u-root/cmds/core/elvish \\\n    ./cpu/cmds/cpud\n\n# Also works for offline builds with `go work vendor` (Go 1.22 feature):\ngo work vendor\n\nmakebb \\\n    ./u-root/cmds/core/init \\\n    ./u-root/cmds/core/elvish \\\n    ./cpu/cmds/cpud\n```\n\nFor a shortcut to specify many commands sharing common path elements (e.g. from\nthe same repository), the `GBB_PATH` environment variable exists. Paths are\nconcatenated with every colon-separated element of `GBB_PATH` from left to right\nand checked for existence.\n\n```shell\nGBB_PATH=$(pwd)/u-root:$(pwd)/cpu makebb \\\n    cmds/core/init \\\n    cmds/core/elvish \\\n    cmds/cpud\n\n# matches:\n#   $(pwd)/u-root/cmds/core/init\n#   $(pwd)/u-root/cmds/core/elvish\n#   $(pwd)/cpu/cmds/cpud\n```\n\n### goanywhere\n\n```shell\ngo install github.com/u-root/gobusybox/src/cmd/goanywhere@latest\n```\n\n`goanywhere` creates a Go workspace temporarily on the fly from the packages'\nmodules in the given local file paths.\n\ngoanywhere then executes the command given after \"--\" in the workspace\ndirectory and amends the packages as args.\n\nFor example,\n\n```\n# -o $(pwd) is needed since goanywhere executes the command in the workspace\n# directory, and by default the Go binary is created in the current wd.\ngoanywhere ./u-root/cmds/core/{init,gosh} -- go build -o $(pwd)\ngoanywhere ./u-root/cmds/core/{init,gosh} -- makebb -o $(pwd)\ngoanywhere ./u-root/cmds/core/{init,gosh} -- mkuimage [other args]\ngoanywhere ./u-root/cmds/core/{init,gosh} -- u-root [other args]\n```\n\nLike `makebb`, `goanywhere` supports `GBB_PATH`, globs and shell expansions.\n\n```\nGBB_PATH=$(pwd)/u-root:$(pwd)/cpu goanywhere \\\n    cmds/core/{init,gosh} cmds/cpud -- makebb -o $(pwd)\n```\n\n### makebb with multiple Go modules\n\n`makebb` supports Go workspaces for locally checked out sources, as shown above.\n\nFor multiple Go module command dependencies that aren't checked out locally, we\napply some Go module tricks.\n\nTo depend on commands outside of ones own repository, the easiest way to depend\non Go commands is the following:\n\n```sh\nTMPDIR=$(mktemp -d)\ncd $TMPDIR\ngo mod init foobar\n```\n\nCreate a file with some unused build tag like this to create dependencies on commands:\n\n```go\n//go:build tools\n\npackage something\n\nimport (\n        _ \"github.com/u-root/u-root/cmds/core/ip\"\n        _ \"github.com/u-root/u-root/cmds/core/init\"\n        _ \"github.com/hugelgupf/p9/cmd/p9ufs\"\n)\n```\n\nYou can generate this file for your repo with the `gencmddeps` tool:\n\n```\ngo install github.com/u-root/gobusybox/src/cmd/gencmddeps@latest\n\ngencmddeps -o deps.go -t tools -p something \\\n    github.com/u-root/u-root/cmds/core/{ip,init} \\\n    github.com/hugelgupf/p9/cmd/p9ufs\n```\n\n\u003e [!IMPORTANT]\n\u003e `gencmddeps` does not support file paths or exclusions, as these rely on a\n\u003e `go.mod` already being present to resolve the Go package name.\n\u003e\n\u003e The input to gencmddeps must be full Go package paths.\n\nThe unused build tag keeps it from being compiled, but its existence forces `go\nmod` to add these dependencies to `go.mod`:\n\n```sh\ngo mod tidy\n\nmakebb \\\n  github.com/u-root/u-root/cmds/core/ip \\\n  github.com/u-root/u-root/cmds/core/init \\\n  github.com/hugelgupf/p9/cmd/p9ufs\n\n# Also works with vendored, offline builds:\ngo mod vendor\n\nmakebb \\\n  github.com/u-root/u-root/cmds/core/ip \\\n  github.com/u-root/u-root/cmds/core/init \\\n  github.com/hugelgupf/p9/cmd/p9ufs\n```\n\n## APIs\n\nBesides the makebb CLI command, there is a\n[Go API at src/pkg/bb](https://pkg.go.dev/github.com/u-root/gobusybox/src/pkg/bb).\n\n## Shortcomings\n\n-   Any *imported* packages' `init` functions are run for *every* command.\n\n    For example, if some command imports and uses the `testing` package, all\n    commands in the busybox will have testing's flags registered as a side\n    effect, because `testing`'s init function runs with every command.\n\n    While Go busybox handles every main commands' init functions, it does not\n    handle dependencies' init functions. Done properly, it would rewrite all\n    non-standard-library packages as well as commands. This has not been\n    necessary to implement so far. It would likely be necessary if, for example,\n    two different imported packages register the same flag unconditionally\n    globally.\n\n## How It Works\n\n[src/pkg/bb](src/pkg/bb) implements a Go source-to-source transformation on pure\nGo code (no cgo).\n\nThis AST transformation does the following:\n\n-   Takes a Go command's source files and rewrites them into Go package files\n    (almost) without global side effects.\n-   Writes a `main.go` file with a `main()` that calls into the appropriate Go\n    command package based on `argv[0]` or `argv[1]`.\n\nThis allows you to take two Go commands, such as Go implementations of `dmesg`\nand `strace` and compile them into one binary.\n\nWhich command is invoked is determined by `argv[0]` or `argv[1]` if `argv[0]` is\nnot recognized. Let's say `bb` is the compiled binary; the following are\nequivalent invocations of `dmesg` and `strace`:\n\n```sh\n(cd ./src/cmd/makebb \u0026\u0026 go install)\n(cd ./test/nested \u0026\u0026 makebb ./cmd/dmesg ./cmd/strace)\n\n# Make a symlink dmesg -\u003e bb\nln -s bb dmesg\n./dmesg\n\n# Make a symlink strace -\u003e bb\nln -s bb strace\n./strace echo \"hi\"\n```\n\n```sh\n./bb dmesg\n./bb strace echo \"hi\"\n```\n\n### Command Transformation\n\nPrincipally, the AST transformation moves all global side-effects into callable\npackage functions. E.g. `main` becomes `registeredMain`, each `init` becomes\n`initN`, and global variable assignments are moved into their own `initN`. A\n`registeredInit` calls each `initN` function in the correct init order.\n\nThen, these `registeredMain` and `registeredInit` functions can be registered\nwith a global map of commands by name and used when called upon.\n\nLet's say a command `github.com/org/repo/cmds/sl` contains the following\n`main.go`:\n\n```go\npackage main\n\nimport (\n  \"flag\"\n  \"log\"\n)\n\nvar name = flag.String(\"name\", \"\", \"Gimme name\")\n\nfunc init() {\n  log.Printf(\"init %s\", *name)\n}\n\nfunc main() {\n  log.Printf(\"train\")\n}\n```\n\nThis would be rewritten to be:\n\n```go\npackage sl // based on the directory name\n\nimport (\n  \"flag\"\n  \"log\"\n\n  \"../bb/pkg/bbmain\" // generated import path\n)\n\n// Type has to be inferred through type checking.\nvar name *string\n\nfunc init0() {\n  log.Printf(\"init %s\", *name)\n}\n\nfunc init1() {\n  name = flag.String(\"name\", \"\", \"Gimme name\")\n}\n\nfunc registeredInit() {\n  // Order is determined by go/types.Info.InitOrder.\n  init1()\n  init0()\n}\n\nfunc registeredMain() {\n  log.Printf(\"train\")\n}\n\nfunc init() {\n  bbmain.Register(\"sl\", registeredInit, registeredMain)\n}\n```\n\n### Generated main.go\n\nThe main.go file is generated from\n[./src/pkg/bb/bbmain/cmd/main.go](./src/pkg/bb/bbmain/cmd/main.go).\n\n```go\npackage main\n\nimport (\n  \"os\"\n  \"log\"\n  \"path/filepath\"\n\n  // Side-effect import so init in sl calls bbmain.Register\n  _ \"github.com/org/repo/cmds/sl\"\n\n  \"../bb/pkg/bbmain\"\n)\n\nfunc main() {\n  bbmain.Run(filepath.Base(os.Argv[0]))\n}\n```\n\n### Directory Structure\n\nAll files are written into a temporary directory. All dependency Go packages are\nalso written there.\n\nThe directory structure we generate resembles a $GOPATH-based source tree, even\nif we are combining module-based Go commands. Regardless of whether the original\ncommands are based on Go modules, Go workspaces, or GOPATH, we generate the same\nstructure and compiled with `GOPATH=$tmpdir GO111MODULE=off`.\n\nThis means that in all cases, traditionally offline compilations remain offline\n(e.g. GOPATH, or vendored modules / workspaces).\n\n```\n/tmp/bb-$NUM/\n└── src\n    ├── bb.u-root.com\n    │   └── bb\n    │       ├── main.go               \u003c\u003c ./src/pkg/bb/bbmain/cmd/main.go (with edits)\n    │       └── pkg\n    │           └── bbmain\n    │               └── register.go   \u003c\u003c ./src/pkg/bb/bbmain/register.go\n    └── github.com\n        └── u-root\n            ├── uio\n            │   ├── uio               \u003c\u003c dependency used by both\n            │   └── ulog              \u003c\u003c dependency used by both\n            ├── u-bmc\n            │   ├── cmd\n            │   │   ├── fan           \u003c\u003c generated command package\n            │   │   ├── login         \u003c\u003c generated command package\n            │   │   └── socreset      \u003c\u003c generated command package\n            │   └── pkg\n            │       ├── acme          \u003c\u003c dependency copied from u-bmc\n            │       ├── aspeed        \u003c\u003c dependency copied from u-bmc\n            │       ├── gpiowatcher   \u003c\u003c dependency copied from u-bmc\n            │       └── mtd           \u003c\u003c dependency copied from u-bmc\n            └── u-root\n                ├── cmds\n                │   └── core\n                │       ├── cat       \u003c\u003c generated command package\n                │       ├── ip        \u003c\u003c generated command package\n                │       └── ls        \u003c\u003c generated command package\n                └── pkg\n                    ├── curl          \u003c\u003c dependency copied from u-root\n                    ├── dhclient      \u003c\u003c dependency copied from u-root\n                    ├── ip            \u003c\u003c dependency copied from u-root\n                    ├── ls            \u003c\u003c dependency copied from u-root\n                    └── uio           \u003c\u003c dependency copied from u-root\n```\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fu-root%2Fgobusybox","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fu-root%2Fgobusybox","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fu-root%2Fgobusybox/lists"}