Data

Tools

Indexes

Applications

Experiments

Awesome

An open API service indexing awesome lists of open source software.

https://github.com/earentir/pbuild

Build go projects
https://github.com/earentir/pbuild

Last synced: 4 months ago
JSON representation

Build go projects

Awesome Lists containing this project

README 

          
# pbuild



Build go projects, it is very specific to my project but can be used by anyone.



A cross-compilation tool for Go projects that builds for multiple target platforms with automatic `.gitignore` management.



## Features



- Cross-compile Go projects for multiple platforms

- Automatic `.gitignore` management (adds `builds/` directory if missing)

- Parallel builds with configurable workers

- Compression support (gzip, zstd)

- Checksum generation (SHA256, SHA512)

- Build metadata and reporting

- Flexible build strategies (purego, flexible, traditional)

- **Reproducible builds** (`--reproducible`): same code produces the same binary hash; forces `-trimpath` and deterministic gzip

- **Vendor support** (`--vendor`): build with `-mod=vendor`; creates `vendor/` if missing; prompts to re-vendor and retry when builds fail due to out-of-date vendored packages

- **GPG signing** (`--sign`): create detached signatures (`.sig` or `.asc`) for each artifact using a pure-Go OpenPGP implementation; each signature is verified after creation

- **SLSA provenance** (`--provenance`): write a single `provenance.intoto.jsonl` (one in-toto Statement per artifact) for OpenSSF/supply-chain verification; can be signed with `--sign`

- **Profile** (`--profile`): use a saved target list from `builds/pbuild-profile.json`; on first run the file is created with all targets enabled—edit it to set `"enabled": false` for targets you don't want; subsequent builds use only enabled targets

- **GitHub release upload**: upload build artifacts to a GitHub release. Use `--upload-release` on the build command (build and upload in one run), or the **`upload` subcommand** to upload an existing build later and choose which build to push (by version, path, or `--list`)



## Installation



```bash

go install github.com/earentir/pbuild@latest

```



## Usage



### Basic Usage



Build for current platform:

```bash

pbuild

```



Build for all predefined targets:

```bash

pbuild --all

```



Build with verbose output:

```bash

pbuild --verbose

```



Reproducible builds (same code → same binary hash; recommended with `--vendor` for fully deterministic builds):

```bash

pbuild --reproducible --vendor

```



Use vendored dependencies (creates `vendor/` on first run if missing; prompts to re-vendor on failure if packages are out of date):

```bash

pbuild --vendor

```



Sign release artifacts with GPG (requires a key file; see [Generating a GPG key pair](#generating-a-gpg-key-pair)):

```bash

pbuild --all --sign --signing-key-file ./release-key.asc

```



Use a profile to build only the targets you want (first run creates `builds/pbuild-profile.json` with all targets enabled; edit to disable, then run again):

```bash

pbuild --profile

```



Upload build artifacts to an existing GitHub release (repo from git remote; tag defaults to v + version, e.g. v1.3.27-ab5bb44):

```bash

GITHUB_TOKEN=ghp_xxx pbuild --all --upload-release

```



Create the release if it does not exist, then upload:

```bash

GITHUB_TOKEN=ghp_xxx pbuild --all --upload-release --release-create

```



**Build now, upload later:** Build and upload are separate. Build once, then push to GitHub when ready using the `upload` subcommand and choose which build to upload:



```bash

pbuild --all                                    # build only

pbuild upload --list                            # list available builds

GITHUB_TOKEN=ghp_xxx pbuild upload 1.3.27-ab5bb44 --release-create   # upload that build

```



You can specify the build with a positional version (`pbuild upload 1.3.27-ab5bb44`), `--version`, or `--version-dir`; use `--list` to see available version directories.



### Example Runs



#### 1. Basic Build (Current Platform)

```bash

$ pbuild

builds/ directory already in .gitignore file

Building version 1.1.7-abc123



 BUILD CONFIG │ VALUE     CPU LEVELS │   VALUE           BEHAVIOR      │ VALUE 

──────────────┼────────  ────────────┼───────────  ────────────────────┼───────

 Strategy     │ purego    AMD64      │ v2           Parallel Workers   │ 6     

──────────────┼────────  ────────────┼───────────  ────────────────────┼───────

 Build Mode   │ auto      ARM64      │ v8.0         Clean Cache        │ false 

                         ────────────┼───────────  ────────────────────┼───────

                          ARM        │ 7            Skip Cleanup       │ false 

                         ────────────┼───────────  ────────────────────┼───────

                          MIPS       │ hardfloat    Stop on Error      │ false 

                         ────────────┼───────────  ────────────────────┼───────

                          PPC64      │ power8       Verbose            │ false 

                         ────────────┼───────────  ────────────────────┼───────

                          RISC-V     │ rva20u64     Generate Checksums │ true  



Building for: linux/amd64 -> /path/to/project/builds/1.1.7-abc123/myapp

  SUCCESS



Artifacts for myapp, version 1.1.7-abc123

stored in /path/to/project/builds/1.1.7-abc123



  FILE  │   TARGET    │       SIZE        │                             SHA 256                              │ STATUS 

────────┼─────────────┼───────────────────┼──────────────────────────────────────────────────────────────────┼────────

 myapp  │ linux/amd64 │ 2.1 MiB (2201234) │ a1b2c3d4e5f6789012345678901234567890abcdef1234567890abcdef123456 │  ✓      



Build summary: Total: 1  Success: 1  Failed: 0



Build metadata written to: /path/to/project/builds/1.1.7-abc123/build-metadata.json

```



#### 2. Cross-Platform Build (All Targets)

```bash

$ pbuild --all

builds/ directory already in .gitignore file

Building version 1.1.7-abc123



[Configuration tables shown above]



Building for: linux/amd64 -> /path/to/project/builds/1.1.7-abc123/myapp

  SUCCESS



Building for: linux/arm64 -> /path/to/project/builds/1.1.7-abc123/myapp

  SUCCESS



Building for: windows/amd64 -> /path/to/project/builds/1.1.7-abc123/myapp.exe

  SUCCESS



Building for: darwin/amd64 -> /path/to/project/builds/1.1.7-abc123/myapp

  SUCCESS



Building for: darwin/arm64 -> /path/to/project/builds/1.1.7-abc123/myapp

  SUCCESS



Artifacts for myapp, version 1.1.7-abc123

stored in /path/to/project/builds/1.1.7-abc123



  FILE      │   TARGET    │       SIZE        │                             SHA 256                              │ STATUS 

────────────┼─────────────┼───────────────────┼──────────────────────────────────────────────────────────────────┼────────

 myapp      │ linux/amd64 │ 2.1 MiB (2201234) │ a1b2c3d4e5f6789012345678901234567890abcdef1234567890abcdef123456 │  ✓      

 myapp      │ linux/arm64 │ 1.8 MiB (1887654) │ b2c3d4e5f6789012345678901234567890abcdef1234567890abcdef1234567a │  ✓      

 myapp.exe  │ windows/amd64│ 2.2 MiB (2309876) │ c3d4e5f6789012345678901234567890abcdef1234567890abcdef1234567ab2 │  ✓      

 myapp      │ darwin/amd64 │ 2.0 MiB (2105432) │ d4e5f6789012345678901234567890abcdef1234567890abcdef1234567abc3 │  ✓      

 myapp      │ darwin/arm64 │ 1.7 MiB (1765432) │ e5f6789012345678901234567890abcdef1234567890abcdef1234567abcd4 │  ✓      



Build summary: Total: 5  Success: 5  Failed: 0



Build metadata written to: /path/to/project/builds/1.1.7-abc123/build-metadata.json

```



#### 3. Verbose Build with Compression

```bash

$ pbuild --all --verbose --compress zstd

builds/ directory already in .gitignore file

Building version 1.1.7-abc123



[Configuration tables shown above]



[Worker 0] Building for: linux/amd64 -> /path/to/project/builds/1.1.7-abc123/myapp

  Command: go build -trimpath -buildmode=exe -tags purego -ldflags -s -w -X main.appVersion=1.1.7-abc123 -o /path/to/project/builds/1.1.7-abc123/myapp .

  Environment: GOOS=linux GOARCH=amd64 GOAMD64=v2

[Worker 0]   SUCCESS

[Worker 0]   Compressed to /path/to/project/builds/1.1.7-abc123/myapp.zst



[Worker 1] Building for: linux/arm64 -> /path/to/project/builds/1.1.7-abc123/myapp

  Command: go build -trimpath -buildmode=exe -tags purego -ldflags -s -w -X main.appVersion=1.1.7-abc123 -o /path/to/project/builds/1.1.7-abc123/myapp .

  Environment: GOOS=linux GOARCH=arm64 GOARM64=v8.0

[Worker 1]   SUCCESS

[Worker 1]   Compressed to /path/to/project/builds/1.1.7-abc123/myapp.zst



[Additional workers...]



Artifacts for myapp, version 1.1.7-abc123

stored in /path/to/project/builds/1.1.7-abc123



  FILE      │   TARGET    │       SIZE        │                             SHA 256                              │ STATUS 

────────────┼─────────────┼───────────────────┼──────────────────────────────────────────────────────────────────┼────────

 myapp.zst  │ linux/amd64 │ 890 KiB (911234)  │ f6789012345678901234567890abcdef1234567890abcdef1234567abcde5 │  ✓      

 myapp.zst  │ linux/arm64 │ 756 KiB (774321)  │ 789012345678901234567890abcdef1234567890abcdef1234567abcdef6 │  ✓      

 myapp.zst  │ windows/amd64│ 912 KiB (934567)  │ 89012345678901234567890abcdef1234567890abcdef1234567abcdef78 │  ✓      

 myapp.zst  │ darwin/amd64 │ 845 KiB (865432)  │ 9012345678901234567890abcdef1234567890abcdef1234567abcdef789 │  ✓      

 myapp.zst  │ darwin/arm64 │ 723 KiB (740123)  │ 012345678901234567890abcdef1234567890abcdef1234567abcdef7890 │  ✓      



Build summary: Total: 5  Success: 5  Failed: 0



Build metadata written to: /path/to/project/builds/1.1.7-abc123/build-metadata.json

```



#### 4. .gitignore Management Examples



When `.gitignore` doesn't exist:

```bash

$ pbuild

No .gitignore file found - skipping builds/ directory check

Building version 1.1.7-abc123

[... rest of build output ...]

```



When `builds/` is missing from existing `.gitignore`:

```bash

$ pbuild

Added builds/ to .gitignore file

Building version 1.1.7-abc123

[... rest of build output ...]

```



When `builds/` already exists in `.gitignore`:

```bash

$ pbuild

builds/ directory already in .gitignore file

Building version 1.1.7-abc123

[... rest of build output ...]

```



## Command Line Options



```bash

pbuild [TARGET_DIR] [flags]



Flags:

      --all                  build for all predefined targets

      --amd64-level string   GOAMD64 level: v1, v2, v3, v4 (default "v2")

      --arm-level string     GOARM level: 5, 6, 7 (default "7")

      --arm64-level string   GOARM64 level: v8.0, v8.1, v8.2, v8.3, v8.4, v8.5, v8.6, v8.7, v8.8, v8.9, v9.0, v9.1, v9.2, v9.3, v9.4, v9.5 (default "v8.0")

      --build-flags string   additional go build flags (default: -trimpath)

      --buildmode string     build mode: auto (exe), pie (requires CGO), exe, c-archive, c-shared (default "auto")

      --checksums            generate SHA256 and SHA512 checksums (default true)

      --clean-cache          clean Go build cache before building

      --compress string      compress binaries: zstd, gzip

      --ldflags string       custom ldflags (default: -s -w -X main.appVersion)

      --mips-level string    GOMIPS level: hardfloat, softfloat (default "hardfloat")

      --name string          override inferred project name

      --output-dir string    directory for build artifacts (default "builds")

      --parallel int         number of parallel builds (0 = sequential) (default 6)

      --ppc64-level string   GOPPC64 level: power8, power9, power10 (default "power8")

      --reproducible         ensure reproducible builds (same code → same binary hash); forces -trimpath and deterministic gzip

      --riscv-level string   GORISCV64 level: rva20u64, rva22u64 (default "rva20u64")

      --skip-cleanup         skip cleaning previous build directory

      --stop-on-error        stop building others when one fails

      --strategy string      build strategy: flexible, purego, traditional (default "purego")

      --tags string          additional build tags (comma-separated)

      --verbose              show actual go build commands

      --vendor               use vendored dependencies (-mod=vendor); create vendor/ if missing; prompt to re-vendor on failure if out of date

      --sign                  create GPG detached signatures (.sig or .asc) for each artifact and verify after signing

      --signing-key-file path path to armored private key file (required when --sign)

      --signing-key id        key ID (hex) when key file contains multiple keys

      --sign-armor            output ASCII-armored signatures (.asc) instead of binary (.sig)

      --provenance            write SLSA provenance (provenance.intoto.jsonl) for OpenSSF/supply-chain verification

      --profile               use target list from builds/pbuild-profile.json (create on first run; edit to disable targets)

      --set-version string    override embedded version tag

      --upload-release        upload build artifacts to a GitHub release (release must exist unless --release-create)

      --release-create        create the release if it does not exist (only with --upload-release)

      --release-repo string   GitHub repo as owner/name (default: from git remote origin)

      --release-tag string    release tag (default: v + version, e.g. v1.3.27-ab5bb44)

      --release-draft         when creating, create as draft release

      --release-notes string  release notes body when creating

      --release-notes-file string  path to file containing release notes when creating

```



**Subcommands:** `pbuild upload [VERSION]` — upload an existing build to a GitHub release (no build step). Specify which build with VERSION, `--version`, or `--version-dir`; use `--list` to see available builds. See [Upload subcommand](#upload-subcommand-build-now-push-later).



## Profile (saved target list)



With `--profile`, pbuild uses a config file in the output directory to decide which OS/arch targets to build. This lets you avoid `--all` while still building a fixed set of targets every time.



1. **First run:** `pbuild --profile` creates `builds/pbuild-profile.json` with every predefined target and `"enabled": true`, then exits (no build). Edit the file, then run again.

2. **Edit the file:** Set `"enabled": false` for any target you don't want (e.g. drop `freebsd`, `openbsd`, or specific arches).

3. **Next runs:** `pbuild --profile` builds only the targets that still have `"enabled": true`.



Example profile (after editing):



```json

{

  "version": 1,

  "targets": [

    { "os": "linux", "arch": "amd64", "enabled": true },

    { "os": "linux", "arch": "arm64", "enabled": true },

    { "os": "darwin", "arch": "amd64", "enabled": false },

    { "os": "darwin", "arch": "arm64", "enabled": true },

    { "os": "windows", "arch": "amd64", "enabled": true }

  ]

}

```



The profile file lives under your output dir (default `builds/`), which is typically in `.gitignore`, so it is not committed. You can commit it if you want to share the same target list with others.



## Generating a GPG key pair



pbuild does **not** create or manage keys; it only signs with a key you provide. Use [GnuPG](https://gnupg.org/) to create a key and export it for pbuild.



### 1. Create a new key (GnuPG 2.1+)



```bash

gpg --full-generate-key

```



- Choose **RSA and RSA** (or **EdDSA** if you prefer).

- Set key size (e.g. 4096 for RSA).

- Enter your name, email, and an optional comment.

- Set a **passphrase** to protect the private key (recommended).



### 2. List your secret keys



```bash

gpg --list-secret-keys --keyid-format=long

```



Example output:



```

sec   rsa4096/ABCD1234EF567890 2024-01-15 [SC]

      XXXX...

uid                 [ultimate] Your Name 

```



The key ID is the part after the slash (e.g. `ABCD1234EF567890`).



### 3. Export the private key to a file (for pbuild)



Export the key in **armored** form so pbuild can read it:



```bash

gpg --export-secret-keys -a KEY_ID > release-key.asc

```



Replace `KEY_ID` with your key ID (e.g. `ABCD1234EF567890`). Keep `release-key.asc` secure and **do not commit it** to version control. Add it to `.gitignore`:



```

release-key.asc

```



### 4. Use the key with pbuild



```bash

pbuild --all --sign --signing-key-file ./release-key.asc

```



- If the key file contains **multiple keys**, specify which one with `--signing-key KEY_ID`.

- Use `--sign-armor` to produce `.asc` (ASCII) signatures instead of binary `.sig`.



### 5. Password-protected keys



If your exported key is **passphrase-protected** (recommended when creating the key), pbuild needs the passphrase to decrypt it:



- **Interactive:** Run pbuild from a terminal; when signing starts, it will prompt (like GPG):

  ```

  Passphrase for signing key: 

  ```

  Type the passphrase and press Enter. **The passphrase is not echoed** to the terminal (secure input).



- **Non-interactive / CI:** Set the environment variable **`PBUILD_SIGNING_PASSPHRASE`** to the passphrase so pbuild never prompts:

  ```bash

  export PBUILD_SIGNING_PASSPHRASE='your-passphrase'

  pbuild --all --sign --signing-key-file ./release-key.asc

  ```

  Avoid committing this or logging it; use a secret manager in CI.



- **No passphrase:** If the key was exported without a passphrase, leave the env unset and press Enter at the prompt (empty passphrase).



- **Not a terminal:** If stdin is not a terminal (e.g. CI, pipe), pbuild will not prompt; you must set `PBUILD_SIGNING_PASSPHRASE` or signing will fail with a clear error.



### 6. Public key in the build dir



When you sign with `--sign`, pbuild writes the **public key** into the same build directory (version folder) as **`release-key.asc`**. That way anyone can verify the signatures without your private key. This matches common practice for GitHub releases and similar: ship the public key next to the signed artifacts so users can run `gpg --import release-key.asc` then `gpg --verify myapp.sig myapp`.



### 7. Verify signatures (downstream users)



Anyone can verify a signed artifact with GnuPG. Use the **release-key.asc** from the same release dir:



```bash

gpg --import release-key.asc

gpg --verify myapp.sig myapp

```



## Build Artifacts



The tool creates a structured output directory:



```

builds/

└── 1.1.7-abc123/           # Version-specific directory

    ├── myapp               # Linux/Unix binaries

    ├── myapp.exe           # Windows binaries

    ├── myapp.zst           # Compressed binaries (if --compress used)

    # SHA256/SHA512 checksums are in build-metadata.json under artifacts[] (if --checksums enabled)

    ├── myapp.sig           # GPG detached signatures (if --sign used; .asc with --sign-armor)

    ├── release-key.asc     # Public key for verification (if --sign used)

    ├── provenance.intoto.jsonl   # SLSA provenance (if --provenance used; one Statement per line)

    ├── provenance.intoto.jsonl.sig  # Signature for provenance (if --sign and --provenance)

    └── build-metadata.json # Build information and configuration

```



## SLSA provenance and OpenSSF



pbuild supports [OpenSSF](https://openssf.org/) (Open Source Security Foundation) goals for signed releases and supply-chain transparency. For **OpenSSF-aligned releases**, use **both** `--sign` and `--provenance` when building:



- **`--sign`**: Produces GPG detached signatures for every artifact (and for the provenance file). This is what the OpenSSF Scorecard **Signed-Releases** check looks for; without signed assets, that check does not pass.

- **`--provenance`**: Produces SLSA build provenance so consumers can verify how and where each artifact was built. Signed provenance gives full supply-chain attestation.



**Recommended for releases:**



```bash

pbuild --all --sign --signing-key-file ./release-key.asc --provenance

```



Upload the resulting binaries, their `.sig`/`.asc` files, `release-key.asc`, and `provenance.intoto.jsonl` (plus `provenance.intoto.jsonl.sig`) to your release page so the OpenSSF Scorecard and downstream verifiers can use them.



---



With `--provenance`, pbuild writes a single **SLSA build provenance** file (`provenance.intoto.jsonl`) in the version directory. This helps with supply-chain verification and aligns with [OpenSSF](https://openssf.org/) and [SLSA](https://slsa.dev/) practices.



- **Format**: [in-toto Statement](https://github.com/in-toto/attestation) v1 with predicate type `https://slsa.dev/provenance/v1`. One JSON Line per built artifact (same build predicate; each line’s `subject` is one artifact).

- **Content**: Build definition (build type, external/internal parameters), run details (builder ID, invocation ID, timestamps), and per-artifact subject (name + sha256/sha512 digest).

- **Build type URI**: `https://github.com/earentir/pbuild/slsa-buildtype/v1` — identifies pbuild as the build system so verifiers can validate expectations.

- **Signing**: If you also use `--sign`, pbuild signs `provenance.intoto.jsonl` (e.g. `provenance.intoto.jsonl.sig`) with the same GPG key so the attestations are verifiable.



**OpenSSF Scorecard**: The “Signed-Releases” check looks for **signed release assets** (e.g. `.sig`/`.asc` next to binaries). Use `--sign` for that. Provenance does not replace signing; it adds verifiable build metadata.



**Verification**: Downstreams can open `provenance.intoto.jsonl`, verify its signature (if present), then parse each line as a JSON Statement and check `subject[].digest` against the corresponding artifact. Tools like [slsa-verifier](https://github.com/slsa-framework/slsa-verifier) can be used with a policy that accepts pbuild’s build type and builder ID.



## GitHub release upload



With `--upload-release`, pbuild uploads **all files** in the version build directory (binaries, checksums, signatures, provenance, build-metadata.json, etc.) to a GitHub release. This uses the GitHub REST API; no `gh` CLI is required.



- **Default (update only)**: The release for the given tag must already exist. pbuild gets that release and uploads each file as an asset. If no release exists, pbuild exits with an error and suggests using `--release-create`.

- **`--release-create`**: If the release does not exist, pbuild creates it (with optional `--release-draft` and `--release-notes` or `--release-notes-file`), then uploads assets. The tag is created at `HEAD` if it does not exist.

- **Repo**: Inferred from `git config remote.origin.url` (supports `https://github.com/owner/repo` and `git@github.com:owner/repo.git`). Override with `--release-repo owner/repo`.

- **Tag**: Defaults to the actual version with a `v` prefix (e.g. `v1.3.27-ab5bb44`). Use `--release-tag` only if you want a different tag (e.g. `v1.3.27`).

- **Auth**: Set the `GITHUB_TOKEN` environment variable (e.g. a fine-grained or classic PAT with `repo` scope). Required when uploading.



### Build and upload in one run



Example: build, sign, add provenance, then upload (release tag defaults to e.g. `v1.3.27-ab5bb44`):



```bash

GITHUB_TOKEN=ghp_xxx pbuild --all --sign --signing-key-file ./release-key.asc --provenance --upload-release

```



To create the release if it does not exist:



```bash

GITHUB_TOKEN=ghp_xxx pbuild --all --upload-release --release-create --release-notes "First release."

```



### Upload subcommand: build now, push later



Use the **`upload`** subcommand to upload an **existing** build without rebuilding. You choose which build to push:



- **Positional version:** `pbuild upload 1.3.27-ab5bb44` — uploads from `builds/1.3.27-ab5bb44`

- **`--version` / `-v`:** Same as positional, e.g. `pbuild upload --version 1.3.27-ab5bb44`

- **`--version-dir`:** Path to the build directory (absolute or relative)

- **`--list`:** List available version directories under the output dir and exit (no upload)



One of these is required (or use `--list` to see options). Release flags (`--release-create`, `--release-repo`, `--release-tag`, etc.) work the same as on the build command.



```bash

pbuild --all                                    # build only

pbuild upload --list                            # list available builds

GITHUB_TOKEN=ghp_xxx pbuild upload 1.3.27-ab5bb44 --release-create   # upload that build

```



Use `--release-tag` only when you want a different tag (e.g. `--release-tag v1.3.27`).



## .gitignore Management



The tool automatically manages the `builds/` directory in your `.gitignore` file:



- **If `.gitignore` exists but lacks `builds/`**: Adds the entry

- **If `.gitignore` doesn't exist**: Skips the check (doesn't create the file)

- **If `builds/` already exists**: Confirms it's present



This ensures your build artifacts are properly ignored by git without cluttering your repository.