{"id":51131547,"url":"https://github.com/francescobianco/packbase","last_synced_at":"2026-06-25T13:01:14.540Z","repository":{"id":351330470,"uuid":"1210449981","full_name":"francescobianco/packbase","owner":"francescobianco","description":"Packbase is a self-hosted distribution layer for software packages","archived":false,"fork":false,"pushed_at":"2026-04-15T20:15:53.000Z","size":26869,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-05-16T16:29:03.635Z","etag":null,"topics":["git-tag","package","package-manager"],"latest_commit_sha":null,"homepage":"https://zub.javanile.org","language":"Zig","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/francescobianco.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null},"funding":{"github":"francescobianco"}},"created_at":"2026-04-14T12:34:01.000Z","updated_at":"2026-04-15T20:15:57.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/francescobianco/packbase","commit_stats":null,"previous_names":["francescobianco/packbase"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/francescobianco/packbase","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/francescobianco%2Fpackbase","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/francescobianco%2Fpackbase/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/francescobianco%2Fpackbase/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/francescobianco%2Fpackbase/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/francescobianco","download_url":"https://codeload.github.com/francescobianco/packbase/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/francescobianco%2Fpackbase/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34775934,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-25T02:00:05.521Z","response_time":101,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["git-tag","package","package-manager"],"created_at":"2026-06-25T13:01:07.989Z","updated_at":"2026-06-25T13:01:14.525Z","avatar_url":"https://github.com/francescobianco.png","language":"Zig","funding_links":["https://github.com/sponsors/francescobianco"],"categories":[],"sub_categories":[],"readme":"# packbase\n\n[![CI](https://github.com/francescoalemanno/packbase/actions/workflows/ci.yml/badge.svg)](https://github.com/francescoalemanno/packbase/actions/workflows/ci.yml)\n[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE)\n[![Zig](https://img.shields.io/badge/Zig-0.15-orange.svg)](https://ziglang.org)\n[![Docker](https://img.shields.io/badge/Docker-ready-2496ED.svg)](Dockerfile)\n\n**packbase** is a self-hosted distribution layer for Zig packages built around one idea that must stay fixed:\nthe primary artifact is the tarball, not the Git repository.\n\nThis project matters because it defines a stricter and more reliable contract for package distribution:\n- packbase stores deterministic release tarballs under `/p/\u003cpkg\u003e/tag/\u003ctag\u003e.tar.gz`\n- `update` and `fetch` build and refresh the internal state of the registry\n- the backend must know tags, versions, and package metadata from that internal state\n- the Git-facing surface is only a pseudo-Git compatibility layer for clients such as `zig fetch`\n- packbase must not depend on hosting or exposing persistent mirrored Git repositories as its source of truth\n- packbase must not execute Git operations at request time just to discover what package data exists\n\nIn other words: Git is an ingress protocol and a compatibility interface, not the backend data model.\nThe backend truth is the tarball set plus the state derived from updates.\n\nThis distinction is the core value of the project. It makes packbase closer to a real package registry:\nstable, cacheable, inspectable, and operationally simpler than a Git mirror disguised as one.\n\n```\nupstream Git  ──►  packbase fetch/update  ──►  internal state + tarballs\n                                                      │\n                                                      ├─► /p/\u003cpkg\u003e/tag/\u003ctag\u003e.tar.gz\n                                                      └─► pseudo-Git interface for clients\n```\n\n`zig fetch` should be able to talk to packbase as if it were speaking to Git, while packbase internally remains a tarball-first registry.\n\n---\n\n## Quick start\n\n```bash\n# Build and run\ndocker build -t packbase .\ndocker run -p 8080:8080 \\\n  -e PACKBASE_TOKEN=secret \\\n  -e PACKBASE_ROOT=/data \\\n  -v packbase-data:/data \\\n  packbase\n```\n\nL'immagine di produzione non include fixture pre-caricati. Se vuoi avere sempre\nalmeno un pacchetto disponibile su un deployment Compose, `compose.yml` monta i\nfixture di test e li materializza sul volume dati al bootstrap del container.\nSe i fixture montati non sono presenti, il bootstrap genera comunque un\npacchetto seed `hello` incorporato.\n\n### Mirror a package\n\n```bash\ncurl -X POST http://localhost:8080/api/fetch \\\n  -H \"Authorization: Bearer secret\" \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\"url\":\"git+https://github.com/OrlovEvgeny/serde.zig\"}'\n# {\"status\":\"ok\",\"package\":\"serde.zig\",\"tag\":\"v0.3.0\",\"url\":\"/p/serde.zig/tag/v0.3.0.tar.gz\"}\n```\n\n### Install from packbase in your project\n\n```bash\nzig fetch --save http://localhost:8080/p/serde.zig/tag/v0.3.0.tar.gz\n```\n\n`build.zig.zon` becomes:\n\n```zig\n.dependencies = .{\n    .@\"serde.zig\" = .{\n        .url = \"http://localhost:8080/p/serde.zig/tag/v0.3.0.tar.gz\",\n        .hash = \"122059e3…\",\n    },\n},\n```\n\n---\n\n## Environment variables\n\n| Variable | Default | Description |\n|---|---|---|\n| `PACKBASE_ROOT` | `public` | Root directory for served files and materialised packages |\n| `PACKBASE_PORT` | `8080` | Listening port |\n| `PACKBASE_TOKEN` | *(unset)* | Bearer token for `POST /api/fetch`. When unset, auth is disabled |\n\n---\n\n## API\n\n### `POST /api/fetch`\n\nMirror an upstream Git repository.\n\n**Headers**\n- `Authorization: Bearer \u003ctoken\u003e` — required when `PACKBASE_TOKEN` is set\n- `Content-Type: application/json`\n\n**Body**\n```json\n{\"url\": \"git+https://github.com/owner/repo\"}\n```\n\n**Response `200`**\n```json\n{\n  \"status\": \"ok\",\n  \"package\": \"repo\",\n  \"tag\": \"v1.2.3\",\n  \"url\": \"/p/repo/tag/v1.2.3.tar.gz\",\n  \"tarballs_created\": 4,\n  \"tarballs_present\": 0,\n  \"tarball_count\": 4\n}\n```\n\n**Error codes**\n\n| Code | Meaning |\n|---|---|\n| `400` | Missing or malformed JSON body |\n| `401` | Missing Authorization header |\n| `403` | Invalid token |\n| `422` | Repository has no tags |\n| `502` | upstream fetch failed (network or URL error) |\n\n### `GET /p/\u003cpackage\u003e/tag/\u003ctag\u003e.tar.gz`\n\nDownload a previously mirrored tarball.\n\n### `GET /api/list`\n\nRestituisce:\n- `packages`: unione dei pacchetti locali e di quelli registrati tramite `PACKBASE_SOURCE`\n- `local_packages`: pacchetti realmente materializzati nell'istanza\n- `registered_packages`: pacchetti presenti nell'ultimo snapshot sincronizzato del source remoto\n\n**Response `200`**\n```json\n{\n  \"packages\": [\"hello\", \"remote-only\", \"serde.zig\"],\n  \"local_packages\": [\"hello\", \"serde.zig\"],\n  \"registered_packages\": [\"hello\", \"remote-only\"]\n}\n```\n\n### `GET /api/info/\u003cpackage\u003e`\n\nRestituisce lo snapshot persistito dell'ultimo `POST /api/update` per un pacchetto:\n- visibilita del pacchetto nell'istanza\n- presenza nel source catalog\n- materializzazione locale\n- tarball disponibili e loro dimensione\n- dimensione totale occupata dal pacchetto nell'istanza\n- esito dell'ultima verifica di fetchability pseudo-Git calcolata durante `update`\n\nLe informazioni non vengono calcolate on demand: se manca lo snapshot, va eseguita prima `POST /api/update`.\n\n**Response `200`**\n```json\n{\n  \"package\": \"hello\",\n  \"available\": true,\n  \"registered\": false,\n  \"local\": true,\n  \"tarball_dir_present\": true,\n  \"tarball_count\": 1,\n  \"latest_tag\": \"v0.1.0\",\n  \"latest_size_bytes\": 371,\n  \"size_bytes\": 371,\n  \"tarballs\": [{\n    \"tag\": \"v0.1.0\",\n    \"size_bytes\": 371,\n    \"manifest_present\": true,\n    \"git_commit_oid\": \"ce919ccf45951856a762ffdb8ef850301cd8c588\",\n    \"git_tree_oid\": \"ce919ccf45951856a762ffdb8ef850301cd8c588\",\n    \"tarball_sha256\": \"…\",\n    \"tarball_md5\": \"…\",\n    \"tarball_crc32\": \"…\"\n  }],\n  \"smart_http_ready\": true,\n  \"pseudo_git_fetchable\": true,\n  \"healthy\": true\n}\n```\n\n**Response `404`**\n```json\n{\n  \"status\": \"not_found\",\n  \"package\": \"missing-package\"\n}\n```\n\n### `GET /api/status`\n\nRestituisce metadati dell'istanza, incluso l'identificativo di rilascio della\nbuild servita e lo stato persistito dell'ultima `update`.\n\n**Response `200`**\n```json\n{\n  \"service\": \"packbase\",\n  \"release\": \"r0007\",\n  \"packages_total\": 79,\n  \"packages_healthy\": 74,\n  \"packages_unhealthy\": 5,\n  \"update\": {\n    \"state\": \"idle\",\n    \"started_at\": 1776183143,\n    \"updated_at\": 1776183143,\n    \"source_packages\": 79\n  }\n}\n```\n\n### `POST /api/update`\n\nRiallinea in modo soft lo stato interno in modo pubblico e idempotente:\n- materializza i tarball mancanti sotto `/p` per i pacchetti del source catalog che non sono `fresh`\n- mantiene il supporto ai repository fixture locali\n- aggiorna `update-server-info`\n- scarica `PACKBASE_SOURCE`, conserva lo snapshot locale, calcola un diff con lo snapshot precedente e aggiorna la lista dei pacchetti registrati\n- aggiorna lo snapshot persistito dei package sotto `.packbase/package-info.json`, includendo size e fetchability pseudo-Git\n- applica un cooldown per evitare carico eccessivo quando viene chiamata ripetutamente\n\nL'endpoint è pubblico e non richiede token.\n\n**Response `200`**\n```json\n{\n  \"status\": \"ok\",\n  \"repos_scanned\": 1,\n  \"packages_synced\": 1,\n  \"tarballs_created\": 1,\n  \"tarballs_present\": 0,\n  \"source_changed\": true,\n  \"source_packages\": 2,\n  \"source_added\": 2,\n  \"source_updated\": 0,\n  \"source_removed\": 0\n}\n```\n\n### `GET /git/\u003crepo\u003e.git/…`\n\nDumb-HTTP Git endpoint for pre-baked fixture repositories (used internally by CI).\n\n### `GET /\u003crepo\u003e/…`\n\nAlias del repository Git esposto in radice. Questo consente di clonare un\nrepository ospitato da packbase senza il prefisso `/git` e senza il suffisso\n`.git`, ad esempio:\n\n```bash\ngit clone https://pb.yafb.net/miopacchetto\n```\n\nSe il consumer usa URL VCS con prefisso `git+https://`, il path resta lo stesso:\n\n```text\ngit+https://pb.yafb.net/miopacchetto\n```\n\n---\n\n## Running the smoke test\n\n```bash\nmake test-smoke\n```\n\nThe smoke test:\n1. Builds the Docker image.\n2. Starts packbase with a test token.\n3. Verifies the dumb-HTTP Git endpoint with `git clone`.\n4. Calls `POST /api/fetch` to mirror `serde.zig` from GitHub.\n5. Runs `zig fetch --save git+http://.../hello` inside a container, confirming the pseudo-Git smart-HTTP path works through `git-upload-pack`.\n6. Checks the service logs for a chunked `git-upload-pack` request.\n7. Runs `zig build` against the fetched dependency so source resolution is verified, not just metadata fetch.\n\nTo verify the short Git URL directly, run:\n\n```bash\nbash test/remote.sh pb.yafb.net hello r0007\n```\n\nOr:\n\n```bash\nPACKBASE_REMOTE_DOMAIN=pb.yafb.net PACKBASE_EXPECTED_RELEASE=r0007 bash test/remote.sh\n```\n\nThe remote smoke now checks these things against the deployed instance behind Caddy:\n- root-level `git clone https://.../\u003crepo\u003e`\n- `POST /api/update` followed by `/api/info/\u003crepo\u003e` for persisted integrity metadata\n- `zig fetch --save git+https://.../\u003crepo\u003e`\n- `zig build` against the fetched dependency\n- liveness after a second `zig fetch`, so regressions that panic after the first request are visible\n- batch installation of the 10 smallest healthy packages selected from `/api/list` and validated with `/api/info/\u003cpkg\u003e`\n\nArtefacts survive in `test/tmp/` for inspection after the run.\n\n---\n\n## Building a distributed registry with packbase\n\npackbase is intentionally minimal: one binary, one HTTP server, files on disk.  \nThat simplicity makes it easy to compose into a **distributed, multi-tier registry**.\n\n### Topology\n\n```\n              ┌─────────────────────────────────────────────┐\n              │              upstream (GitHub, etc.)         │\n              └────────────────────┬────────────────────────┘\n                                   │ git+https://\n                    ┌──────────────▼──────────────┐\n                    │   Central packbase node      │\n                    │   (one per org / region)     │\n                    │   POST /api/fetch            │\n                    │   stores tarballs on S3/NFS  │\n                    └──────┬──────────────┬────────┘\n                           │              │\n              ┌────────────▼──┐    ┌──────▼───────────┐\n              │  Edge node A  │    │   Edge node B     │\n              │  (on-prem DC) │    │   (CI farm)       │\n              └───────┬───────┘    └────────┬──────────┘\n                      │                     │\n               zig fetch --save      zig fetch --save\n```\n\n### How it works\n\n**1. Central node pulls from upstream once**\n\nA cron job or webhook calls `POST /api/fetch` on the central node whenever a new tag appears upstream.  The central node clones the repo, creates a deterministic tarball, and stores it.\n\n**2. Edge nodes serve from local cache**\n\nEdge nodes point `PACKBASE_ROOT` at a replicated volume (S3 bucket, NFS share, or a nightly `rsync` from the central node).  They only serve `GET` requests; they never clone from GitHub.  Developer machines and CI runners always resolve packages from the nearest edge node.\n\n**3. `build.zig.zon` pins a packbase URL**\n\n```zig\n.httpx = .{\n    .url = \"https://packages.example.com/p/httpx/tag/v1.4.2.tar.gz\",\n    .hash = \"1220…\",\n},\n```\n\nSwapping the base URL (e.g. for a closer edge node) does not affect the hash, so reproducibility is preserved.\n\n**4. Immutability guarantee**\n\nTag URLs never change.  Once `/p/httpx/tag/v1.4.2.tar.gz` exists on the central node it is never overwritten.  Edge nodes replicate the blob by content address, so builds remain reproducible even if the upstream tag moves or the repository disappears.\n\n**5. Offline / air-gapped builds**\n\nOnce all dependencies are mirrored, the CI network can be locked down.  `zig fetch` resolves everything from the edge node on the internal network.  The upstream internet is no longer on the critical path.\n\n### Deployment recipe (minimal)\n\n```yaml\n# docker-compose.yml for a central + one edge node\nservices:\n  packbase-central:\n    image: packbase\n    environment:\n      PACKBASE_TOKEN: \"${PACKBASE_TOKEN}\"\n      PACKBASE_ROOT: /data\n    volumes:\n      - packages:/data\n    ports: [\"8080:8080\"]\n\n  packbase-edge:\n    image: packbase\n    environment:\n      PACKBASE_ROOT: /data   # read-only replica, no token needed\n    volumes:\n      - packages:/data:ro\n    ports: [\"8081:8080\"]\n\nvolumes:\n  packages:\n```\n\nMirror a package on the central node, then serve it from the edge:\n\n```bash\n# mirror once\ncurl -X POST http://central:8080/api/fetch \\\n  -H \"Authorization: Bearer ${PACKBASE_TOKEN}\" \\\n  -d '{\"url\":\"git+https://github.com/owner/mylib\"}'\n\n# install from the edge (in build.zig.zon or via CLI)\nzig fetch --save http://edge:8081/p/mylib/tag/v1.0.0.tar.gz\n```\n\n---\n\n## Design\n\nSee [DESIGN.md](DESIGN.md) for the full architecture document.\n\n## License\n\n[MIT](LICENSE)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffrancescobianco%2Fpackbase","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffrancescobianco%2Fpackbase","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffrancescobianco%2Fpackbase/lists"}