{"id":49488149,"url":"https://github.com/cloudflare/artifact-fs","last_synced_at":"2026-06-06T00:00:55.748Z","repository":{"id":351812383,"uuid":"1195644813","full_name":"cloudflare/artifact-fs","owner":"cloudflare","description":"ArtifactFS is a filesystem driver designed to mount large git repos as quickly as possible, hydrating file contents on-the-fly instead of blocking on the initial clone. It's ideal for agents, sandboxes, containers and other use-cases where startup time is critical.","archived":false,"fork":false,"pushed_at":"2026-05-07T17:22:36.000Z","size":269,"stargazers_count":906,"open_issues_count":4,"forks_count":34,"subscribers_count":3,"default_branch":"main","last_synced_at":"2026-06-05T17:02:34.580Z","etag":null,"topics":["cloudflare","filesystem","fuse","git"],"latest_commit_sha":null,"homepage":"http://workers.cloudflare.com/product/artifacts","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/cloudflare.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":"AGENTS.md","dco":null,"cla":null}},"created_at":"2026-03-29T23:02:49.000Z","updated_at":"2026-06-05T14:19:10.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/cloudflare/artifact-fs","commit_stats":null,"previous_names":["cloudflare/artifact-fs"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/cloudflare/artifact-fs","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cloudflare%2Fartifact-fs","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cloudflare%2Fartifact-fs/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cloudflare%2Fartifact-fs/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cloudflare%2Fartifact-fs/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/cloudflare","download_url":"https://codeload.github.com/cloudflare/artifact-fs/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cloudflare%2Fartifact-fs/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33964367,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-05T02:00:06.157Z","response_time":120,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cloudflare","filesystem","fuse","git"],"created_at":"2026-05-01T03:00:36.153Z","updated_at":"2026-06-06T00:00:55.699Z","avatar_url":"https://github.com/cloudflare.png","language":"Go","funding_links":[],"categories":["Go"],"sub_categories":[],"readme":"\u003cp align=\"center\"\u003e\n  \u003cimg src=\"artifact-fs.png\" alt=\"ArtifactFS\" width=\"720\"\u003e\n\u003c/p\u003e\n\n# ArtifactFS\n\n[![Build \u0026 Test](https://github.com/cloudflare/artifact-fs/actions/workflows/build-test.yml/badge.svg)](https://github.com/cloudflare/artifact-fs/actions/workflows/build-test.yml)\n\n\u003e This is a beta release of ArtifactFS. Your mileage may vary.\n\nArtifactFS is a Git-backed filesystem daemon (FUSE driver) in Go that mounts repositories as normal working trees while avoiding eager blob downloads.\n\nIt exposes the tree quickly, then hydrates file contents on demand. That makes it useful for sandboxes, agents, and other short-lived environments where waiting for a full clone is too expensive.\n\nIn practice:\n\n* The operating system sees the full tree almost immediately, while the FUSE driver fetches file contents in the background. It prioritizes package manifests, dependency manifests, and source files ahead of large blobs.\n* ArtifactFS is part of [Cloudflare Artifacts](http://workers.cloudflare.com/product/artifacts), a versioned filesystem that speaks git, but it also works with any git repo.\n* ArtifactFS is optional. You can clone an Artifact repo directly, but larger repos still take time to clone. ArtifactFS lets you mount the repo and fetch blob contents as they are needed.\n\n## What are Cloudflare Artifacts?\n\n[Cloudflare Artifacts](https://workers.cloudflare.com/product/artifacts) is a versioned filesystem that speaks git. It is built for agent toolchains, sandboxes, and CI/CD systems that need fast access to code repositories.\n\nArtifactFS is the optional FUSE driver -- it lets you mount an Artifact (or any git repo) as a local filesystem without waiting for a full clone.\n\n## Build and Install\n\nRequires Go 1.24+ and a FUSE implementation:\n\n- **macOS** -- [macFUSE](https://osxfuse.github.io/)\n- **Linux** -- `fuse3` (`apt install fuse3` on Debian/Ubuntu, `dnf install fuse3` on Fedora)\n\nInstall the CLI from the module:\n\n```bash\ngo install github.com/cloudflare/artifact-fs/cmd/artifact-fs@latest\n```\n\nOr build it directly from the module path:\n\n```bash\ngo build -o artifact-fs github.com/cloudflare/artifact-fs/cmd/artifact-fs\n```\n\nQuick start against a public repo:\n\n```bash\nexport ARTIFACT_FS_ROOT=/tmp/artifact-fs-test\n\n# Register and clone (returns immediately)\n./artifact-fs add-repo \\\n  --name workers-sdk \\\n  --remote https://github.com/cloudflare/workers-sdk.git \\\n  --branch main \\\n  --mount-root /tmp\n\n# Start the daemon (mounts via FUSE, blocks until killed)\n./artifact-fs daemon --root /tmp \u0026\nDAEMON_PID=$!\n\n# Use the repo\nls /tmp/workers-sdk/\ncat /tmp/workers-sdk/README.md\ngit -C /tmp/workers-sdk log --oneline -5\n\n# Cleanup\nkill $DAEMON_PID\n```\n\n## Monitoring hydration and repo status\n\nCheck the state of a mounted repo with `status`:\n\n```bash\n./artifact-fs status --name workers-sdk\n# repo=workers-sdk state=mounted head=d4c61587... ref=main ahead=0 behind=0 diverged=false last_fetch=2026-03-27T12:00:00Z result=ok overlay_dirty=false\n```\n\n| Field | Meaning |\n|-------|---------|\n| `state` | `mounted` or `unmounted` |\n| `head` | Current HEAD commit OID |\n| `ref` | Tracked branch |\n| `ahead` / `behind` | Commits ahead/behind the remote tracking branch |\n| `overlay_dirty` | `true` if there are local writes (created, modified, or deleted files) |\n| `last_fetch` / `result` | Timestamp and outcome of the last background fetch |\n\nHydration (blob downloading) is transparent: the file tree is visible immediately after mount, and reads block only until the requested blob is fetched. The daemon prioritizes code and manifests (`package.json`, `go.mod`, `README.md`) over binary files.\n\nTo monitor hydration activity, watch the daemon's JSON log output:\n\n```bash\n./artifact-fs daemon --root /tmp 2\u003e/tmp/daemon.log \u0026\n# In another terminal:\ntail -f /tmp/daemon.log | grep -i hydrat\n```\n\nUse `--hydration-concurrency` to control the number of parallel blob-fetch workers (default 4). Each worker maintains a persistent `git cat-file --batch` process, so higher values trade memory for faster bulk hydration:\n\n```bash\n./artifact-fs daemon --root /tmp --hydration-concurrency 8\n```\n\n## Sandboxes and Containers\n\n[`examples/Dockerfile`](examples/Dockerfile) builds artifact-fs and starts a FUSE-mounted repo inside a container. The container requires `--cap-add SYS_ADMIN --device /dev/fuse` for FUSE access.\n\n```bash\n# Build the image\ndocker build -t artifact-fs-example -f examples/Dockerfile .\n\n# Run with the default repo (cloudflare/workers-sdk)\ndocker run --rm --cap-add SYS_ADMIN --device /dev/fuse artifact-fs-example\n\n# Run with a private repo\ndocker run --rm --cap-add SYS_ADMIN --device /dev/fuse \\\n  -e REPO_REMOTE_URL=https://\u003ctoken\u003e@github.com/org/private-repo.git \\\n  artifact-fs-example\n\n# Run a command inside the mounted repo\ndocker run --rm --cap-add SYS_ADMIN --device /dev/fuse \\\n  artifact-fs-example git log --oneline -5\n```\n\nThe entrypoint registers the repo, starts the daemon, waits for the mount, then either runs the provided command or keeps the container alive.\n\nOn hosts with AppArmor enabled (Ubuntu default), add `--security-opt apparmor:unconfined` to the `docker run` flags.\n\n## Architecture\n\nArtifactFS has two distinct phases: a one-shot **setup** (`add-repo`) that performs a fast blobless clone of a repo, and a long-running **daemon** that mounts it via FUSE and serves file operations.\n\n```\n                         ┌─────────────────────────────────────────────────┐\n                         │                    Daemon                       │\n                         │                                                 │\n  ┌──────────┐  clone    │  ┌──────────┐    ls-tree      ┌──────────────┐  │\n  │  Remote  │◄──────────┼──│ GitStore │────────────────►│   Snapshot   │  │\n  │   repo   │  fetch    │  │          │  cat-file       │   (SQLite)   │  │\n  └──────────┘           │  │ batch    │  --batch-check  │              │  │\n                         │  │ pool     │                 │  base_nodes  │  │\n                         │  └────┬─────┘                 │  per gen     │  │\n                         │       │ cat-file              └──────┬───────┘  │\n                         │       │ --batch                      │          │\n                         │       ▼                              ▼          │\n                         │  ┌──────────┐                 ┌──────────────┐  │\n                         │  │  Blob    │                 │   Resolver   │  │\n                         │  │  Cache   │                 │              │  │\n                         │  │  (disk)  │◄────hydrate─────│ snap + ovl   │  │\n                         │  └──────────┘                 │  merged view │  │\n                         │       ▲                       └──────┬───────┘  │\n                         │       │                              │          │\n                         │  ┌────┴─────┐   prefetch       ┌─────┴────────┐ │\n                         │  │ Hydrator │◄─────────────────│    Engine    │ │\n                         │  │          │                  │              │ │\n                         │  │ priority │   ensureOverlay  │ read / write │ │\n                         │  │ queue    │   copy-on-write  │ create / rm  │ │\n                         │  └──────────┘                  └─────┬────────┘ │\n                         │                                      │          │\n                         │  ┌──────────┐                 ┌──────┴───────┐  │\n                         │  │ Overlay  │◄────────────────│  FUSE Layer  │  │\n                         │  │ (SQLite  │  write ops      │  (macFUSE /  │  │\n                         │  │  + upper │                 │   /dev/fuse) │  │\n                         │  │  dir)    │                 └──────┬───────┘  │\n                         │  └──────────┘                        │          │\n                         │                                      │          │\n                         │  ┌──────────┐  HEAD poll       ┌─────┴────────┐ │\n                         │  │ Watcher  │─────────────────►│ Mount point  │ │\n                         │  │ (500ms)  │  re-index +      │ /tmp/myrepo  │ │\n                         │  └──────────┘  reconcile       └──────────────┘ │\n                         └─────────────────────────────────────────────────┘\n```\n\n### Data flow\n\n1. **Clone** -- `add-repo` runs `git clone --filter=blob:none` (blobless). Only commits, trees, and refs are fetched. No file content is downloaded.\n\n2. **Index** -- `git ls-tree -r -t -z HEAD` enumerates every path in the tree. Sizes are resolved locally via `git cat-file --batch-check` with `GIT_NO_LAZY_FETCH=1` to avoid network round-trips. The result is bulk-inserted into a SQLite `base_nodes` table as a new generation.\n\n3. **Mount** -- The FUSE layer exposes the tree immediately. A synthesized `.git` gitfile points at the real gitdir so git commands work inside the mount.\n\n4. **Read** -- The Resolver merges the snapshot (base tree) with the overlay (local writes). For base files, reads block until the Hydrator fetches the blob via a persistent `git cat-file --batch` process and streams it to the blob cache.\n\n5. **Write** -- The Engine promotes base files to the overlay via copy-on-write (hydrate, then copy to the `upper/` directory). Subsequent reads come from the overlay. Deletes are recorded as whiteouts.\n\n6. **Background** -- A watcher polls HEAD/refs every 500ms. On HEAD changes (commit, branch switch, fetch), the daemon re-indexes the tree, publishes a new snapshot generation, reconciles stale overlay entries, and refreshes the git index.\n\n### Subsystems\n\n| Package | Role |\n|---------|------|\n| `daemon` | Orchestrates repo lifecycle, refresh loop, watcher callbacks |\n| `fusefs` | FUSE adapter (inode management, op dispatch), Resolver (merged view), Engine (read/write logic) |\n| `gitstore` | Git CLI wrapper: clone, fetch, ls-tree, batch pool for `cat-file --batch` |\n| `snapshot` | SQLite store for `base_nodes` keyed by `(generation, path)` |\n| `overlay` | SQLite metadata + `upper/` directory for local writes, whiteouts, reconciliation |\n| `hydrator` | Priority queue with deduped waiters; workers block on a `workReady` channel |\n| `watcher` | Polls gitdir mtimes (HEAD, index, refs) at 500ms intervals |\n| `registry` | SQLite-backed repo config persistence |\n| `model` | Shared types and canonical interfaces (`GitStore`, `SnapshotStore`, `OverlayStore`, `Hydrator`) |\n\n## Supported git operations\n\nWork in progress. The table below reflects operations tested against [cloudflare/workers-sdk](https://github.com/cloudflare/workers-sdk) mounted via macFUSE.\n\n### Filesystem operations\n\n| Operation | Status | Notes |\n|-----------|--------|-------|\n| `ls` (root and subdirectories) | Supported | Includes synthesized `.git` gitfile |\n| `cat` / read file | Supported | Triggers on-demand hydration for unhydrated blobs |\n| `stat` (file size, mode) | Supported | Sizes resolved via `git cat-file --batch-check` |\n| `mkdir` | Supported | Persisted in writable overlay |\n| Create new file | Supported | Persisted in writable overlay |\n| Write / append to file | Supported | Copy-on-write for tracked files |\n| Rename file | Supported | Works for both overlay and tracked (snapshot-only) files |\n| Delete file (`rm`) | Supported | Whiteout recorded in overlay |\n| `rmdir` | Supported | Checks directory is empty first |\n| Truncate | Supported | Hydrates blob before truncating |\n| Symlink read (`readlink`) | Supported | Symlink target read from blob content |\n\n### Git operations\n\n| Operation | Status | Notes |\n|-----------|--------|-------|\n| `git log` | Supported | Reads from pack objects |\n| `git branch` | Supported | |\n| `git rev-parse HEAD` | Supported | |\n| `git show` | Supported | |\n| `git remote -v` | Supported | Credentials stripped from output |\n| `git stash list` | Supported | |\n| `git status` | Supported | ~7s on 5800-entry repo |\n| `git diff` | Supported | Shows correct unified diff for modified files |\n| `git add` | Supported | Stages modified files |\n| `git reset` | Supported | ~6.5s index refresh |\n| `git commit` | Supported | Watcher detects HEAD change; overlay reconciles stale entries |\n| `git checkout` | Supported | Re-indexes tree, reconciles overlay, refreshes git index |\n| `git fetch` | Supported | Background refresh loop fetches periodically |\n\n### Known limitations\n\n| Issue | Impact |\n|-------|--------|\n| `git status` takes ~7s on large repos (5800+ entries) | Performance -- full tree walk through FUSE |\n| `git reset` takes ~6.5s for index refresh | Performance -- same root cause as `git status` |\n\n## Testing\n\nUnit tests:\n\n```bash\ngo test ./...\n```\n\nEnd-to-end tests mount a git repo via FUSE and exercise filesystem + git operations (including commit and overlay reconciliation). They require a FUSE implementation (macFUSE on macOS, `fuse3` on Linux) and are off by default.\n\nBy default, e2e tests create a local bare repo -- no network required. Set `AFS_E2E_REPO` to test against a real remote.\n\n```bash\n# Run e2e tests (uses a local test repo by default)\nAFS_RUN_E2E_TESTS=1 go test -v -run TestE2E -count=1 -timeout 10m .\n\n# Run against a specific remote repo\nAFS_RUN_E2E_TESTS=1 \\\n  AFS_E2E_REPO=https://github.com/cloudflare/workers-sdk.git \\\n  go test -v -run TestE2E -count=1 -timeout 10m .\n```\n\n### Environment variables\n\n| Variable | Default | Description |\n|----------|---------|-------------|\n| `AFS_RUN_E2E_TESTS` | `0` | Set to `1` to enable end-to-end tests |\n| `AFS_E2E_REPO` | local bare repo | Git remote URL for e2e tests. When unset, a local bare repo is created automatically. Set to an HTTPS URL to test against a real remote (accepts authenticated URLs). |\n| `ARTIFACT_FS_ROOT` | `~/.local/share/artifact-fs` (macOS) or `/var/lib/artifact-fs` (Linux) | Runtime data root for the daemon and CLI |\n\n## Contributing\n\nContributions are welcome, but not all contributions will be accepted. As guidance:\n\n1. **Ensure you open an issue describing your change** - why it's a problem, how to reproduce it (if it's a bug)\n2. **Your PR should be clear and concise** - including why it should be upstreamed.\n3. **You are expected to have self-reviewed** - any PRs that are straight from automation with glaring issues, that don't build, or don't add good tests are likely to be closed.\n\nAI/LLM submissions are welcome, but overall issue/PR quality is ultimately the responsibility of the submitter, and the codebase is the responsibility (and long term maintenance burden) of the maintainers.\n\nSee [AGENTS.md](AGENTS.md) for build commands, architecture details, and conventions. Run `go test ./...` and `go vet ./...` before submitting changes.\n\n## Credits\n\nThe ArtifactFS FUSE driver takes inspiration from and draws from implementation details in:\n\n* [TigrisFS](https://github.com/tigrisdata/tigrisfs/)\n* [gitfs](https://github.com/presslabs/gitfs)\n* [SlothFS](https://gerrit.googlesource.com/gitfs/)\n\n## License\n\n(c) Cloudflare, 2026. Apache-2.0 licensed.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcloudflare%2Fartifact-fs","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcloudflare%2Fartifact-fs","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcloudflare%2Fartifact-fs/lists"}