{"id":50910377,"url":"https://github.com/roerohan/flarbor","last_synced_at":"2026-06-16T09:32:51.526Z","repository":{"id":354633520,"uuid":"1218064502","full_name":"roerohan/flarbor","owner":"roerohan","description":"Cloudflare harbor","archived":false,"fork":false,"pushed_at":"2026-05-11T21:05:26.000Z","size":470,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-05-11T22:31:48.784Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/roerohan.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":"AGENTS.md","dco":null,"cla":null}},"created_at":"2026-04-22T13:52:05.000Z","updated_at":"2026-05-11T21:05:30.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/roerohan/flarbor","commit_stats":null,"previous_names":["roerohan/flarbor"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/roerohan/flarbor","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/roerohan%2Fflarbor","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/roerohan%2Fflarbor/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/roerohan%2Fflarbor/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/roerohan%2Fflarbor/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/roerohan","download_url":"https://codeload.github.com/roerohan/flarbor/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/roerohan%2Fflarbor/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34400451,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-16T02:00:06.860Z","response_time":126,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-06-16T09:32:51.396Z","updated_at":"2026-06-16T09:32:51.511Z","avatar_url":"https://github.com/roerohan.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Flarbor\n\nFlarbor is a toolchain for building, running, and benchmarking RL environments for AI agents and model-driven workflows.\n\nAn RL environment is the loop around a model: it gives the model a task, exposes tools and state, records the model's actions, and scores the result with reward functions. For agentic software tasks, that environment often needs durable state, a filesystem, git, logs, evaluation code, and repeatable trials.\n\nFlarbor is designed for input/output-heavy environments where most of the work is reading, writing, diffing, scoring, and coordinating state rather than running long-lived OS processes. It runs the core workflow on Cloudflare primitives: Durable Objects for state, Workspaces for filesystems, Dynamic Workers for sandboxed code execution, and the Cloudflare Agents SDK for the agent loop.\n\nFlarbor is inspired by [Harbor](https://www.harborframework.com/docs), which showed how useful isolated RL environments can be. Harbor runs each environment in a Docker container, which can be slow to spin up and expensive to run in large parallel batches. Flarbor explores the same environment model on Cloudflare primitives, using containers only when an environment needs `npm build`, `npm test`, compilers, or other OS-level toolchains via the [Sandbox SDK](https://developers.cloudflare.com/sandbox/).\n\n## Why\n\nRL environments will soon be democratized. People will train and fine-tune models for their specific use cases by running task-specific RL loops with reward functions tailored to their domain.\n\nThe bottleneck is environment infrastructure. Harbor proved the concept with Docker containers, but containers are heavy, slow to cold-start, and expensive at scale. The core agentic workflow - clone a repo, read files, make changes, push - doesn't need an OS. It needs a filesystem, git, and an LLM.\n\nFlarbor runs that workflow on Cloudflare's execution ladder: Durable Objects for state, Workspaces for filesystems, Dynamic Workers for sandboxed code execution, and isomorphic-git for version control. No containers, no VMs, no cold starts. A thousand idle environments cost nothing.\n\nWhen you _do_ need a real OS — for `npm test`, compilers, build toolchains — Flarbor spins up a container on demand via the Sandbox SDK. Containers as a last resort, not the default.\n\n## Quick start\n\n```bash\npnpm install\n\n# Verify the local workspace\npnpm typecheck\npnpm test\n\n# Set secrets\nwrangler secret put ANTHROPIC_API_KEY --name code-change-flarbor\nwrangler secret put GITHUB_TOKEN --name code-change-flarbor\n\n# Run locally\npnpm dev\n\n# Or deploy\npnpm deploy\n```\n\nIf your local npm config routes `@cloudflare/*` packages to a private registry, install with a command-local public registry override:\n\n```bash\npnpm install --config.@cloudflare:registry=https://registry.npmjs.org/\n```\n\nTrigger a PR replay task:\n\n```bash\ncurl http://localhost:8787/tasks\n\ncurl -X POST http://localhost:8787/run \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\n    \"taskId\": \"zod-5855\",\n    \"branch\": \"flarbor/zod-5855\"\n  }'\n```\n\nYou can also set defaults via wrangler env vars (`TASK_ID`, `MAX_STEPS`, etc.) and send a minimal or empty POST body. Request fields override env var defaults.\n\n## Repository structure\n\n```\nflarbor/\n├── packages/\n│   ├── flarbor/              Core library: FlarborEnvironment, GitWorkspace, runTask\n│   ├── flarbor-shared/       Shared types, glob matching, dispatch, test helpers\n│   ├── flarbor-container/    Sandbox-backed container offload for build/test commands\n│   ├── flarbor-job/          Batch job orchestration and optional JobObject helper\n│   ├── flarbor-reward/       Reward/scoring kit for evaluating agent outputs\n│   └── flarbor-verify/       Verifier orchestration and Harbor-compatible checks\n└── environments/\n    ├── code-change/\n    │   ├── flarbor/          PR-replay benchmark on Workers + Durable Objects\n    │   └── harbor/           Docker/Harbor comparison implementation\n    ├── repo-audit/\n    │   ├── flarbor/          Read-only repository audit with batch jobs\n    │   └── harbor/           Docker/Harbor comparison implementation\n    └── support-simulation/\n        └── flarbor/          Customer-support simulation and scoring environment\n```\n\n### `packages/flarbor`\n\nThe core library. Exports `FlarborEnvironment`, an abstract base class extending `@cloudflare/think` that handles infrastructure: lifecycle hooks, token tracking, error capture, chat recovery, git workspace, and code execution wiring.\n\nEnvironments subclass it and implement `getEnvironmentConfig()` with their domain-specific model, prompt, tools, and safety rules.\n\n### `packages/flarbor-shared`\n\nZero-dependency shared package for common types and behavior used across packages: token usage, trial results, criterion context, glob matching, budget decay, dispatch helpers, runtime trial-result validation, and test helpers.\n\n### `packages/flarbor-reward`\n\nComposable reward/scoring kit. Define criteria (file checks, diff analysis, token budgets, LLM-as-judge), group them into rewards with aggregation strategies, and score trials.\n\n```typescript\nimport { run, reward, fileExists, trialSuccess, tokenBudget } from \"flarbor-reward\";\n\nconst result = await run(\n  [\n    reward({ name: \"correctness\", criteria: [fileExists(\"output.txt\"), trialSuccess(3.0)] }),\n    reward({ name: \"efficiency\", criteria: [tokenBudget(50_000)] }),\n  ],\n  { workspace, filesChanged, success: true, usage },\n);\n```\n\n### `packages/flarbor-container`\n\nCore library for offloading heavyweight build/test commands from a Durable Object agent loop to Cloudflare Sandbox/Containers. It exports a `ContainerRunner`, workspace sync helpers, command allowlist validation, and an AI SDK-compatible `createContainerCommandTool()`.\n\nThe package is intentionally not a deployable Worker. Environments provide the Sandbox binding, container image, command policy, and prompt wiring.\n\n### `packages/flarbor-job`\n\nBatch orchestration for running task sets across agent targets with bounded concurrency, retries, lifecycle hooks, and aggregate stats. The pure `runJob()` runner is in-request; the optional `JobObject` Durable Object helper adds RPC methods for persisted `start()`, `get()`, and `cancel()` state.\n\n### `packages/flarbor-verify`\n\nVerifier orchestration for turning executable checks into reward-shaped results. It supports deterministic criteria such as command success/output, file checks, JSON paths, HTTP checks, CSV/XLSX/SQLite checks, image checks, and dynamic verifier scripts. It can execute through Cloudflare Sandbox via `flarbor-container`.\n\n### `environments/code-change/`\n\nSide-by-side Flarbor and Harbor implementations for PR replay. The Flarbor environment exposes `GET /tasks` and `POST /run`, clones a repository at a captured base commit, gives the agent the original PR task, verifies the result with `flarbor-verify`, and returns a `TrialResult` with reward scores.\n\n### `environments/repo-audit/`\n\nRead-only repository audit environment. It exposes `POST /run`, in-request batch jobs at `POST /jobs/run`, and persisted Durable Object job state through `POST /jobs`, `GET /jobs/:id`, and `POST /jobs/:id/cancel`.\n\n### `environments/support-simulation/`\n\nCustomer-support conversation simulation. It exposes `GET /scenarios`, `POST /run`, and the same batch/persisted job endpoints as repo audit. It scores outcomes, tool correctness, policy compliance, conversation quality, and efficiency.\n\n## Concept mapping\n\n| Harbor                               | Flarbor                                     | Cloudflare primitive                                                                                                                           |\n| ------------------------------------ | ------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------- |\n| `BaseEnvironment` (Docker container) | `FlarborEnvironment` (Durable Object)       | [Durable Objects](https://developers.cloudflare.com/durable-objects/) + [`@cloudflare/shell`](https://www.npmjs.com/package/@cloudflare/shell) |\n| `BaseAgent`                          | Think subclass                              | [`@cloudflare/think`](https://developers.cloudflare.com/agents/api-reference/think/)                                                           |\n| Task (instruction + test)            | `TaskConfig` via `POST /run`                | Think chat turn                                                                                                                                |\n| Trial (one attempt)                  | `runTask()` -\u003e `TrialResult`                | DO instance lifecycle                                                                                                                          |\n| Container filesystem                 | `GitWorkspace` (Workspace + isomorphic-git) | [`Workspace`](https://www.npmjs.com/package/@cloudflare/shell)                                                                                 |\n| `docker exec`                        | `@cloudflare/shell` state API               | Dynamic Workers                                                                                                                                |\n| `git` CLI                            | `@cloudflare/shell/git`                     | Pure-JS git                                                                                                                                    |\n| `npm test` / `npm build`             | Sandbox SDK                                 | [`@cloudflare/sandbox`](https://developers.cloudflare.com/sandbox/)                                                                            |\n\n## Execution ladder\n\n| Tier                    | What                                     | Container? |\n| ----------------------- | ---------------------------------------- | ---------- |\n| 0: Workspace            | Durable virtual filesystem (SQLite + R2) | No         |\n| 1: Dynamic Worker       | Sandboxed V8 isolate, no network         | No         |\n| 2: Dynamic Worker + npm | Isolate with bundled packages            | No         |\n| 3: Browser              | Headless Chrome via Browser Run          | No         |\n| 4: Sandbox              | Full Linux container                     | Yes        |\n\nTiers 0-2 handle the core workflow. Tier 4 is only for build/test.\n\n## Comparing Flarbor vs Harbor\n\nThe code-change and repo-audit environments both include Flarbor and Harbor implementations, so you can benchmark Workers/Durable Objects against Docker/Harbor directly.\n\n### Running both\n\n```bash\n# Flarbor\npnpm dev\ncurl http://localhost:8787/tasks\ntime curl -s -X POST http://localhost:8787/run \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\"taskId\":\"zod-5855\"}' | jq .\n\n# Harbor\ncd environments/code-change/harbor\nexport TASK_ID=\"zod-5855\"\nexport GITHUB_TOKEN=\"ghp_...\"\nexport ANTHROPIC_API_KEY=\"sk-ant-...\"\ntime ./run.sh\n```\n\n### Cost comparison\n\nBoth use the same model (`claude-opus-4-6` via Anthropic API directly), so LLM costs are identical.\n\n**Flarbor (Cloudflare)**:\n\n- DO compute: $12.50/M requests + $0.10/GB-s wall-clock\n- DO storage: $0.20/M reads, $1.00/M writes, $0.75/GB stored\n- Idle cost: **$0** (DOs hibernate)\n\n**Harbor (Docker)**:\n\n- Local Docker: free (your machine)\n- Cloud sandboxes: per-second billing (Daytona, Modal, E2B)\n- Idle cost: $0 locally, per-second on cloud\n\n### What to measure\n\n| Metric             | How                                                           |\n| ------------------ | ------------------------------------------------------------- |\n| Wall-clock latency | `time curl` / `time ./run.sh`                                 |\n| Token usage        | `usage` field in result JSON                                  |\n| Cold start         | First request after deploy (Flarbor ~50-200ms; Harbor 10-60s) |\n| Files changed      | `filesChanged` in result                                      |\n| Success rate       | Run N trials, compare `success` rate                          |\n\nRun 3-5 trials per side with the same repo and instruction. LLM output is non-deterministic.\n\n## Roadmap\n\n- [x] Core library (`FlarborEnvironment`, `GitWorkspace`, task runner)\n- [x] Shared package (`flarbor-shared`) for common types and utilities\n- [x] Reward/scoring kit (`flarbor-reward`)\n- [x] `flarbor-container` — Sandbox SDK integration for build/test offload\n- [x] `flarbor-job` — batch runner plus optional persisted Job DO helper\n- [x] `flarbor-verify` — verifier orchestration and Harbor-compatible checks\n- [x] Code-change PR replay environment with Harbor comparison\n- [x] Repo-audit environment with Harbor comparison\n- [x] Support-simulation environment\n- [x] Root `typecheck` and test verification scripts\n- [ ] End-to-end validation on deployed Workers\n- [ ] Eval runner environment (run task datasets, collect results)\n- [ ] Queue-backed job orchestration for larger/background workloads\n- [ ] Task/dataset registry\n\n## License\n\nApache-2.0\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Froerohan%2Fflarbor","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Froerohan%2Fflarbor","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Froerohan%2Fflarbor/lists"}