{"id":49362185,"url":"https://github.com/openclaw/crabpot","last_synced_at":"2026-05-01T21:00:48.902Z","repository":{"id":353840663,"uuid":"1221111886","full_name":"openclaw/crabpot","owner":"openclaw","description":"Compatibility testbed for OpenClaw community plugins and plugin seams 🦀","archived":false,"fork":false,"pushed_at":"2026-04-30T19:57:15.000Z","size":1854,"stargazers_count":4,"open_issues_count":3,"forks_count":2,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-04-30T20:20:34.311Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/openclaw.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":"AGENTS.md","dco":null,"cla":null},"funding":{"github":["moltbot"]}},"created_at":"2026-04-25T19:12:03.000Z","updated_at":"2026-04-30T09:53:39.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/openclaw/crabpot","commit_stats":null,"previous_names":["openclaw/crabpot"],"tags_count":5,"template":false,"template_full_name":null,"purl":"pkg:github/openclaw/crabpot","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openclaw%2Fcrabpot","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openclaw%2Fcrabpot/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openclaw%2Fcrabpot/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openclaw%2Fcrabpot/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/openclaw","download_url":"https://codeload.github.com/openclaw/crabpot/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openclaw%2Fcrabpot/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32512670,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-30T13:12:12.517Z","status":"online","status_checked_at":"2026-05-01T02:00:05.856Z","response_time":64,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-04-27T17:00:28.468Z","updated_at":"2026-05-01T21:00:48.858Z","avatar_url":"https://github.com/openclaw.png","language":"JavaScript","funding_links":["https://github.com/sponsors/moltbot"],"categories":["Skills \u0026 Plugins"],"sub_categories":["Notable Skills \u0026 Plugins"],"readme":"# crabpot\n\nCompatibility trap for OpenClaw plugin contracts.\n\n`crabpot` keeps a curated set of real community plugins pinned under `plugins/`\nand runs seam-focused compatibility checks against OpenClaw plugin APIs. The goal\nis to catch contract drift before external plugin authors do.\n\n\n\u003c!-- crabpot-summary:start --\u003e\n## Dashboard\n\nLast dashboard update: Apr 28, 2026, 09:32 UTC\n\nState: PASS\n\nMode: check\n\nOpenClaw: openclaw/openclaw@main\n\nRun: https://github.com/openclaw/crabpot/actions/runs/25045088692\n\n### Result Grid\n\n| Metric                 | Result                                               |\n| ---------------------- | ---------------------------------------------------- |\n| Fixtures               | 28                                                   |\n| Hard breakages         | 0                                                    |\n| Warnings               | 57                                                   |\n| Suggestions            | 99                                                   |\n| Issues                 | 156                                                  |\n| P0 issues              | [🔴 P0 5](reports/crabpot-issues.md#p0-live-issues)  |\n| P1 issues              | [🟠 P1 30](reports/crabpot-issues.md#triage-summary) |\n| Live issues            | 5 total / 5 P0                                       |\n| Compat gaps            | 4                                                    |\n| Deprecation warnings   | 22                                                   |\n| Inspector gaps         | 101                                                  |\n| Upstream metadata      | 24                                                   |\n| Contract probes        | 151                                                  |\n| Policy failures        | 0                                                    |\n| Policy warnings        | 3                                                    |\n| Ref diff failures      | 0                                                    |\n| Profile failures       | 0                                                    |\n| Execution probes       | 6 pass / 0 fail / 2 blocked                          |\n| Synthetic probes       | 155 ready / 0 blocked / 155 total                    |\n| Cold import            | 0 ready / 31 blocked / 31 entrypoints                |\n| Workspace plan         | 31 entrypoints / 20 installs / 8 builds              |\n| Platform risks         | 158 Windows / 49 container                           |\n| Jiti loader candidates | 18                                                   |\n| Import loop            | p50 153ms / p95 165ms / max RSS 46.3MB / CPU 27ms    |\n| Runtime profile        | p50 649ms / p95 1329ms / max RSS 68.7MB              |\n\n### Top Discovered Issues\n\n| Severity | Class         | Fixture          | Code                     | Decision            | Title                                                                                                                           |\n| -------- | ------------- | ---------------- | ------------------------ | ------------------- | ------------------------------------------------------------------------------------------------------------------------------- |\n| 🔴 P0    | live-issue    | clawmetry        | sdk-export-missing       | core-compat-adapter | [clawmetry: plugin SDK import aliases are missing from target package exports](reports/crabpot-issues.md#p0-live-issues)        |\n| 🔴 P0    | live-issue    | codex-app-server | sdk-export-missing       | core-compat-adapter | [codex-app-server: plugin SDK import aliases are missing from target package exports](reports/crabpot-issues.md#p0-live-issues) |\n| 🔴 P0    | live-issue    | honcho           | sdk-export-missing       | core-compat-adapter | [honcho: plugin SDK import aliases are missing from target package exports](reports/crabpot-issues.md#p0-live-issues)           |\n| 🔴 P0    | live-issue    | hyperspell       | unknown-hook-name        | core-compat-adapter | [hyperspell: fixture uses a hook missing from target OpenClaw](reports/crabpot-issues.md#p0-live-issues)                        |\n| 🔴 P0    | live-issue    | yuanbao          | sdk-export-missing       | core-compat-adapter | [yuanbao: plugin SDK import aliases are missing from target package exports](reports/crabpot-issues.md#p0-live-issues)          |\n| 🟠 P1    | inspector-gap | a2a-gateway      | registration-capture-gap | inspector-follow-up | [a2a-gateway: runtime registrations need capture before contract judgment](reports/crabpot-issues.md#inspector-proof-gaps)      |\n| 🟠 P1    | compat-gap    | clawmetry        | missing-compat-record    | core-compat-adapter | [clawmetry: compat-dependent behavior lacks registry coverage](reports/crabpot-issues.md#compat-gaps)                           |\n| 🟠 P1    | inspector-gap | clawmetry        | registration-capture-gap | inspector-follow-up | [clawmetry: runtime registrations need capture before contract judgment](reports/crabpot-issues.md#inspector-proof-gaps)        |\n| 🟠 P1    | compat-gap    | codex-app-server | missing-compat-record    | core-compat-adapter | [codex-app-server: compat-dependent behavior lacks registry coverage](reports/crabpot-issues.md#compat-gaps)                    |\n| 🟠 P1    | inspector-gap | codex-app-server | registration-capture-gap | inspector-follow-up | [codex-app-server: runtime registrations need capture before contract judgment](reports/crabpot-issues.md#inspector-proof-gaps) |\n\u003c!-- crabpot-summary:end --\u003e\n## What this tests\n\n- plugin manifests and install metadata\n- native tool registration and dynamic tool schemas\n- channel registration and message delivery seams\n- lifecycle hooks such as `gateway_start`, `gateway_stop`, and `before_install`\n- agent hooks such as `before_tool_call`, `before_prompt_build`, `llm_input`,\n  `llm_output`, and `agent_end`\n- provider capability registration such as speech/TTS\n- plugin-owned services, routes, subprocesses, and async job patterns\n\n## Layout\n\n```text\ncrabpot/\n  crabpot.config.json        fixture manifest and seam tags\n  plugins/                   external plugin repositories as git submodules\n  reports/                   generated compatibility report artifacts\n  scripts/                   manifest and fixture helpers\n  test/                      repo-level checks\n  docs/                      operating notes and seam matrix\n```\n\n## Quick start\n\n```bash\nnpm test\nnode scripts/list-fixtures.mjs\nnode scripts/sync-fixtures.mjs --check\nnpm run report\nnpm run contract:capture\nnpm run contract:synthetic\nnpm run cold-import\nnpm run workspace:plan\nnpm run platform:probes\nnpm run import:profile\nnpm run execution:report\nnpm run profile\nnpm run contract:coverage\nnpm run readme:summary\n```\n\nTo materialize the fixture repos as submodules:\n\n```bash\nnode scripts/sync-fixtures.mjs --materialize\ngit submodule update --init --recursive\n```\n\nThat command mutates `.gitmodules` and `plugins/*`. Commit those changes when\nyou intentionally pin or update fixture revisions.\n\n## Compatibility report\n\nStart with the dashboard at the top of this README. It is the condensed view of\nthe generated reports: fixture count, breakages, warnings, issue backlog, probe\ncoverage, cold-import blockers, workspace execution shape, and runtime profile.\n\nFor deeper review, open the reports in this order:\n\n| Need | Command | Primary report |\n| --- | --- | --- |\n| Main compatibility triage, decision matrix, issue backlog | `npm run report` | `reports/crabpot-report.md` |\n| Stable issue list for compat-layer planning | `npm run report` | `reports/crabpot-issues.md` |\n| Hooks, registrars, SDK imports, and entrypoints that need capture | `npm run contract:capture` | `reports/crabpot-capture.md` |\n| Executable synthetic hook/registration probe plan | `npm run contract:synthetic` | `reports/crabpot-synthetic-probes.md` |\n| Why plugin entrypoints cannot be safely cold-imported yet | `npm run cold-import` | `reports/crabpot-cold-import.md` |\n| Isolated install/build/capture commands Crabpot would run | `npm run workspace:plan` | `reports/crabpot-workspace-plan.md` |\n| Results from opt-in isolated fixture execution | `npm run execution:report` | `reports/crabpot-execution-results.md` |\n| Boot time and RSS against the target OpenClaw registry surface | `npm run profile` | `reports/crabpot-runtime-profile.md` |\n| README dashboard refresh from all generated JSON reports | `npm run readme:summary` | `README.md` |\n\nEach Markdown report has a matching JSON file beside it for CI, dashboards, and\nfuture inspector tooling. The JSON is the contract; the Markdown is the review\nsurface.\n\nUse the main compatibility report like this:\n\n| Section | What to do with it |\n| --- | --- |\n| Hard Breakages | Treat as release-blocking contract drift. |\n| Warnings | Review for target OpenClaw compatibility gaps or plugin metadata drift. |\n| Suggestions To OpenClaw Compat Layer | Convert into compat-layer work, inspector follow-ups, or upstream plugin requests. |\n| Issue Findings | Use stable `CRABPOT-*` ids for tracking and comparison across runs. |\n| Contract Probe Backlog | Turn into tests before changing a plugin-facing seam. |\n| Decision Matrix | Decide whether the fix belongs in core compat, the future inspector, or the plugin upstream. |\n\nBy default, reports target the OpenClaw checkout configured in\n`crabpot.config.json`. Point a run at a branch, tag, SHA checkout, or local fork\nwith `--openclaw`:\n\n```bash\nnode scripts/generate-report.mjs --openclaw ../openclaw\nnode scripts/generate-report.mjs --check --openclaw ../openclaw\n```\n\nCrabpot does not execute third-party plugin code during default checks. The\nworkspace plan is dry planning unless you explicitly opt into isolated execution.\nPreview a fixture lane first:\n\n```bash\nnpm run workspace:execute -- --fixture wecom --dry-run\n```\n\nThen run isolated execution only when you want install/build/import side effects\ninside Crabpot's generated workspace:\n\n```bash\nCRABPOT_EXECUTE_ISOLATED=1 npm run workspace:execute -- --fixture wecom\nnpm run execution:report\n```\n\n## Manual OpenClaw ref CI\n\nThe `OpenClaw Ref Compatibility` workflow can be run from GitHub Actions with\nan OpenClaw branch, tag, or SHA. Set `openclaw_repository` when testing a fork,\nand `openclaw_ref` to the exact ref under review.\n\nThe default job runs the static contract suite against that checkout and uploads\nthe generated reports. The optional isolated job runs one fixture lane when\n`run_isolated_fixture` is enabled and `fixture` is set, then uploads\n`.crabpot/results/` plus the execution summary report.\n\n## Fixture policy\n\nFixtures should earn their spot by covering a distinct seam. Popularity is a\nuseful signal, but a small plugin that exercises a rare hook is more valuable\nthan the fourth web-search wrapper.\n\nThe first fixture set intentionally covers channels, dynamic tools, LLM\nobservation, diagnostics, gateway-owned services, async jobs, provider\ncapabilities, and security/policy hooks.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopenclaw%2Fcrabpot","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fopenclaw%2Fcrabpot","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopenclaw%2Fcrabpot/lists"}