https://github.com/ginaecho/skill-quartermaster

Non-destructive skill manager — compiles the right skill loadout per project, then demotes and hides unused skills to keep your context window lean. Never deletes without your approval.
https://github.com/ginaecho/skill-quartermaster

agent-skills agentic-ai ai-agents context-engineering context-window context-window-optimization developer-tools llm prompt-engineering skill-management skill-marketplace token-optimization

Last synced: 15 days ago
JSON representation

Non-destructive skill manager — compiles the right skill loadout per project, then demotes and hides unused skills to keep your context window lean. Never deletes without your approval.

• Host: GitHub
• URL: https://github.com/ginaecho/skill-quartermaster
• Owner: ginaecho
• License: mit
• Created: 2026-06-20T09:43:25.000Z (about 1 month ago)
• Default Branch: main
• Last Pushed: 2026-06-20T10:55:54.000Z (about 1 month ago)
• Last Synced: 2026-06-20T12:27:19.187Z (about 1 month ago)
• Topics: agent-skills, agentic-ai, ai-agents, context-engineering, context-window, context-window-optimization, developer-tools, llm, prompt-engineering, skill-management, skill-marketplace, token-optimization
• Language: Python
• Homepage:
• Size: 63.5 KB
• Stars: 0
• Watchers: 0
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • Changelog: CHANGELOG.md
  • License: LICENSE

Awesome Lists containing this project

README

          
# skill-quartermaster

Non-destructive skill manager — compiles the right skill loadout per project, then demotes and hides unused skills to keep your context window lean. Never deletes without your approval.

🎖️ Quartermaster




  A non-destructive skill manager for coding agents.


  Compiles the right skill loadout for your project, then quietly demotes and hides the skills you aren't using —


  so your context window stays lean and your skill set stays relevant. Nothing is ever deleted without your yes.





  

  

  

  

  



---

## The problem

The skill ecosystem exploded. Mega-marketplaces ship hundreds of skills in one install; aggregators index tens of thousands of plugin repos. Two pains follow:

- **Context cost & noise.** A large installed set clutters the model's selection space and degrades tool-selection accuracy past a few dozen skills.

- **Curation burden.** You have to decide what to keep, what to drop, and when to write new skills — and today there isn't even a clean "turn this one off." The common workaround is to *rename `SKILL.md` to `_SKILL.md`* so the parser misses it.

People avoid cleanup tools because of one fear: **"what if it deletes the wrong thing?"** Quartermaster is built so that can't happen.

## What it does

Measured on **851 real open-source skills** (Anthropic + community hubs — see

[`benchmark/`](./benchmark/)):

```

851 skills installed  →  30 loaded for this project  →  ~57.5k tokens saved (96%)  →  0 deleted

```

| Claim | Evidence on the 851-skill corpus |

|---|---|

| Cuts the model's selection set | auto-select set **851 → 30** (28× smaller, at the ~30 sweet spot) |

| Saves context tokens | **~59.7k → ~2.2k** indexed tokens (~57.5k saved, 96%) |

| Never deletes | **851 → 851** files on disk, **0** deleted |

| Fully reversible | demote→restore **byte-identical on 200/200** sampled skills |

| Usage-driven curation | flags **809** stale skills to demote, then **737** to hide — fresh skills left active |

| **Keeps what the task needs** | trimmed loadout is **~10× more relevant than random**; **100% recall** of specifically-needed skills at cap=30 (vs 3.5% random) |

| **Doesn't hurt the agent** | live skill-selection A/B (`claude-opus-4-8`, 60 tasks): loadout **97%** vs full set **93%** (**+3 pts**, 100% recall) — trimming doesn't degrade which skill the model picks |

Full reports: [`BENCHMARK.md`](./BENCHMARK.md) (lifecycle + savings),

[`PERFORMANCE.md`](./PERFORMANCE.md) (capability retention — proof the token

savings don't drop the skills a task needs), and the live

[A/B selection eval](./benchmark/ab_eval/) (proof the loadout doesn't degrade

which skill the model picks). All reproducible from [`benchmark/`](./benchmark/).

### Does trimming hurt the agent? (live A/B)

Saving tokens is only worth it if the agent still picks the right skill. The

[`benchmark/ab_eval/`](./benchmark/ab_eval/) harness puts a real model

(`claude-opus-4-8`) in the loop and measures **skill-selection accuracy** on

tasks with known-correct skills, comparing the **full 851-skill set** against the

**~30-skill loadout**. Because both conditions share the same task wording, the

shared confounds cancel in the A−B difference — the hypothesis is **loadout ≥

full**.

```bash

pip install anthropic && export ANTHROPIC_API_KEY=sk-ant-...

# build the corpus once (clones real skill repos, dedupes into /SKILL.md)

git clone --depth 1 https://github.com/davila7/claude-code-templates /tmp/c8

git clone --depth 1 https://github.com/anthropics/skills              /tmp/corpus

python3 benchmark/build_corpus.py /tmp/c8 /tmp/corpus --out /tmp/skillhub

# run the A/B (uses /tmp/c8 for the gold category labels)

python3 benchmark/ab_eval/run_ab_eval.py /tmp/skillhub /tmp/c8 --n 60

# writes benchmark/ab_eval/AB_RESULTS.md

```

Provider-agnostic — pick any model/provider, not just Anthropic:

`--provider openai --model gpt-4o`, any OpenAI-compatible endpoint via

`--base-url` (OpenRouter/Groq/Azure, or a **local** Ollama/vLLM/LM Studio), or

drive a real agent CLI with `--provider command --command "claude -p"` /

`"copilot -p"`. See [`benchmark/ab_eval/`](./benchmark/ab_eval/).

**Live run** — `claude-opus-4-8`, 60 tasks (seed 7), full menu 840 skills, loadout cap 30:

| Condition | menu size | gold-hit accuracy |

|---|--:|--:|

| **A — full installed set** | 840 | **93%** |

| **B — Quartermaster loadout** | ≤30 | **97%** |

| Δ (loadout − full) | | **+3 pts** |

Recall (gold skill present in the loadout): **100%** · selection accuracy given present: **97%**.

**Trimming to the loadout did not hurt selection (97% vs 93%)** — the agent finds the

right skill at least as often with ~30 skills as with 840, at a fraction of the context

cost. Full report: [`benchmark/ab_eval/AB_RESULTS.md`](./benchmark/ab_eval/AB_RESULTS.md).

Quartermaster manages the **lifecycle** of your skills instead of their content. It moves skills along a tiered ladder based on what you actually use — and keeps a human veto on anything irreversible.

| State | In context? | Auto-loadable? | You can invoke? | On disk? |

|---|:---:|:---:|:---:|:---:|

| **active** | ✅ indexed | ✅ | ✅ | ✅ |

| **demoted** | ✅ indexed | ❌ | ✅ manual | ✅ |

| **hidden** | ❌ | ❌ | ❌ | ✅ |

| **archived** | ❌ | ❌ | ❌ | ✅ *(outside active roots)* |

| **deleted** | ❌ | — | — | ❌ *(only after you approve)* |

Every transition is **logged and reversible**. Demote and hide happen automatically; **delete never does**.

## Quick start

```bash

# Add the marketplace

/plugin marketplace add /skill-quartermaster

# Install

/plugin install quartermaster@skill-quartermaster

```

Once installed you get the `quartermaster` skill plus slash commands

(`/qm-status`, `/qm-compile`, `/qm-review`, `/qm-restore`) and the `qm` CLI:

```bash

qm status              # show every skill, its state, last-used, token cost

qm status --layers     # include metadata layer and priority

qm runtimes            # list supported runtime adapters

qm runtime-setup codex # write local setup files for a runtime adapter

qm compile ""  # build an active loadout for this project

qm review              # see proposed demotions/promotions and approve them

qm restore      # bring anything back from demoted/hidden

qm demote       # take a skill out of auto-selection (manual-only)

qm hide         # remove a skill from context entirely

qm archive  --yes # move a skill to reversible archive storage

qm log                 # print the audit trail of every change

qm history      # inspect historical usage/selection metadata

qm conflicts           # report explicit/inferred skill conflicts

qm sources             # list curated external skill repos to consider

qm intake  --dry-run # safely scan a local external skill repo

qm delete  --yes  # human-gated removal (the only destructive action)

# Authoring arm — turn recurring gaps into new skills

qm gap ""        # record a capability gap (a need with no matching skill)

qm gaps                # cluster gaps; recommend new skills to author

qm author        # scaffold a probationary skill (hand off to skill-creator)

qm graduate     # end probation once a new skill has proven useful

# Feedback & undo

qm feedback ""  # route a plain-language complaint to the right lever

qm revert              # undo the last automatic change (one-click revert)

```

> Quartermaster only ever *toggles states* and *proposes* changes. It will not remove a skill from disk unless you explicitly confirm with `--yes`.

### Try it without installing

The CLI is pure-Python (stdlib only). Point it at any folder of skills:

```bash

export QM_SKILLS_DIR=~/.claude/skills      # or your project's .claude/skills

python3 bin/qm status

```

Quartermaster is runtime-aware. Claude Code remains the default, and Phase 0

also exposes adapter names for Codex, GitHub Copilot CLI, VS Code, and generic

command agents:

```bash

qm runtimes

qm runtime-setup codex

qm runtime-setup --all

qm --runtime claude status

qm --runtime generic status

QM_RUNTIME=codex qm status

```

Non-Claude adapters use Quartermaster-owned state metadata (`qm-state`),

workspace-local setup files, and exported loadout manifests as the compatibility

layer. Runtime setup writes local guidance/manifests without needing vendor

credentials:

```bash

qm runtime-setup codex

qm runtime-setup copilot

qm runtime-setup vscode

qm runtime-setup generic

```

Skills can optionally declare Quartermaster metadata in frontmatter:

```yaml

qm-layer: guardrail

qm-priority: 100

qm-tags: security, secrets

qm-risk: network, production

qm-provides: secret-scan

qm-requires-guardrails: security-review

qm-conflicts-with: unsafe-deploy

```

All metadata is optional. Existing skills without these keys still load, and

Quartermaster infers a conservative default layer for status/reporting.

### Trusted external skill intake

Quartermaster can grow your skill shelf from public Git repositories, but it

does not blindly install them. Clone candidate repos yourself, then scan the

local checkout:

```bash

qm sources

git clone --depth 1  /tmp/skills-source

qm intake /tmp/skills-source --dry-run

qm intake /tmp/skills-source --import-to .claude/skills --yes

```

The intake scanner never executes candidate code. It reads `SKILL.md` files,

scores high-value skills, flags suspicious install/shell/exfiltration patterns,

and imports only accepted candidates after explicit `--yes`.

`qm compile` now explains loadouts by layer and writes a runtime loadout

manifest under `$QM_HOME/loadouts/` when applied, so non-Claude runtimes can

consume the same selected skill set through their adapter.

Local state (usage telemetry, audit log, and the historical skill dictionary)

lives under `~/.quartermaster/`

(override with `QM_HOME`). Nothing ever leaves your machine.

## How it works

```mermaid

flowchart LR

    A[Project intent + style] --> B[Intent Compiler]

    B --> C[Active loadout ~30 skills]

    C --> D[Run tasks]

    D --> E[Usage telemetry + feedback]

    E --> F[Policy Engine - proposals only]

    F --> G[Human approval gate]

    G --> H[Update registry / state]

    H --> C

    F -. capability gap .-> I[Authoring arm]

    I --> H

```

1. **Registry** — an index of every skill on disk with its state, description embedding, and last-used timestamp.

2. **Intent compiler** — selects an initial active set from your project intent + style file, kept near the ~30-skill accuracy sweet spot.

3. **Telemetry** — logs which skills actually fire per task (via skill hooks). Local-only; nothing leaves your machine.

4. **Historical dictionary** — records first/last seen, usage count, selection count, state transitions, metadata, and useful intents per skill.

5. **Policy engine** — *proposes* demotions (unused for N days), promotions (you keep invoking a demoted skill), and authoring (repeated gaps with no matching skill).

6. **Human gate** — batched approvals; deletion only after long-demoted **and** explicit confirmation.

7. **Authoring arm** — hands genuine gaps to `skill-creator`, admitting new skills as `active, probationary`.

Under the hood these states map to the selected runtime's primitives where

available. Claude Code uses its native skill frontmatter flags; generic and

future adapters use Quartermaster-owned metadata until native integration is

implemented.

## Why non-destructive matters

You're trusting a tool to touch your skills. Quartermaster's entire design is built around that trust:

- **Demote, don't delete** — unused skills drop out of the model's attention and out of context, but stay fully on disk and recoverable.

- **One-command restore** — `qm restore ` reverses any single change; `qm revert` walks back the last N automatic changes from the audit trail.

- **Human-gated deletion** — the only path to removal runs through an explicit approval; `qm revert` deliberately refuses to undo a deletion or silently delete a skill.

- **Full audit log** — every state change (including each revert) is recorded and inspectable via `qm log`.

## Roadmap

| Phase | Scope | Status |

|---|---|---|

| **v0** | Lifecycle core: registry, state toggles, `qm status` + token-saved report | ✅ shipped |

| **v0.2** | Usage telemetry (PreToolUse hook) + demote-if-unused proposals + batched approvals (`qm review`) | ✅ shipped |

| **v0.3** | Intent compiler (`qm compile` — keyword loadout from project intent) | ◐ basic |

| **v0.4** | Authoring arm: gap detection (`qm gap`/`qm gaps`) → `skill-creator` handoff (`qm author`) → probationary admission + graduation | ✅ shipped |

| **v0.5** | Natural-language feedback (`qm feedback`) → style file / gap / promote / demote signals | ✅ shipped |

| **v1.0** | One-click revert (`qm revert`), full audit trail, marketplace listing; semantic-embedding compiler + dashboard | ◐ partial |

We ship the lifecycle half first on purpose — the compiler and authoring arm only earn their place once the simple half has users.

## Project layout

```

.claude-plugin/

  plugin.json          # plugin manifest (skill + commands + hooks)

  marketplace.json     # one-command install manifest

skills/quartermaster/  # the meta-skill that teaches Claude to drive qm

commands/              # /qm-status, /qm-compile, /qm-review, /qm-restore

hooks/                 # PreToolUse usage telemetry (local-only)

qm/                    # the pure-Python CLI

  registry.py          #   the shelf: scan skills, derive state from frontmatter

  transitions.py       #   non-destructive state changes + audit logging

  policy.py            #   the policy engine — proposes, never executes

  compile.py           #   intent compiler (keyword loadout)

  authoring.py         #   authoring arm: gap clustering + skill scaffolding

  feedback.py          #   route plain-language complaints to the right lever

  history.py           #   one-click revert from the audit trail

  store.py             #   local audit log + usage telemetry

  report.py            #   status table + token-saved report

  cli.py               #   argparse dispatch

bin/qm                 # zero-install entry point

tests/                 # pytest suite

```

Run the tests with `python3 -m pytest`.

## Contributing

Contributions welcome — especially telemetry hooks, policy rules, and adapters for other harnesses (Cursor, Copilot, Zed) via the open Agent Skills standard. Please open an issue describing the change before large PRs.

## License

MIT — see [LICENSE](./LICENSE).

---

Quartermaster manages your skills. It never loses them.