{"id":46576724,"url":"https://github.com/thekevinscott/skillet","last_synced_at":"2026-03-07T10:05:21.368Z","repository":{"id":334115501,"uuid":"1114758271","full_name":"thekevinscott/skillet","owner":"thekevinscott","description":"Evaluation-driven Claude Code skill development","archived":false,"fork":false,"pushed_at":"2026-02-16T03:08:03.000Z","size":1526,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-02-16T08:50:27.901Z","etag":null,"topics":["claude-code","evaluation","llm"],"latest_commit_sha":null,"homepage":"https://skillet.run","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/thekevinscott.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":"ROADMAP.md","authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-12-11T21:00:24.000Z","updated_at":"2026-02-16T02:31:53.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/thekevinscott/skillet","commit_stats":null,"previous_names":["thekevinscott/skillet"],"tags_count":13,"template":false,"template_full_name":null,"purl":"pkg:github/thekevinscott/skillet","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thekevinscott%2Fskillet","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thekevinscott%2Fskillet/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thekevinscott%2Fskillet/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thekevinscott%2Fskillet/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/thekevinscott","download_url":"https://codeload.github.com/thekevinscott/skillet/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thekevinscott%2Fskillet/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":30212021,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-07T09:02:10.694Z","status":"ssl_error","status_checked_at":"2026-03-07T09:02:08.429Z","response_time":53,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["claude-code","evaluation","llm"],"created_at":"2026-03-07T10:05:20.863Z","updated_at":"2026-03-07T10:05:21.360Z","avatar_url":"https://github.com/thekevinscott.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# skillet\n\n[![CI](https://github.com/thekevinscott/skillet/actions/workflows/test.yml/badge.svg)](https://github.com/thekevinscott/skillet/actions/workflows/test.yml)\n[![License](https://img.shields.io/github/license/thekevinscott/skillet)](LICENSE)\n[![Python](https://img.shields.io/badge/python-3.12+-blue.svg)](https://www.python.org/downloads/)\n\nEvaluation-driven Claude Code skill development.\n\n## Install\n\n```bash\npip install pyskillet\n```\n\n## Why\n\nAnthropic [recommends building evaluations before writing skills](https://platform.claude.com/docs/en/agents-and-tools/agent-skills/best-practices#build-evaluations-first):\n\n\u003e Create evaluations BEFORE writing extensive documentation. This ensures your Skill solves real problems rather than documenting imagined ones.\n\nBut they don't provide tooling:\n\n\u003e We do not currently provide a built-in way to run these evaluations.\n\nskillet fills that gap.\n\n## Workflow\n\n### 1. Capture evals with `/skillet:add`\n\nWhen Claude does something wrong, capture it:\n\n```\n\u003e Write a code review comment for this SQL query...\n\nClaude: This code has a SQL injection vulnerability...\n\n\u003e /skillet:add\n\nClaude: What did you expect instead?\n\n\u003e Should start with **issue** (blocking): using conventional comments format\n\nClaude: Eval saved to ~/.skillet/evals/conventional-comments/001.yaml\n```\n\n### 2. Run baseline eval\n\n```bash\nskillet eval conventional-comments\n```\n\n```\nEval Results (baseline, no skill)\n==================================\nEvals: 5\nSamples: 3 per eval\nTotal runs: 15\n\nPass rate: 0% (0/15)\n```\n\n### 3. Create the skill\n\n```bash\nskillet create conventional-comments\n```\n\n```\nFound 5 evals for 'conventional-comments', drafting SKILL.md...\n\nCreated ~/.claude/skills/conventional-comments/\n└── SKILL.md (draft from 5 evals)\n```\n\n### 4. Eval with skill\n\n```bash\nskillet eval conventional-comments ~/.claude/skills/conventional-comments\n```\n\n```\nEval Results (with skill)\n=========================\nSkill: ~/.claude/skills/conventional-comments\nEvals: 5\nSamples: 3 per eval\nTotal runs: 15\n\nPass rate: 80% (12/15)\n```\n\n### 5. Tune the skill\n\n```bash\nskillet tune conventional-comments ~/.claude/skills/conventional-comments\n```\n\n```\nRound 1/5: Pass rate 80% (12/15)\n  Improving skill...\nRound 2/5: Pass rate 93% (14/15)\n  Improving skill...\nRound 3/5: Pass rate 100% (15/15)\n  Target reached!\n\nBest skill saved to ~/.claude/skills/conventional-comments/SKILL.md\n```\n\n## Commands\n\n```bash\nskillet eval \u003cname\u003e [skill]         # run evals (baseline or with skill)\nskillet create \u003cname\u003e               # create skill from evals\nskillet tune \u003cname\u003e \u003cskill\u003e         # iteratively improve skill\nskillet compare \u003cname\u003e \u003cskill\u003e      # compare baseline vs skill from cache\nskillet show \u003cname\u003e                 # inspect cached eval results\nskillet lint \u003cpath\u003e                 # lint a SKILL.md for common issues\nskillet generate-evals \u003cskill\u003e      # generate candidate evals from a skill\n```\n\n## Evals\n\nEvals are stored in `~/.skillet/evals/\u003cname\u003e/`:\n\n```yaml\n# ~/.skillet/evals/conventional-comments/001.yaml\ntimestamp: 2025-01-15T10:30:00Z\nname: conventional-comments\nprompt: \"Write a code review comment for...\"\nexpected: \"Should start with **issue** (blocking):\"\n```\n\n## Python API\n\n```python\nimport asyncio\nfrom pathlib import Path\nfrom skillet import evaluate\n\nasync def main():\n    # Baseline (no skill)\n    baseline = await evaluate(\"conventional-comments\")\n    print(f\"Baseline: {baseline['pass_rate']}%\")\n\n    # With skill\n    result = await evaluate(\n        \"conventional-comments\",\n        skill_path=Path(\"~/.claude/skills/conventional-comments\").expanduser(),\n    )\n    print(f\"With skill: {result['pass_rate']}%\")\n\nasyncio.run(main())\n```\n\nTune a skill programmatically:\n\n```python\nfrom skillet import tune\nfrom skillet.tune import TuneConfig\n\nresult = await tune(\n    \"conventional-comments\",\n    Path(\"~/.claude/skills/conventional-comments\").expanduser(),\n    config=TuneConfig(max_rounds=10, target_pass_rate=90.0),\n)\nprint(f\"Final pass rate: {result.result.final_pass_rate}%\")\n```\n\nSee the [Python API reference](https://skillet.run/reference/python-api) for all functions and options.\n\n## Documentation\n\nFull documentation available at the [docs site](https://skillet.run):\n\n- [Getting Started](https://skillet.run/getting-started)\n- [Concepts](https://skillet.run/concepts/skills-vs-agents) — Skills vs agents, capability vs regression, balanced problem sets\n- [CLI Reference](https://skillet.run/reference/cli)\n- [Eval Format](https://skillet.run/reference/eval-format)\n- [Python API](https://skillet.run/reference/python-api)\n\n## Roadmap\n\nSee [ROADMAP.md](ROADMAP.md) for future ideas and planned features.\n\n## License\n\nMIT\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthekevinscott%2Fskillet","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fthekevinscott%2Fskillet","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthekevinscott%2Fskillet/lists"}