{"id":42952723,"url":"https://github.com/effortlessmetrics/tokmd","last_synced_at":"2026-04-15T02:04:53.313Z","repository":{"id":334631520,"uuid":"1141779061","full_name":"EffortlessMetrics/tokmd","owner":"EffortlessMetrics","description":"Code intelligence for humans, machines, and LLMs: receipts, metrics, and insights from your codebase.","archived":false,"fork":false,"pushed_at":"2026-02-14T14:12:28.000Z","size":2637,"stargazers_count":0,"open_issues_count":80,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-02-14T19:39:15.332Z","etag":null,"topics":["ai-tools","ci","cli","code-metrics","code-statistics","csv","devex","github-actions","jsonl","llm","llmops","loc","markdown","monorepo","repository-analysis","rust","sloc","tokei","tsv"],"latest_commit_sha":null,"homepage":"https://www.tokmd.org/","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/EffortlessMetrics.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":".github/FUNDING.yml","license":"LICENSE-APACHE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":"CITATION.cff","codeowners":".github/CODEOWNERS","security":"SECURITY.md","support":null,"governance":null,"roadmap":"ROADMAP.md","authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":"AGENTS.md","dco":null,"cla":null},"funding":{"github":["EffortlessMetrics"]}},"created_at":"2026-01-25T12:20:09.000Z","updated_at":"2026-02-10T17:02:56.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/EffortlessMetrics/tokmd","commit_stats":null,"previous_names":["effortlessmetrics/tokmd"],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/EffortlessMetrics/tokmd","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/EffortlessMetrics%2Ftokmd","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/EffortlessMetrics%2Ftokmd/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/EffortlessMetrics%2Ftokmd/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/EffortlessMetrics%2Ftokmd/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/EffortlessMetrics","download_url":"https://codeload.github.com/EffortlessMetrics/tokmd/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/EffortlessMetrics%2Ftokmd/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29573398,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-18T07:57:19.261Z","status":"ssl_error","status_checked_at":"2026-02-18T07:57:18.820Z","response_time":162,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai-tools","ci","cli","code-metrics","code-statistics","csv","devex","github-actions","jsonl","llm","llmops","loc","markdown","monorepo","repository-analysis","rust","sloc","tokei","tsv"],"created_at":"2026-01-30T21:33:49.307Z","updated_at":"2026-04-15T02:04:53.306Z","avatar_url":"https://github.com/EffortlessMetrics.png","language":"Rust","funding_links":["https://github.com/sponsors/EffortlessMetrics"],"categories":[],"sub_categories":[],"readme":"# tokmd\n\n\u003e Deterministic repo receipts, analysis, and review artifacts for humans, CI, and LLM workflows.\n\n[![Crates.io](https://img.shields.io/crates/v/tokmd)](https://crates.io/crates/tokmd)\n[![GitHub Release](https://img.shields.io/github/v/release/EffortlessMetrics/tokmd?display_name=tag)](https://github.com/EffortlessMetrics/tokmd/releases)\n[![Docs.rs](https://img.shields.io/docsrs/tokmd)](https://docs.rs/tokmd)\n[![CI](https://github.com/EffortlessMetrics/tokmd/actions/workflows/ci.yml/badge.svg)](https://github.com/EffortlessMetrics/tokmd/actions/workflows/ci.yml)\n[![License](https://img.shields.io/crates/l/tokmd)](https://crates.io/crates/tokmd)\n[![Downloads](https://img.shields.io/crates/d/tokmd)](https://crates.io/crates/tokmd)\n\n`tokmd` turns a source tree into stable receipts you can diff, analyze, archive, gate, and pack for downstream automation. It starts with code inventory, then keeps going: saved artifacts, derived metrics, PR review surfaces, policy checks, sensor outputs, and browser-safe context bundles.\n\n## GitHub Action\n\nUse the root composite action when you want a workflow-friendly receipt and PR summary without scripting `tokmd` installation yourself.\n\n- Installs a released `tokmd` binary for the current runner.\n- Generates `tokmd-summary.md` from `tokmd module`.\n- Generates a structured receipt file from `tokmd export`.\n- Optionally uploads both files as workflow artifacts.\n- Optionally posts the summary as a pull request comment.\n\n```yaml\nname: tokmd receipt\n\non:\n  pull_request:\n\npermissions:\n  contents: read\n  pull-requests: write\n\njobs:\n  receipt:\n    runs-on: ubuntu-latest\n    steps:\n      - uses: actions/checkout@v4\n\n      - uses: EffortlessMetrics/tokmd@v1\n        with:\n          version: 'x.y.z'\n          paths: .\n          module-roots: crates,packages\n          top: '20'\n          format: json\n          artifact: 'true'\n          comment: 'true'\n```\n\nInputs:\n\n| Input | Required | Default | Purpose |\n| :---- | :------- | :------ | :------ |\n| `version` | no | `latest` | `tokmd` release to install. Pass an explicit version if you want the action ref and binary version to stay aligned. |\n| `paths` | no | `.` | Paths to scan. Space/newline-delimited list; each entry is passed as a separate argument. |\n| `module-roots` | no | `crates,packages` | Module root prefixes for `tokmd module` and `tokmd export`. |\n| `top` | no | `20` | Number of rows shown in `tokmd-summary.md`. |\n| `format` | no | `json` | Receipt export format: `json`, `jsonl`, or `csv`. |\n| `artifact` | no | `true` | Upload `tokmd-summary.md` and the receipt as workflow artifacts. |\n| `comment` | no | `true` | Post `tokmd-summary.md` as a pull request comment when running on `pull_request` events. |\n\nOutputs:\n\n| Output | Description |\n| :----- | :---------- |\n| `receipt` | Path to the generated receipt file. |\n\nNotes:\n\n- PR commenting needs `pull-requests: write` and only runs for `pull_request` events.\n- The action currently installs the latest `tokmd` release by default. If you publish the action under `@v1` and want a specific binary version, set `with: version: 'x.y.z'` explicitly.\n- Release asset support is Linux/macOS `amd64` and `arm64`, plus Windows `amd64`.\n- To scan multiple paths, pass whitespace-separated values (for example, `paths: \"src crates\"`), or use a multiline input:\n\n  ```yaml\n  paths: |\n    src\n    packages\n  ```\n\n## The Problem\n\nRaw LOC counts are easy to produce and hard to reuse.\n\n- Terminal output is awkward to diff and archive.\n- CI needs artifacts and gates, not screenshots of a table.\n- LLM workflows need bounded, deterministic context instead of pasted summaries.\n- Review workflows need stable before/after surfaces, not one-off shell output.\n\n`tokmd` exists to turn repository shape into repeatable, machine-friendly truth.\n\n## What `tokmd` Gives You\n\n- Deterministic receipts for `lang`, `module`, `export`, `run`, `diff`, and `analyze`.\n- Review surfaces such as `cockpit`, `gate`, `baseline`, and `sensor`.\n- Saved artifacts in Markdown, JSON, JSONL, CSV, CycloneDX, HTML, SVG, Mermaid, and tree formats.\n- LLM-oriented workflows through `context`, `handoff`, redaction, token budgeting, and tool-schema generation.\n- Multiple product surfaces: CLI, Rust facade, Python bindings, Node bindings, and a browser/WASM slice.\n\n## Start Here\n\nInstall:\n\n```bash\ncargo install tokmd --locked\n# or\nnix run github:EffortlessMetrics/tokmd -- --version\n```\n\nRun the common paths:\n\n```bash\n# Summarize the current repo\ntokmd --format md --top 8\n\n# Save a deterministic run directory for CI or later diffing\ntokmd run --analysis receipt --output-dir .runs/current\n\n# Compare two states\ntokmd diff main HEAD\n\n# Generate a risk-oriented analysis view\ntokmd analyze --preset risk --format md\n\n# Pack code for an LLM budget\ntokmd context --budget 128k --mode bundle --output context.txt\n```\n\n## Choose a Path\n\n| If you need to... | Start with... | Typical output |\n| :---------------- | :------------ | :------------- |\n| summarize a repo or PR | `tokmd`, `diff`, `cockpit` | Markdown summary, diff tables, review plan |\n| save deterministic artifacts | `run`, `export` | JSON/JSONL/CSV/CycloneDX receipts |\n| analyze code health or risk | `analyze` | Markdown, JSON, HTML, SVG, Mermaid |\n| estimate effort between refs | `analyze --preset estimate` | effort report with optional base/head delta |\n| gate policy in CI | `gate`, `baseline`, `sensor` | policy verdicts, ratchets, `sensor.report.v1` |\n| pack context for an LLM | `context`, `handoff` | bounded bundle text, JSON receipts, handoff directory |\n\n## What It Looks Like\n\nThese are live GitHub Actions badges from this repository:\n\n[![CI](https://github.com/EffortlessMetrics/tokmd/actions/workflows/ci.yml/badge.svg)](https://github.com/EffortlessMetrics/tokmd/actions/workflows/ci.yml)\n[![Release Workflow](https://github.com/EffortlessMetrics/tokmd/actions/workflows/release.yml/badge.svg)](https://github.com/EffortlessMetrics/tokmd/actions/workflows/release.yml)\n\nExample summary output from this repository:\n\n```md\n|Lang|Code|Lines|Bytes|Tokens|\n|---|---:|---:|---:|---:|\n|Rust|377263|470341|15334354|3833170|\n|JSON|5405|5405|284012|70997|\n|Markdown|3067|17273|567919|141930|\n|JavaScript|1979|2233|64463|16111|\n|TOML|1947|2387|71514|17855|\n|Other|1978|2758|78806|19691|\n|**Total**|391639|500397|16401068|4099754|\n```\n\n## Generated Badges\n\nThe badges above are GitHub-hosted workflow badges. `tokmd badge` produces repo-local SVG badges from your own code data:\n\n```bash\ntokmd badge --metric lines --output badge-lines.svg\ntokmd badge --metric hotspot --preset risk --output badge-hotspot.svg\n```\n\nEmbed them in your own README:\n\n```markdown\n![Lines of Code](badge-lines.svg)\n![Hotspot](badge-hotspot.svg)\n```\n\n## Command Surface\n\n| Command | Purpose |\n| :------ | :------ |\n| `tokmd` | Language summary for a repo or path set |\n| `tokmd module` | Group stats by module roots such as `crates/` or `packages/` |\n| `tokmd export` | File-level dataset for downstream pipelines |\n| `tokmd run` | Save a full receipt set to a run directory |\n| `tokmd analyze` | Derived metrics and enrichments |\n| `tokmd badge` | Render SVG badges from receipt metrics |\n| `tokmd diff` | Compare two runs, receipts, or refs deterministically |\n| `tokmd context` | Pack code into an LLM context window |\n| `tokmd handoff` | Build an LLM handoff bundle |\n| `tokmd cockpit` | PR-review metrics with risk and evidence gates |\n| `tokmd gate` | Evaluate TOML policy rules and ratchets |\n| `tokmd baseline` | Capture a baseline for later ratchet comparisons |\n| `tokmd sensor` | Emit a `sensor.report.v1` envelope |\n| `tokmd tools` | Generate tool definitions for OpenAI, Anthropic, and JSON Schema consumers |\n| `tokmd init` | Generate a `.tokeignore` template |\n| `tokmd check-ignore` | Explain why a path is being ignored |\n| `tokmd completions` | Generate shell completions |\n\n## Browser And WASM\n\n`tokmd-wasm` and `web/runner` expose a narrower browser-safe slice of the product.\n\n- Supported today: `lang`, `module`, `export`, and browser-safe `analyze` presets on ordered in-memory inputs.\n- Public repo ingestion uses GitHub tree and contents APIs with built-in caching, progress tracking, and auth handling.\n- Git-history enrichers and full native filesystem flows remain native-first.\n\n## What `tokmd` Is Not\n\n- It is not a formatter, linter, or build system.\n- It is not a developer-scoring tool.\n- It is not a vulnerability database or SAST replacement.\n- It does not ask you to trust prose where a receipt can be emitted instead.\n\n## Go Deeper\n\n### Tutorial\n\n- [Tutorial](docs/tutorial.md) for first-run setup and basic workflows\n\n### How-To\n\n- [Recipes](docs/recipes.md) for practical usage patterns\n- [Troubleshooting](docs/troubleshooting.md) for common problems and fixes\n- [Contributing](CONTRIBUTING.md) for local development and release work\n\n### Reference\n\n- [CLI Reference](docs/reference-cli.md) for flags, formats, and config\n- [Schema](docs/SCHEMA.md) for receipt contracts\n- [tokmd responsibilities](tokmd-role.md) for the wider sensor/receipt stack\n\n### Explanation\n\n- [Philosophy](docs/explanation.md) for the design stance\n- [Architecture](docs/architecture.md) for the crate graph and boundaries\n- [Design](docs/design.md) for system concepts and invariants\n- [Roadmap](ROADMAP.md) for the active horizon\n\n## License\n\nMIT or Apache-2.0.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Feffortlessmetrics%2Ftokmd","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Feffortlessmetrics%2Ftokmd","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Feffortlessmetrics%2Ftokmd/lists"}