{"id":51508538,"url":"https://github.com/blackwell-systems/betterthantoon","last_synced_at":"2026-07-08T03:30:38.026Z","repository":{"id":363890890,"uuid":"1264297669","full_name":"blackwell-systems/betterthantoon","owner":"blackwell-systems","description":"GCF vs TOON benchmark data","archived":false,"fork":false,"pushed_at":"2026-06-23T21:59:11.000Z","size":58,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-06-23T23:21:23.666Z","etag":null,"topics":["ai-agents","benchmark","comparison","data-serialization","evaluation","gcf","graph-compact-format","json","json-alternative","landing-page","llm","llm-benchmark","mcp","model-context-protocol","structured-data","token-efficiency","token-optimization","toon","web","wire-format"],"latest_commit_sha":null,"homepage":"http://betterthantoon.com/","language":"HTML","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/blackwell-systems.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null},"funding":{"github":["blackwell-systems"]}},"created_at":"2026-06-09T18:51:25.000Z","updated_at":"2026-06-23T21:59:15.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/blackwell-systems/betterthantoon","commit_stats":null,"previous_names":["blackwell-systems/betterthantoon"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/blackwell-systems/betterthantoon","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/blackwell-systems%2Fbetterthantoon","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/blackwell-systems%2Fbetterthantoon/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/blackwell-systems%2Fbetterthantoon/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/blackwell-systems%2Fbetterthantoon/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/blackwell-systems","download_url":"https://codeload.github.com/blackwell-systems/betterthantoon/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/blackwell-systems%2Fbetterthantoon/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":35251015,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-07-08T02:00:06.796Z","response_time":61,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai-agents","benchmark","comparison","data-serialization","evaluation","gcf","graph-compact-format","json","json-alternative","landing-page","llm","llm-benchmark","mcp","model-context-protocol","structured-data","token-efficiency","token-optimization","toon","web","wire-format"],"created_at":"2026-07-08T03:30:37.330Z","updated_at":"2026-07-08T03:30:38.019Z","avatar_url":"https://github.com/blackwell-systems.png","language":"HTML","funding_links":["https://github.com/sponsors/blackwell-systems"],"categories":[],"sub_categories":[],"readme":"# betterthantoon.com\n\n**GCF is better than TOON on every dimension.** Their datasets, their tokenizer, their methodology. Full benchmark data proving it.\n\n## Why TOON fails\n\nTOON claims ~40% token savings over JSON on tabular data. What they don't measure: whether LLMs can actually read it at scale, or produce valid output in it.\n\n- **Comprehension**: TOON averages 68.8% at 500 records. GCF averages 91.2%.\n- **Generation**: TOON's own decoder rejects LLM-generated output on 7 of 9 models. The error is always the same: `toon: cannot assign string to int`. The model writes \"target\" where TOON expects the integer `0`.\n- **Token efficiency**: GCF wins all 6 datasets on TOON's own benchmark.\n\nTOON's flat tabular design forces two failure modes that GCF eliminates structurally:\n\n1. **Distance grouping failure**: without section headers, models must scan 500 rows and filter by column value. They guess round numbers instead. TOON median error: 53.\n2. **Semantic-to-integer encoding**: TOON encodes categories as integers with no structural cue. Models can't perform this mapping unprompted.\n\n## Token efficiency (TOON's own benchmark)\n\nUsing TOON's benchmark code, TOON's tokenizer (o200k_base), and TOON's datasets:\n\n| Dataset | GCF | TOON | Result |\n|:---|:---|:---|:---|\n| Semi-uniform event logs | **108,158** | 154,032 | **GCF 42% smaller** |\n| E-commerce orders | **61,593** | 73,246 | **GCF 19% smaller** |\n| Deeply nested config | **616** | 618 | **GCF 0.3% smaller** |\n| Employee records | **49,055** | 49,966 | **GCF 2% smaller** |\n| Analytics time-series | **8,398** | 9,127 | **GCF 8% smaller** |\n| GitHub repos | **8,576** | 8,744 | **GCF 2% smaller** |\n\nGCF wins all 6. Including the flat tabular datasets TOON was designed for.\n\n## Comprehension (can the model read it?)\n\n500 symbols, 200 edges, 13 extraction questions. No format instructions. No system prompt.\n\n| Model | GCF | TOON | JSON |\n|:---|:---|:---|:---|\n| Claude Opus 4.6 | **96.2%** | 84.6% | 73.1% |\n| Claude Sonnet 4.6 | **100%** | 73.1% | 53.8% |\n| Claude Haiku 4.5 | **96.2%** | 69.2% | 57.7% |\n| GPT-5.5 | **84.1%** | 67.7% | 45.8% |\n| GPT-5.4 | **78.0%** | 56.0% | 44.1% |\n| GPT-5.4-mini | **71.8%** | 64.1% | 54.2% |\n| Gemini 2.5 Pro | **100%** | 76.9% | 58.3% |\n| Gemini 3.1 Pro | **100%** | 76.9% | 46.2% |\n| Gemini 3.5 Flash | **100%** | 61.5% | 46.2% |\n| Gemini 2.5 Flash | **80.6%** | 54.6% | 57.0% |\n\n23 runs, 10 models, 3 providers. GCF wins 22, ties 1, loses 0. Four models hit 100%.\n\n## Generation (can the model write it?)\n\n3-line primer, validated through real decoders. Same data, same prompt structure.\n\n| Model | GCF | TOON | JSON |\n|:---|:---|:---|:---|\n| Claude Opus 4.6 | **5/5** | 0/5 | 5/5 |\n| Claude Sonnet 4.6 | **5/5** | 2-3/5 | 5/5 |\n| Claude Haiku 4.5 | **5/5** | 1-3/5 | 5/5 |\n| GPT-5.5 | **4-5/5** | 1-2/5 | 5/5 |\n| GPT-5.4 | **5/5** | 0/5 | 5/5 |\n| GPT-5.4-mini | **5/5** | 0/5 | 5/5 |\n| Gemini 2.5 Pro | **5/5** | 1/5 | 5/5 |\n| Gemini 3.1 Pro | **5/5** | 0/5 | 5/5 |\n\nNo model has ever been trained on GCF. Every frontier model produces valid output on first exposure. TOON has been published for months and fails on 7 of 9 models.\n\n## The TOON generation failure\n\nEvery TOON generation failure produces the same error:\n\n```\nINVALID: symbols: index 0: distance: toon: cannot assign string to int\n```\n\nThe model writes `target` in the distance column. TOON expects `0`. The model would need to know, unprompted, that \"target\" maps to 0, \"related\" maps to 1, \"extended\" maps to 2. No model does this because the format gives no structural cue.\n\nGCF expresses distance through section placement: targets go in `## targets`, related in `## related`. No integer mapping required.\n\nEven when TOON is hand-held (pre-encoded integers in the prompt), GCF output is still 28% smaller.\n\n## What GCF has that TOON doesn't\n\n- **Session deduplication**: symbols from prior calls become bare references. 92.7% savings by the 5th call. TOON re-serializes everything every time.\n- **Delta encoding**: 81.2% additional savings on re-queries. TOON has no equivalent.\n- **Hierarchical grouping**: section headers eliminate the distance-filtering failures that plague TOON at scale.\n- **Local IDs**: edges cost ~4 tokens each regardless of identifier length. TOON repeats full qualified names.\n- **Streaming**: zero-buffering with trailer summary. TOON has no streaming support.\n\n## We opened a PR on TOON's repo\n\nWe added GCF as a formatter to TOON's official benchmark. 9 lines of code. The PR is public:\n\n[toon-format/toon#319](https://github.com/toon-format/toon/pull/319)\n\nThe fork with full results:\n\n[blackwell-systems/toon](https://github.com/blackwell-systems/toon)\n\n## How to try GCF\n\n```bash\npip install gcf-proxy\n```\n\nWrap any MCP server. Zero code changes. Or use the libraries directly:\n\n```bash\npip install gcf-python        # Python\nnpm install @blackwell-systems/gcf  # TypeScript\ngo get github.com/blackwell-systems/gcf-go  # Go\ncargo add gcf                 # Rust\n```\n\n## Links\n\n- [Live site](https://betterthantoon.com)\n- [Full benchmarks](https://gcformat.com/guide/benchmarks.html)\n- [GCF vs TOON (detailed)](https://gcformat.com/guide/vs-toon.html)\n- [Full eval data](https://gcformat.com/guide/eval-results.html)\n- [GCF Specification](https://github.com/blackwell-systems/gcf)\n- [GCF Proxy](https://github.com/blackwell-systems/gcf-proxy)\n- [Playground (live three-way comparison)](https://gcformat.com/playground.html)\n- [Whitepaper (DOI: 10.5281/zenodo.20579817)](https://doi.org/10.5281/zenodo.20579817)\n- [TOON benchmark fork](https://github.com/blackwell-systems/toon)\n\n## License\n\nMIT - [Dayna Blackwell](https://github.com/blackwell-systems)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fblackwell-systems%2Fbetterthantoon","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fblackwell-systems%2Fbetterthantoon","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fblackwell-systems%2Fbetterthantoon/lists"}