{"id":46244342,"url":"https://github.com/toon-format/toon-rust","last_synced_at":"2026-03-03T21:01:59.710Z","repository":{"id":321965169,"uuid":"1087780616","full_name":"toon-format/toon-rust","owner":"toon-format","description":"🦀 Community-driven Rust implementation of TOON","archived":false,"fork":false,"pushed_at":"2026-02-16T07:50:02.000Z","size":158,"stargazers_count":106,"open_issues_count":0,"forks_count":16,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-02-16T14:38:11.428Z","etag":null,"topics":["rust","serialization","toon"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/toon-format.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-11-01T16:22:56.000Z","updated_at":"2026-02-16T12:58:59.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/toon-format/toon-rust","commit_stats":null,"previous_names":["toon-format/toon-rust"],"tags_count":17,"template":false,"template_full_name":null,"purl":"pkg:github/toon-format/toon-rust","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/toon-format%2Ftoon-rust","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/toon-format%2Ftoon-rust/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/toon-format%2Ftoon-rust/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/toon-format%2Ftoon-rust/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/toon-format","download_url":"https://codeload.github.com/toon-format/toon-rust/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/toon-format%2Ftoon-rust/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":30060631,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-03T18:21:05.932Z","status":"ssl_error","status_checked_at":"2026-03-03T18:20:59.341Z","response_time":61,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["rust","serialization","toon"],"created_at":"2026-03-03T21:01:58.052Z","updated_at":"2026-03-03T21:01:59.705Z","avatar_url":"https://github.com/toon-format.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# TOON Format for Rust\n\n[![Crates.io](https://img.shields.io/crates/v/toon-format.svg)](https://crates.io/crates/toon-format)\n[![Documentation](https://docs.rs/toon-format/badge.svg)](https://docs.rs/toon-format)\n[![Spec v3.0](https://img.shields.io/badge/spec-v3.0-brightgreen.svg)](https://github.com/toon-format/spec/blob/main/SPEC.md)\n[![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](./LICENSE)\n[![Tests](https://img.shields.io/badge/tests-%20passing-success.svg)]()\n\n**Token-Oriented Object Notation (TOON)** is a compact, human-readable format designed for passing structured data to Large Language Models with significantly reduced token usage.\n\nThis crate provides the official, **spec-compliant Rust implementation** of TOON v3.0, offering both a library (`toon-format`) and a full-featured command-line tool (`toon`).\n\n## Quick Example\n\n**JSON** (16 tokens, 40 bytes):\n```json\n{\n  \"users\": [\n    { \"id\": 1, \"name\": \"Alice\" },\n    { \"id\": 2, \"name\": \"Bob\" }\n  ]\n}\n```\n\n**TOON** (13 tokens, 28 bytes) - **18.75% token savings**:\n```toon\nusers[2]{id,name}:\n  1,Alice\n  2,Bob\n```\n\n## Features\n\n- **Generic API**: Works with any `Serialize`/`Deserialize` type - custom structs, enums, JSON values, and more\n- **Spec-Compliant**: Fully compliant with [TOON Specification v3.0](https://github.com/toon-format/spec/blob/main/SPEC.md)\n- **Key Folding \u0026 Path Expansion**: Collapse and expand dotted key paths\n- **Safe \u0026 Performant**: Built with safe, fast Rust\n- **Powerful CLI**: Full-featured command-line tool\n- **Strict Validation**: Enforces all spec rules (configurable)\n- **Well-Tested**: Comprehensive test suite with unit tests, spec fixtures, and real-world scenarios\n\n## Installation\n\n### As a Library\n\n```bash\ncargo add toon-format\n```\n\n### As a CLI Tool\n\n```bash\ncargo install toon-format\n```\n\n---\n\n## Library Usage\n\n### Basic Encode \u0026 Decode\n\nThe `encode` and `decode` functions work with any type implementing `Serialize`/`Deserialize`:\n\n**With custom structs:**\n\n```rust\nuse serde::{Serialize, Deserialize};\nuse toon_format::{encode_default, decode_default};\n\n#[derive(Serialize, Deserialize, Debug, PartialEq)]\nstruct User {\n    name: String,\n    age: u32,\n    email: String,\n}\n\nfn main() -\u003e Result\u003c(), toon_format::ToonError\u003e {\n    let user = User {\n        name: \"Alice\".to_string(),\n        age: 30,\n        email: \"alice@example.com\".to_string(),\n    };\n\n    // Encode to TOON\n    let toon = encode_default(\u0026user)?;\n    println!(\"{}\", toon);\n    // Output:\n    // name: Alice\n    // age: 30\n    // email: alice@example.com\n\n    // Decode back to struct\n    let decoded: User = decode_default(\u0026toon)?;\n    assert_eq!(user, decoded);\n\n    Ok(())\n}\n```\n\n**With JSON values:**\n\n```rust\nuse serde_json::{json, Value};\nuse toon_format::{encode_default, decode_default};\n\nfn main() -\u003e Result\u003c(), toon_format::ToonError\u003e {\n    let data = json!({\n        \"users\": [\n            {\"id\": 1, \"name\": \"Alice\"},\n            {\"id\": 2, \"name\": \"Bob\"}\n        ]\n    });\n\n    // Encode to TOON\n    let toon_str = encode_default(\u0026data)?;\n    println!(\"{}\", toon_str);\n    // Output:\n    // users[2]{id,name}:\n    //   1,Alice\n    //   2,Bob\n\n    // Decode back to JSON\n    let decoded: Value = decode_default(\u0026toon_str)?;\n    assert_eq!(decoded, data);\n    \n    Ok(())\n}\n```\n---\n\n## API Reference\n\n### Encoding\n\n#### `encode\u003cT: Serialize\u003e(\u0026value, \u0026options) -\u003e Result\u003cString, ToonError\u003e`\n\nEncode any serializable type to TOON format. Works with custom structs, enums, collections, and `serde_json::Value`.\n\n```rust\nuse toon_format::{encode, EncodeOptions, Delimiter, Indent};\nuse serde_json::json;\n\nlet data = json!({\"items\": [\"a\", \"b\", \"c\"]});\n\n// Default encoding\nlet toon = encode(\u0026data, \u0026EncodeOptions::default())?;\n// items[3]: a,b,c\n\n// Custom delimiter\nlet opts = EncodeOptions::new()\n    .with_delimiter(Delimiter::Pipe);\nlet toon = encode(\u0026data, \u0026opts)?;\n// items[3|]: a|b|c\n\n// Custom indentation\nlet opts = EncodeOptions::new()\n    .with_indent(Indent::Spaces(4));\nlet toon = encode(\u0026data, \u0026opts)?;\n```\n\n#### `EncodeOptions`\n\n| Method | Description | Default |\n|--------|-------------|---------|\n| `with_delimiter(d)` | Set delimiter: `Comma`, `Tab`, or `Pipe` | `Comma` |\n| `with_indent(i)` | Set indentation (spaces only) | `Spaces(2)` |\n| `with_spaces(n)` | Shorthand for `Indent::Spaces(n)` | `2` |\n| `with_key_folding(mode)` | Enable key folding (v1.5) | `Off` |\n| `with_flatten_depth(n)` | Set max folding depth | `usize::MAX` |\n\n### Decoding\n\n#### `decode\u003cT: Deserialize\u003e(\u0026input, \u0026options) -\u003e Result\u003cT, ToonError\u003e`\n\nDecode TOON format into any deserializable type. Works with custom structs, enums, collections, and `serde_json::Value`.\n\n**With custom structs:**\n```rust\nuse serde::Deserialize;\nuse toon_format::{decode, DecodeOptions};\n\n#[derive(Deserialize)]\nstruct Config {\n    host: String,\n    port: u16,\n}\n\nlet toon = \"host: localhost\\nport: 8080\";\nlet config: Config = decode(toon, \u0026DecodeOptions::default())?;\n```\n\n**With JSON values:**\n```rust\nuse serde_json::Value;\nuse toon_format::{decode, DecodeOptions};\n\nlet toon = \"name: Alice\\nage: 30\";\n\n// Default (strict) decode\nlet json: Value = decode(toon, \u0026DecodeOptions::default())?;\n\n// Non-strict mode (relaxed validation)\nlet opts = DecodeOptions::new().with_strict(false);\nlet json: Value = decode(toon, \u0026opts)?;\n\n// Disable type coercion\nlet opts = DecodeOptions::new().with_coerce_types(false);\nlet json: Value = decode(\"active: true\", \u0026opts)?;\n// With coercion: {\"active\": true}\n// Without: {\"active\": \"true\"}\n```\n\n**Helper functions:**\n- `encode_default\u003cT\u003e(\u0026value)` - Encode with default options\n- `decode_default\u003cT\u003e(\u0026input)` - Decode with default options\n\n#### `DecodeOptions`\n\n| Method | Description | Default |\n|--------|-------------|---------|\n| `with_strict(b)` | Enable strict validation | `true` |\n| `with_coerce_types(b)` | Auto-convert strings to types | `true` |\n| `with_expand_paths(mode)` | Enable path expansion (v1.5) | `Off` |\n\n---\n\n## v1.5 Features\n\n### Key Folding (Encoder)\n\n**New in v1.5**: Collapse single-key object chains into dotted paths to reduce tokens.\n\n**Standard nesting:**\n```toon\ndata:\n  metadata:\n    items[2]: a,b\n```\n\n**With key folding:**\n```toon\ndata.metadata.items[2]: a,b\n```\n\n**Example:**\n\n```rust\nuse serde_json::json;\nuse toon_format::{encode, EncodeOptions, KeyFoldingMode};\n\nlet data = json!({\n    \"data\": {\n        \"metadata\": {\n            \"items\": [\"a\", \"b\"]\n        }\n    }\n});\n\n// Enable key folding\nlet opts = EncodeOptions::new()\n    .with_key_folding(KeyFoldingMode::Safe);\n\nlet toon = encode(\u0026data, \u0026opts)?;\n// Output: data.metadata.items[2]: a,b\n```\n\n#### With Depth Control\n\n```rust\nlet data = json!({\"a\": {\"b\": {\"c\": {\"d\": 1}}}});\n\n// Fold only 2 levels\nlet opts = EncodeOptions::new()\n    .with_key_folding(KeyFoldingMode::Safe)\n    .with_flatten_depth(2);\n\nlet toon = encode(\u0026data, \u0026opts)?;\n// Output:\n// a.b:\n//   c:\n//     d: 1\n```\n\n#### Safety Features\n\nKey folding only applies when:\n- All segments are valid identifiers (`a-z`, `A-Z`, `0-9`, `_`)\n- Each level contains exactly one key\n- No collision with sibling literal keys\n- Within the specified `flatten_depth`\n\nKeys like `full-name`, `user.email` (if quoted), or numeric keys won't be folded.\n\n### Path Expansion (Decoder)\n\n**New in v1.5**: Automatically expand dotted keys into nested objects.\n\n**Compact input:**\n```toon\na.b.c: 1\na.b.d: 2\na.e: 3\n```\n\n**Expanded output:**\n```json\n{\n  \"a\": {\n    \"b\": {\n      \"c\": 1,\n      \"d\": 2\n    },\n    \"e\": 3\n  }\n}\n```\n\n**Example:**\n\n```rust\nuse serde_json::Value;\nuse toon_format::{decode, DecodeOptions, PathExpansionMode};\n\nlet toon = \"a.b.c: 1\\na.b.d: 2\";\n\n// Enable path expansion\nlet opts = DecodeOptions::new()\n    .with_expand_paths(PathExpansionMode::Safe);\n\nlet json: Value = decode(toon, \u0026opts)?;\n// {\"a\": {\"b\": {\"c\": 1, \"d\": 2}}}\n```\n\n**Round-Trip Example:**\n\n```rust\nuse serde_json::{json, Value};\nuse toon_format::{encode, decode, EncodeOptions, DecodeOptions, KeyFoldingMode, PathExpansionMode};\n\nlet original = json!({\n    \"user\": {\n        \"profile\": {\n            \"name\": \"Alice\"\n        }\n    }\n});\n\n// Encode with folding\nlet encode_opts = EncodeOptions::new()\n    .with_key_folding(KeyFoldingMode::Safe);\nlet toon = encode(\u0026original, \u0026encode_opts)?;\n// Output: \"user.profile.name: Alice\"\n\n// Decode with expansion\nlet decode_opts = DecodeOptions::new()\n    .with_expand_paths(PathExpansionMode::Safe);\nlet restored: Value = decode(\u0026toon, \u0026decode_opts)?;\n\nassert_eq!(restored, original); // Perfect round-trip!\n```\n\n**Quoted Keys Remain Literal:**\n\n```rust\nuse serde_json::Value;\nuse toon_format::{decode, DecodeOptions, PathExpansionMode};\n\nlet toon = r#\"a.b: 1\n\"c.d\": 2\"#;\n\nlet opts = DecodeOptions::new()\n    .with_expand_paths(PathExpansionMode::Safe);\nlet json: Value = decode(toon, \u0026opts)?;\n// {\n//   \"a\": {\"b\": 1},\n//   \"c.d\": 2        \u003c- quoted key preserved\n// }\n```\n\n---\n\n## Interactive TUI\n\nTOON includes a full-featured Terminal User Interface for interactive conversions!\n\n```bash\n# Launch interactive mode\ntoon --interactive\n# or\ntoon -i\n```\n\n### Features:\n- Real-time conversion as you type\n- Live statistics (tokens, bytes, savings)\n- Interactive settings - adjust all options on-the-fly\n- File browser with visual navigation\n- Side-by-side diff viewer\n- Conversion history tracking\n- File operations (open, save, new)\n- Clipboard integration (copy/paste)\n- REPL mode for command-line interaction\n- Round-trip testing\n- Theme support (Dark/Light)\n- Built-in help with keyboard shortcuts\n\n**Perfect for:**\n- Learning TOON format interactively\n- Testing conversions in real-time\n- Experimenting with different settings\n- Visual before/after comparisons\n- Quick data transformations\n\nSee [docs/TUI.md](docs/TUI.md) for complete documentation and keyboard shortcuts!\n\n---\n\n## CLI Usage\n\n### Basic Commands\n\n```bash\n# Auto-detect from extension\ntoon data.json        # Encode\ntoon data.toon        # Decode\n\n# Force mode\ntoon -e data.txt      # Force encode\ntoon -d output.txt    # Force decode\n\n# Pipe from stdin\ncat data.json | toon\necho '{\"name\": \"Alice\"}' | toon -e\n```\n\n### Encode Options\n\n```bash\n# Custom delimiter\ntoon data.json --delimiter pipe\ntoon data.json --delimiter tab\n\n# Custom indentation\ntoon data.json --indent 4\n\n# Key folding (v1.5)\ntoon data.json --fold-keys\ntoon data.json --fold-keys --flatten-depth 2\n\n# Show statistics\ntoon data.json --stats\n```\n\n### Decode Options\n\n```bash\n# Pretty-print JSON\ntoon data.toon --json-indent 2\n\n# Relaxed validation\ntoon data.toon --no-strict\n\n# Disable type coercion\ntoon data.toon --no-coerce\n\n# Path expansion (v1.5)\ntoon data.toon --expand-paths\n```\n\n### Full Example\n\n```bash\n$ echo '{\"data\":{\"meta\":{\"items\":[\"x\",\"y\"]}}}' | toon --fold-keys --stats\n\ndata.meta.items[2]: x,y\n\nStats:\n+--------------+------+------+---------+\n| Metric       | JSON | TOON | Savings |\n+======================================+\n| Tokens       | 13   | 8    | 38.46%  |\n|--------------+------+------+---------|\n| Size (bytes) | 38   | 23   | 39.47%  |\n+--------------+------+------+---------+\n```\n\n---\n\n## Testing\n\nThe library includes a comprehensive test suite covering core functionality, edge cases, spec compliance, and real-world scenarios.\n\n```bash\n# Run all tests\ncargo test\n\n# Run specific test suites\ncargo test --test spec_fixtures\ncargo test --lib\n\n# With output\ncargo test -- --nocapture\n```\n\n## Error Handling\n\nAll operations return `Result\u003cT, ToonError\u003e` with descriptive error messages:\n\n```rust\nuse serde_json::Value;\nuse toon_format::{decode_strict, ToonError};\n\nmatch decode_strict::\u003cValue\u003e(\"items[3]: a,b\") {\n    Ok(value) =\u003e println!(\"Success: {:?}\", value),\n    Err(ToonError::LengthMismatch { expected, found, .. }) =\u003e {\n        eprintln!(\"Array length mismatch: expected {}, found {}\", expected, found);\n    }\n    Err(e) =\u003e eprintln!(\"Error: {}\", e),\n}\n```\n\n### Error Types\n\n- `ParseError` - Syntax errors with line/column info\n- `LengthMismatch` - Array length doesn't match header\n- `TypeMismatch` - Unexpected value type\n- `InvalidStructure` - Malformed TOON structure\n- `SerializationError` / `DeserializationError` - Conversion failures\n\n---\n\n\n## Examples\nRun with `cargo run --example examples` to see all examples:\n- `structs.rs` - Custom struct serialization\n- `tabular.rs` - Tabular array formatting\n- `arrays.rs` - Various array formats\n- `arrays_of_arrays.rs` - Nested arrays\n- `objects.rs` - Object encoding\n- `mixed_arrays.rs` - Mixed-type arrays\n- `delimiters.rs` - Custom delimiters\n- `round_trip.rs` - Encode/decode round-trips\n- `decode_strict.rs` - Strict validation\n- `empty_and_root.rs` - Edge cases\n\n---\n\n## Resources\n\n- 📖 [TOON Specification v3.0](https://github.com/toon-format/spec/blob/main/SPEC.md)\n- 📦 [Crates.io Package](https://crates.io/crates/toon-format)\n- 📚 [API Documentation](https://docs.rs/toon-format)\n- 🔧 [Main Repository (JS/TS)](https://github.com/toon-format/toon)\n- 🎯 [Benchmarks \u0026 Performance](https://github.com/toon-format/toon#benchmarks)\n\n---\n\n## Contributing\n\nContributions are welcome! Please see [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines.\n\n### Development\n\n```bash\n# Clone the repository\ngit clone https://github.com/your-org/toon-rust.git\ncd toon-rust\n\n# Run tests\ncargo test --all\n\n# Run lints\ncargo clippy -- -D warnings\n\n# Format code\ncargo fmt\n\n# Build docs\ncargo doc --open\n```\n\n---\n\n## License\n\nMIT License © 2025-PRESENT [Johann Schopplich](https://github.com/johannschopplich) and [Shreyas K S](https://github.com/shreyasbhat0)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftoon-format%2Ftoon-rust","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftoon-format%2Ftoon-rust","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftoon-format%2Ftoon-rust/lists"}