https://github.com/jassielof/ztoon
A Zig implementation of the TOON (Token-Oriented Object Notation) format.
https://github.com/jassielof/ztoon
ai toon toon-format zig
Last synced: 7 months ago
JSON representation
A Zig implementation of the TOON (Token-Oriented Object Notation) format.
- Host: GitHub
- URL: https://github.com/jassielof/ztoon
- Owner: jassielof
- License: agpl-3.0
- Created: 2025-11-06T05:05:06.000Z (8 months ago)
- Default Branch: main
- Last Pushed: 2025-11-10T18:16:12.000Z (7 months ago)
- Last Synced: 2025-11-10T19:13:21.579Z (7 months ago)
- Topics: ai, toon, toon-format, zig
- Language: Zig
- Homepage:
- Size: 33.2 KB
- Stars: 2
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE.txt
Awesome Lists containing this project
README
# Z-TOON: Zig TOON
A Zig implementation of the TOON (Token-Oriented Object Notation) format, version 2.0.
## What is TOON?
TOON is a line-oriented, indentation-based text format that encodes the JSON data model with explicit structure and minimal quoting. It's particularly efficient for arrays of uniform objects, providing a more compact and readable alternative to JSON for structured data.
See the [full specification](https://github.com/toon-format/spec) for details in depth.
## Features
- [x] **Core Encoding/Decoding**: Full JSON ↔ TOON conversion
- [x] **Primitives**: strings, numbers, booleans, null with smart quoting
- [x] **Objects**: Nested objects with indentation-based structure
- [x] **Arrays**: Both inline (primitives) and multi-line (objects/nested)
- [x] **Tabular Arrays**: Compact `[N]{field1,field2}:` format for uniform object arrays
- [x] **Alternative Delimiters**: Comma (default), tab (`\t`), and pipe (`|`) support
- [x] **Delimiter Detection**: Automatic delimiter detection in array headers `[N]`
- [x] **CLI Tool**: Encode and decode via command line or pipes
## Building
Requires Zig 0.15.2 or later:
```
zig build
```
The binary will be available at `./zig-out/bin/ztoon`.
## Usage
### CLI
**Encode JSON to TOON:**
```bash
echo '{"name": "Alice", "age": 30}' | ./zig-out/bin/ztoon encode
```
Output:
```
name: Alice
age: 30
```
**Decode TOON to JSON:**
```bash
echo 'name: Alice
age: 30' | ./zig-out/bin/ztoon decode
```
Output:
```json
{
"name": "Alice",
"age": 30
}
```
**From files:**
```bash
./zig-out/bin/ztoon encode input.json
./zig-out/bin/ztoon decode input.toon
```
### Library
```zig
const std = @import("std");
const toon = @import("ztoon");
pub fn main() !void {
var gpa = std.heap.GeneralPurposeAllocator(.{}){};
defer _ = gpa.deinit();
const allocator = gpa.allocator();
// Create a value
var obj = std.StringHashMap(toon.Value).init(allocator);
defer obj.deinit();
try obj.put("name", toon.Value{ .string = "Alice" });
try obj.put("age", toon.Value{ .number = 30 });
const value = toon.Value{ .object = obj };
defer value.deinit(allocator);
// Encode to TOON
const encoded = try toon.encode(allocator, value, .{});
defer allocator.free(encoded);
std.debug.print("TOON: {s}\n", .{encoded});
// Decode from TOON
var decoded = try toon.decode(allocator, encoded, .{});
defer decoded.deinit(allocator);
}
```
## Examples
### Simple Object
```json
{"name": "Alice", "age": 30, "active": true}
```
↓
```toon
name: Alice
age: 30
active: true
```
### Nested Objects
```json
{"user": {"name": "Alice", "settings": {"theme": "dark"}}}
```
↓
```toon
user:
name: Alice
settings:
theme: dark
```
### Tabular Array (Uniform Objects)
```json
{"items": [
{"sku": "A1", "qty": 2, "price": 9.99},
{"sku": "B2", "qty": 1, "price": 14.5}
]}
```
↓
```toon
items:[2]{sku,price,qty}:
A1,9.99,2
B2,14.5,1
```
### Pipe-Delimited Tabular Array
```json
{"users": [
{"name": "Alice", "role": "admin"},
{"name": "Bob", "role": "dev"}
]}
```
↓
```toon
users:[2|]{name|role}:
Alice|admin
Bob|dev
```
### Primitive Array
```json
{"items": [1, 2, 3, 4, 5]}
```
↓
```toon
items[5]: 1,2,3,4,5
```
### Array of Objects (Non-Uniform)
```json
{"users": [{"name": "Alice", "id": 1}, {"name": "Bob", "id": 2}]}
```
↓
```toon
users[2]:
-
name: Alice
id: 1
-
name: Bob
id: 2
```
## Status & Roadmap
### Implemented ✅
- Core TOON encoder and decoder
- Nested objects with indentation
- Tabular arrays with field lists
- Alternative delimiters (comma, tab, pipe)
- Delimiter detection from array headers
- Smart string quoting
- CLI tool
- Field order preservation (deterministic output)
- Comprehensive error messages with line/column information
### Planned 📋
- Key folding (`keyFolding="safe"` mode) - collapse single-key object chains into dotted notation
- Path expansion (`expandPaths="safe"` mode) - split dotted keys into nested objects
- Strict mode validation (length mismatches, malformed headers)
- Conformance test suite from `spec/tests/fixtures/`
- Performance optimizations
- Benchmarks vs JSON
### Known Limitations ⚠️
- Some edge cases from spec may not be fully covered
- Non-strict mode has limited validation
## Next Steps
To continue improving this implementation:
1. **Conformance Tests**: Load and run test fixtures from `spec/tests/fixtures/` for full spec compliance
2. **Strict Mode**: Implement validation for length mismatches, invalid characters, and malformed headers
3. **Key Folding**: Implement optional `keyFolding="safe"` mode for dotted-path notation
4. **Path Expansion**: Implement optional `expandPaths="safe"` mode for splitting dotted keys
5. **Better Delimiters**: Auto-select optimal delimiter on encode based on content analysis
6. **Performance**: Profile and optimize hot paths, especially for large tabular arrays
7. **Documentation**: Add inline documentation and more usage examples
## Testing
Currently tested manually with various inputs. Run some quick tests:
```bash
# Test nested objects
echo '{"a":1,"b":{"c":2}}' | ./zig-out/bin/ztoon encode | ./zig-out/bin/ztoon decode
# Test tabular arrays
echo '{"items":[{"id":1,"name":"Alice"},{"id":2,"name":"Bob"}]}' | ./zig-out/bin/ztoon encode
# Test with different delimiters
printf 'items:[2|]{name|role}:\n Alice|admin\n Bob|dev' | ./zig-out/bin/ztoon decode
```