{"id":36501360,"url":"https://github.com/tonl-dev/tonl","last_synced_at":"2026-05-13T13:53:19.484Z","repository":{"id":323944843,"uuid":"1088700895","full_name":"tonl-dev/tonl","owner":"tonl-dev","description":"TONL (Token-Optimized Notation Language)","archived":false,"fork":false,"pushed_at":"2025-12-20T18:43:49.000Z","size":3558,"stargazers_count":789,"open_issues_count":0,"forks_count":29,"subscribers_count":6,"default_branch":"main","last_synced_at":"2025-12-22T19:28:25.168Z","etag":null,"topics":["json","llm","token"],"latest_commit_sha":null,"homepage":"https://tonl.dev","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tonl-dev.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-11-03T10:39:08.000Z","updated_at":"2025-12-22T00:43:07.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/tonl-dev/tonl","commit_stats":null,"previous_names":["ersinkoc/tonl","tonl-dev/tonl"],"tags_count":15,"template":false,"template_full_name":null,"purl":"pkg:github/tonl-dev/tonl","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tonl-dev%2Ftonl","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tonl-dev%2Ftonl/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tonl-dev%2Ftonl/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tonl-dev%2Ftonl/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tonl-dev","download_url":"https://codeload.github.com/tonl-dev/tonl/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tonl-dev%2Ftonl/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28332026,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-12T00:36:25.062Z","status":"online","status_checked_at":"2026-01-12T02:00:08.677Z","response_time":98,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["json","llm","token"],"created_at":"2026-01-12T02:21:55.884Z","updated_at":"2026-01-12T02:21:57.351Z","avatar_url":"https://github.com/tonl-dev.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cdiv align=\"center\"\u003e\n\n![TONL - Token-Optimized Notation Language](readme.png)\n\n\u003c/div\u003e\n\n# TONL (Token-Optimized Notation Language)\n\n**TONL** is a production-ready data platform that combines compact serialization with powerful query, modification, indexing, and streaming capabilities. Designed for LLM token efficiency while providing a rich API for data access and manipulation.\n\n## 🎉 Latest Release: v2.5.2 - Documentation \u0026 Testing Excellence\n\n### 📚 **v2.5.2 (December 20, 2025)**\n- **216 new tests** - Total 698 tests across 162 suites with 100% pass rate\n- **Browser documentation** - Complete docs/BROWSER.md and docs/ERROR_HANDLING.md\n- **4 browser examples** - React 18 and Vue 3 interactive demos\n- **5 security fixes** - All npm vulnerabilities resolved\n- **Updated dependencies** - All packages at latest versions\n\n### 🔧 **v2.5.1 (December 11, 2025)**\n- 8 critical bug fixes including DoS prevention and async handling\n\n### 🛡️ **v2.5.0 (December 3, 2025)**\n- Enterprise security hardening and optimization module\n\n### 🧪 **Testing Excellence:**\n- **698 Comprehensive Tests** - All passing with 100% success rate\n- **96 Security Tests** - Covering all attack vectors\n- **Concurrency Tests** - Thread safety validation\n- **Browser Tests** - Cross-platform compatibility\n\n[![npm version](https://badge.fury.io/js/tonl.svg)](https://www.npmjs.com/package/tonl)\n[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)\n[![TypeScript](https://img.shields.io/badge/TypeScript-100%25-blue.svg)](https://www.typescriptlang.org/)\n\n**🏠 Homepage**: [tonl.dev](https://tonl.dev)\n**📦 GitHub**: [github.com/tonl-dev/tonl](https://github.com/tonl-dev/tonl)\n**📖 Documentation**: [Complete Guides](docs/)\n\n## 📋 Table of Contents\n- [Why TONL?](#why-tonl)\n- [Quick Start](#-quick-start)\n- [Format Overview](#-format-overview)\n- [Feature Set](#-complete-feature-set)\n- [Performance](#-performance-comparison)\n- [Security](#-security--quality)\n- [Use Cases](#-use-cases)\n- [Browser Usage](#-browser-usage)\n- [API Reference](#-complete-api-reference)\n- [Development](#-development)\n- [Roadmap](#-roadmap)\n- [Documentation](#-documentation)\n- [Contributing](#-contributing)\n- [License](#-license)\n\n---\n\n## Why TONL?\n\n🗜️ **Up to 60% Smaller** - Reduce JSON size and LLM token costs\n👁️ **Human-Readable** - Clear text format, not binary\n🚀 **Blazingly Fast** - 10-1600x faster than targets\n🔒 **Production Secure** - 100% security hardened (v2.0.3)\n🛠️ **TypeScript-First** - Full type safety \u0026 IntelliSense\n📦 **Zero Dependencies** - Pure TypeScript, no bloat\n🌐 **Browser Ready** - 10.5 KB gzipped bundle (IIFE/UMD)\n✅ **100% Tested** - 496/496 tests passing (core functionality)\n\n---\n\n## 🚀 Quick Start\n\n### Installation\n\n```bash\nnpm install tonl\n```\n\n### Basic Usage\n\n```typescript\nimport { TONLDocument, encodeTONL, decodeTONL } from 'tonl';\n\n// Create from JSON\nconst doc = TONLDocument.fromJSON({\n  users: [\n    { id: 1, name: \"Alice\", role: \"admin\", age: 30 },\n    { id: 2, name: \"Bob\", role: \"user\", age: 25 }\n  ]\n});\n\n// Query with JSONPath-like syntax\ndoc.get('users[0].name');                          // 'Alice'\ndoc.query('users[*].name');                        // ['Alice', 'Bob']\ndoc.query('users[?(@.role == \"admin\")]');          // [{ id: 1, ... }]\ndoc.query('$..age');                               // All ages recursively\n\n// Aggregation (v2.4.0)\ndoc.count('users[*]');                             // 2\ndoc.sum('users[*]', 'age');                        // 55\ndoc.avg('users[*]', 'age');                        // 27.5\ndoc.groupBy('users[*]', 'role');                   // { admin: [...], user: [...] }\ndoc.aggregate('users[*]').stats('age');            // { count, sum, avg, min, max, stdDev }\n\n// Fuzzy Matching (v2.4.0)\nimport { fuzzySearch, soundsLike } from 'tonl/query';\nfuzzySearch('Jon', ['John', 'Jane', 'Bob']);       // [{ value: 'John', score: 0.75 }]\nsoundsLike('Smith', 'Smyth');                      // true\n\n// Temporal Queries (v2.4.0)\nimport { parseTemporalLiteral, isDaysAgo } from 'tonl/query';\nparseTemporalLiteral('@now-7d');                   // 7 days ago\nisDaysAgo(someDate, 30);                           // within last 30 days?\n\n// Modify data\ndoc.set('users[0].age', 31);\ndoc.push('users', { id: 3, name: \"Carol\", role: \"editor\", age: 28 });\n\n// Navigate and iterate\nfor (const [key, value] of doc.entries()) {\n  console.log(key, value);\n}\n\ndoc.walk((path, value, depth) =\u003e {\n  console.log(`${path}: ${value}`);\n});\n\n// Export\nconst tonl = doc.toTONL();\nconst json = doc.toJSON();\nawait doc.save('output.tonl');\n\n// Classic API (encode/decode)\nconst data = { users: [{ id: 1, name: \"Alice\" }] };\nconst tonlText = encodeTONL(data);\nconst restored = decodeTONL(tonlText);\n\n// Advanced Optimization (v2.0.1+)\nimport { AdaptiveOptimizer, BitPacker, DeltaEncoder } from 'tonl/optimization';\n\n// Automatic optimization\nconst optimizer = new AdaptiveOptimizer();\nconst result = optimizer.optimize(data);  // Auto-selects best strategies\n\n// Specific optimizers\nconst packer = new BitPacker();\nconst packed = packer.packBooleans([true, false, true]);\n\nconst delta = new DeltaEncoder();\nconst timestamps = [1704067200000, 1704067201000, 1704067202000];\nconst compressed = delta.encode(timestamps, 'timestamp');\n```\n\n### CLI Usage\n\n#### 🎮 **Interactive CLI (NEW v2.3.1)**\n```bash\n# Interactive stats dashboard\ntonl stats data.json --interactive\ntonl stats data.json -i --theme neon\n\n# File comparison mode\ntonl stats data.json --compare --theme matrix\n\n# Interactive exploration\ntonl stats --interactive  # Launch without file for menu-driven exploration\n```\n\n#### 📊 **Standard Commands**\n```bash\n# Get started (shows help)\ntonl\n\n# Version info\ntonl --version\n\n# Encode JSON to TONL (perfect round-trip, quotes special keys)\ntonl encode data.json --out data.tonl --smart --stats\n\n# Encode with preprocessing (clean, readable keys)\ntonl encode data.json --preprocess --out data.tonl\n\n# Decode TONL to JSON\ntonl decode data.tonl --out data.json\n\n# Query data\ntonl query users.tonl \"users[?(@.role == 'admin')]\"\ntonl get data.json \"user.profile.email\"\n\n# Validate against schema\ntonl validate users.tonl --schema users.schema.tonl\n\n# Format and prettify\ntonl format data.tonl --pretty --out formatted.tonl\n\n# Compare token costs\ntonl stats data.json --tokenizer gpt-5\n```\n\n#### 🎨 **Interactive Themes (v2.3.1)**\n```bash\n# Available themes: default, neon, matrix, cyberpunk\ntonl stats data.json -i --theme neon        # Bright neon colors\ntonl stats data.json -i --theme matrix      # Green matrix style\ntonl stats data.json -i --theme cyberpunk   # Cyan/purple cyberpunk\ntonl stats data.json -i --theme default     # Clean terminal colors\n```\n\n#### ⚖️ **File Comparison (v2.3.1)**\n```bash\n# Compare JSON and TONL files side-by-side\ntonl stats data.json --compare\ntonl stats data.json --compare --theme neon\n\n# Interactive comparison mode\ntonl stats data.json -i --compare\n```\n\n---\n\n## 📊 Format Overview\n\n### Arrays of Objects (Tabular Format)\n\n**JSON** (245 bytes, 89 tokens):\n```json\n{\n  \"users\": [\n    { \"id\": 1, \"name\": \"Alice\", \"role\": \"admin\" },\n    { \"id\": 2, \"name\": \"Bob, Jr.\", \"role\": \"user\" },\n    { \"id\": 3, \"name\": \"Carol\", \"role\": \"editor\" }\n  ]\n}\n```\n\n**TONL** (158 bytes, 49 tokens - **45% reduction**):\n```tonl\n#version 1.0\nusers[3]{id:u32,name:str,role:str}:\n  1, Alice, admin\n  2, \"Bob, Jr.\", user\n  3, Carol, editor\n```\n\n### Nested Objects\n\n**JSON**:\n```json\n{\n  \"user\": {\n    \"id\": 1,\n    \"name\": \"Alice\",\n    \"contact\": {\n      \"email\": \"alice@example.com\",\n      \"phone\": \"+123456789\"\n    },\n    \"roles\": [\"admin\", \"editor\"]\n  }\n}\n```\n\n**TONL**:\n```tonl\n#version 1.0\nuser{id:u32,name:str,contact:obj,roles:list}:\n  id: 1\n  name: Alice\n  contact{email:str,phone:str}:\n    email: alice@example.com\n    phone: +123456789\n  roles[2]: admin, editor\n```\n\n---\n\n## ✨ Complete Feature Set\n\n### 🔄 Core Serialization\n- **Compact Format** - 32-45% smaller than JSON (bytes + tokens)\n- **Human-Readable** - Clear text format with minimal syntax\n- **Round-Trip Safe** - Perfect bidirectional JSON conversion\n- **Smart Encoding** - Auto-selects optimal delimiters and formatting\n- **Type Hints** - Optional schema information for validation\n\n### 🔍 Query \u0026 Navigation API\n- **JSONPath Queries** - `users[?(@.age \u003e 25)]`, `$..email`\n- **Filter Expressions** - `==`, `!=`, `\u003e`, `\u003c`, `\u0026\u0026`, `||`, `contains`, `matches`\n- **Wildcard Support** - `users[*].name`, `**.email`\n- **Tree Traversal** - `entries()`, `keys()`, `values()`, `walk()`\n- **LRU Cache** - \u003e90% cache hit rate on repeated queries\n\n### ✏️ Modification API\n- **CRUD Operations** - `set()`, `get()`, `delete()`, `push()`, `pop()`\n- **Bulk Operations** - `merge()`, `update()`, `removeAll()`\n- **Change Tracking** - `diff()` with detailed change reports\n- **Snapshots** - Document versioning and comparison\n- **Atomic File Edits** - Safe saves with automatic backups\n\n### ⚡ Performance \u0026 Indexing\n- **Hash Index** - O(1) exact match lookups\n- **BTree Index** - O(log n) range queries\n- **Compound Index** - Multi-field indexing\n- **Stream Processing** - Handle multi-GB files with \u003c100MB memory\n- **Pipeline Operations** - Chainable filter/map/reduce transformations\n\n### 🗜️ Advanced Optimization\n- **Dictionary Encoding** - Value compression via lookup tables (30-50% savings)\n- **Delta Encoding** - Sequential data compression (40-60% savings)\n- **Run-Length Encoding** - Repetitive value compression (50-80% savings)\n- **Bit Packing** - Boolean and small integer bit-level compression (87.5% savings)\n- **Numeric Quantization** - Precision reduction for floating-point numbers (20-40% savings)\n- **Schema Inheritance** - Reusable column schemas across data blocks (20-40% savings)\n- **Hierarchical Grouping** - Common field extraction for nested structures (15-30% savings)\n- **Tokenizer-Aware** - LLM tokenizer optimization for minimal token usage (5-15% savings)\n- **Column Reordering** - Entropy-based ordering for better compression\n- **Adaptive Optimizer** - Automatic strategy selection based on data patterns\n\n### ✅ Schema \u0026 Validation\n- **Schema Definition** - `.schema.tonl` files with TSL (TONL Schema Language)\n- **13 Constraints** - `required`, `min`, `max`, `pattern`, `unique`, `email`, etc.\n- **TypeScript Generation** - Auto-generate types from schemas\n- **Runtime Validation** - Validate data programmatically or via CLI\n- **Strict Mode** - Enforce schema compliance\n\n### 🛠️ Developer Tools\n- **🎮 Interactive CLI Dashboard** - Real-time file analysis with themes and progress visualization\n- **⚖️ File Comparison System** - Side-by-side JSON/TONL comparison with detailed metrics\n- **🎨 Visual Customization** - Multiple terminal themes (default, neon, matrix, cyberpunk)\n- **Interactive REPL** - Explore data interactively in terminal\n- **Modular CLI Suite** - `encode`, `decode`, `query`, `validate`, `format`, `stats` with Command Pattern architecture\n- **Browser Support** - ESM, UMD, IIFE builds (8.84 KB gzipped)\n- **VS Code Extension** - Syntax highlighting for `.tonl` files\n- **TypeScript-First** - Full IntelliSense and type safety\n\n---\n\n## 📊 Performance Comparison\n\n| Metric | JSON | TONL | TONL Smart | Improvement |\n|--------|------|------|------------|-------------|\n| **Size (bytes)** | 245 | 167 | 158 | **36% smaller** |\n| **Tokens (GPT-5)** | 89 | 54 | 49 | **45% fewer** |\n| **Encoding Speed** | 1.0x | 15x | 12x | **12-15x faster** |\n| **Decoding Speed** | 1.0x | 10x | 10x | **10x faster** |\n| **Query Speed** | - | - | 1600x | **Target: \u003c1ms** |\n\n*Benchmarks based on typical e-commerce product catalog data*\n\n---\n\n## 🔒 Security \u0026 Quality\n\n```\n✅ Tests:          698+ tests passing (100% coverage)\n✅ Security:       All vulnerabilities fixed (100%)\n✅ Security Tests: 96 security tests passing\n✅ Code Quality:   TypeScript strict mode\n✅ Dependencies:   0 runtime dependencies\n✅ Bundle Size:    10.5 KB gzipped (browser)\n✅ Performance:    10-1600x faster than targets\n✅ Production:     Ready \u0026 Fully Secure\n```\n\n**Security:**\n- ✅ ReDoS, Path Traversal, Buffer Overflow protection\n- ✅ Prototype Pollution, Command Injection prevention\n- ✅ Integer Overflow, Type Coercion fixes\n- ✅ Comprehensive input validation and resource limits\n\nSee [SECURITY.md](SECURITY.md) and [CHANGELOG.md](CHANGELOG.md) for details.\n\n---\n\n## 🎯 Use Cases\n\n### LLM Prompts\nReduce token costs by 32-45% when including structured data in prompts:\n```typescript\nconst prompt = `Analyze this user data:\\n${doc.toTONL()}`;\n// 45% fewer tokens = lower API costs\n```\n\n### Configuration Files\nHuman-readable configs that are compact yet clear:\n```tonl\nconfig{env:str,database:obj,features:list}:\n  env: production\n  database{host:str,port:u32,ssl:bool}:\n    host: db.example.com\n    port: 5432\n    ssl: true\n  features[3]: auth, analytics, caching\n```\n\n### API Responses\nEfficient data transmission with schema validation:\n```typescript\napp.get('/api/users', async (req, res) =\u003e {\n  const doc = await TONLDocument.load('users.tonl');\n  const filtered = doc.query('users[?(@.active == true)]');\n  res.type('text/tonl').send(encodeTONL(filtered));\n});\n```\n\n### Data Pipelines\nStream processing for large datasets:\n```typescript\nimport { createEncodeStream, createDecodeStream } from 'tonl/stream';\n\ncreateReadStream('huge.json')\n  .pipe(createDecodeStream())\n  .pipe(transformStream)\n  .pipe(createEncodeStream({ smart: true }))\n  .pipe(createWriteStream('output.tonl'));\n```\n\n### Log Aggregation\nCompact structured logs:\n```tonl\nlogs[1000]{timestamp:i64,level:str,message:str,metadata:obj}:\n  1699564800, INFO, \"User login\", {user_id:123,ip:\"192.168.1.1\"}\n  1699564801, ERROR, \"DB timeout\", {query:\"SELECT...\",duration:5000}\n  ...\n```\n\n---\n\n## 🌐 Browser Usage\n\n### ESM (Modern Browsers)\n```html\n\u003cscript type=\"module\"\u003e\n  import { encodeTONL, decodeTONL } from 'https://cdn.jsdelivr.net/npm/tonl@2.4.1/+esm';\n\n  const data = { users: [{ id: 1, name: \"Alice\" }] };\n  const tonl = encodeTONL(data);\n  console.log(tonl);\n\u003c/script\u003e\n```\n\n### UMD (Universal)\n```html\n\u003cscript src=\"https://unpkg.com/tonl@2.4.1/dist/browser/tonl.umd.js\"\u003e\u003c/script\u003e\n\u003cscript\u003e\n  const tonl = TONL.encodeTONL({ hello: \"world\" });\n  console.log(tonl);\n\u003c/script\u003e\n```\n\n**Bundle Sizes:**\n- ESM: 15.5 KB gzipped\n- UMD: 10.7 KB gzipped\n- IIFE: 10.6 KB gzipped\n\n**Examples:**\nSee [examples/browser/](examples/browser/) for interactive React and Vue examples.\n\n---\n\n## 📚 Complete API Reference\n\n### TONLDocument Class\n\n```typescript\n// Creation\nTONLDocument.fromJSON(data)\nTONLDocument.parse(text)                           // Parse TONL string\nTONLDocument.fromFile(filepath)                    // Async file load\nTONLDocument.fromFileSync(filepath)                // Sync file load\n\n// Query\ndoc.get(path: string)                              // Single value\ndoc.query(query: string)                           // Multiple values\ndoc.exists(path: string)                           // Check existence\n\n// Modification\ndoc.set(path: string, value: any)                  // Set value\ndoc.delete(path: string)                           // Delete value\ndoc.push(path: string, value: any)                 // Append to array\ndoc.pop(path: string)                              // Remove last from array\ndoc.merge(path: string, value: object)             // Deep merge objects\n\n// Navigation\ndoc.entries()                                      // Iterator\u003c[key, value]\u003e\ndoc.keys()                                         // Iterator\u003cstring\u003e\ndoc.values()                                       // Iterator\u003cany\u003e\ndoc.walk(callback: WalkCallback)                   // Tree traversal\ndoc.find(predicate: Predicate)                     // Find single value\ndoc.findAll(predicate: Predicate)                  // Find all matching\ndoc.some(predicate: Predicate)                     // Any match\ndoc.every(predicate: Predicate)                    // All match\n\n// Indexing\ndoc.createIndex(name: string, path: string, type?) // Create index\ndoc.dropIndex(name: string)                        // Remove index\ndoc.getIndex(name: string)                         // Get index\n\n// Export\ndoc.toTONL(options?: EncodeOptions)                // Export as TONL\ndoc.toJSON()                                       // Export as JSON\ndoc.save(filepath: string, options?)               // Save to file\ndoc.size()                                         // Size in bytes\ndoc.stats()                                        // Statistics object\n```\n\n### Encode/Decode API\n\n```typescript\n// Encoding\nencodeTONL(data: any, options?: {\n  delimiter?: \",\" | \"|\" | \"\\t\" | \";\";\n  includeTypes?: boolean;\n  version?: string;\n  indent?: number;\n  singleLinePrimitiveLists?: boolean;\n}): string\n\n// Smart encoding (auto-optimized)\nencodeSmart(data: any, options?: EncodeOptions): string\n\n// Decoding\ndecodeTONL(text: string, options?: {\n  delimiter?: \",\" | \"|\" | \"\\t\" | \";\";\n  strict?: boolean;\n}): any\n```\n\n### Schema API\n\n```typescript\nimport { parseSchema, validateTONL } from 'tonl/schema';\n\n// Parse schema\nconst schema = parseSchema(schemaText: string);\n\n// Validate data\nconst result = validateTONL(data: any, schema: Schema);\n\nif (!result.valid) {\n  result.errors.forEach(err =\u003e {\n    console.error(`${err.field}: ${err.message}`);\n  });\n}\n```\n\n### Streaming API\n\n```typescript\nimport { createEncodeStream, createDecodeStream, encodeIterator, decodeIterator } from 'tonl/stream';\n\n// Node.js streams\ncreateReadStream('input.json')\n  .pipe(createEncodeStream({ smart: true }))\n  .pipe(createWriteStream('output.tonl'));\n\n// Async iterators\nfor await (const line of encodeIterator(dataStream)) {\n  console.log(line);\n}\n```\n\n---\n\n## ✅ Schema Validation\n\nDefine schemas with the TONL Schema Language (TSL):\n\n```tonl\n@schema v1\n@strict true\n@description \"User management schema\"\n\n# Define custom types\nUser: obj\n  id: u32 required\n  username: str required min:3 max:20 pattern:^[a-zA-Z0-9_]+$\n  email: str required pattern:email lowercase:true\n  age: u32? min:13 max:150\n  roles: list\u003cstr\u003e required min:1 unique:true\n\n# Root schema\nusers: list\u003cUser\u003e required min:1\ntotalCount: u32 required\n```\n\n**13 Built-in Constraints:**\n- `required` - Field must exist\n- `min` / `max` - Numeric range or string/array length\n- `length` - Exact length\n- `pattern` - Regex validation (or shortcuts: `email`, `url`, `uuid`)\n- `unique` - Array elements must be unique\n- `nonempty` - String/array cannot be empty\n- `positive` / `negative` - Number sign\n- `integer` - Must be integer\n- `multipleOf` - Divisibility check\n- `lowercase` / `uppercase` - String case enforcement\n\nSee [docs/SCHEMA_SPECIFICATION.md](docs/SCHEMA_SPECIFICATION.md) for complete reference.\n\n---\n\n## 🛠️ Development\n\n### Build \u0026 Test\n\n```bash\n# Install dependencies\nnpm install\n\n# Build TypeScript\nnpm run build\n\n# Run all tests (698+ tests)\nnpm test\n\n# Watch mode\nnpm run dev\n\n# Clean build artifacts\nnpm run clean\n```\n\n### Benchmarking\n\n```bash\n# Byte size comparison\nnpm run bench\n\n# Token estimation (GPT-5, Claude 3.5, Gemini 2.0, Llama 4)\nnpm run bench-tokens\n\n# Comprehensive performance analysis\nnpm run bench-comprehensive\n```\n\n### CLI Development\n\n```bash\n# Install CLI locally\nnpm run link\n\n# Test commands\ntonl encode test.json\ntonl query data.tonl \"users[*].name\"\ntonl format data.tonl --pretty\n\n# Test interactive features (v2.3.1+)\ntonl stats data.json --interactive\ntonl stats data.json -i --theme neon\ntonl stats data.json --compare\n```\n\n---\n\n## 🗺️ Roadmap\n\n**✅ v2.5.1 - Complete (Latest)**\n- ✅ Critical bug fixes (Array expansion DoS, JSON.stringify vulnerability, async handling)\n- ✅ 482 tests with 100% pass rate\n- ✅ Enhanced stability and error handling\n\n**✅ v2.5.0 - Complete**\n- ✅ Aggregation Functions (count, sum, avg, groupBy, stats, median, percentile)\n- ✅ Fuzzy String Matching (Levenshtein, Jaro-Winkler, Soundex, Metaphone)\n- ✅ Temporal Queries (@now-7d, before, after, sameDay, daysAgo)\n- ✅ 763+ comprehensive tests with 100% success rate\n\n**✅ v2.2+ - Complete**\n- ✅ Revolutionary Interactive CLI Dashboard with real-time analysis\n- ✅ Complete Modular Architecture Transformation (735→75 lines)\n- ✅ File Comparison System with side-by-side analysis\n- ✅ Visual Themes (default, neon, matrix, cyberpunk)\n\n**✅ v2.0+ - Complete**\n- ✅ Advanced optimization module (60% additional compression)\n- ✅ Complete query, modification, indexing, streaming APIs\n- ✅ Schema validation \u0026 TypeScript generation\n- ✅ Browser support (10.5 KB bundles)\n- ✅ 100% test coverage \u0026 security hardening\n\n**🚀 Future**\n- Enhanced VS Code extension (IntelliSense, debugging)\n- Web playground with live conversion\n- Python, Go, Rust implementations\n- Binary TONL format for extreme compression\n\nSee [ROADMAP.md](ROADMAP.md) for our comprehensive development vision.\n\n---\n\n## 📖 Documentation\n\n### For Users\n- **[Getting Started Guide](docs/GETTING_STARTED.md)** - Beginner-friendly tutorial with examples\n- **[API Reference](docs/API.md)** - Complete API documentation with examples\n- **[CLI Documentation](docs/CLI.md)** - Command-line tool guide\n- **[Browser API](docs/BROWSER.md)** - Browser usage with ESM, UMD, IIFE builds\n- **[Error Handling](docs/ERROR_HANDLING.md)** - Error classes and troubleshooting\n- **[Query API](docs/QUERY_API.md)** - JSONPath-like query syntax reference\n- **[Modification API](docs/MODIFICATION_API.md)** - CRUD operations guide\n- **[Navigation API](docs/NAVIGATION_API.md)** - Tree traversal methods\n- **[Use Cases](docs/USE_CASES.md)** - Real-world scenarios and solutions\n\n### For Implementers (Other Languages)\n- **[Implementation Reference](docs/IMPLEMENTATION_REFERENCE.md)** - Language-agnostic implementation guide\n- **[Transformation Examples](docs/TRANSFORMATION_EXAMPLES.md)** - 20+ JSON↔TONL conversion examples\n- **[Format Specification](docs/SPECIFICATION.md)** - Technical format specification\n- **[Schema Specification](docs/SCHEMA_SPECIFICATION.md)** - TSL (TONL Schema Language) spec\n\n**Implementing TONL in Python, Go, Rust, or another language?** Check out the [Implementation Reference](docs/IMPLEMENTATION_REFERENCE.md) for complete algorithms, pseudo-code, and test requirements!\n\n---\n\n## 🤝 Contributing\n\nContributions are welcome! Please read [CONTRIBUTING.md](CONTRIBUTING.md) for:\n- Development setup\n- Code style guidelines\n- Testing requirements\n- Pull request process\n- Architecture overview\n\n---\n\n## 📄 License\n\nMIT License - see [LICENSE](LICENSE) file for details.\n\n---\n\n## 🌟 Links\n\n- **Website**: [tonl.dev](https://tonl.dev)\n- **npm Package**: [npmjs.com/package/tonl](https://www.npmjs.com/package/tonl)\n- **GitHub**: [github.com/tonl-dev/tonl](https://github.com/tonl-dev/tonl)\n- **Issues**: [github.com/tonl-dev/tonl/issues](https://github.com/tonl-dev/tonl/issues)\n- **Discussions**: [github.com/tonl-dev/tonl/discussions](https://github.com/tonl-dev/tonl/discussions)\n- **VS Code Extension**: [Coming Soon]\n\n---\n\n\u003cdiv align=\"center\"\u003e\n\n**TONL**: Making structured data LLM-friendly without sacrificing readability. 🚀\n\n*Built with ❤️ by [Ersin Koc](https://github.com/ersinkoc)*\n\n\u003c/div\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftonl-dev%2Ftonl","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftonl-dev%2Ftonl","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftonl-dev%2Ftonl/lists"}