{"id":33918835,"url":"https://github.com/lnmplang/lnmp-protocol","last_synced_at":"2026-04-05T01:32:29.555Z","repository":{"id":324712881,"uuid":"1098212132","full_name":"lnmplang/lnmp-protocol","owner":"lnmplang","description":"LNMP – A protocol that carries meaning, enabling efficient and semantically consistent communication between LLM-native and autonomous systems.","archived":false,"fork":false,"pushed_at":"2026-02-09T18:46:31.000Z","size":2613,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2026-02-09T22:39:52.768Z","etag":null,"topics":["a2a-protocol","anti-monopoly","autonomous-systems","future-of-ai","llm-protocol","lnmp-protocol","m2m-protocol","real-time-systems","semantic-protocol"],"latest_commit_sha":null,"homepage":"https://lnmp.ai","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/lnmplang.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":"CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null},"funding":{"github":null,"patreon":null,"open_collective":null,"ko_fi":null,"tidelift":null,"community_bridge":null,"liberapay":null,"issuehunt":null,"lfx_crowdfunding":null,"polar":null,"buy_me_a_coffee":"madraka","thanks_dev":null,"custom":null}},"created_at":"2025-11-17T12:00:03.000Z","updated_at":"2026-02-09T18:47:24.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/lnmplang/lnmp-protocol","commit_stats":null,"previous_names":["lnmplang/lnmp-protocol"],"tags_count":9,"template":false,"template_full_name":null,"purl":"pkg:github/lnmplang/lnmp-protocol","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lnmplang%2Flnmp-protocol","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lnmplang%2Flnmp-protocol/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lnmplang%2Flnmp-protocol/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lnmplang%2Flnmp-protocol/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/lnmplang","download_url":"https://codeload.github.com/lnmplang/lnmp-protocol/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lnmplang%2Flnmp-protocol/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31421869,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-05T00:25:07.052Z","status":"ssl_error","status_checked_at":"2026-04-05T00:25:05.923Z","response_time":60,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["a2a-protocol","anti-monopoly","autonomous-systems","future-of-ai","llm-protocol","lnmp-protocol","m2m-protocol","real-time-systems","semantic-protocol"],"created_at":"2025-12-12T08:34:51.296Z","updated_at":"2026-04-05T01:32:29.539Z","avatar_url":"https://github.com/lnmplang.png","language":"Rust","funding_links":["https://buymeacoffee.com/madraka"],"categories":[],"sub_categories":[],"readme":"# LNMP Protocol\n\nLNMP (LLM Native Minimal Protocol) is a minimal, tokenizer-friendly, semantic-ID-based data format designed for data exchange with large language models (LLMs).\n\n[![Crates.io](https://img.shields.io/crates/v/lnmp.svg)](https://crates.io/crates/lnmp)\n[![Documentation](https://docs.rs/lnmp/badge.svg)](https://docs.rs/lnmp)\n[![License](https://img.shields.io/badge/license-MIT%2FApache--2.0-blue.svg)](LICENSE)\n[![Rust](https://img.shields.io/badge/rust-1.91%2B-orange.svg)](https://www.rust-lang.org)\n[![Downloads](https://img.shields.io/crates/d/lnmp.svg)](https://crates.io/crates/lnmp)\n\n**Official domain:** [lnmp.ai](https://lnmp.ai)\n\n\u003e **📋 FID Registry:** All Field IDs have official definitions in [`registry/fids.yaml`](registry/fids.yaml). See [`spec/fid-governance.md`](spec/fid-governance.md) for governance rules.\n\n## Features\n\n- 🚀 **Minimal syntax** - Token reduction compared to JSON\n- 🔢 **Semantic IDs** - Uses numeric field IDs for efficient learning\n- 📖 **Human-readable** - Text-based format that's easy to read and debug\n- 🦀 **Pure Rust** - Memory-safe implementation with minimal dependencies\n- 🏗️ **Nested structures** - Support for hierarchical data modeling (v0.3)\n- 🔒 **Semantic checksums** - SC32 checksums for drift prevention (v0.3)\n- 🎯 **LLM-optimized** - Prompt visibility optimization and ShortForm encoding (v0.3)\n- ⚡ **Binary protocol** - Efficient binary encoding with 30-50% size reduction (v0.4)\n- 🔄 **Bidirectional conversion** - Seamless text ↔ binary transformation (v0.4)\n- 📦 **VarInt encoding** - Space-efficient integer representation (v0.4)\n- 🌳 **Binary nested structures** - Recursive encoding for hierarchical data (v0.5)\n- 📡 **Streaming support** - Chunked transmission for large payloads (v0.5)\n- 🤝 **Schema negotiation** - Capability exchange and version negotiation (v0.5)\n- 🔄 **Delta encoding** - Bandwidth-efficient incremental updates (v0.5)\n- 🧠 **LLB2 optimization** - Enhanced LLM context optimization (v0.5)\n- 🧬 **Embedding support** - Native vector embeddings with delta encoding (v0.5)\n- 🛡️ **Input sanitization** - Security-focused input validation and sanitization (v0.5)\n- 📍 **Spatial awareness** - Physical coordinates and transformations with hybrid protocol (v0.5.2)\n- ✅ **Well-tested** - Comprehensive test suite with multi-language compliance tests\n- 📉 **Quantization** - Adaptive vector quantization (FP16, Int8, Int4, Binary) with batch processing (v0.5.4)\n- 🔧 **Generic Arrays** - IntArray, FloatArray, BoolArray for efficient numeric data (v0.5.5)\n- 🎯 **Strict Profiles** - Configurable validation levels (Loose/Standard/Strict) (v0.5.5)\n- 🏗️ **RecordBuilder** - Fluent API for canonical record construction (v0.5.5)\n- 🧠 **Context Profiling** - Automatic scoring for LLM prioritization (freshness, importance, risk, confidence) (v0.5.7)\n- 🌐 **Network Intelligence** - `lnmp-net` layer for message classification, QoS, and intelligent routing (v0.5.7)\n- 🚀 **Transport bindings** - Standard mappings for HTTP, Kafka, gRPC, NATS with W3C Trace Context and OpenTelemetry integration (v0.5.7)\n\n## Quick Start\n\n### Installation\n\nAdd LNMP to your `Cargo.toml`:\n\n```toml\n[dependencies]\nlnmp = \"0.5.15\"\n```\n\nOr use cargo:\n\n```bash\ncargo add lnmp\n```\n\n### Basic Usage\n\n```rust\nuse lnmp::prelude::*;\n\nfn main() -\u003e Result\u003c(), Box\u003cdyn std::error::Error\u003e\u003e {\n    // Parse LNMP text\n    let text = \"F12=14532;F7=1;F23=[admin,dev]\";\n    let mut parser = Parser::new(text)?;\n    let record = parser.parse_record()?;\n    \n    // Access fields\n    if let Some(field) = record.get_field(12) {\n        println!(\"User ID: {:?}\", field.value);\n    }\n    \n    // Create records with RecordBuilder\n    let new_record = RecordBuilder::new()\n        .add_field(LnmpField { fid: 12, value: LnmpValue::Int(14532) })\n        .add_field(LnmpField { fid: 7, value: LnmpValue::Bool(true) })\n        .build();\n    \n    // Encode to text\n    let encoder = Encoder::new();\n    let output = encoder.encode(\u0026new_record);\n    println!(\"{}\", output); // F7=1\\nF12=14532\n    \n    Ok(())\n}\n```\n\nFor more examples, see the `examples/` directory in individual crates or the [lnmp-examples](https://github.com/lnmplang/lnmp-examples) repository.\n\n## Project Structure\n\nThis is a **Pure Protocol** repository containing the core Rust crates:\n- **lnmp-core** (v0.5.13): Core type definitions and data structures\n- **lnmp-codec** (v0.5.13): Parser and encoder implementations\n- **lnmp-embedding** (v0.5.13): Vector embedding support\n- **lnmp-envelope** (v0.5.13): Operational metadata envelope\n- **lnmp-spatial** (v0.5.13): Spatial awareness types\n- **lnmp-quant** (v0.5.13): Adaptive quantization\n- **lnmp-llb** (v0.5.13): LNMP-LLM Bridge Layer\n- **lnmp-sfe** (v0.5.13): Semantic Fidelity Engine\n- **lnmp-net** (v0.5.13): Network behavior layer\n- **lnmp-sanitize** (v0.5.13): Input sanitization\n- **lnmp-transport** (v0.5.13): Transport bindings\n\n## 🌍 Ecosystem\n\nThe LNMP ecosystem consists of several repositories:\n\n### SDKs\n- **Python**: [`lnmp-sdk-python`](https://github.com/lnmplang/lnmp-sdk-python)\n- **TypeScript/JS**: [`lnmp-sdk-js`](https://github.com/lnmplang/lnmp-sdk-js)\n- **Rust**: [`lnmp-sdk-rust`](https://github.com/lnmplang/lnmp-sdk-rust)\n- **Go**: [`lnmp-sdk-go`](https://github.com/lnmplang/lnmp-sdk-go)\n\n### Tools\n- **CLI**: [`lnmp-cli`](https://github.com/lnmplang/lnmp-cli)\n- **MCP Server**: [`lnmp-mcp`](https://github.com/lnmplang/lnmp-mcp)\n- **VS Code Extension**: [`lnmp-vscode`](https://github.com/lnmplang/lnmp-vscode)\n\n### Examples\n- **Showcase**: [`lnmp-showcase`](https://github.com/lnmplang/lnmp-showcase) - City Pulse Demo\n- **Examples**: [`lnmp-examples`](https://github.com/lnmplang/lnmp-examples)\n\n\n## Quick Start\n\n### Recommended: Use the Meta Crate (All-in-One)\n\nThe easiest way to get started is with the `lnmp` meta crate, which includes all LNMP modules:\n\n```toml\n[dependencies]\nlnmp = \"0.5.15\"\n```\n\nThen use the convenient prelude:\n\n```rust\nuse lnmp::prelude::*;\n\n// Parse LNMP text\nlet lnmp_text = r#\"F12=14532;F7=1;F23=[\"admin\",\"dev\"]\"#;\nlet mut parser = Parser::new(lnmp_text).unwrap();\nlet record = parser.parse_record().unwrap();\n\n// Encode LNMP text\nlet mut record = LnmpRecord::new();\nrecord.add_field(LnmpField {\n    fid: 12,\n    value: LnmpValue::Int(14532),\n});\n\nlet encoder = Encoder::new();\nprintln!(\"{}\", encoder.encode(\u0026record)); // F12=14532\n```\n\n**All modules are accessible:**\n\n```rust\nuse lnmp::{core, codec, embedding, spatial, quant, llb, sfe, sanitize};\n```\n\n### Alternative: Individual Crates\n\nFor fine-grained control, you can use individual crates:\n\n```toml\n[dependencies]\nlnmp-core = \"0.5.7\"\nlnmp-codec = \"0.5.7\"\n```\n\n\u003e Developing against a local checkout? Replace the version strings with `path = \"../lnmp-protocol/crates/…\"`. See `REPO-STRUCTURE.md` for details.\n\n### Parsing LNMP Text Format.\n\n```rust\nuse lnmp_codec::Parser;\n\nlet lnmp_text = r#\"F12=14532;F7=1;F23=[\"admin\",\"dev\"]\"#;\n\nlet mut parser = Parser::new(lnmp_text).unwrap();\nlet record = parser.parse_record().unwrap();\n\nprintln!(\"User ID: {:?}\", record.get_field(12).unwrap().value);\n```\n\n### Encoding LNMP Text Format\n\n```rust\nuse lnmp_codec::Encoder;\nuse lnmp_core::{LnmpField, LnmpRecord, LnmpValue};\n\nlet mut record = LnmpRecord::new();\nrecord.add_field(LnmpField {\n    fid: 12,\n    value: LnmpValue::Int(14532),\n});\n\nlet encoder = Encoder::new();\nprintln!(\"{}\", encoder.encode(\u0026record)); // F12=14532\n```\n\n### Binary Format (v0.4+)\n\n```rust\nuse lnmp_codec::binary::{BinaryEncoder, BinaryDecoder};\n\n// Encode text to binary\nlet text = \"F7=1\\nF12=14532\\nF23=[admin,dev]\";\nlet encoder = BinaryEncoder::new();\nlet binary = encoder.encode_text(text).unwrap();\n\n// Decode binary to text\nlet decoder = BinaryDecoder::new();\nlet decoded_text = decoder.decode_to_text(\u0026binary).unwrap();\n\n// Round-trip conversion maintains data integrity\nassert_eq!(text, decoded_text);\n```\n\n### Binary Nested Structures (v0.5)\n\n```rust\nuse lnmp_codec::binary::{BinaryNestedEncoder, BinaryNestedDecoder, NestedEncoderConfig};\nuse lnmp_core::{LnmpField, LnmpRecord, LnmpValue};\n\n// Create nested record (F70=nested_data, F12=user_id from registry)\nlet mut inner = LnmpRecord::new();\ninner.add_field(LnmpField { fid: 12, value: LnmpValue::Int(42) });\n\nlet mut outer = LnmpRecord::new();\nouter.add_field(LnmpField {\n    fid: 70,  // F70=nested_data\n    value: LnmpValue::NestedRecord(Box::new(inner)),\n});\n\n// Encode with nested support\nlet config = NestedEncoderConfig::new().with_max_depth(32);\nlet encoder = BinaryNestedEncoder::with_config(config);\nlet binary = encoder.encode_nested_record(\u0026outer).unwrap();\n```\n\n### Streaming Large Payloads (v0.5)\n\n```rust\nuse lnmp_codec::binary::{StreamingEncoder, StreamingConfig};\n\nlet config = StreamingConfig::new()\n    .with_chunk_size(4096)\n    .with_checksums(true);\n\nlet mut encoder = StreamingEncoder::with_config(config);\n\n// Stream in chunks\nlet begin = encoder.begin_stream().unwrap();\nlet chunk = encoder.write_chunk(\u0026data).unwrap();\nlet end = encoder.end_stream().unwrap();\n```\n\n### Delta Encoding (v0.5)\n\n```rust\nuse lnmp_codec::binary::{DeltaEncoder, DeltaConfig};\n\nlet config = DeltaConfig::new().with_delta_enabled(true);\nlet encoder = DeltaEncoder::with_config(config);\n\n// Compute delta between records\nlet delta_ops = encoder.compute_delta(\u0026old_record, \u0026new_record).unwrap();\nlet delta_binary = encoder.encode_delta(\u0026delta_ops).unwrap();\n\n// 50%+ bandwidth savings for typical updates\n```\n\n## LNMP Format\n\nLNMP uses field assignments in the format `F\u003cfield_id\u003e=\u003cvalue\u003e`:\n\n```\n# Inline format (semicolon-separated)\nF12=14532;F7=1;F23=[\"admin\",\"dev\"]\n\n# Multiline format (canonical)\nF7=1\nF12=14532\nF20=\"Halil\"\nF23=[\"admin\",\"dev\"]\n\n# With type hints\nF12:i=14532\nF7:b=1\nF23:sa=[\"admin\",\"dev\"]\n\n# With checksums (v0.3)\nF12:i=14532#36AAE667\nF7:b=1#A3F2B1C4\n\n# Nested structures (v0.3)\n# F70=nested_data, F71=record_list, F20=name from registry\nF70={F12=1;F7=1}\nF71=[{F20=alice},{F20=bob}]\n```\n\n### Supported Types\n\n**Primitive Types:**\n- **Integers**: `F1=42`, `F2=-123`\n- **Floats**: `F40=3.14`, `F41=-2.5`\n- **Booleans**: `F7=1` (true), `F8=0` (false)\n- **Strings**: `F20=\"hello world\"`, `F21=simple_string`\n- **Arrays**: `F23=[\"admin\",\"dev\"]` (StringArray)\n- *\u003e Typed numeric arrays (IntArray, FloatArray, BoolArray) now parse from text when the corresponding `:ia/:fa/:ba` hints are supplied, matching the codec and binary representations.*\n\n**Nested Types (v0.3):**\n- **Nested Records**: `F70={F12=1;F7=1}` - Records within records (F70=nested_data)\n- **Record Arrays**: `F71=[{F20=alice},{F20=bob}]` - Arrays of records (F71=record_list)\n\n### Escape Sequences\n\nQuoted strings support escape sequences:\n- `\\\"` - Double quote\n- `\\\\` - Backslash\n- `\\n` - Newline\n- `\\r` - Carriage return\n- `\\t` - Tab\n\n## v0.3 Features\n\n### Semantic Fidelity Engine (SFE)\n\nPrevent LLM input drift with semantic checksums:\n\n```rust\nuse lnmp_codec::{Encoder, EncoderConfig};\n\nlet config = EncoderConfig {\n    enable_checksums: true,\n    ..Default::default()\n};\nlet encoder = Encoder::with_config(config);\nlet output = encoder.encode(\u0026record);\n// Output: F12:i=14532#36AAE667\n```\n\n**Features:**\n- SC32 checksums (32-bit CRC32-based)\n- Value normalization (booleans, floats, strings)\n- Equivalence mapping (synonym recognition)\n\n### Structural Extensibility Layer (SEL)\n\nModel hierarchical data with nested structures:\n\n```rust\n// Nested record (F70=nested_data from registry)\nlet input = \"F70={F12=1;F7=1}\";\n\n// Record array (F71=record_list, F20=name from registry)\nlet input = \"F71=[{F20=alice},{F20=bob}]\";\n\n// Deep nesting\nlet input = \"F70={F20=user;F70={F30=nested;F31=data}}\";\n```\n\n**Features:**\n- Arbitrary nesting depth\n- Structural canonicalization\n- Deterministic field ordering at all levels\n\n### LNMP-LLM Bridge Layer (LLB)\n\nOptimize for LLM tokenization:\n\n```rust\n// Explain mode - human-readable comments\nF12:i=14532  # user_id\nF7:b=1       # is_active\n\n// ShortForm - extreme token reduction\n12=14532 7=1 23=[admin,dev]\n```\n\n**Features:**\n- Prompt visibility optimization\n- Explain mode encoding\n- ShortForm encoding (7-12× reduction vs JSON)\n\n### Zero-Ambiguity Grammar (ZAG)\n\nFormal grammar specification for multi-language implementations:\n\n- PEG grammar specification\n- EBNF specification\n- Error classification system\n- Multi-language compliance test suite (Rust, Python, TypeScript, C++)\n\n## v0.5 Features\n\n### Binary Nested Structures (BNS)\n\nEncode hierarchical data in binary format with depth and size validation:\n\n```rust\nuse lnmp_codec::binary::{BinaryNestedEncoder, NestedEncoderConfig};\n\nlet config = NestedEncoderConfig::new()\n    .with_max_depth(32)\n    .with_max_record_size(Some(1_000_000));\n\nlet encoder = BinaryNestedEncoder::with_config(config);\nlet binary = encoder.encode_nested_record(\u0026nested_record).unwrap();\n```\n\n**Features:**\n- TypeTag 0x06 for nested records\n- TypeTag 0x07 for nested arrays\n- Configurable depth limits (default: 32)\n- Optional size limits for security\n- Automatic canonical ordering\n\n### Streaming Frame Layer (SFL)\n\nStream large payloads in chunks with integrity checking:\n\n```rust\nuse lnmp_codec::binary::{StreamingEncoder, StreamingConfig};\n\nlet config = StreamingConfig::new()\n    .with_chunk_size(4096)\n    .with_checksums(true);\n\nlet mut encoder = StreamingEncoder::with_config(config);\n\n// BEGIN frame\nlet begin = encoder.begin_stream().unwrap();\n\n// CHUNK frames\nlet chunk = encoder.write_chunk(\u0026data).unwrap();\n\n// END frame\nlet end = encoder.end_stream().unwrap();\n```\n\n**Features:**\n- Frame types: BEGIN (0xA0), CHUNK (0xA1), END (0xA2), ERROR (0xA3)\n- XOR checksum validation\n- Backpressure flow control\n- Configurable chunk size (default: 4KB)\n- Error recovery\n\n### Schema Negotiation Layer (SNL)\n\nNegotiate capabilities before data exchange:\n\n```rust\nuse lnmp_codec::binary::{SchemaNegotiator, Capabilities, FeatureFlags};\n\nlet features = FeatureFlags {\n    supports_nested: true,\n    supports_streaming: true,\n    supports_delta: true,\n    supports_llb: true,\n    requires_checksums: true,\n    requires_canonical: true,\n};\n\nlet caps = Capabilities {\n    version: 5,\n    features,\n    supported_types: vec![/* ... */],\n};\n\nlet mut negotiator = SchemaNegotiator::new(caps);\nlet msg = negotiator.initiate().unwrap();\n```\n\n**Features:**\n- Capability exchange\n- Feature flag negotiation\n- FID conflict detection\n- Type mismatch detection\n- Protocol version negotiation\n- Graceful degradation\n\n### Dynamic FID Discovery (v0.5.14)\n\nQuery and synchronize FID registries between peers:\n\n```rust\nuse lnmp_codec::binary::{SchemaNegotiator, FidDefinition, FidDefStatus, TypeTag};\nuse lnmp_core::registry::RegistrySync;\n\n// Request peer's FID registry\nlet mut negotiator = SchemaNegotiator::v0_5()\n    .with_registry_version(\"1.0.0\".into());\nlet request = negotiator.request_registry(None);\n\n// Check FID support after discovery\nif negotiator.peer_supports_fid(12) {\n    println!(\"Peer supports user_id field\");\n}\n\n// Multi-peer registry sync\nlet mut sync = RegistrySync::with_embedded();\nsync.register_peer(\"peer-1\".into(), \"0.9.0\".into());\nif sync.is_ahead_of(\"peer-1\") {\n    let delta_fids = sync.delta_fids_for(\"peer-1\");\n}\n```\n\n**Features:**\n- RequestRegistry/RegistryResponse protocol\n- Incremental RegistryDelta sync\n- Peer FID support queries\n- Version-aware registry sync\n\n### Delta Encoding \u0026 Partial Update Layer (DPL)\n\nSend only changed fields for bandwidth efficiency:\n\n```rust\nuse lnmp_codec::binary::{DeltaEncoder, DeltaConfig};\n\nlet config = DeltaConfig::new().with_delta_enabled(true);\nlet encoder = DeltaEncoder::with_config(config);\n\nlet delta_ops = encoder.compute_delta(\u0026old_record, \u0026new_record).unwrap();\nlet delta_binary = encoder.encode_delta(\u0026delta_ops).unwrap();\n```\n\n**Features:**\n- Delta operations: SET_FIELD, DELETE_FIELD, UPDATE_FIELD, MERGE_RECORD\n- 50%+ bandwidth savings for typical updates\n- Nested record merging\n- Incremental update chains\n\n### LLM Optimization Layer v2 (LLB2)\n\nEnhanced context optimization with binary integration:\n\n```rust\nuse lnmp_llb::{LlbConverter, LlbConfig};\n\nlet config = LlbConfig::new()\n    .with_flattening(true)\n    .with_semantic_hints(true)\n    .with_collision_safe_ids(true);\n\nlet converter = LlbConverter::new(config);\n\n// Binary to ShortForm\nlet shortform = converter.binary_to_shortform(\u0026binary).unwrap();\n\n// Flatten nested structures\nlet flattened = converter.flatten_nested(\u0026nested_record).unwrap();\n```\n\n**Features:**\n- Binary ↔ ShortForm conversion\n- Binary ↔ FullText conversion\n- Nested structure flattening\n- Semantic hint embedding\n- Collision-safe ID generation\n\n## Building\n\n```bash\n# Build all crates\ncargo build\n\n# Run tests\ncargo test\n\n# Run examples\ncargo run --example parse_example\ncargo run --example encode_example\ncargo run --example round_trip\n\n# Run v0.3 examples\ncargo run -p lnmp-core --example nested_structures\ncargo run -p lnmp-core --example semantic_checksums\ncargo run -p lnmp-core --example explain_mode\ncargo run -p lnmp-core --example shortform\n\n# Run v0.4 binary format examples\ncargo run -p lnmp-codec --example binary_encoding\ncargo run -p lnmp-codec --example binary_roundtrip\n\n# Run v0.5 advanced protocol examples\ncargo run -p lnmp-codec --example v05_nested_binary\ncargo run -p lnmp-codec --example v05_streaming\ncargo run -p lnmp-core --example v05_schema_negotiation\ncargo run -p lnmp-codec --example v05_delta_encoding\ncargo run -p lnmp-llb --example v05_llb2_binary\n\n# Run compliance tests\ncargo test --package lnmp-codec -- spec_compatibility\n\n# Generate documentation\ncargo doc --open\n```\n\n## Examples\n\nSee the `examples/` directory in individual crates for complete examples:\n\n**Basic Examples:**\n- `parse_example.rs` - Parsing LNMP text\n- `encode_example.rs` - Creating and encoding records\n- `round_trip.rs` - Parse → Encode → Parse round-trip\n\n**v0.2 Examples:**\n- `type_hints.rs` - Type hint usage\n- `strict_vs_loose.rs` - Parsing mode comparison\n- `deterministic_serialization.rs` - Canonical format\n\n**v0.3 Examples:**\n- `nested_structures.rs` - Nested records and arrays\n- `semantic_checksums.rs` - SC32 checksum usage\n- `explain_mode.rs` - Explain mode encoding\n- `shortform.rs` - ShortForm encoding/parsing\n- `structural_canonicalization.rs` - Structural canonicalization\n\n**v0.4 Examples (Binary Format):**\n- `binary_encoding.rs` - Basic binary encoding and decoding\n- `binary_roundtrip.rs` - Round-trip conversion and data integrity\n\n**v0.5 Examples (Advanced Protocol):**\n- `v05_nested_binary.rs` - Binary nested structures with depth/size validation\n- `v05_streaming.rs` - Streaming Frame Layer with backpressure control\n- `v05_schema_negotiation.rs` - Capability negotiation and conflict detection\n- `v05_delta_encoding.rs` - Delta operations with bandwidth savings\n- `v05_llb2_binary.rs` - LLB2 integration with binary format\n\n## Requirements\n\n- Rust 1.70.0 or later\n- Cargo (comes with Rust)\n\n## Documentation\n\n- **v0.5 Specification**: See `specs/lnmp-v0.5-advanced-protocol/`\n  - `requirements.md` - Formal requirements using EARS patterns\n  - `design.md` - Architecture and component design\n- **[Migration Guides](./docs/migration/)** - Version upgrade guides (v0.3 → v0.4 → v0.5)\n- **[API Reference](./docs/api/)** - Complete API documentation for Rust, TypeScript, and other SDKs\n- **v0.3 Specification**: See `specs/lnmp-v0.3-semantic-fidelity/`\n- **[Grammar Specification](./spec/grammar.md)** - Formal grammar for LNMP v0.5EG/EBNF specification\n- **Error Classes**: See `spec/error-classes.md` for error classification\n\n## Testing\n\nThe project includes comprehensive tests:\n\n- **Unit tests**: Core functionality tests in all crates\n- **Integration tests**: Round-trip and cross-crate tests\n- **Spec compatibility**: Tests verifying spec compliance\n- **Compliance tests**: Multi-language test suite (Rust, Python, TypeScript, C++)\n\n```bash\n# Run all tests\ncargo test\n\n# Run specific test suite\ncargo test --package lnmp-codec\ncargo test --package lnmp-core\ncargo test --package lnmp-sfe\ncargo test --package lnmp-llb\n\n# Run compliance tests\ncd tests/compliance/rust \u0026\u0026 cargo test\ncd tests/compliance/python \u0026\u0026 pytest\ncd tests/compliance/typescript \u0026\u0026 npm test\ncd tests/compliance/cpp \u0026\u0026 make test\n```\n\n## Performance\n\n**Zero-Copy Decoding (v0.5.15):**\n\n| Payload Size | Standard | Zero-Copy | Speedup |\n|--------------|----------|-----------|---------|\n| Small (3 fields) | 248 ns | 92 ns | **2.70x** |\n| Medium (7 fields) | 610 ns | 164 ns | **3.71x** |\n| Large (embeddings) | 18.3 μs | 2.1 μs | **8.91x** |\n\n**Throughput:**\n- Large records: **1.04 GiB/s** (zero-copy) vs 0.12 GiB/s (standard)\n- Batch processing: **3x faster** (1000 records in 55 μs)\n\n**Other Performance Metrics:**\n- No regex dependencies, hand-written lexer\n- Minimal allocations during parsing\n- SC32 checksum computation: \u003c1μs per field\n- Nested parsing: \u003c10μs for 3-level nesting\n- Token reduction: 7-12× vs JSON (ShortForm mode)\n- Delta encoding savings: \u003e50% for typical updates (v0.5)\n\n## Roadmap\n\n### v0.3 - Semantic Fidelity \u0026 Structural Extensibility ✅\n- Semantic checksums (SC32)\n- Nested structures (records and arrays)\n- Value normalization and equivalence mapping\n- Formal PEG/EBNF grammar\n- Multi-language compliance test suite\n- LLM-optimized encoding (explain mode, ShortForm)\n\n### v0.4 - Binary Protocol \u0026 Transport Layer ✅\n- Binary protocol format with version 0x04\n- VarInt encoding for space-efficient integers\n- Bidirectional text ↔ binary conversion\n- 30-50% size reduction compared to text format\n- Canonical binary form with deterministic field ordering\n- Type-safe binary encoding with explicit type tags\n- Round-trip conversion guarantees\n\n### v0.5 - Advanced Protocol \u0026 M2M Transport ✅ (Current)\n- **Binary Nested Structures (BNS)**: Recursive encoding with TypeTag 0x06/0x07\n- **Streaming Frame Layer (SFL)**: Chunked transmission with BEGIN/CHUNK/END frames\n- **Schema Negotiation Layer (SNL)**: Capability exchange and version negotiation\n- **Delta Encoding \u0026 Partial Update Layer (DPL)**: Bandwidth-efficient incremental updates\n- **LLM Optimization Layer v2 (LLB2)**: Enhanced context optimization with binary integration\n- Depth validation (default 32 levels)\n- Size limits for security\n- XOR checksum validation\n- Backpressure flow control\n- FID conflict detection\n- Type mismatch detection\n- 50%+ bandwidth savings with delta encoding\n\n### v0.6 (0.5.2) - LNMP-Spatial Protocol ✅ (Current)\n- **Spatial Type System**: Position, Rotation, Velocity, Acceleration, Quaternion, BoundingBox\n- **Binary Spatial Codec**: 2-3ns encode/decode latency\n- **Delta Encoding**: Position/Rotation deltas with 99% bandwidth reduction\n- **Hybrid Protocol**: Automatic ABS/DELTA mixing (1% ABS, 99% DELTA)\n- **Predictive Delta**: Dead reckoning for packet loss resilience\n- **Frame Integrity**: CRC32 checksums and nanosecond timestamps\n- **High Frequency**: Verified at 1kHz control loops\n- **Streaming Telemetry**: Continuous robot state transmission\n- **Safety Features**: Configurable prediction limits, drift correction\n- **Use Cases**: Robot control, autonomous vehicles, AR/VR, simulation\n\n#### LNMP-Spatial Quick Start\n\n```rust\nuse lnmp_spatial::protocol::{SpatialStreamer, SpatialStreamerConfig};\n\n// Configure hybrid protocol\nlet config = SpatialStreamerConfig {\n    abs_interval: 100,        // ABS frame every 100 frames (drift correction)\n    enable_prediction: true,   // Enable predictive delta (packet loss handling)\n    max_prediction_frames: 3,  // Max 3 consecutive predictions\n};\n\nlet mut streamer = SpatialStreamer::with_config(config);\n\n// Sender: Generate frames\nlet frame = streamer.next_frame(\u0026robot_state, timestamp_ns)?;\n// Automatically uses DELTA (99%) or ABS (1%) based on sequence\n\n// Receiver: Process frames\nlet state = streamer.process_frame(\u0026frame)?;\n// Automatically handles packet loss with prediction fallback\n```\n\n**Performance:**\n- Encode/Decode: **2-3 ns** per operation\n- Bandwidth: **99% reduction** with DELTA\n- Frequency: **1kHz** control loop verified\n- Packet Loss: **Predictive fallback** maintains smooth operation\n\n**Examples:**\n```bash\ncargo run -p lnmp-spatial --example robot        # Robot + Embedding integration\ncargo run -p lnmp-spatial --example stream       # Continuous telemetry\ncargo run -p lnmp-spatial --example jitter_sim   # 1kHz control loop\ncargo run -p lnmp-spatial --example reflex_sim   # Prediction vs non-prediction\n```\n\n\n### v0.7 (0.5.4) - Quantization \u0026 Compression ✅ (Current)\n- **Adaptive Quantization**: Auto-select scheme based on accuracy/compression targets\n- **FP16 Passthrough**: Near-lossless 2x compression (~99.9% accuracy)\n- **QInt8**: 4x compression with high fidelity (~99% similarity)\n- **QInt4**: 8x compression for balanced efficiency (~95% similarity)\n- **Binary**: 32x compression for maximum density (~85% similarity)\n- **Batch Processing**: Efficient API for processing multiple vectors\n- **Zero Overhead**: Adaptive selection compiles to direct calls\n\n#### LNMP-Quant Quick Start\n\n```rust\nuse lnmp_quant::adaptive::{quantize_adaptive, AccuracyTarget};\nuse lnmp_quant::batch::quantize_batch;\n\n// Adaptive: Select best scheme for target\nlet q = quantize_adaptive(\u0026emb, AccuracyTarget::High)?; // Uses QInt8\n\n// Batch: Process multiple vectors efficiently\nlet results = quantize_batch(\u0026embeddings, QuantScheme::QInt8);\nprintln!(\"Processed {} vectors in {:?}\", results.stats.total, results.stats.total_time);\n```\n\n**Performance:**\n- **Adaptive Overhead**: Zero (negligible)\n- **Batch Overhead**: \u003c150ns per vector\n- **Throughput**: \u003e2M vectors/sec (FP16)\n\n### v0.5.4 - Determinism \u0026 Generic Arrays ✅ (Current)\n- **Generic Arrays**: `IntArray`, `FloatArray`, `BoolArray` for efficient numeric data\n- **Strict Determinism**: `LnmpProfile` (Loose, Standard, Strict) for validation\n- **RecordBuilder**: Fluent API for canonical record construction\n- **Enhanced Validation**: Runtime field ordering checks\n- **Enhanced Validation**: Runtime field ordering checks\n- **Performance**: Sub-microsecond operations for all core types\n\n### v0.5.7 - LNMP-Net \u0026 Intelligent Routing ✅ (Current)\n- **Semantic Message Types**: Event, State, Command, Query, Alert\n- **QoS Primitives**: Priority (0-255) and TTL for network behavior\n- **ECO Profile**: Energy/Token Optimization routing (90% reduction in LLM calls)\n- **Intelligent Routing**: Route based on importance score + freshness\n- **Transport Integration**: Header mappings for HTTP, Kafka, NATS, gRPC\n\n## Migration Guide\n\n### From v0.4 to v0.5\n\nv0.5 is fully backward compatible with v0.4 and v0.3. All existing code continues to work without changes.\n\n**New Features (Optional):**\n- Binary nested structures: `BinaryNestedEncoder` and `BinaryNestedDecoder`\n- Streaming support: `StreamingEncoder` and `StreamingDecoder`\n- Schema negotiation: `SchemaNegotiator` with `Capabilities`\n- Delta encoding: `DeltaEncoder` and `DeltaDecoder`\n- Enhanced LLB2: `LlbConverter` with binary integration\n\n**No Breaking Changes:**\n- All v0.4 binary format remains valid\n- All v0.3 text format remains valid\n- Existing encoders/decoders work unchanged\n- New features are opt-in via configuration\n\n**Recommended Updates:**\n1. Use binary nested structures for hierarchical data\n2. Enable streaming for large payloads (\u003e4KB)\n3. Add schema negotiation for new integrations\n4. Use delta encoding for frequent updates\n5. Apply LLB2 flattening for LLM consumption\n\n**See `MIGRATION_V05.md` for detailed migration guide.**\n\n### From v0.2 to v0.3\n\nv0.3 is fully backward compatible with v0.2. All v0.2 code continues to work without changes.\n\n**New Features (Optional):**\n- Enable checksums: `EncoderConfig { enable_checksums: true, .. }`\n- Use nested structures: `LnmpValue::NestedRecord` and `LnmpValue::NestedArray`\n- Apply normalization: `EncoderConfig { normalization_config: Some(..), .. }`\n- Add equivalence mapping: `EncoderConfig { equivalence_mapper: Some(..), .. }`\n\nSee individual crate READMEs for detailed migration guides.\n\n## Contributing\n\nContributions are welcome! Please see the following:\n\n- **Issues**: Report bugs or request features on GitHub\n- **Pull Requests**: Submit PRs with tests and documentation\n- **Compliance Tests**: Add test cases to `tests/compliance/test-cases.yaml`\n- **Language Implementations**: Follow the compliance test suite for new languages\n\n## Acknowledgments\n\nLNMP is part of **Cloudflare Project Alexandria**.  \nCloudflare sponsors the **Cloudflare Pro plan** for the `lnmp.ai` domain under the program.\n\n## License\n\nMIT OR Apache-2.0\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flnmplang%2Flnmp-protocol","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flnmplang%2Flnmp-protocol","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flnmplang%2Flnmp-protocol/lists"}