{"id":39510133,"url":"https://github.com/creationix/n2","last_synced_at":"2026-01-18T06:01:20.610Z","repository":{"id":318570825,"uuid":"976954173","full_name":"creationix/n2","owner":"creationix","description":"Simple and Compact Random Access Serialization Format","archived":false,"fork":false,"pushed_at":"2026-01-05T04:20:32.000Z","size":320,"stargazers_count":3,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2026-01-07T22:36:25.754Z","etag":null,"topics":["binary-data","binary-serialization","persistent-data-structure","serialization","serialization-format","unstructured-data"],"latest_commit_sha":null,"homepage":"https://n2.fyi","language":"Lua","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/creationix.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-05-03T04:56:38.000Z","updated_at":"2026-01-05T04:20:36.000Z","dependencies_parsed_at":"2025-10-08T01:08:36.901Z","dependency_job_id":"81707e70-6bc2-428b-aa9a-a2cb9510d612","html_url":"https://github.com/creationix/n2","commit_stats":null,"previous_names":["creationix/n2"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/creationix/n2","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/creationix%2Fn2","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/creationix%2Fn2/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/creationix%2Fn2/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/creationix%2Fn2/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/creationix","download_url":"https://codeload.github.com/creationix/n2/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/creationix%2Fn2/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28531991,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-18T00:39:45.795Z","status":"online","status_checked_at":"2026-01-18T02:00:07.578Z","response_time":98,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["binary-data","binary-serialization","persistent-data-structure","serialization","serialization-format","unstructured-data"],"created_at":"2026-01-18T06:01:19.996Z","updated_at":"2026-01-18T06:01:20.604Z","avatar_url":"https://github.com/creationix.png","language":"Lua","readme":"# N₂: Data, Distilled\n\n![N2 Logo](www/n2logo.jpg)\n\n[![N2 LuaJIT Tests](https://github.com/creationix/n2/actions/workflows/lua-tests.yaml/badge.svg?event=push)](https://github.com/creationix/n2/actions/workflows/lua-tests.yaml)\n[![N2 Bun Tests](https://github.com/creationix/n2/actions/workflows/bun-tests.yaml/badge.svg?event=push)](https://github.com/creationix/n2/actions/workflows/bun-tests.yaml)\n\n**N₂** (Nitrogen) is a binary format designed for **instant access to massive datasets**. It enables applications—especially remote workers, edge functions, and CLI tools—to query gigabytes of data with **near-zero startup latency** by reading only the bytes strictly required for the query.\n\n### The Problem: Latency at Scale\nLoading large datasets (e.g., 500MB+) in serverless environments or remote workers is prohibitive. Traditional formats like JSON, MessagePack, or Protocol Buffers require **parsing the entire file** before you can access a single value. This causes massive CPU and memory spikes, killing cold-start performance.\n\n### The Solution: On-Demand Access\nN₂ eliminates this bottleneck using a **Reverse TLV** architecture and separate schema storage:\n\n- ⚡ **Zero-Parse Startup**: \"Open\" a 10GB file and read a nested value in microseconds.\n- 📡 **Network Efficient**: Fetch *only* the byte ranges needed for your query (perfect for HTTP Range requests).\n- 💾 **Lazy Loading**: Complete massive datasets can be accessed virtually, with data loaded bit-by-bit only when requested.\n- 🔄 **Incremental Updates**: Modify data by appending to the end of the file—no expensive rewrites.\n\n### Architecture: Dumb Server, Smart Client\nN₂ shifts the \"database engine\" from the server to the client. Instead of a running database process (Postgres, SQLite) that burns CPU parsing queries, N₂ allows you to host your data on **dumb, static storage** (S3, R2, CDN).\n\n- **Server**: Serves raw bytes via HTTP Range requests. Zero CPU overhead. Infinite scalability.\n- **Client**: Uses the cached N₂ index to know exactly which bytes to fetch.\n- **Result**: The first lookup takes ~2 RTTs. Every subsequent lookup is **1 RTT** (just fetching bytes), bypassing backend logic entirely.\n\n---\n\n## Use Cases\n\nN₂ is ideal for:\n\n- **Configuration Management**: Store app configs with atomic rollback capability\n- **Data Distribution**: Efficiently sync large datasets with incremental updates\n- **Caching**: Compact storage with fast random access to specific values\n- **Game Save Files**: Versioned, compact saves with instant load/rollback\n- **API Responses**: Bandwidth-efficient alternative to JSON with deduplication\n\n---\n\n## Implementations\n\nThis repository contains N₂ encoders in multiple languages:\n\n### TypeScript (Reference Implementation)\n\n- **Location**: `ts/n2.ts`\n- **Tests**: `ts/n2.test.ts`\n- **Runtime**: Bun\n- **Purpose**: Reference encoder with comprehensive test suite\n\nRun TypeScript tests:\n\n```bash\ncd ts \u0026\u0026 bun test\n```\n\n### Lua (Production Implementation)\n\n- **Location**: `lua/n2.lua`\n- **Tests**: `lua/n2.test.lua`\n- **Runtime**: LuaJIT\n- **Purpose**: High-performance encoder using FFI for native performance\n\nRun Lua tests:\n\n```bash\nluajit lua/n2.test.lua\n```\n\n---\n\n## Specification\n\nFor detailed information on the binary format, type system, and encoding rules, please see [SPEC.md](SPEC.md).\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcreationix%2Fn2","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcreationix%2Fn2","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcreationix%2Fn2/lists"}