{"id":25424397,"url":"https://github.com/justrach/dhi","last_synced_at":"2026-07-06T15:31:12.673Z","repository":{"id":318006725,"uuid":"1069464598","full_name":"justrach/dhi","owner":"justrach","description":"⚡ Ultra-fast data validation for Python \u0026 TypeScript. Drop-in Pydantic/Zod replacement powered by Zig + SIMD WASM.","archived":false,"fork":false,"pushed_at":"2026-06-12T13:27:00.000Z","size":4242,"stargazers_count":376,"open_issues_count":3,"forks_count":6,"subscribers_count":1,"default_branch":"main","last_synced_at":"2026-06-12T14:06:52.463Z","etag":null,"topics":["bun","python","typescript","zig"],"latest_commit_sha":null,"homepage":"https://trilok.ai","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/justrach.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-10-04T01:48:41.000Z","updated_at":"2026-06-12T13:27:03.000Z","dependencies_parsed_at":"2025-10-04T14:19:10.971Z","dependency_job_id":"2b20c8dd-5279-431c-ab8b-7e2eb396f10d","html_url":"https://github.com/justrach/dhi","commit_stats":null,"previous_names":["justrach/satya-zig","justrach/dhi-zig","justrach/dhi"],"tags_count":28,"template":false,"template_full_name":null,"purl":"pkg:github/justrach/dhi","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/justrach%2Fdhi","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/justrach%2Fdhi/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/justrach%2Fdhi/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/justrach%2Fdhi/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/justrach","download_url":"https://codeload.github.com/justrach/dhi/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/justrach%2Fdhi/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":35197534,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-07-06T02:00:07.184Z","response_time":106,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bun","python","typescript","zig"],"created_at":"2025-02-16T23:16:00.774Z","updated_at":"2026-07-06T15:31:12.657Z","avatar_url":"https://github.com/justrach.png","language":"TypeScript","funding_links":[],"categories":["\u003ca name=\"TypeScript\"\u003e\u003c/a\u003eTypeScript"],"sub_categories":[],"readme":"\u003cp align=\"center\"\u003e\n  \u003cimg src=\"assets/logo.png\" alt=\"dhi logo\" width=\"200\"/\u003e\n\u003c/p\u003e\n\n# dhi\n\n**33M Python batch rows/sec. 461M direct int validations/sec. 13.8x faster than Zod 4 on the TypeScript suite. Same API.**\n\nOne validation core. Three ecosystems. Zero compromise.\n\n[![npm](https://img.shields.io/npm/v/dhi)](https://www.npmjs.com/package/dhi)\n[![PyPI](https://img.shields.io/pypi/v/dhi)](https://pypi.org/project/dhi/)\n[![MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)\n\n```\nPython:     33.2M rows/sec on name+email+age batches; 461M ints/sec direct range batches\nTypeScript: 13.8x faster than Zod 4 on the full suite; URL and IPv4 hot paths are 2.9x/2.4x faster in 1.3.1\nZig:        Zero-cost. Comptime. No runtime.\n```\n\n---\n\n## What is dhi?\n\n**dhi** (Sanskrit: *intellect, insight*) is a validation library with a single Zig core compiled to three targets, so the *same* validation rules and error semantics run everywhere you do:\n\n- **Python** — a drop-in [Pydantic](https://docs.pydantic.dev) replacement (`BaseModel`, `Field`, 80+ types) backed by a native C extension over SIMD batch validators.\n- **TypeScript** — a drop-in [Zod 4](https://zod.dev) replacement (`z.object`, full API parity) running on a 28KB SIMD WASM module in browsers/edge, or an N-API native addon on Node.js.\n- **Zig** — `comptime`-generated models with zero runtime cost, zero allocations on the happy path.\n\nYou keep your existing code and API; dhi swaps in the engine underneath. No schema rewrites, no new mental model — and **zero runtime dependencies** on either registry.\n\n### Contents\n\n- [Drop-in usage](#you-dont-have-to-change-your-code) — Python, TypeScript, Zig\n- [Benchmarks](#the-numbers) — the numbers, vs Zod 4 and the Python field\n- [Install](#install)\n- [Repo structure](#repo-structure)\n- [Zod 4 feature parity](#full-zod-4-feature-parity)\n- [Pydantic-compatible types](#80-pydantic-compatible-types)\n- [Why is it this fast?](#why-is-it-this-fast)\n- [Run the benchmarks yourself](#run-the-benchmarks-yourself)\n\n---\n\n## You don't have to change your code.\n\n### Python — drop-in Pydantic replacement\n\n```python\nfrom dhi import BaseModel, Field, EmailStr\nfrom typing import Annotated\n\nclass User(BaseModel):\n    name: Annotated[str, Field(min_length=1, max_length=100)]\n    email: EmailStr\n    age: Annotated[int, Field(gt=0, le=150)]\n\nuser = User(name=\"Alice\", email=\"alice@example.com\", age=28)\nuser.model_dump()   # {'name': 'Alice', 'email': 'alice@example.com', 'age': 28}\n```\n\n### TypeScript — drop-in Zod 4 replacement\n\n```diff\n- import { z } from 'zod';\n+ import { z } from 'dhi';\n```\n\n```typescript\nconst User = z.object({\n  id: z.string().uuid(),\n  name: z.string().min(1).max(100),\n  email: z.email(),              // Zod 4 top-level shortcuts\n  age: z.int().positive(),       // Zod 4 number formats\n  createdAt: z.iso.datetime(),   // Zod 4 ISO namespace\n});\n\nconst user = User.parse(data); // same API, 13.8x faster on the benchmark suite\n```\n\n**Two backends — same API:**\n\n| Backend | Import | Best for |\n|---------|--------|----------|\n| WASM (default) | `import { z } from 'dhi'` | Browsers, edge runtimes, Cloudflare Workers |\n| N-API native | `import { z } from 'dhi/napi'` | Node.js servers — 1.7–2x faster on string formats |\n\n\n### Zig — compile-time validated, zero overhead\n\n```zig\nconst dhi = @import(\"model\");\n\nconst User = dhi.Model(\"User\", .{\n    .name = dhi.Str(.{ .min_length = 1, .max_length = 100 }),\n    .email = dhi.EmailStr,\n    .age = dhi.Int(i32, .{ .gt = 0, .le = 150 }),\n});\n\nconst user = try User.parse(.{\n    .name = \"Alice\",\n    .email = \"alice@example.com\",\n    .age = @as(i32, 28),\n});\n// Validation is inlined at compile time. Zero allocations. Zero dispatch.\n```\n\n**Same validation rules. Same error behavior. Three languages. One core.**\n\n---\n\n## The Numbers\n\n### TypeScript (vs Zod 4)\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"docs/benchmarks/speedup-all.png\" alt=\"dhi vs Zod Performance\" width=\"800\"/\u003e\n\u003c/p\u003e\n\n| Category | Speedup vs Zod 4 |\n|----------|------------------|\n| Overall benchmark suite | **13.8x faster** |\n| URL validator | **30.17M/s** — **8.0x vs Zod**; **2.9x faster than pre-1.3.1** |\n| IPv4 validator | **22.28M/s** — **1.1x vs Zod**; **2.4x faster than pre-1.3.1** |\n| Number Formats | **30-50x faster** |\n| StringBool | **32x faster** |\n| Coercion | **23-56x faster** |\n| String Formats | **12-27x faster** |\n| ISO Formats | **12-22x faster** |\n| Objects | **4-7x faster** |\n| Arrays | **8x faster** |\n\n\u003e Release 1.3.1 benchmark command: `bun run benchmarks/benchmark-vs-all.ts`. Pre-1.3.1 comparison used the previous release-branch commit before the TypeScript fast paths.\n\n### Python\n\n| Workload | Throughput | Improvement |\n|----------|------------|-------------|\n| `name+email+age` batch | **33.18M rows/sec** | **~28% faster** than the earlier 26M rows/sec baseline |\n| `name+email+age+url` batch | **25.35M rows/sec** | **~27% faster** than the earlier 20M rows/sec baseline |\n| `name+email+age+url+uuid` batch | **20.51M rows/sec** | **~28% faster** than the earlier 16M rows/sec baseline |\n| `all 6 (+ ipv4)` batch | **16.13M rows/sec** | **~47% faster** than the earlier 11M rows/sec baseline |\n| UUID-only batch | **112.78M rows/sec** | Vectorized UUID hot path |\n| IPv4-only batch | **68.98M rows/sec** | C-inlined IPv4 hot path |\n| Direct int range list | **461.19M ints/sec** | New native list batch path |\n\nBaseModel layer: 546K model_validate/sec | 6.4M model_dump/sec\n\n\u003e **Note on msgspec:** Plain msgspec (~5.8M/sec) does type-checked JSON decoding but lacks field-level validators (email, URL, positive int). msgspec-ext adds those 26 validators via Python `dec_hook` callbacks, bringing it to ~777K/sec — a fairer apples-to-apples comparison with dhi's validated parsing.\n\n---\n\n## Install\n\n```bash\npip install dhi          # Python 3.9–3.14 (incl. free-threading); wheels for macOS arm64 + Linux x86_64/aarch64\nnpm install dhi          # TypeScript (Node 18+ / Bun / Deno)\n```\n\n**Zero runtime dependencies** on both PyPI and npm. A pure-Python fallback is bundled, so `pip install dhi` works even where the native extension can't build.\n\n---\n\n## Repo Structure\n\ndhi is a Zig core with bindings for JS and Python:\n\n```\nsrc/              → Zig core: validators, SIMD, C API, WASM API (the engine)\njs-bindings/      → npm package: Zod 4 drop-in replacement\npython-bindings/  → PyPI package: Pydantic drop-in replacement\ndocs/             → Benchmark charts, shared docs\n.github/          → CI, release workflows\n```\n\n**Which package should I use?**\n- **TypeScript/JS**: `npm install dhi` — replaces Zod, works in browsers + edge + Node.js\n- **Python**: `pip install dhi` — replaces Pydantic-style validation with 33M+ row/sec native batches\n- **Zig**: Import `src/` directly — zero-cost comptime validation\n\n## Full Zod 4 Feature Parity\n\ndhi implements **100% of the Zod 4 API**, including all new Zod 4 features:\n\n### Top-Level String Format Shortcuts (New in Zod 4)\n\n```typescript\nz.email()      z.uuid()       z.url()        z.ipv4()       z.ipv6()\nz.jwt()        z.nanoid()     z.ulid()       z.cuid()       z.cuid2()\nz.base64()     z.e164()       z.mac()        z.cidrv4()     z.cidrv6()\nz.hex()        z.hostname()   z.hash('sha256')\n```\n\n### ISO Namespace \u0026 Number Formats (New in Zod 4)\n\n```typescript\n// ISO namespace\nz.iso.datetime()  z.iso.date()  z.iso.time()  z.iso.duration()\n\n// Number formats\nz.int()     z.float()    z.float32()   z.float64()\nz.int8()    z.uint8()    z.int16()     z.uint16()\nz.int32()   z.uint32()   z.int64()     z.uint64()\n```\n\n### Additional Zod 4 Features\n\n```typescript\nz.stringbool()                           // \"true\"/\"yes\"/\"1\" → true\nz.templateLiteral(['user-', z.number()]) // Template literal types\nz.json()                                 // Recursive JSON schema\nz.file().mime('image/png').max(5_000_000) // File validation\nz.registry()                             // Schema registry\nz.prettifyError(error)                   // Pretty error formatting\n```\n\n---\n\n## 80+ Pydantic-compatible types\n\n| Category | Types |\n|----------|-------|\n| **Model** | `BaseModel`, `Field()`, `@field_validator`, `@model_validator` |\n| **Numeric** | `PositiveInt`, `NegativeFloat`, `FiniteFloat`, `conint()`, `confloat()` |\n| **String** | `EmailStr`, `constr()`, pattern, length, strip/lower/upper transforms |\n| **Network** | `HttpUrl`, `AnyUrl`, `PostgresDsn`, `RedisDsn`, `MongoDsn`, +8 DSN types |\n| **Special** | `UUID4`, `FilePath`, `Base64Str`, `Json`, `ByteSize`, `SecretStr` |\n| **Datetime** | `PastDate`, `FutureDate`, `AwareDatetime`, `NaiveDatetime` |\n| **Constraints** | `Gt`, `Ge`, `Lt`, `Le`, `MultipleOf`, `MinLength`, `MaxLength`, `Pattern` |\n\nFull `model_validate()`, `model_dump()`, `model_dump_json()`, `model_json_schema()`, `model_copy()` support.\n\n---\n\n## Why is it this fast?\n\ndhi is written in [Zig](https://ziglang.org) — a systems language with compile-time code generation, no garbage collector, and direct hardware access. The same source compiles to:\n\n| Target | What it does |\n|--------|-------------|\n| `libdhi.dylib/.so` | Python C extension — extracts from dicts, no copies |\n| `dhi.wasm` (28KB) | TypeScript WASM — 128-bit SIMD, works in browsers + edge |\n| `dhi_native.node` | TypeScript N-API — direct native calls, 1.7–2x faster on Node.js |\n| Native `.zig` import | Zig — zero-cost comptime validation, fully inlined |\n\n**Key tricks:**\n- **Comptime models** — Validation logic is generated at compile time. No vtables, no reflection, no hash lookups.\n- **SIMD batch validation** — Process 4 values per cycle on 256-bit vectors.\n- **Single FFI call** — Python batch validation crosses the FFI boundary once, not per-item.\n- **N-API over WASM** — For Node.js, the native addon skips alloc/encode/dealloc per string call: 1.7–2x faster for format validators (url, ipv4, datetime).\n- **No allocations** — The happy path never allocates. Errors are stack-returned.\n\n```\n┌─────────────────────────────────────────────────────┐\n│            Zig Core (comptime + SIMD)                │\n└──────┬────────────────┬──────────┬────────────────┬──┘\n       │                │          │                │\n  ┌────▼─────┐    ┌────▼────┐ ┌───▼──────┐    ┌───▼──────┐\n  │  Python  │    │  WASM   │ │  N-API   │    │   Zig    │\n  │  C ext   │    │  28KB   │ │  .node   │    │  Native  │\n  └────┬─────┘    └────┬────┘ └───┬──────┘    └───┬──────┘\n       │                │          │                │\n  BaseModel        z.object()  z.object()      Model()\n  Pydantic API    Zod 4 (edge) Zod 4 (Node)  comptime API\n```\n\n---\n\n## Run the benchmarks yourself\n\n```bash\n# Python — full comparison (dhi vs msgspec vs msgspec-ext vs satya vs Pydantic)\ngit clone https://github.com/justrach/dhi.git \u0026\u0026 cd dhi\ncd python-bindings\npip install -e .\npip install msgspec msgspec-ext 'pydantic[email]' satya\npython benchmarks/benchmark_vs_all.py\n\n# TypeScript — dhi vs Zod 4\ncd js-bindings \u0026\u0026 bun install \u0026\u0026 bun run benchmark-vs-zod.ts\n\n# Zig\nzig build bench -Doptimize=ReleaseFast\n```\n\n### Methodology\n\nThe Python benchmark validates **10,000 user objects** with 4 field-level validators each:\n\n| Field | Validator |\n|-------|-----------|\n| `name` | string, min_length=1, max_length=100 |\n| `email` | RFC 5321 email format |\n| `age` | positive integer |\n| `website` | URL format |\n\nEach library uses its idiomatic equivalent:\n- **dhi**: `_dhi_native.validate_batch_direct()` — single FFI call, Zig SIMD validators\n- **msgspec**: `msgspec.json.Decoder` — C-level JSON decode + type check (no format validators)\n- **msgspec-ext**: `msgspec.json.decode()` with `dec_hook` — adds EmailStr, HttpUrl, PositiveInt validators on top of msgspec\n- **satya**: `model_validate_json_array_bytes()` — Rust JSON parse + validation\n- **Pydantic v2**: `model_validate()` per item — Rust-backed, with EmailStr + Field constraints\n\nTiming uses `time.perf_counter()` over 5 runs after a warmup pass. Results are averaged. Tested on Python 3.13 and 3.14 (no significant difference — hot paths are native code).\n\n---\n\n## The Experiment\n\ndhi started as a question: *can Zig's type system unify validation across language boundaries?*\n\nThe hypothesis: Zig's `comptime` can generate the same validation semantics that Pydantic builds with metaclasses and Zod builds with method chains — but at compile time, with zero runtime cost, targeting any platform via its C ABI and WASM backends.\n\n**Results:**\n\n| Claim | Status |\n|-------|--------|\n| Pydantic-level DX in Zig | `Model(\"User\", .{ .name = Str(.{}) })` — yes |\n| One core, three ecosystems | Python FFI + WASM + native Zig — yes |\n| 10-100x faster | 523x (Python), 20x avg (TypeScript) — yes |\n| Reasonable binary size | 28KB WASM, ~200KB native — yes |\n| Comptime replaces reflection | No runtime type inspection needed — yes |\n\n---\n\n## License\n\nMIT\n\n---\n\n**dhi** — the fastest validation library for every language you use.\n\n## Contributing\n\nSee [CONTRIBUTING.md](CONTRIBUTING.md) for Rach's Agentic Contribution Template before opening a PR.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjustrach%2Fdhi","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjustrach%2Fdhi","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjustrach%2Fdhi/lists"}