{"id":41950048,"url":"https://github.com/paradedb/decimal-bytes","last_synced_at":"2026-02-06T00:12:34.919Z","repository":{"id":334287560,"uuid":"1140855572","full_name":"paradedb/decimal-bytes","owner":"paradedb","description":"Arbitrary precision decimals with lexicographically sortable byte encoding","archived":false,"fork":false,"pushed_at":"2026-02-03T03:39:46.000Z","size":172,"stargazers_count":7,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-02-03T10:47:08.780Z","etag":null,"topics":["encoding","numeric","postgresql","rust","search"],"latest_commit_sha":null,"homepage":"https://crates.io/crates/decimal-bytes","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/paradedb.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null},"funding":{"github":["paradedb"],"patreon":null,"open_collective":null,"ko_fi":null,"tidelift":null,"community_bridge":null,"liberapay":null,"issuehunt":null,"otechie":null,"custom":null}},"created_at":"2026-01-23T20:59:56.000Z","updated_at":"2026-02-03T03:39:50.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/paradedb/decimal-bytes","commit_stats":null,"previous_names":["paradedb/decimal-bytes"],"tags_count":3,"template":false,"template_full_name":null,"purl":"pkg:github/paradedb/decimal-bytes","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/paradedb%2Fdecimal-bytes","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/paradedb%2Fdecimal-bytes/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/paradedb%2Fdecimal-bytes/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/paradedb%2Fdecimal-bytes/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/paradedb","download_url":"https://codeload.github.com/paradedb/decimal-bytes/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/paradedb%2Fdecimal-bytes/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29140052,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-05T23:14:48.546Z","status":"ssl_error","status_checked_at":"2026-02-05T23:14:35.724Z","response_time":65,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["encoding","numeric","postgresql","rust","search"],"created_at":"2026-01-25T20:25:51.345Z","updated_at":"2026-02-06T00:12:34.911Z","avatar_url":"https://github.com/paradedb.png","language":"Rust","funding_links":["https://github.com/sponsors/paradedb"],"categories":[],"sub_categories":[],"readme":"# decimal-bytes\n\n[![Crates.io](https://img.shields.io/crates/v/decimal-bytes.svg)](https://crates.io/crates/decimal-bytes)\n[![codecov](https://codecov.io/gh/paradedb/decimal-bytes/graph/badge.svg)](https://codecov.io/gh/paradedb/decimal-bytes)\n[![CI](https://github.com/paradedb/decimal-bytes/actions/workflows/ci.yml/badge.svg)](https://github.com/paradedb/decimal-bytes/actions/workflows/ci.yml)\n[![Documentation](https://docs.rs/decimal-bytes/badge.svg)](https://docs.rs/decimal-bytes)\n[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](https://opensource.org/licenses/MIT)\n\nArbitrary precision decimals with lexicographically sortable byte encoding.\n\n## Overview\n\nThis crate provides three decimal types optimized for database storage:\n\n- **`Decimal`**: Variable-length arbitrary precision (up to 131,072 digits)\n- **`Decimal64`**: Fixed 8-byte representation with embedded scale (precision ≤ 16 digits)\n- **`Decimal64NoScale`**: Fixed 8-byte representation with external scale (precision ≤ 18 digits)\n\nAll types support PostgreSQL special values (NaN, ±Infinity) with correct sort ordering.\n\n**Why not use `rust_decimal` or `bigdecimal`?** Those libraries are excellent for arithmetic, but their byte representations are not lexicographically sortable. You cannot compare their serialized bytes to determine numerical order - you must deserialize first. `decimal-bytes` solves this by providing a byte encoding where `bytes(a) \u003c bytes(b)` if and only if `a \u003c b` numerically.\n\n## When to Use Which\n\n| Type | Precision | Scale | Storage | Best For |\n|------|-----------|-------|---------|----------|\n| `Decimal64NoScale` | ≤ **18** digits | External | 8 bytes | **Columnar storage, aggregates** |\n| `Decimal64` | ≤ 16 digits | Embedded | 8 bytes | Self-contained values |\n| `Decimal` | Unlimited | Unlimited | Variable | Scientific, very large numbers |\n\n## Features\n\n- **Three storage options**: Fixed 8-byte (`Decimal64`, `Decimal64NoScale`) or variable-length (`Decimal`)\n- **Columnar-friendly**: `Decimal64NoScale` enables correct aggregates with external scale\n- **Lexicographic ordering**: Byte comparison matches numerical comparison\n- **PostgreSQL NUMERIC compatibility**: Full support for precision, scale (including negative), and special values\n- **Special values**: Infinity, -Infinity, and NaN with correct PostgreSQL sort order\n\n## Decimal64 Usage\n\nFor most financial and business applications where precision ≤ 16 digits:\n\n```rust\nuse decimal_bytes::Decimal64;\n\n// Create with scale\nlet price = Decimal64::new(\"99.99\", 2).unwrap();\nassert_eq!(price.to_string(), \"99.99\");\nassert_eq!(price.scale(), 2);\n\n// Parse with automatic scale detection\nlet d: Decimal64 = \"123.456\".parse().unwrap();\nassert_eq!(d.scale(), 3);\n\n// Access raw components\nlet value = price.value();  // 9999 (scaled integer)\nlet scale = price.scale();  // 2\n\n// Special values (PostgreSQL compatible)\nlet inf = Decimal64::infinity();\nlet neg_inf = Decimal64::neg_infinity();\nlet nan = Decimal64::nan();\n\n// Correct sort order: -Infinity \u003c numbers \u003c +Infinity \u003c NaN\nassert!(neg_inf \u003c price);\nassert!(price \u003c inf);\nassert!(inf \u003c nan);\n\n// NaN equals NaN (PostgreSQL semantics)\nassert_eq!(nan, Decimal64::nan());\n```\n\n### Decimal64 with Precision and Scale (PostgreSQL NUMERIC)\n\n`Decimal64` fully supports PostgreSQL's `NUMERIC(precision, scale)` semantics:\n\n```rust\nuse decimal_bytes::Decimal64;\n\n// NUMERIC(5, 2) - up to 5 digits total, 2 after decimal\nlet d = Decimal64::with_precision_scale(\"123.456\", Some(5), Some(2)).unwrap();\nassert_eq!(d.to_string(), \"123.46\"); // Rounded to 2 decimal places\n\n// Precision overflow - truncates from left (PostgreSQL behavior)\nlet d = Decimal64::with_precision_scale(\"12345.67\", Some(5), Some(2)).unwrap();\nassert_eq!(d.to_string(), \"345.67\"); // Keeps rightmost 5 digits\n\n// NUMERIC(2, -3) - negative scale rounds to powers of 10\nlet d = Decimal64::with_precision_scale(\"12345\", Some(2), Some(-3)).unwrap();\nassert_eq!(d.to_string(), \"12000\"); // Rounded to nearest 1000\n```\n\n### Decimal64 Storage Layout\n\n```text\n64-bit packed representation:\n┌──────────────────┬─────────────────────────────────────────────────────┐\n│ Scale (8 bits)   │ Value (56 bits, signed)                             │\n│ Byte 0           │ Bytes 1-7                                           │\n└──────────────────┴─────────────────────────────────────────────────────┘\n```\n\n- **Scale byte**: 0-18 for normal values, 253/254/255 for -Infinity/+Infinity/NaN\n- **Value**: 56-bit signed integer (-2^55 to 2^55-1, ~16 significant digits)\n\n### Decimal64 Benefits\n\n- **Fixed 8 bytes**: Predictable storage, no heap allocation, cache-friendly\n- **PostgreSQL compatible**: Full NUMERIC(p,s) semantics including NaN, ±Infinity\n- **Fast operations**: Single i64 comparison and serialization\n\n## Decimal64NoScale Usage (Recommended for Columnar Storage)\n\n`Decimal64NoScale` stores the raw scaled value without embedding the scale, enabling:\n- **18 digits of precision** (vs 16 for Decimal64)\n- **Correct aggregates** (SUM, MIN, MAX work directly on raw i64 values)\n- **Columnar storage compatibility** (scale stored once in schema metadata)\n\n```rust\nuse decimal_bytes::Decimal64NoScale;\n\n// Scale is provided externally (e.g., from schema metadata)\nlet scale = 2;\nlet a = Decimal64NoScale::new(\"100.50\", scale).unwrap();\nlet b = Decimal64NoScale::new(\"200.25\", scale).unwrap();\n\n// Raw values can be summed directly!\nlet sum = a.value() + b.value();  // 30075\nassert_eq!(sum, 30075);\n\n// Interpret result with scale\nlet result = Decimal64NoScale::from_raw(sum);\nassert_eq!(result.to_string_with_scale(scale), \"300.75\");\n\n// 18 digits supported (more than Decimal64's 16)\nlet big = Decimal64NoScale::new(\"123456789012345678\", 0).unwrap();\nassert_eq!(big.value(), 123456789012345678);\n```\n\n### Why Decimal64NoScale for Aggregates?\n\n`Decimal64` embeds scale in the i64, which **corrupts aggregate results**:\n\n```text\nDecimal64:        packed = (scale \u003c\u003c 56) | mantissa\n                  SUM(a, b) = adds scale bits → WRONG!\n\nDecimal64NoScale: stored = value * 10^scale\n                  SUM(a, b) = (a+b)*scale → divide by scale → CORRECT!\n```\n\n### Decimal64NoScale Storage Layout\n\n```text\n64-bit representation:\n┌─────────────────────────────────────────────────────────────────┐\n│ Value (64 bits, signed) - represents value * 10^scale           │\n└─────────────────────────────────────────────────────────────────┘\n```\n\n- **Value**: Full 64-bit signed integer (±9.99×10^17, ~18 significant digits)\n- **Scale**: Stored externally (e.g., in database schema)\n- **Special values**: `i64::MIN` (NaN), `i64::MIN+1` (-Infinity), `i64::MAX` (+Infinity)\n\n## Decimal Usage (Arbitrary Precision)\n\n```rust\nuse decimal_bytes::Decimal;\n\n// Create decimals from strings\nlet a = Decimal::from_str(\"123.456\").unwrap();\nlet b = Decimal::from_str(\"123.457\").unwrap();\n\n// Byte comparison matches numerical comparison\nassert!(a.as_bytes() \u003c b.as_bytes());\nassert!(a \u003c b);\n\n// With precision and scale constraints (SQL NUMERIC semantics)\nlet d = Decimal::with_precision_scale(\"123.456\", Some(10), Some(2)).unwrap();\nassert_eq!(d.to_string(), \"123.46\"); // Rounded to 2 decimal places\n\n// Negative scale (rounds to left of decimal point)\nlet d = Decimal::with_precision_scale(\"12345\", Some(10), Some(-3)).unwrap();\nassert_eq!(d.to_string(), \"12000\"); // Rounded to nearest 1000\n\n// Efficient byte access (primary representation)\nlet bytes: \u0026[u8] = d.as_bytes();\n\n// Reconstruct from bytes\nlet restored = Decimal::from_bytes(bytes).unwrap();\nassert_eq!(d, restored);\n```\n\n## Special Values\n\nPostgreSQL-compatible special values with correct sort ordering:\n\n```rust\nuse decimal_bytes::Decimal;\n\n// Create special values\nlet pos_inf = Decimal::infinity();\nlet neg_inf = Decimal::neg_infinity();\nlet nan = Decimal::nan();\n\n// Or parse from strings (case-insensitive)\nlet inf = Decimal::from_str(\"Infinity\").unwrap();\nlet inf = Decimal::from_str(\"inf\").unwrap();\nlet nan = Decimal::from_str(\"NaN\").unwrap();\n\n// Check for special values\nassert!(pos_inf.is_infinity());\nassert!(pos_inf.is_pos_infinity());\nassert!(neg_inf.is_neg_infinity());\nassert!(nan.is_nan());\nassert!(!pos_inf.is_finite());\n\n// Sort order: -Infinity \u003c negatives \u003c zero \u003c positives \u003c Infinity \u003c NaN\nassert!(neg_inf \u003c Decimal::from_str(\"-1000000\").unwrap());\nassert!(Decimal::from_str(\"1000000\").unwrap() \u003c pos_inf);\nassert!(pos_inf \u003c nan);\n```\n\n### PostgreSQL vs IEEE 754 Semantics\n\nThis library follows **PostgreSQL semantics** for special values, which differ from IEEE 754 floating-point:\n\n| Behavior | PostgreSQL / decimal-bytes | IEEE 754 float |\n|----------|---------------------------|----------------|\n| `NaN == NaN` | `true` | `false` |\n| `NaN` ordering | Greatest value (\u003e Infinity) | Unordered |\n| `Infinity == Infinity` | `true` | `true` |\n\n```rust\nuse decimal_bytes::Decimal;\n\nlet nan1 = Decimal::nan();\nlet nan2 = Decimal::nan();\nlet inf = Decimal::infinity();\n\n// NaN equals itself (PostgreSQL behavior, unlike IEEE 754)\nassert_eq!(nan1, nan2);\n\n// NaN is greater than everything, including Infinity\nassert!(nan1 \u003e inf);\n```\n\nThis makes `Decimal` suitable for use in indexes, sorting, and deduplication where consistent ordering and equality semantics are required.\n\n## PostgreSQL Compatibility\n\nThis crate implements the PostgreSQL NUMERIC specification:\n\n| Feature | Support |\n|---------|---------|\n| Max digits before decimal | 131,072 |\n| Max digits after decimal | 16,383 |\n| Precision constraint | ✓ |\n| Scale constraint (positive) | ✓ |\n| Scale constraint (negative) | ✓ |\n| Infinity | ✓ |\n| -Infinity | ✓ |\n| NaN | ✓ |\n| Rounding (ties away from zero) | ✓ |\n\n## Storage Efficiency\n\nThe encoding matches PostgreSQL's storage efficiency (2 bytes per 4 decimal digits):\n\n- 1 byte for sign\n- 2 bytes for exponent  \n- ~N/2 bytes for N-digit mantissa (BCD encoding: 2 digits per byte)\n- Special values: 3 bytes each\n\nExample: A 9-digit number like `123456789` requires only ~8 bytes total.\n\n## Sort Order\n\nThe lexicographic byte order matches the PostgreSQL NUMERIC sort order:\n\n```\n-Infinity \u003c negative numbers \u003c zero \u003c positive numbers \u003c +Infinity \u003c NaN\n```\n\nThis enables efficient range queries in sorted key-value stores without decoding.\n\n## Performance\n\n### Type Comparison Summary\n\n| Type | Max Precision | Parse | Aggregates | Best For |\n|------|---------------|-------|------------|----------|\n| `Decimal64NoScale` | **18 digits** | ~85 µs/1000 | **✓ Correct, 17 Gelem/s** | Columnar storage |\n| `Decimal64` | 16 digits | ~136 µs/1000 | ✗ Wrong (scale corrupts) | Self-contained values |\n| `Decimal` | Unlimited | ~134 µs/1000 | N/A | Arbitrary precision |\n\n### Memory Usage\n\n| Type | Stack | Heap | Total |\n|------|-------|------|-------|\n| Decimal64NoScale | 8 bytes | 0 | **8 bytes** |\n| Decimal64 | 8 bytes | 0 | **8 bytes** |\n| Decimal | 24 bytes | ~9 bytes | ~33 bytes |\n\n### Decimal64NoScale Operations (Recommended for Columnar)\n\n| Operation | Time | Notes |\n|-----------|------|-------|\n| Parse (`new`) | 60-85 ns | Scales with digit count |\n| `to_string_with_scale()` | 18-25 ns | Scales with digit count |\n| `from_raw()` | **\u003c1 ns** | Trivial (just wrap i64) |\n| Equality (`==`) | **\u003c1 ns** | Direct i64 comparison |\n| SUM 1000 values | **~59 ns** | 17 Gelem/s - just sum raw i64s |\n| MIN/MAX 1000 values | **~230 ns** | 4.3 Gelem/s - direct comparison |\n| `to_be_bytes()` | \u003c1 ns | Trivial conversion |\n| `from_be_bytes()` | \u003c1 ns | Trivial conversion |\n\n### Decimal64 Operations\n\n| Operation | Time | Notes |\n|-----------|------|-------|\n| Parse (`new`) | 64-71 ns | Scales with digit count |\n| `to_string()` | 19-88 ns | Scales with digit count |\n| Equality (`==`) | 0.5 ns | Single i64 comparison |\n| Comparison (same scale) | 1.6 ns | Direct value comparison |\n| Comparison (diff scale) | 2 ns | Requires normalization |\n| `to_be_bytes()` | 0.9 ns | Trivial conversion |\n| `from_be_bytes()` | 0.8 ns | Trivial conversion |\n| `is_nan()` / `is_infinity()` | 0.3 ns | Fast special value checks |\n\n### Decimal Operations (Arbitrary Precision)\n\n| Operation | Time | Notes |\n|-----------|------|-------|\n| Byte comparison | ~4 ns | The key use case - compare without decoding |\n| `from_str` (parse) | 84-312 ns | Scales with digit count |\n| `to_string` | 61-89 ns | Scales with digit count |\n| `from_bytes` | 58-261 ns | With validation |\n| `from_bytes_unchecked` | ~15 ns | Skip validation if bytes are trusted |\n| `is_nan()` / `is_infinity()` | ~1.3 ns | Fast special value checks |\n\n### Aggregate Performance (Key Differentiator)\n\nFor columnar storage where aggregates are important:\n\n| Operation | Decimal64NoScale | Decimal64 | Speedup |\n|-----------|------------------|-----------|---------|\n| SUM 1000 values | **59 ns** (17 Gelem/s) | 275 ns (3.6 Gelem/s) | **4.7x** |\n| MIN/MAX 1000 values | **230 ns** (4.3 Gelem/s) | 1001 ns (1 Gelem/s) | **4.3x** |\n| Create 1000 values | **85 µs** | 136 µs | **1.6x** |\n| Results correct? | **✓ Yes** | **✗ No** | - |\n\n**Why is Decimal64NoScale faster?**\n- `Decimal64NoScale.value()` returns raw i64 directly\n- `Decimal64.value()` must unpack/mask the 56-bit value from the packed format\n\nRun `cargo bench` locally to reproduce benchmarks on your hardware.\n\n## Arithmetic Operations\n\nThis library focuses on storage and comparison, not arithmetic. Existing Rust decimal libraries (`rust_decimal`, `bigdecimal`) provide arithmetic but their byte representations are **not lexicographically sortable** - you cannot compare their serialized bytes to determine numerical order. That's the gap `decimal-bytes` fills: efficient storage with byte-level ordering for databases and search engines.\n\nFor calculations, use an established decimal library and convert:\n\n### With `rust_decimal` (recommended for most use cases)\n\n```toml\n[dependencies]\ndecimal-bytes = { version = \"0.1\", features = [\"rust_decimal\"] }\n```\n\n```rust\nuse rust_decimal::Decimal as RustDecimal;\nuse decimal_bytes::Decimal;\n\n// Convert from rust_decimal for storage\nlet rd = RustDecimal::new(12345, 2); // 123.45\nlet stored: Decimal = rd.try_into().unwrap();\n\n// Do arithmetic with rust_decimal\nlet a: RustDecimal = (\u0026stored).try_into().unwrap();\nlet b = RustDecimal::new(1000, 2); // 10.00\nlet sum = a + b; // 133.45\n\n// Convert back for storage\nlet result: Decimal = sum.try_into().unwrap();\n```\n\n### With `bigdecimal` (for arbitrary precision arithmetic)\n\n```toml\n[dependencies]\ndecimal-bytes = { version = \"0.1\", features = [\"bigdecimal\"] }\n```\n\n```rust\nuse bigdecimal::BigDecimal;\nuse decimal_bytes::Decimal;\nuse std::str::FromStr;\n\n// Convert between types\nlet bd = BigDecimal::from_str(\"123.456789012345678901234567890\").unwrap();\nlet stored: Decimal = bd.try_into().unwrap();\nlet restored: BigDecimal = (\u0026stored).try_into().unwrap();\n```\n\n## License\n\nMIT License - see [LICENSE](LICENSE) for details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fparadedb%2Fdecimal-bytes","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fparadedb%2Fdecimal-bytes","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fparadedb%2Fdecimal-bytes/lists"}