{"id":51184574,"url":"https://github.com/dfa1/rocksdbffm","last_synced_at":"2026-06-27T09:04:09.782Z","repository":{"id":349414145,"uuid":"1202165140","full_name":"dfa1/rocksdbffm","owner":"dfa1","description":"proposal for rocksdb bindings for modern Java using FFM instead of JNI","archived":false,"fork":false,"pushed_at":"2026-06-17T08:45:46.000Z","size":756,"stargazers_count":5,"open_issues_count":3,"forks_count":1,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-06-17T10:25:09.215Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dfa1.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-04-05T17:24:37.000Z","updated_at":"2026-06-17T08:45:43.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/dfa1/rocksdbffm","commit_stats":null,"previous_names":["dfa1/rocksdbffm"],"tags_count":5,"template":false,"template_full_name":null,"purl":"pkg:github/dfa1/rocksdbffm","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dfa1%2Frocksdbffm","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dfa1%2Frocksdbffm/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dfa1%2Frocksdbffm/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dfa1%2Frocksdbffm/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dfa1","download_url":"https://codeload.github.com/dfa1/rocksdbffm/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dfa1%2Frocksdbffm/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34847288,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-27T02:00:06.362Z","response_time":126,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-06-27T09:04:08.591Z","updated_at":"2026-06-27T09:04:09.773Z","avatar_url":"https://github.com/dfa1.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"# RocksDB FFM\n\n[![Maven Central](https://img.shields.io/maven-central/v/io.github.dfa1/rocksdbffm-core.svg)](https://central.sonatype.com/artifact/io.github.dfa1/rocksdbffm-core)\n![PURL](https://img.shields.io/badge/purl-pkg%3Amaven%2Fio.github.dfa1%2Frocksdbffm--core%400.5-blue)\n![RocksDB](https://img.shields.io/badge/RocksDB-11.0.4-green.svg)\n![MacOS](https://img.shields.io/badge/macOS-fully_supported-green.svg)\n![Linux](https://img.shields.io/badge/linux-fully_supported-green.svg)\n![Linux aarch64](https://img.shields.io/badge/linux_aarch64-fully_supported-green.svg)\n![Windows](https://img.shields.io/badge/windows-not_supported-red.svg)\n[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)\n[![CI](https://github.com/dfa1/rocksdbffm/workflows/CI/badge.svg?branch=main)](https://github.com/dfa1/rocksdbffm/actions?query=branch=main)\n\n**rocksdbffm** is an experimental Java wrapper for [RocksDB](https://rocksdb.org/) using the **Foreign\nFunction \u0026 Memory (FFM) API**.\n\nThe project aims to provide a more maintainable alternative to the traditional JNI-based `rocksdbjni`.\nThe target is JDK 25+ because of `java.lang.foreign`.\n\n\u003e **AI-assisted development:** This project uses [Claude Code](https://claude.ai/code) heavily for implementation\n\u003e work — C header mapping, test generation, and documentation. **Architecture, API design, and all decisions are\n\u003e human-driven.**\n\nThe native library is built from the RocksDB source via **`zig cc` / `zig c++`** as a drop-in C/C++ compiler\n(`PORTABLE=1 make shared_lib`). Zig bundles clang and libc++ for every target, enabling hermetic cross-compilation\nwithout a separate sysroot or system toolchain.\n\n## 📦 Coordinates\n\nThe library comes with core (pure `Java`) and one additional native artifact per OS/Architecture. Windows is not yet supported: RocksDB's POSIX `Makefile` cannot cross-compile to a `.dll` via `zig cc`, and a CMake-based build path is needed. Contributions welcome.\n\n### Maven (BOM — recommended)\n\nImport the BOM once; all artifact versions are managed automatically:\n\n```xml\n\u003cdependencyManagement\u003e\n  \u003cdependencies\u003e\n    \u003cdependency\u003e\n      \u003cgroupId\u003eio.github.dfa1\u003c/groupId\u003e\n      \u003cartifactId\u003erocksdbffm-bom\u003c/artifactId\u003e\n      \u003cversion\u003ex.y.z\u003c/version\u003e\n      \u003ctype\u003epom\u003c/type\u003e\n      \u003cscope\u003eimport\u003c/scope\u003e\n    \u003c/dependency\u003e\n  \u003c/dependencies\u003e\n\u003c/dependencyManagement\u003e\n\n\u003c!-- No version needed — managed by the BOM --\u003e\n\u003cdependencies\u003e\n  \u003cdependency\u003e\n    \u003cgroupId\u003eio.github.dfa1\u003c/groupId\u003e\n    \u003cartifactId\u003erocksdbffm-core\u003c/artifactId\u003e\n  \u003c/dependency\u003e\n  \u003c!-- choose 1 or more native package --\u003e\n  \u003cdependency\u003e\n    \u003cgroupId\u003eio.github.dfa1\u003c/groupId\u003e\n    \u003cartifactId\u003erocksdbffm-native-osx-aarch64\u003c/artifactId\u003e\n  \u003c/dependency\u003e\n  \u003cdependency\u003e\n    \u003cgroupId\u003eio.github.dfa1\u003c/groupId\u003e\n    \u003cartifactId\u003erocksdbffm-native-linux-x86_64\u003c/artifactId\u003e\n  \u003c/dependency\u003e\n  \u003cdependency\u003e\n    \u003cgroupId\u003eio.github.dfa1\u003c/groupId\u003e\n    \u003cartifactId\u003erocksdbffm-native-linux-aarch64\u003c/artifactId\u003e\n  \u003c/dependency\u003e\n\u003c/dependencies\u003e\n```\n\n### Gradle\n\n```kotlin\nimplementation(platform(\"io.github.dfa1:rocksdbffm-bom:x.y.z\"))\nimplementation(\"io.github.dfa1:rocksdbffm-core\")\n// choose 1 or more native package\nimplementation(\"io.github.dfa1:rocksdbffm-native-osx-aarch64\")\nimplementation(\"io.github.dfa1:rocksdbffm-native-linux-x86_64\")\nimplementation(\"io.github.dfa1:rocksdbffm-native-linux-aarch64\")\n```\n\n### 🔐 Supply‑chain \u0026 SBOM\n\nThis project publishes a Software Bill of Materials (SBOM) using the CycloneDX format.\n\n- Package URL (PURL): `pkg:maven/io.github.dfa1/rocksdbffm-core@0.5`\n- Ecosystem: Maven\n- Coordinates: `io.github.dfa1:rocksdbffm-core:0.4`\n\nPURLs allow SCA tools (Syft, Grype, Trivy, osv.dev, GitHub Advisory DB) to uniquely identify this artifact.\n\n## Contributing\n\n### Requirements\n\n- JDK 25+.\n- [Zig](https://ziglang.org/) (any 0.15.x build).\n\n### Build and Test\n\n```bash\n# Clone the rocksdb submodule (first time)\ngit submodule update --init --recursive\n\n# Build RocksDB from the submodule (first time or after a clean)\n./mvnw generate-resources -Pnative-build\n\n# Run unit tests\n./mvnw test\n```\n\n## Why This Project Exists\n\n### 1. Reducing JNI Maintenance Lag\n\nThere is often a significant delay between new features appearing in the RocksDB C++ core and their availability in\nthe Java JNI wrappers. This is largely due to the complexity of maintaining C++ glue code. By using FFM, we can map\nC headers directly in Java, simplifying the process of supporting new C++ features.\n\n### 2. Safety\n\nFFM improves safety over JNI on the Java side: accessing a closed or out-of-bounds `MemorySegment`\nthrows an exception rather than silently corrupting memory. However, a bad pointer passed into a\nnative call can still crash the JVM — FFM does not sandbox native execution.\n\n## Design Choices\n\nSeveral deliberate decisions set this library apart from `rocksdbjni`.\n\n### Modern Java\n\nThe API uses `java.lang.foreign` (FFM), records, sealed types, and pattern matching where they reduce\nboilerplate or improve safety. There is no legacy compatibility shim.\n\n### Expose only valid operations\n\nEvery type of RocksDB instance exposes only relevant operations in Java.\nFor example, `rocksdb_open_for_read_only` is exposed as `ReadOnlyDB`, which\ndoes not expose any `put` or `delete` method.\n\nIn `rocksdbjni`, the same `RocksDB` type is used for both read-write and read-only opens.\nCalling `put()` on a read-only instance compiles and runs, but fails at runtime:\n\n1. `RocksDB.put(byte[], byte[])` calls through JNI into `db-\u003ePut(...)`.\n2. The underlying C++ object is a `DBImplReadOnly`, which overrides every write method to return `Status::NotSupported(\"Not supported operation in read only mode.\")`.\n3. The JNI layer converts that status into a thrown `RocksDBException`.\n\nHere the constraint is enforced at compile time — `ReadOnlyDB` simply has no `put`, `delete`, `merge`, or `write` method, so an invalid call is a build error rather than a runtime failure.\n\n### Exceptions for all errors\n\nEvery operation that can fail throws `RocksDBException` (an unchecked exception). `rocksdbjni` historically returned\n`null`, `-1`, or relied on status objects that callers could silently ignore. Here a failure is always loud.\n\n### Domain primitives instead of raw scalars\n\nRaw numeric types carry no unit information and cannot be validated at construction time.\n\n| Concept              | rocksdbjni               | rocksdbffm            |\n|:---------------------|:-------------------------|:----------------------|\n| Cache / buffer sizes | `long` (bytes, silently) | `MemorySize.ofMB(64)` |\n| Snapshot position    | `long`                   | `SequenceNumber`      |\n\nBoth types are immutable, `Comparable`, and reject invalid values at construction — an illegal value cannot be created\nand therefore cannot be passed anywhere.\n\n### `Path` for filesystem operations\n\nAll methods that accept a filesystem location (open, checkpoint, backup, …) take `java.nio.file.Path` instead of\n`String`. This prevents confusion between absolute and relative paths, integrates naturally with the NIO file API, and\nrules out accidentally passing non-path strings.\n\n### Performance through Zero-Copy\n\n- **Pinnable Slices:** Uses `rocksdb_get_pinned` to avoid intermediate copies from the block cache.\n- **MemorySegment \u0026 ByteBuffer:** Support for `java.lang.foreign.MemorySegment` and direct `ByteBuffer` for data\n  transfer between Java and native code.\n\n## Performance Results\n\nBenchmarks performed on JDK 25 (Apple M5), RocksDB v11.0.4. Each tier uses the same pre-seeded key so the\nnumbers reflect pure call overhead, not cache miss variance.\n\n| Operation              | API tier           | FFM (ops/s) | JNI (ops/s) |   Gain    |\n|:-----------------------|:-------------------|:-----------:|:-----------:|:---------:|\n| Reads                  | `byte[]`           |  7,196,554  |  3,619,125  | **+99%**  |\n| Reads                  | `DirectByteBuffer` |  8,077,135  |  3,656,113  | **+121%** |\n| Reads                  | `MemorySegment`    |  8,149,510  |      —      |     —     |\n| Writes                 | `byte[]`           |   671,213   |   608,496   | **+10%**  |\n| Writes                 | `DirectByteBuffer` |   694,166   |   590,923   | **+17%**  |\n| Writes                 | `MemorySegment`    |   686,889   |      —      |     —     |\n| Batch writes (100 ops) | `byte[]`           |   23,936    |   16,813    | **+42%**  |\n\n*Both libraries use `PinnableSlice` for reads. Read gains (~2×) come from the absence of JNI frame setup and\nthread-state transitions — FFM downcall stubs are JIT-compiled directly. `MemorySegment` is the fastest read tier\nbecause segments backed by a confined arena carry no GC scope-check overhead on the hot path. Write gains are smaller\nbecause WAL/memtable I/O dominates. Batch write gains multiply because per-call overhead is paid 100× per iteration.*\n\n### Running benchmarks\n\n```bash\n./scripts/benchmark.sh\n```\n\nBuilds everything, runs both FFM and JNI suites, and prints a side-by-side comparison table.\n\n## Roadmap\n\nThis project is currently experimental. The table below tracks parity with `rocksdbjni`.\n\n| Feature                    | Status | Notes                                                                                                                                            |\n|:---------------------------|:------:|:-------------------------------------------------------------------------------------------------------------------------------------------------|\n| DB Open/Create             |   ✅    | Options, CreateIfMissing, ReadOnly                                                                                                               |\n| Put/Get/Delete             |   ✅    | byte[], ByteBuffer, MemorySegment; zero-copy via PinnableSlice                                                                                   |\n| WriteBatch                 |   ✅    | Atomic multi-op writes                                                                                                                           |\n| Transactions (pessimistic) |   ✅    | TransactionDB, savepoints, get-for-update                                                                                                        |\n| Checkpoints                |   ✅    | Point-in-time on-disk snapshot                                                                                                                   |\n| Table Options              |   ✅    | BlockBasedTableConfig, LRUCache, FilterPolicy (Bloom)                                                                                            |\n| Iterators                  |   ✅    | seekToFirst/Last, seek, seekForPrev, next/prev; all three access tiers                                                                           |\n| Snapshots                  |   ✅    | Point-in-time consistent reads; `ReadOptions.setSnapshot`, sequence numbers                                                                      |\n| Flush                      |   ✅    | `flush(FlushOptions)`, `flushWal(boolean sync)`; sync/async modes                                                                                |\n| DB Properties              |   ✅    | `getProperty(DBProperty)` → `Optional\u003cString\u003e`, `getLongProperty(DBProperty)` → `OptionalLong`                                                   |\n| Statistics                 |   ✅    | TickerType, HistogramType, StatsLevel                                                                                                            |\n| Compression                |   ✅    | `CompressionType` enum (NO/Snappy/zlib/bz2/LZ4/LZ4HC/Xpress/Zstd); `Options.setCompression`; `CompressionType.getSupportedTypes()` runtime probe |\n| Column Families            |   ✅    | `openWithColumnFamilies`, `listColumnFamilies`, `createColumnFamily`, `dropColumnFamily`; `ColumnFamilyHandle`, `ColumnFamilyDescriptor`; put/get/delete/deleteRange/keyMayExist/flush/getProperty/newIterator all three tiers; `WriteBatch` CF overloads; CF overloads on `ReadOnlyDB`, `TtlDB`, `TransactionDB`, `OptimisticTransactionDB`; `Transaction` put/delete/get/getForUpdate/newIterator per-CF; multi-CF open for all DB types |\n| MultiGet                   |   ❌    | Bulk reads                                                                                                                                       |\n| DeleteRange                |   ✅    | Range tombstones; `deleteRange` on `RocksDB` and `WriteBatch`; all three access tiers                                                            |\n| Compaction control         |   ✅    | `compactRange` (all three tiers + `CompactOptions`), `suggestCompactRange`, `disableFileDeletions`, `enableFileDeletions`                        |\n| SST File Ingest            |   ✅    | `SstFileWriter` (put/delete/deleteRange/merge), `RocksDB.ingestExternalFile`; `IngestExternalFileOptions`                                        |\n| Backup Engine              |   ✅    | `BackupEngine`, `BackupEngineOptions`, `RestoreOptions`, `BackupInfo`, `BackupId`; incremental backup/restore; purge; verify                     |\n| TTL DB                     |   ✅    | `openWithTtl(path, Duration)`; lazy expiry via compaction; full API available                                                                    |\n| Optimistic Transactions    |   ✅    | `OptimisticTransactionDB`; conflict detection at commit; `OptimisticTransactionOptions`                                                          |\n| Merge / MergeOperator      |   🚧    | `merge` on `RocksDB`, `WriteBatch`, `SstFileWriter`; `setUInt64AddMergeOperator` on `Options`; custom `MergeOperator` via FFM upcall stubs       |\n| CompactionFilter           |   ❌    | Custom compaction logic                                                                                                                          |\n| WAL Iterator               |   ✅    | `WalIterator`, `WalBatchResult`; `getUpdatesSince(SequenceNumber)`, `getLatestSequenceNumber`; CDC/replication/auditing                          |\n| Rate Limiter               |   ✅    | `RateLimiter`; writes-only, reads-only, all-IO modes; auto-tuned variant; `Options.setRateLimiter`                                               |\n| Env                        |   ✅    | `Env.defaultEnv()`, `Env.memEnv()`; background thread pools (`setBackgroundThreads`, `setHighPriorityBackgroundThreads`); `Options.setEnv`       |\n| SST File Manager           |   ✅    | `SstFileManager`; disk-space limits, trash-deletion rate, compaction buffer; `Options.setSstFileManager`                                         |\n| Secondary DB               |   ✅    | `SecondaryDB`; `tryCatchUpWithPrimary`, get, iterator, snapshot, properties                                                                      |\n| Blob DB                    |   ✅    | `BlobDB`; blob options on `Options`; blob properties (`BLOB_STATS`, `NUM_BLOB_FILES`, …); `PrepopulateBlobCache`                                 |\n| Logger                     |   ✅    | Logger + callback                                                                                                                                |\n| Custom Comparators         |   ❌    | Custom comparators                                                                                                                               |\n| Advanced column family     |   ❌    |                                                                                                                                                  |\n| Advanced memtable config   |   ❌    |                                                                                                                                                  |\n| Perf Context               |   ✅    | `PerfContext`, `PerfLevel`, `PerfMetric`; `setPerfLevel`, `reset`, `metric`, `report`                                                            |\n| Persistent Cache           |   🚫    | Not exposed in `rocksdb/c.h` — C++ only (`NewPersistentCache`); requires a custom C shim to bridge                                               |\n| Wide Columns               |   🚫    | Not exposed in `rocksdb/c.h` — C++ only (`PutEntity`, `GetEntity`, `WideColumns`); requires a custom C shim to bridge                            |\n| Background Jobs            |   🚧    | Tier 1: `cancelAllBackgroundWork`, `disableManualCompaction`, `enableManualCompaction`, `waitForCompact(WaitForCompactOptions)`; Tier 3–5 (Options tuning, FIFO/Universal options) pending                  |\n\n## Releasing\n\n```bash\n./mvnw --batch-mode release:clean release:prepare\ngit push \u0026\u0026 git push --tags\n```\n\nGitHub Actions picks up the tag and deploys to Maven Central.\n\n## License\n\nThis project is licensed under the same terms as RocksDB (LevelDB/Apache 2.0).\n\n## See Also\n\n- [Expanding RocksDB's Java FFI](https://rocksdb.org/blog/2024/02/20/foreign-function-interface.html)\n- [Rocksjava: present and future](https://evolvedbinary.slides.com/adamretter/rocksjava-present-and-future#/1)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdfa1%2Frocksdbffm","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdfa1%2Frocksdbffm","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdfa1%2Frocksdbffm/lists"}