https://github.com/marschall/native-bytebuffers
Native ByteBuffers for Java
https://github.com/marschall/native-bytebuffers
bytebuffer java jni off-heap
Last synced: about 1 month ago
JSON representation
Native ByteBuffers for Java
- Host: GitHub
- URL: https://github.com/marschall/native-bytebuffers
- Owner: marschall
- Created: 2020-02-16T16:37:16.000Z (over 5 years ago)
- Default Branch: master
- Last Pushed: 2022-07-16T09:00:32.000Z (almost 3 years ago)
- Last Synced: 2025-03-27T02:43:13.687Z (about 2 months ago)
- Topics: bytebuffer, java, jni, off-heap
- Language: Java
- Size: 132 KB
- Stars: 6
- Watchers: 4
- Forks: 0
- Open Issues: 5
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
Native ByteBuffers [](https://maven-badges.herokuapp.com/maven-central/com.github.marschall/native-bytebuffers) [](https://www.javadoc.io/doc/com.github.marschall/native-bytebuffers)
==================
ByteBuffers that are allocated and released directly with:
* `malloc` and `free`
* `calloc` and `free`
* `aglined_alloc` and `free`
* `memfd_secret` and `close`
* `mmap` and `munmap`
* HugeTLB page support on Linux
* superpage support on macOS
Usage
-----
```xml
com.github.marschall
native-bytebuffers
0.4.2
```
```java
ByteBuffer buffer = Stdlib.malloc(size);
try {
// do things with the buffer
} finally {
Stdlib.free(buffer);
}
```
```java
ByteBuffer buffer = Mman.mmap(length);
try {
// do things with the buffer
} finally {
Mman.munmap(buffer);
}
```
For best startup performance it is recommended to extract the `.so` from the JAR and add it to a folder present in the `LD_LIBRARY_PATH` environment variable or the `java.library.path` system property. Otherwise this library will extract the `.so` to a temporary folder the first time it is called.
This library has been tested on Linux AMD64, macOS x86-64 as well as macOS AArch64.
It should be possible to port it other Unix like *BSD.
Why would you want this?
------------------------
You want a direct (off-heap) `java.nio.ByteBuffer` but you:
* don't want to go though the slow allocation path of the JVM for `java.nio.ByteBuffer#allocateDirect` that ends up calling `java.lang.System#gc()` and `java.lang.Thread#sleep` in a loop
* don't want to rely on the slow release mechanism of the JVM for `java.nio.ByteBuffer` that relies on garbage collection
* don't want to use internals like `sun.misc.Unsafe`
* don't want to rely on libffi
This project instead uses official [JNI ByteBuffer APIs](https://docs.oracle.com/en/java/javase/11/docs/specs/jni/functions.html#nio-support) and is therefore portable across JVMs.
Why would you not want this?
----------------------------
This obviously brings all the issues of manual memory management to Java like:
* memory leaks by not calling `free`/`munmap`
* double free
* use after free
These issues can crash the JVM.
As this project uses JNI a native library is required. We ship one for Linux x86-64 but for every other platform you have to build it yourself.
Links
-----
- [nosecmem](https://github.com/JonathonReinhart/nosecmem)
- [Using Linux's memfd_secret syscall from the JVM with JEP-419](https://blog.arkey.fr/2022/05/16/linux_memfd_secret_with_jep-419/)