Data

Tools

Indexes

Applications

Experiments

Awesome

An open API service indexing awesome lists of open source software.

https://github.com/openhft/chronicle-values


https://github.com/openhft/chronicle-values

Last synced: 5 months ago
JSON representation

Awesome Lists containing this project

README 

          
= Chronicle-Values

Chronicle Software

:css-signature: demo

:toc: macro

:toclevels: 3



image:https://maven-badges.herokuapp.com/maven-central/net.openhft/chronicle-values/badge.svg[link=https://maven-badges.herokuapp.com/maven-central/net.openhft/chronicle-values]

image:https://javadoc.io/badge2/net.openhft/chronicle-values/javadoc.svg[link=https://www.javadoc.io/doc/net.openhft/chronicle-values/latest/index.html]

image:https://img.shields.io/badge/release%20notes-subscribe-brightgreen[link=https://chronicle.software/release-notes/]

image:https://sonarcloud.io/api/project_badges/measure?project=OpenHFT_Chronicle-Values&metric=alert_status[link=https://sonarcloud.io/dashboard?id=OpenHFT_Chronicle-Values]



toc::[]



== About



Generation of constantly-sized flyweight accessors to https://github.com/OpenHFT/Chronicle-Bytes[Chronicle Bytes] and simple bean-style on-heap implementations from interfaces.

Interfaces that can be processed by Chronicle-Values generation are called *value interfaces*.



*Project status: Stable* - the feature matrix (see below) is very vast and not expected to be implemented further.

Please write the value interface according to the specification below, with *unit tests for your interface*.

If the generation from the value interface (according to spec) doesn't work, please report the case via GitHub issues.



All classes in the package `net.openhft.chronicle.values.internal` and its subpackages are internal implementation details.

They may change without notice and must not be referenced directly.



=== Value interface specification



Value interfaces must belong to a non-empty package (they cannot sit in the default package).



.Simple example

[source,java]

----

package test;



interface Balance {

    long getDollars();

    void setDollars(long dollars);

    long addDollars(long addition);



    int getCents();

    void setCents(int cents);

}

----

The above is compiled into either a 12-byte flyweight or a regular bean with `long dollars` and `int cents`.



==== Supported field types



===== All Java primitives

`byte`, `boolean`, `char`, `short`, `int`, `long`, `float`, `double`



===== String or CharSequence

Must be annotated with `@MaxUtf8Length`:



[source,java]

----

interface Client {

    CharSequence getName();

    void setName(@MaxUtf8Length(20) CharSequence name);



    CharSequence getStateCode();

    void setStateCode(@NotNull @MaxUtf8Length(2) CharSequence stateCode);

}

----



The value supplied to `@MaxUtf8Length` is the allowed size once the text is encoded as UTF-8. The native form checks this limit when writing to the backing `BytesStore`.

Heap implementations assume the caller supplies a compliant value and any breach is detected only when copied to a native instance.



===== Another value interface

Allows nested structures:



[source,java]

----

interface Point {

    double getX();

    void setX(double x);



    double getY();

    void setY(double y);

}



interface Circle {

    Point getCenter();

    void getUsingCenter(Point using);

    void setCenter(Point center);



    double getRadius();

    void setRadius(double radius);

}

----

(Self-references are forbidden.)



===== Any Java enum type

[source,java]

----

interface Order {

    enum State { NEW, CANCELLED, FILLED }



    State getState();

    void setState(@NotNull State state);

}

----

The generated classes keep a reference to the array of constants for each enum field.

The `Enums` helper obtains this array via reflection from `EnumSet.getUniverse(Class)`.

Caching the array lets the marshalling code translate ordinals without calling `values()` on every access.



===== java.util.Date



===== Array fields

Use the `-At` suffix and make the first parameter `int index`:



[source,java]

----

interface SomeStats {

    @Array(length = 100)

    long getPercentFreqAt(int index);

    void setPercentFreqAt(int index, long percentFreq);

    long addPercentFreqAt(int index, long addition);

}

----



==== Supported methods



===== Simple get/set

`[get]FieldName[At]`, `[set]FieldName[At]`, or `isFieldName` for boolean fields.

You may omit the `get`/`set` prefixes:



[source,java]

----

interface Point {

    double x();

    void x(double x);



    double y();

    void y(double y);

}

----



===== Volatile get/set

`getVolatileFieldName[At]`, `setVolatileFieldName[At]`



===== Ordered set

`setOrderedFieldName[At]` (similar to `AtomicInteger.lazySet`)



===== Simple add

`type addFieldName[At]([int index, ]type addition)`

Works for numeric primitives.



===== Atomic add

`type addAtomicFieldName[At]([int index, ]type addition)`

Atomic variant for numeric primitives.



===== Compare-and-swap

`boolean compareAndSwapFieldName[At]([int index, ]type expected, type newValue)`

Works for primitives, enums, and `Date`.



===== getUsing

`getUsingFieldName[At]([int index, ]Type using)`

For `String`/`CharSequence`, `using` must be a `StringBuilder`.

For nested value interfaces, `using` is that interface.



==== Table of supported methods



[cols="1,^1,^1,^1,^1,^1,^1,^1",options="header"]

|===

| | Integer (`byte`-`long`) | `float`, `double` | `boolean` | Char sequence | Value interface | `enum` | `Date`



| get / set

| ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓



| Volatile get/set, ordered set

| ✓ | ✓ | ✓ |  |  | ✓ | ✓



| Compare-and-swap

| ✓ | ✓ | ✓ |  |  | ✓ | ✓



| Simple add, atomic add

| ✓ | ✓ |  |  |  |  |



| getUsing

|  |  |  | ✓ | ✓ |  |

|===



=== Field configuration via annotations



==== Field ordering in flyweight layout

Fields are laid out group by group.

Use `@Group` to cluster fields and control their ordering.

Fields without `@Group` form the default group, which always comes first:



[source,java]

----

interface Complex {

    @Group(1)

    double real();

    void real(double real);



    @Group(2)

    double imaginary();

    void imaginary(double imaginary);

}

----



==== Field nullability

`enum` and string fields are nullable by default; mark with `@NotNull` to forbid `null`:



[source,java]

----

interface Instrument {

    CharSequence getSymbol();

    void setSymbol(@NotNull @MaxUtf8Length(5) CharSequence symbol);

}

----



When a field is nullable the generated native form stores a marker instead of the text.

Heap implementations use an extra boolean for `CharSequence` fields to track a null value.

With `@NotNull` the setters emit a `NullPointerException` if given `null`.



==== Numeric field ranges

Annotate with `@Range(min=, max=)` to reduce flyweight size:



[source,java]

----

interface Transaction {

    int getSecondFromDayStart();

    void setSecondFromDayStart(@Range(min = 0, max = 24 * 60 * 60) int secondFromDayStart);

}

----



==== Field alignment

Ensure a field does not cross a cache-line boundary, for example:



[source,java]

----

interface Message {



    // ... other fields ...



    @Align(dontCross = 64)

    long getImportantField();

    void setImportantField(long importantValue);

}

----

See the Javadocs of `@Align` and `@Array` for details.



== Using Chronicle Values



[source,java]

----

// Flyweight

Point offHeapPoint = Values.newNativeReference(Point.class);

((Byteable) offHeapPoint).bytesStore(bytesStore, offset, 16);

offHeapPoint.setX(0);

offHeapPoint.setY(0);



// On-heap

Point onHeapPoint = Values.newHeapInstance(Point.class);

onHeapPoint.setX(1);

onHeapPoint.setY(2);

----



Generated classes implement:



* `Copyable` - e.g. `onHeapPoint.copyFrom(offHeapPoint)`

* `BytesMarshallable` (Chronicle-Bytes)

* Correct `equals`, `hashCode`, `toString`

* `Byteable` (no-op on the heap implementation)



The `Copyable` feature allows an on-heap value to take a snapshot of a native instance or vice versa.

This provides a simple way to shuttle data between the two representations.



You may extend these in the interface to avoid casting:



[source,java]

----

interface Point extends Byteable, BytesMarshallable, Copyable { ... }



Point offHeapPoint = Values.newNativeReference(Point.class);

// no cast required

offHeapPoint.bytesStore(bytesStore, offset, offHeapPoint.maxSize());

----



== Why Use Chronicle-Values and How It Fits In



Chronicle-Values empowers you to work with structured data with high performance and type safety, especially when interfacing with low-latency and off-heap memory systems.

It achieves this by generating efficient data accessors from Java interfaces.



=== Core Benefits



Chronicle-Values is designed to provide:



* **High Performance & Low Latency:** Access and manipulate data with minimal overhead, crucial for systems where every microsecond counts. This is particularly true when using its "native" (off-heap) implementations.

* **Minimized GC Impact:** Through the use of off-heap 'flyweight' objects, which act as lightweight views onto raw byte data. This significantly reduces garbage collection pressure by avoiding the creation of numerous heap objects.

* **Type Safety:** Define your data structures using standard Java interfaces. This provides compile-time checking, clear API contracts, and reduces runtime errors.

* **Developer Convenience:** The library automatically generates the necessary accessor boilerplate (getters, setters, etc.) based on your interface definition and intuitive annotations. This saves development time and reduces manual coding errors.

* **"Zero-Deserialization" Access:** Read data directly from memory without the traditional cost of hydrating full heap objects, allowing for faster processing.

* **Seamless Integration:** Designed to work smoothly with other Chronicle libraries like Chronicle Map (for off-heap key-value stores) and Chronicle Queue (for low-latency messaging), enabling efficient data storage and exchange.



=== Primary Use Cases



Chronicle-Values is ideal when you need to work with data that has a **fixed, well-defined structure at compile time**.

Common scenarios include:



* Representing financial messages (e.g., FIX, SBE, or custom binary protocols).

* Defining event payloads for event-sourcing architectures.

* Creating complex, fixed-layout keys or values for use with Chronicle Map or Chronicle Queue.

* Any situation where you need performant, typed access to structured binary data, whether on-heap or off-heap.



=== How Chronicle-Values Complements Chronicle-Bytes



Chronicle-Values operates on top of Chronicle-Bytes, which provides the foundational layer for byte manipulation.

While Values offers a higher-level, typed abstraction for _structured_ data, Chronicle-Bytes remains essential for:



* Low-level, direct manipulation of byte sequences.

* Implementing custom serialization or deserialization logic for arbitrary formats.

* Parsing and constructing binary network protocols from scratch.

* Handling dynamic, unstructured, or schema-less binary data.



**Typical Workflow:**

A common pattern is to use Chronicle-Bytes for initial I/O operations (e.g., reading from a network socket or a file) or to parse the outer envelope of a raw data stream.

Once you've identified a segment of that stream that corresponds to a known, fixed structure defined by a value interface, you can then map a Chronicle-Values native reference (flyweight) over that region for typed, convenient, and performant access.



=== Heap vs. Native (Flyweight) Implementations



For every value interface, Chronicle-Values can generate two types of implementations:



* **Native (Flyweight) References (`Values.newNativeReference(..)`):**

** These are lightweight views onto a `BytesStore` (which can be on-heap or off-heap memory).

** They provide maximum performance and are central to off-heap data strategies.

** They do not store data themselves; they merely provide typed accessors to the underlying bytes.

** Careful management of the backing `BytesStore`'s lifecycle is crucial (see section on Lifecycle Management).

* **Heap Instances (`Values.newHeapInstance(..)`):**

** These are standard Java objects (POJOs) that store their data on the Java heap.

** Useful for interacting with other parts of your system that expect POJOs, or when off-heap performance is not the primary concern for a particular data structure.

** Data can be easily copied between native and heap instances using the generated `copyFrom()` method (from the `Copyable` interface).