https://github.com/milenkovicm/adhesive

Apache Datafusion JVM User Defined Functions (UDF), integration nobody asked for 😀
https://github.com/milenkovicm/adhesive

arrow bytecode-compiler compiler datafusion java jni jvm rust sql udf udf-libraries userdefined-functions

Last synced: 3 months ago
JSON representation

Apache Datafusion JVM User Defined Functions (UDF), integration nobody asked for 😀

• Host: GitHub
• URL: https://github.com/milenkovicm/adhesive
• Owner: milenkovicm
• License: mit
• Created: 2024-03-16T10:05:58.000Z (10 months ago)
• Default Branch: master
• Last Pushed: 2024-09-26T05:01:32.000Z (3 months ago)
• Last Synced: 2024-09-30T13:04:05.998Z (3 months ago)
• Topics: arrow, bytecode-compiler, compiler, datafusion, java, jni, jvm, rust, sql, udf, udf-libraries, userdefined-functions
• Language: Rust
• Homepage:
• Size: 54.7 KB
• Stars: 2
• Watchers: 2
• Forks: 1
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# Adhesive - Datafusion Integration Nobody Asked For

Very opinionated datafusion user defined functions written in java.

It has been implemented to demonstrate DataFusion `FunctionFactory` functionality ([arrow-datafusion/pull#9333](https://github.com/apache/arrow-datafusion/pull/9333)).

> [!NOTE]

> It has not been envisaged as a actively maintained library.

>

Other project utilizing `FunctionFactory`:

- [Torchfusion, Opinionated Torch Inference on DataFusion](https://github.com/milenkovicm/torchfusion)

- [LightGBM Inference on DataFusion](https://github.com/milenkovicm/lightfusion)

## How To Use

A java user defined function can be defined using `CREATE FUNCTION`:

```sql

CREATE FUNCTION f1(BIGINT, BIGINT)

RETURNS BIGINT

LANGUAGE JAVA

AS '

public class NewClass extends com.github.milenkovicm.adhesive.Adhesive {

    @Override

    public Long compute(org.apache.arrow.vector.table.Row row) {

        return row.getBigInt(0) * row.getBigInt(1); 

    }

}

'

```

which will be compiled. Or, referencing existing compiled class:

```sql

CREATE FUNCTION f2(BIGINT, BIGINT)

RETURNS BIGINT

LANGUAGE CLASS

AS "com.github.milenkovicm.adhesive.example.BasicExample"

```

Note change of `LANGUAGE`. In both cases classes should extend `com.github.milenkovicm.adhesive.Adhesive` abstract class,

which will do transition between rust and java.

`com.github.milenkovicm.adhesive.example.BasicExample` is defined like:

```java

package com.github.milenkovicm.adhesive.example;

import com.github.milenkovicm.adhesive.Adhesive;

import org.apache.arrow.vector.table.Row;

public class BasicExample extends Adhesive {

  @Override

  protected Long compute(Row row) {

    return row.getBigInt(0) + row.getBigInt(1);

  }

}

```

Defined functions can be invoked in SQL:

```sql

SELECT f2(a,b) FROM t

```

## Setup

```rust

// JAR containing required libraries and additional classes 

const JAR_PATH: &str ="adhesive-1.0-jar-with-dependencies.jar";

let ctx = SessionContext::new()

    .with_function_factory(Arc::new(JvmFunctionFactory::new_with_jar(JAR_PATH)?));

```