https://github.com/numq/text-generation
JVM library for text generation, written in Kotlin and based on the C++ library llama.cpp
https://github.com/numq/text-generation
cpp java jni jvm kotlin llama ml text-generation text-to-text
Last synced: 2 months ago
JSON representation
JVM library for text generation, written in Kotlin and based on the C++ library llama.cpp
- Host: GitHub
- URL: https://github.com/numq/text-generation
- Owner: numq
- License: apache-2.0
- Created: 2025-01-30T23:24:50.000Z (over 1 year ago)
- Default Branch: master
- Last Pushed: 2025-02-24T22:52:45.000Z (over 1 year ago)
- Last Synced: 2025-02-24T23:28:38.680Z (over 1 year ago)
- Topics: cpp, java, jni, jvm, kotlin, llama, ml, text-generation, text-to-text
- Language: Kotlin
- Homepage:
- Size: 181 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
Text generation
JVM library for text generation, written in Kotlin and based on the C++ library llama.cpp
## Features
- Generate text from a string
## Installation
- Download latest [release](https://github.com/numq/text-generation/releases)
- Add library dependency
```kotlin
dependencies {
implementation(file("/path/to/jar"))
}
```
- Unzip binaries
## Usage
### TL;DR
> See the [example](example) module for implementation details
- Call `generate` to process the string and get a generated output
### Step-by-step
- Load binaries
- CPU
```kotlin
TextGeneration.Llama.loadCPU(
ggmlBase = "/path/to/ggml-base",
ggmlCpu = "/path/to/ggml-cpu",
ggmlRpc = "/path/to/ggml-rpc",
ggml = "/path/to/ggml",
llama = "/path/to/llama",
textGeneration = "/path/to/text-generation",
)
```
- CUDA
```kotlin
TextGeneration.Llama.loadCPU(
ggmlBase = "/path/to/ggml-base",
ggmlCpu = "/path/to/ggml-cpu",
ggmlCuda = "/path/to/ggml-cuda",
ggmlRpc = "/path/to/ggml-rpc",
ggml = "/path/to/ggml",
llama = "/path/to/llama",
textGeneration = "/path/to/text-generation",
)
```
- Create an instance
```kotlin
TextGeneration.Llama.create(
modelPath = "/path/to/model"
)
```
- Call `history` to get the history of text generation
- Call `generate` to process the string and get a generated output
- Call `reset` to reset the internal state and history
- Call `close` to release resources
## Requirements
- JVM version 9 or higher
## License
This project is licensed under the [Apache License 2.0](LICENSE)
## Acknowledgments
- [llama.cpp](https://github.com/ggml-org/llama.cpp)
___