{"id":38480062,"url":"https://github.com/hurui200320/llama-cpp-kt","last_synced_at":"2026-01-17T05:26:00.674Z","repository":{"id":185898675,"uuid":"674285695","full_name":"hurui200320/llama-cpp-kt","owner":"hurui200320","description":"The Kotlin wrapper of llama.cpp, powered by JNA","archived":false,"fork":false,"pushed_at":"2023-08-08T03:59:52.000Z","size":126,"stargazers_count":13,"open_issues_count":0,"forks_count":1,"subscribers_count":3,"default_branch":"master","last_synced_at":"2026-01-11T15:22:47.173Z","etag":null,"topics":["java-native-access","jna","kotlin","llama","llama2","llamacpp"],"latest_commit_sha":null,"homepage":"","language":"Kotlin","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/hurui200320.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-08-03T15:09:59.000Z","updated_at":"2025-10-18T16:03:41.000Z","dependencies_parsed_at":null,"dependency_job_id":"5f9ba349-7fc1-40c0-abf5-254207f59ddd","html_url":"https://github.com/hurui200320/llama-cpp-kt","commit_stats":{"total_commits":11,"total_committers":1,"mean_commits":11.0,"dds":0.0,"last_synced_commit":"ff74ce1e68cc86c2371cfadd27e728597c9e3833"},"previous_names":["hurui200320/llama-cpp-kt"],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/hurui200320/llama-cpp-kt","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hurui200320%2Fllama-cpp-kt","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hurui200320%2Fllama-cpp-kt/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hurui200320%2Fllama-cpp-kt/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hurui200320%2Fllama-cpp-kt/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/hurui200320","download_url":"https://codeload.github.com/hurui200320/llama-cpp-kt/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hurui200320%2Fllama-cpp-kt/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28385798,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-13T12:01:30.995Z","status":"ssl_error","status_checked_at":"2026-01-13T12:00:09.625Z","response_time":56,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["java-native-access","jna","kotlin","llama","llama2","llamacpp"],"created_at":"2026-01-17T05:26:00.608Z","updated_at":"2026-01-17T05:26:00.666Z","avatar_url":"https://github.com/hurui200320.png","language":"Kotlin","readme":"# llama-cpp-kt\n\n[![](https://jitpack.io/v/info.skyblond/llama-cpp-kt.svg)](https://jitpack.io/#info.skyblond/llama-cpp-kt)\n\nThe Kotlin wrapper of [llama.cpp](https://github.com/ggerganov/llama.cpp), powered by JNA.\n\n## Setup\n\nFirst, you need to build your own `libllama.so` from [llama.cpp](https://github.com/ggerganov/llama.cpp), using cmake:\n\n```bash\nmkdir build\ncd build\ncmake .. -DBUILD_SHARED_LIBS=ON \u003cother flags\u003e\ncmake --build . --config Release\n```\n\nYou need the `-DBUILD_SHARED_LIBS=ON` to build shared lib (`.so`), otherwise it will build the static library (`.a`),\nwhich cannot be loaded by JNA.\n\nThen see [jitpack](https://jitpack.io/#info.skyblond/llama-cpp-kt/) document for how to use the build artifacts in your maven or gradle project.\n\n## Usage\n\nFirst to set the JNA library path:\n\n```kotlin\ninit {\n    System.setProperty(\"jna.library.path\", \"./\")\n    lib = LibLLaMa.LIB\n}\n```\n\nThis will load lib `llama` by default, aka the JNA will search for `libllama.so` or `llama.dll`. If you have a different\nfile name, you may use `Native.load(\"llama\", LibLLaMa::class.java) as LibLLaMa` to get your own instance. But do notice that the code requires the default instance to work, since some constant are decided at runtime (for example the `LLAMA_MAX_DEVICES` is 1 when using CPU but will be 16 when using cuda).\n\nThis is a low level binding, which means you get the full control of the C functions, which looks like this:\n\n+ `lib.llama_model_quantize_default_params()`\n+ `lib.llama_load_session_file(ctx, sessionPath, tokens, size, pInt)`\n\nThere are also some high level helper functions like:\n\n+ `lib.initLLaMaBackend()`\n+ `lib.getContextParams(contextSize=1024, rmsNormEps=1e-5f)`\n\nYou can check the [example](https://github.com/hurui200320/llama-cpp-kt/tree/master/examples) subproject to see how to use it. I implemented the original [`quantize.cpp`](https://github.com/hurui200320/llama-cpp-kt/blob/master/examples/src/main/kotlin/info/skyblond/libllama/example/Quantize.kt), [`simple.cpp`](https://github.com/hurui200320/llama-cpp-kt/blob/master/examples/src/main/kotlin/info/skyblond/libllama/example/Simple.kt)\nand the [`main.cpp`](https://github.com/hurui200320/llama-cpp-kt/blob/master/examples/src/main/kotlin/info/skyblond/libllama/example/Main.kt). There are also [a simple multi-session chat server](https://github.com/hurui200320/llama-cpp-kt/blob/master/examples/src/main/kotlin/info/skyblond/libllama/example/ChatServer.kt) which can serve multiple session at once over HTTP (in the example I use single thread computation since I don't have enough ram to do parallel computing).\n\n## Roadmap\n\nCurrently this repo is still very new and I don't have that many ideas on which path to go. So discussions and\ncontributions are welcome. JVM is a fantastic platform but apparently it is underestimated during the machine learning rush. Despite Python and C++ can create powerful and fast computing deep learning models, I still believe JVM is the best platform to develop complex business logic. Here I choose Kotlin because it's much better than Java yet maintained a good interoperability with Java (however I don't think you can use this lib in Java).\n\nOne clear objective is get rid of JNA when Foreign Function \u0026 Memory API is in stable release (Maybe JDK 21?).\n\nAnother objective is make the grammar working. Currently the grammar feature is missing (Now you can use grammar related call, but the parser is not there. So you have to grow your own tree).\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhurui200320%2Fllama-cpp-kt","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhurui200320%2Fllama-cpp-kt","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhurui200320%2Fllama-cpp-kt/lists"}