https://github.com/smallrye/smallrye-llm

Experimentation around LLM and MicroProfile
https://github.com/smallrye/smallrye-llm

Last synced: 6 months ago
JSON representation

Experimentation around LLM and MicroProfile

• Host: GitHub
• URL: https://github.com/smallrye/smallrye-llm
• Owner: smallrye
• License: apache-2.0
• Created: 2024-05-13T15:09:16.000Z (over 1 year ago)
• Default Branch: main
• Last Pushed: 2025-06-11T16:43:12.000Z (7 months ago)
• Last Synced: 2025-06-11T18:09:57.795Z (7 months ago)
• Language: Java
• Size: 835 KB
• Stars: 15
• Watchers: 12
• Forks: 10
• Open Issues: 12
• Metadata Files:
  • Readme: README.adoc
  • Contributing: CONTRIBUTING.md
  • License: LICENSE
  • Code of conduct: CODE_OF_CONDUCT.md
  • Codeowners: CODEOWNERS
  • Security: SECURITY.md

Awesome Lists containing this project

README

          
:ci: https://github.com/smallrye/smallrye-llm/actions?query=workflow%3A%22SmallRye+Build%22

image:https://github.com/smallrye/smallrye-llm/workflows/SmallRye%20Build/badge.svg?branch=main[link={ci}]

image:https://img.shields.io/github/license/smallrye/smallrye-llm.svg["License", link="http://www.apache.org/licenses/LICENSE-2.0"]

image:https://img.shields.io/maven-central/v/io.smallrye.llm/smallrye-llm?color=green["Maven", link="https://central.sonatype.com/search?q=io.smallrye.llm%3Asmallrye-llm-parent"]

= 🚀 Smallrye LLM

Experimentation around LLM and MicroProfile

== How to run examples

=== Use LM Studio

==== Install LM Studio

https://lmstudio.ai/

==== Download model 

Mistral 7B Instruct v0.2

==== Run

On left goto "local server", select the model in dropdown combo on the top, then start server

=== Use Ollama

Running Ollama with the llama3.1 model:

[source,bash]

----

CONTAINER_ENGINE=$(command -v podman || command -v docker)

$CONTAINER_ENGINE run -d --rm --name ollama --replace --pull=always -p 11434:11434 -v ollama:/root/.ollama --stop-signal=SIGKILL docker.io/ollama/ollama

$CONTAINER_ENGINE exec -it ollama ollama run llama3.1

----

=== Run the examples

Go to each example `README.md` to see how to execute the example.

== Contributing

If you want to contribute, please have a look at link:CONTRIBUTING.md[CONTRIBUTING.md].

== License

This project is licensed under the Apache License 2.0 - see the link:LICENSE[LICENSE] file for details.