{"id":14964740,"url":"https://github.com/rsatrio/llm-chatbot-springboot","last_synced_at":"2026-02-17T05:15:21.906Z","repository":{"id":251427779,"uuid":"837382654","full_name":"rsatrio/llm-chatbot-springboot","owner":"rsatrio","description":"LLM Chatbot using Spring Boot 3","archived":false,"fork":false,"pushed_at":"2025-03-29T00:19:41.000Z","size":131,"stargazers_count":9,"open_issues_count":0,"forks_count":11,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-10-05T10:39:28.031Z","etag":null,"topics":["chatbot","java","llamacpp","llm","spring-boot"],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/rsatrio.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-08-02T20:59:02.000Z","updated_at":"2025-08-11T03:21:40.000Z","dependencies_parsed_at":"2024-08-02T23:45:28.746Z","dependency_job_id":"15230ac5-1efc-4a63-b139-edf854cc33a6","html_url":"https://github.com/rsatrio/llm-chatbot-springboot","commit_stats":{"total_commits":3,"total_committers":2,"mean_commits":1.5,"dds":"0.33333333333333337","last_synced_commit":"d4117329ef5f9a796e354e4670e66eeaf7e9cc82"},"previous_names":["rsatrio/llm-chatbot-springboot"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/rsatrio/llm-chatbot-springboot","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rsatrio%2Fllm-chatbot-springboot","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rsatrio%2Fllm-chatbot-springboot/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rsatrio%2Fllm-chatbot-springboot/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rsatrio%2Fllm-chatbot-springboot/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/rsatrio","download_url":"https://codeload.github.com/rsatrio/llm-chatbot-springboot/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rsatrio%2Fllm-chatbot-springboot/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":278778967,"owners_count":26044256,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-07T02:00:06.786Z","response_time":59,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["chatbot","java","llamacpp","llm","spring-boot"],"created_at":"2024-09-24T13:33:42.829Z","updated_at":"2026-02-17T05:15:21.884Z","avatar_url":"https://github.com/rsatrio.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"# LLMCpp Spring Boot Chat\n\nA robust, CLI-based LLM (Large Language Model) chat application built with **Spring Boot 3** and **Java 17**, utilizing [LlamaCpp-Java](https://github.com/kherud/java-llama.cpp) bindings for high-performance inference.\n\nThis project demonstrates how to integrate local LLM inference within a Spring Boot application, supporting GGUF model formats.\n\n![Chatbot LLM](./demo-local-chatbot.gif)\n\n## Features\n\n*   **Interactive CLI Chat**: Real-time chat interface via the command line.\n*   **Local Inference**: Runs GGUF models locally (no API keys required).\n*   **Customizable Prompts**: Support for external prompt templates.\n*   **Configurable Generation**: Fine-tune temperature, top-p, context size, and CPU threads.\n*   **Performance Statistics**: Detailed metrics for every response (tokens/sec, time to first token, total tokens).\n*   **Modular Architecture**: Decoupled I/O and business logic for better testability.\n*   **Comprehensive Tests**: Includes unit tests for services and components.\n*   **Docker Support**: Ready-to-use Dockerfile for containerized deployment.\n\n## Prerequisites\n\n*   **Java**: JDK 21 or higher.\n*   **Maven**: 3.8+ (Wrapper included).\n*   **RAM**: Sufficient RAM to load your chosen GGUF model (e.g., ~1GB for TinyLlama 1.1B Q4).\n\n## Getting Started\n\n### 1. Build from Source\n\nClone the repository and build the application using Maven:\n\n```bash\ngit clone \u003crepository-url\u003e\ncd llm-chatbot-springboot\n./mvnw clean package\n```\n\nThe executable JAR will be located in the `target` directory (e.g., `target/LLMCpp-Chat-SpringBoot.jar`).\n\n### 2. Prepare the Model\n\nDownload a GGUF model file (e.g., from [Hugging Face](https://huggingface.co/models?search=gguf)).\n*   Recommended for testing: [TinyLlama-1.1B-Chat-v1.0-GGUF](https://huggingface.co/TheBloke/TinyLlama-1.1B-Chat-v1.0-GGUF)\n\n### 3. Run the Application\n\nRun the JAR, pointing it to your model file:\n\n```bash\njava -jar target/LLMCpp-Chat-SpringBoot.jar --llamacpp.model=/path/to/your/model.gguf\n```\n\nOr using the default configuration (looks for `tinyllama-1.1b-chat-v1.0.Q6_K.gguf` in the working directory):\n\n```bash\njava -jar target/LLMCpp-Chat-SpringBoot.jar\n```\n\n### 4. Run Tests\n\nYou can run the unit tests using the Maven wrapper:\n\n```bash\n./mvnw test\n```\n\n## Configuration\n\nYou can configure the application via `application.properties`, system properties, or command-line arguments.\n\n| Property | Description | Default Value |\n| :--- | :--- | :--- |\n| `llamacpp.model` | Absolute or relative path to the GGUF model file. | `tinyllama-1.1b-chat-v1.0.Q6_K.gguf` |\n| `llamacpp.prompt.path` | Path to a text file containing the system prompt template. | `llamacpp_prompt.txt` |\n| `llamacpp.temperature` | Controls randomness (0.0 to 1.0). Higher is more creative. | `0.2` |\n| `llamacpp.topp` | Nucleus sampling probability threshold. | `10` |\n| `llamacpp.thread.cpu` | Number of CPU threads to use for inference. | `1` |\n| `llamacpp.number.context` | Context window size (0 uses model default). | `0` |\n| `llamacpp.frequency-penalty` | Penalty for token repetition. | `0.2` |\n| `llamacpp.miro-stat` | MiroStat sampling version (`V0`, `V1`, `V2`). | `V2` |\n| `llamacpp.stop-strings` | List of strings that stop generation. | `\u003c/s\u003e, \u003c|im_end|\u003e, User:` |\n\n### Customizing the Prompt\n\nBy default, the application uses a built-in prompt template suitable for chat-tuned models. To customize it, create a file (e.g., `my_prompt.txt`) and pass it:\n\n```bash\njava -jar target/LLMCpp-Chat-SpringBoot.jar --llamacpp.prompt.path=my_prompt.txt\n```\n\n**Template Variables:**\n*   `{question}`: Will be replaced by the user's input.\n\n**Example Prompt File:**\n```text\n\u003c|system|\u003e\nYou are a helpful coding assistant.\n\u003c|user|\u003e\n{question}\n\u003c|assistant|\u003e\n```\n\n## Docker Usage\n\nBuild the Docker image:\n\n```bash\ndocker build -t chat-cli .\n```\n\nRun the container, mounting the model file:\n\n```bash\ndocker run -it -v /local/path/to/model.gguf:/app/model.gguf chat-cli --llamacpp.model=/app/model.gguf\n```\n\n## Architecture\n\nThe application follows a clean Spring Boot architecture with decoupled concerns:\n\n*   **`ChatRunner`**: Implements `CommandLineRunner` to start the chat service without blocking the application context initialization.\n*   **`ChatServicesImpl`**: Manages the high-level chat loop, using an `IOService` for interaction.\n*   **`ChatbotServicesImpl`**: Handles the business logic for generating responses using the LLM.\n*   **`IOService` / `ConsoleIOService`**: Abstracts I/O operations (CLI), enabling easy unit testing and potential future UI swaps.\n*   **`LlamaCppProperties`**: Centralized, type-safe configuration bean for all `llamacpp.*` properties.\n*   **`LlamaModelComponent`**: Manages the lifecycle of the native `LlamaModel` instance.\n*   **`PromptComponent`**: Loads and formats the prompt template.\n\nSee [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) for more details.\n\n## Feedback\n\nPlease raise issues in the repository for bugs or feature requests.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frsatrio%2Fllm-chatbot-springboot","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frsatrio%2Fllm-chatbot-springboot","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frsatrio%2Fllm-chatbot-springboot/lists"}