{"id":20081789,"url":"https://github.com/llm-solution/minichatbot","last_synced_at":"2026-05-04T18:34:20.034Z","repository":{"id":258282982,"uuid":"873717113","full_name":"LLM-Solution/MiniChatBot","owner":"LLM-Solution","description":"MiniChatBot for LLM Solution web site. The purpose of MiniChatBot is to answer questions about LLM Solution's services.","archived":false,"fork":false,"pushed_at":"2024-12-10T17:28:04.000Z","size":298,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-03-02T13:50:30.515Z","etag":null,"topics":["ai","chatbot","inference","llm","python"],"latest_commit_sha":null,"homepage":"https://llm-solutions.fr/minichatbot-en.html","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/LLM-Solution.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-10-16T15:45:41.000Z","updated_at":"2024-12-10T17:28:53.000Z","dependencies_parsed_at":"2024-11-15T13:37:01.950Z","dependency_job_id":"17ed9270-fedf-4bc1-91fe-9aefe9efd475","html_url":"https://github.com/LLM-Solution/MiniChatBot","commit_stats":null,"previous_names":["llm-solution/minichatbot"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/LLM-Solution/MiniChatBot","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LLM-Solution%2FMiniChatBot","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LLM-Solution%2FMiniChatBot/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LLM-Solution%2FMiniChatBot/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LLM-Solution%2FMiniChatBot/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/LLM-Solution","download_url":"https://codeload.github.com/LLM-Solution/MiniChatBot/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LLM-Solution%2FMiniChatBot/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32619841,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-04T10:08:07.713Z","status":"ssl_error","status_checked_at":"2026-05-04T10:08:02.005Z","response_time":58,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","chatbot","inference","llm","python"],"created_at":"2024-11-13T15:40:18.651Z","updated_at":"2026-05-04T18:34:20.014Z","avatar_url":"https://github.com/LLM-Solution.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# MiniChatBot Backend for LLM Solutions Website\n\nMiniChatBot is a lightweight and efficient chatbot designed to run on a web platform. This repository contains scripts to retrain models, the backend setup, including deployment scripts, dependency management, and configuration for GPU and CPU environments.\n\n## Features\n\n### General\n\n- **Simplified Setup**: Easily configure the backend for both GPU and CPU environments using a `Makefile` or `setup.sh`.\n- **Environment Isolation**: Includes Python virtual environment setup to ensure dependency management and avoid conflicts.\n- **Dynamic GPU Support**: Automatically detects GPU availability and installs appropriate dependencies. Optional GPU driver installation instructions are provided.\n\n### Training\n\n- **Custom Model Fine-Tuning**: Tailor the chatbot's behavior by training or fine-tuning models using the `transformers` library.\n- **Efficient Training Process**: Leverage existing scripts and tools to streamline model customization.\n\n### Inference\n\n- **Lightweight Integration**: Seamless integration with llama.cpp for efficient and fast inference, even in resource-constrained environments.\n- **Secure API Hosting**: Pre-configured Nginx setup for reliable and secure API hosting.\n\n## Installation\n\n### 1. System Requirements\n\nEnsure your system meets the following prerequisites:\n- **Operating System**: Ubuntu 24.04 or later\n- **Python**: Version 3.12 or higher\n- **GPU Support** (Optional): NVIDIA GPU with CUDA\n\n**Tip**: Run `python3 --version` and `nvidia-smi` to check Python and GPU availability.\n\n### 2. Clone the Repository\n\nClone the MiniChatBot repository:\n\n```bash\ngit clone https://github.com/LLM-Solution/MiniChatBot.git\ncd MiniChatBot\n```\n\n### 3. Install GPU Drivers (Optional)\n\nIf you plan to use GPU acceleration, install GPU drivers, use the Makefile:\n\n```bash\nmake install_gpu\n```\n\n**Important**: A system reboot is required after installing GPU drivers.\n\n### 4. Setup Backend\n\nRun the following command to set up the backend using the Makefile:\n\n```bash\nmake setup\n```\n\nThe script will:\n- Update your system and install dependencies.\n- Set up a Python virtual environment and install required Python libraries.\n- Detect GPU availability and install the appropriate PyTorch version.\n- Clone necessary repositories (e.g., `llama.cpp`, `PyLLMSol`).\n- Configure Nginx for your API.\n\n### 5. Start the API\n\nOnce the setup is complete, start the backend server using:\n\n```bash\nmake run\n```\n\n## Makefile Commands\n\n| Command | Description |\n| --- | --- |\n| `make setup` | Complete backend setup, including dependencies and Nginx setup. |\n| `make install_gpu` | Install NVIDIA GPU drivers (optional). |\n| `make run` | Start the backend API server using Gunicorn. |\n| `make status` | Check backend server status. |\n| `make stop` | Stop the backend server. |\n| `make update` | Pull latest code and update dependencies. |\n| `make clean` | Clean up logs and temporary files. |”\n\n## Configuration\n\n### 1. Nginx Configuration\n\nDuring the setup, you’ll be prompted to provide your API’s hostname (e.g., `api.example.com`). This hostname will be used in the Nginx configuration.\n\nTo manually edit or check the configuration:\n\n- Configuration file: `/etc/nginx/sites-available/\u003chostname\u003e`\n- Enable the configuration: sudo ln -s `/etc/nginx/sites-available/\u003chostname\u003e /etc/nginx/sites-enabled/`\n- Test and restart Nginx:\n\n```bash\nsudo nginx -t\nsudo systemctl restart nginx\n```\n\n### 2. API Hostname\n\nIf you need to change the hostname later, edit the Nginx configuration and restart the server.\n\n## Directory Structure\n\n```\nMiniChatBot/\n├── data/                  # Data files for training the model\n├── model/                 # Directory for trained models\n├── Prompts/               # Initial prompts for the chatbot\n├── install_gpu.sh         # Optional script for GPU driver installation\n├── setup.sh               # Main setup script\n├── train_model.sh         # Script to train, merge, and quantize the LLM\n├── Makefile               # Makefile for common tasks\n├── requirements.txt       # Python dependencies\n├── src/                   # Source code directory\n│   ├── logs/              # Directory for log files\n│   ├── api.py             # Script to connect MiniChatBot to the API\n│   ├── config.py          # Configuration file\n│   ├── lora_merger.py     # Script to merge LoRA weights with the base model\n│   ├── make_full_data.py  # Script to gather and prepare data\n│   ├── trainer.py         # Script to retrain the LLM\n│   ├── utils.py           # Utility functions\n│   ├── wsgi.py            # WSGI entry point\n│   └── tests/             # Test module for unit and integration tests\n│       └── ...\n├── PyLLMSol/              # Cloned repository for PyLLMSol\n├── llama.cpp/             # Cloned repository for llama.cpp\n└── README.md              # Project documentation\n\n```\n\n## Common Issues\n\n### 1. GPU Not Detected\n\nIf `nvidia-smi` is not available:\n- Ensure NVIDIA drivers are installed by running:\n```bash\nsudo make install_gpu\n```\n- Reboot the system and verify with:\n```bash\nnvidia-smi\n```\n\nOtherwise you can also run model on CPU only:\n```bash\n./setup.sh --cpu-only\n```\n\n### 2. Dependency Conflicts\n\nIf you encounter Python dependency issues, remove the venv directory and restart the setup:\n\n```bash\nrm -rf ~/venv\nsudo bash setup.sh\n```\n\n## Contributing\n\nWe welcome contributions! Feel free to fork this repository and submit pull requests.\n\n## License\n\nThis project is licensed under the MIT License. See the LICENSE file for details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fllm-solution%2Fminichatbot","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fllm-solution%2Fminichatbot","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fllm-solution%2Fminichatbot/lists"}