{"id":18064334,"url":"https://github.com/llm-solution/pyllmsol","last_synced_at":"2026-05-06T22:05:10.460Z","repository":{"id":260290728,"uuid":"878395126","full_name":"LLM-Solution/PyLLMSol","owner":"LLM-Solution","description":"Python Tool Box for LLM Solutions","archived":false,"fork":false,"pushed_at":"2024-12-20T09:07:37.000Z","size":277,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-04-05T14:13:25.325Z","etag":null,"topics":["chatbot","llamacpp","llm","llm-training","pytorch"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/LLM-Solution.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-10-25T10:06:54.000Z","updated_at":"2024-12-20T09:07:41.000Z","dependencies_parsed_at":"2024-11-21T18:27:55.996Z","dependency_job_id":"7017972c-e94f-468e-8191-f98b720f8b4d","html_url":"https://github.com/LLM-Solution/PyLLMSol","commit_stats":null,"previous_names":["llm-solution/pyllmsol"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LLM-Solution%2FPyLLMSol","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LLM-Solution%2FPyLLMSol/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LLM-Solution%2FPyLLMSol/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LLM-Solution%2FPyLLMSol/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/LLM-Solution","download_url":"https://codeload.github.com/LLM-Solution/PyLLMSol/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247345842,"owners_count":20924102,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["chatbot","llamacpp","llm","llm-training","pytorch"],"created_at":"2024-10-31T06:05:34.377Z","updated_at":"2026-05-06T22:05:05.425Z","avatar_url":"https://github.com/LLM-Solution.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Python Tool Box for [LLM Solutions](https://llm-solutions.fr)\n\n![Pylint](https://github.com/LLM-Solution/PyLLMSol/actions/workflows/pylint.yml/badge.svg)\n![Tests](https://github.com/LLM-Solution/PyLLMSol/actions/workflows/tests.yml/badge.svg)\n[![codecov](https://codecov.io/gh/LLM-Solution/PyLLMSol/graph/badge.svg?token=X2MH94CWGZ)](https://codecov.io/gh/LLM-Solution/PyLLMSol)\n\n**PyLLMSol** is a Python package designed to simplify the training and inference processes for **large language models (LLMs)**. With dedicated modules for both **training** and **inference**, **PyLLMSol** allows users to create checkpoints, manage prompts, and run models using CLI or API interfaces.\n\n## Table of Contents\n- [Installation](#installation)\n- [Features](#features)\n- [Structure](#structure)\n- [Dependencies](#dependencies)\n- [Getting Started](#getting-started)\n- [License](#license)\n- [Author](#author)\n\n## Installation\n\nTo install **PyLLMSol** and its dependencies, follow these steps:\n\n### 1. Install PyTorch\n\nFirst, install **PyTorch** with GPU or CPU support by following the instructions on the [PyTorch official website](https://pytorch.org/get-started/locally/). Choose the appropriate command for your operating system, Python version, and hardware.\n\n### 2. Clone the repository\n\nClone the **PyLLMSol** repository to your local machine:\n\n```bash\ngit clone https://github.com/LLM-Solution/PyLLMSol.git\ncd PyLLMSol\n```\n\n### 3. Install dependencies\n\nInstall the required Python packages listed in the requirements.txt file:\n\n```bash\npip install -r requirements.txt\n```\n\n### 4. Install PyLLMSol\n\nInstall the **PyLLMSol** package using pip:\n\n```bash\npip install .\n```\n\n## Features\n\n- **Training Management**: Handle datasets, manage training steps, and track losses.\n- **Checkpointing**: Save and load checkpoints of models and data at regular intervals.\n- **CLI Interface**: Interact with the model via command-line.\n- **API Support**: Host the model as a REST API with Flask.\n- **Prompt Management**: Handle prompts with truncation and formatting options.\n\n\n## Structure\n\n```plaintext\nPyLLMSol/\n├── setup.py\n├── requirements.txt\n├── pyllmsol/\n│   └── argparser.py\n│   └── _base.py                 # Basis of training and inference modules\n│   ├── data/                    # Data module\n│   │   └── _base_data.py\n│   │   └── chat.py              # Chat objects with LLaMa-3.2 format\n│   │   └── prompt.py\n│   │   └── utils.py\n│   └── inference/               # Inference module\n│   │   └── _base_api.py\n│   │   └── _base_cli.py\n│   │   └── cli_instruct.py      # CLI with LLaMa-3.2 chat format\n│   └── training/                # Training module\n│   │   └── checkpoint.py\n│   │   └── instruct_trainer.py  # Trainer with LLaMa-3.2 chat format\n│   │   └── loss.py\n│   │   └── trainer.py\n│   │   └── utils.py\n│   └── tests/\n│       └── mock.py\n│       └── ...                  # Some tests of PyLLMSol\n└── README.md\n```\n\n### 1. `training` Module\n\nThe `training` module contains tools for managing training workflows and model checkpoints.\n\n- **Trainer**: Manages training loops, handles batch processing, and tracks loss over time.\n- **Checkpoint**: Saves and loads model states and data at specific intervals, enabling easy restoration of the training process.\n- **DataBrowser**: Supports batch processing and iterating over data with customizable parameters.\n\n#### Example Usage\n\n```python\nfrom pyllmsol.training import Trainer, Checkpoint\n\n# Initialize training components\ntrainer = Trainer(llm=my_model, tokenizer=my_tokenizer, dataset=my_data, batch_size=16)\ncheckpoint = Checkpoint(path='./checkpoints')\n\n# Run training with checkpointing\ntrainer.run(device='cuda', checkpoint=checkpoint)\n```\n\n### 2. `inference` Module\n\nThe `inference` module supports generating responses from the model and includes both CLI and API options.\n\n- **_BaseCommandLineInterface**: Offers an interactive command-line interface for chatting with the model.\n- **API**: Provides a REST API using Flask, allowing remote model access.\n\n#### CLI Usage\n\nRun the command-line interface to interact with your LLM:\n\n```bash\npython -m pyllmsol.inference.cli --model_path path/to/model.gguf\n```\n\n#### API Usage\n\nTo launch the API:\n\n```python\nfrom pyllmsol.inference import API, CommandLineInterface\n\ncli = CommandLineInterface.from_path(model_path='path/to/model', init_prompt='Hello! How can I assist you?')\napi = API(cli)\napi.run(host=\"0.0.0.0\", port=5000)\n```\n\n## Dependencies\n\nPyLLMSol requires Python 3.10 or later. Core dependencies include:\n\n- `flask\u003e=3.0.3`\n- `llama-cpp-python\u003e=0.3.1`\n- `matplotlib\u003e=3.9.2`\n- `pandas\u003e=2.2.3`\n- `peft\u003e=0.13.2`\n- `sentencepiece\u003e=0.2.0`\n- `torch\u003e=2.5.0`\n- `transformers\u003e=4.45.2`\n- `tqdm\u003e=4.66.5`\n\nFor a full list, see requirements.txt.\n\n## Getting Started\n\n1. Clone the repository and install dependencies.\n2. Set up your model files and ensure you have the necessary model weights.\n3. Use the CLI or API modules to interact with the model.\n\n## License\n\nThis project is licensed under the MIT License - see the LICENSE file for details.\n\n## Author\n\n[LLM Solutions](https://llm-solutions.fr) - [Arthur Bernard](https://www.linkedin.com/in/arthur-bernard-789955152/) - contact@llm-solutions.fr\n\n___\n\nFor further information, refer to the documentation in the source files.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fllm-solution%2Fpyllmsol","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fllm-solution%2Fpyllmsol","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fllm-solution%2Fpyllmsol/lists"}