{"id":28484285,"url":"https://github.com/py-lama/getllm","last_synced_at":"2026-05-08T15:47:25.020Z","repository":{"id":294670980,"uuid":"987689669","full_name":"py-lama/getllm","owner":"py-lama","description":"getllm is a Python package for managing LLM models with Ollama integration and generating Python code. It allows you to install, list, set the default model, update the model list, and generate code using LLM models. GetLLM is part of the PyLama ecosystem and integrates with LogLama as the primary service for centralized logging and environment man","archived":false,"fork":false,"pushed_at":"2025-05-31T21:25:01.000Z","size":732,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-06-07T22:06:40.762Z","etag":null,"topics":["bielik","dwonload","getllm","huggingface","llm","models","ollama","shell","speakleash","tools"],"latest_commit_sha":null,"homepage":"https://py-lama.github.io/getllm/","language":"HTML","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/py-lama.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-05-21T12:48:45.000Z","updated_at":"2025-05-31T21:25:04.000Z","dependencies_parsed_at":"2025-05-28T10:18:11.842Z","dependency_job_id":"925c6265-fee1-45ce-ae4b-2795ea3f6084","html_url":"https://github.com/py-lama/getllm","commit_stats":null,"previous_names":["py-lama/pyllm","py-lama/getllm"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/py-lama/getllm","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/py-lama%2Fgetllm","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/py-lama%2Fgetllm/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/py-lama%2Fgetllm/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/py-lama%2Fgetllm/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/py-lama","download_url":"https://codeload.github.com/py-lama/getllm/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/py-lama%2Fgetllm/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":262663310,"owners_count":23345032,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bielik","dwonload","getllm","huggingface","llm","models","ollama","shell","speakleash","tools"],"created_at":"2025-06-07T22:06:41.693Z","updated_at":"2026-05-08T15:47:19.993Z","avatar_url":"https://github.com/py-lama.png","language":"HTML","funding_links":["https://github.com/sponsors/tom-sapletta-com","https://opencollective.com/tom-sapletta-com","https://www.paypal.me/softreck/10.00"],"categories":[],"sub_categories":[],"readme":"# getllm\n\n![getllm-interactive.png](getllm-interactive.png)\n\n## PyLama Ecosystem Navigation\n\n| Project | Description | Links |\n|---------|-------------|-------|\n| **GetLLM** | LLM model management and code generation | [GitHub](https://github.com/py-lama/getllm) · [PyPI](https://pypi.org/project/getllm/) · [Docs](https://py-lama.github.io/getllm/) |\n| **DevLama** | Python code generation with Ollama | [GitHub](https://github.com/py-lama/devlama) · [Docs](https://py-lama.github.io/devlama/) |\n| **LogLama** | Centralized logging and environment management | [GitHub](https://github.com/py-lama/loglama) · [PyPI](https://pypi.org/project/loglama/) · [Docs](https://py-lama.github.io/loglama/) |\n| **APILama** | API service for code generation | [GitHub](https://github.com/py-lama/apilama) · [Docs](https://py-lama.github.io/apilama/) |\n| **BEXY** | Sandbox for executing generated code | [GitHub](https://github.com/py-lama/bexy) · [Docs](https://py-lama.github.io/bexy/) |\n| **JSLama** | JavaScript code generation | [GitHub](https://github.com/py-lama/jslama) · [NPM](https://www.npmjs.com/package/jslama) · [Docs](https://py-lama.github.io/jslama/) |\n| **JSBox** | JavaScript sandbox for executing code | [GitHub](https://github.com/py-lama/jsbox) · [NPM](https://www.npmjs.com/package/jsbox) · [Docs](https://py-lama.github.io/jsbox/) |\n| **SheLLama** | Shell command generation | [GitHub](https://github.com/py-lama/shellama) · [PyPI](https://pypi.org/project/shellama/) · [Docs](https://py-lama.github.io/shellama/) |\n| **WebLama** | Web application generation | [GitHub](https://github.com/py-lama/weblama) · [Docs](https://py-lama.github.io/weblama/) |\n\n## Author\n\n**Tom Sapletta** — DevOps Engineer \u0026 Systems Architect\n\n- 💻 15+ years in DevOps, Software Development, and Systems Architecture\n- 🏢 Founder \u0026 CEO at Telemonit (Portigen - edge computing power solutions)\n- 🌍 Based in Germany | Open to remote collaboration\n- 📚 Passionate about edge computing, hypermodularization, and automated SDLC\n\n[![GitHub](https://img.shields.io/badge/GitHub-181717?logo=github\u0026logoColor=white)](https://github.com/tom-sapletta-com)\n[![LinkedIn](https://img.shields.io/badge/LinkedIn-0077B5?logo=linkedin\u0026logoColor=white)](https://linkedin.com/in/tom-sapletta-com)\n[![ORCID](https://img.shields.io/badge/ORCID-A6CE39?logo=orcid\u0026logoColor=white)](https://orcid.org/0009-0000-6327-2810)\n[![Portfolio](https://img.shields.io/badge/Portfolio-000000?style=flat\u0026logo=about.me\u0026logoColor=white)](https://www.digitname.com/)\n\n## Support This Project\n\nIf you find this project useful, please consider supporting it:\n\n- [GitHub Sponsors](https://github.com/sponsors/tom-sapletta-com)\n- [Open Collective](https://opencollective.com/tom-sapletta-com)\n- [PayPal](https://www.paypal.me/softreck/10.00)\n- [Donate via Softreck](https://donate.softreck.dev)\n\n---\n\ngetllm is a Python package for managing LLM models with Ollama integration and generating Python code. It allows you to install, list, set the default model, update the model list, and generate code using LLM models. GetLLM is part of the PyLama ecosystem and integrates with LogLama as the primary service for centralized logging and environment management.\n\n![slides.svg](slides.svg)\n\n## Features\n\n- **Code Generation**: Generate Python code using LLM models\n- **Model Management**: Install, list, and select models\n- **Hugging Face Integration**: Search and install models from Hugging Face\n- **Automatic Model Installation**: Automatically install models when they are not found\n- **Multiple Ollama Installation Options**:\n  - Direct installation using official script\n  - Docker-based installation\n  - Bexy sandbox for testing\n  - Mock mode for development without Ollama\n- **Fallback Mechanisms**: Use fallback models when the requested model is not available\n- **Environment Configuration**: Configure Ollama through environment variables\n- **Special Model Handling**: Special installation process for SpeakLeash Bielik models\n- **Mock Mode**: Support for mock mode without requiring Ollama\n- **Interactive Mode**: Interactive CLI for model selection and code generation\n- **Template System**: Generate code with awareness of platform, dependencies, and more\n- **Code Execution**: Execute generated code directly\n\n## LogLama Integration\n\nPyLLM integrates with LogLama as the primary service in the PyLama ecosystem. This integration provides:\n\n- **Centralized Environment Management**: Environment variables are loaded from the central `.env` file in the `devlama` directory\n- **Shared Configuration**: Model configurations are shared across all PyLama components\n- **Dependency Management**: Dependencies are validated and installed by LogLama\n- **Service Orchestration**: Services are started in the correct order using LogLama CLI\n- **Centralized Logging**: All PyLLM operations are logged to the central LogLama system\n- **Structured Logging**: Logs include component context for better filtering and analysis\n- **Health Monitoring**: LogLama monitors PyLLM service health and availability\n\n---\n\n## General Diagram (Mermaid)\n```mermaid\ngraph TD\n    A[User] --\u003e|CLI/Interactive| B[getllm/cli.py]\n    B --\u003e C[models.py]\n    B --\u003e D[interactive_cli.py]\n    C --\u003e E[LogLama Central .env]\n    C --\u003e F[Ollama API]\n    D --\u003e B\n    G[LogLama] --\u003e E\n```\n\n---\n\n## ASCII Diagram: CLI Command Flow\n```\nUser\n    |\n    v\n+-----------------+\n|   getllm CLI     |\n+-----------------+\n    |\n    v\n+-----------------+\n|   models.py     |\n+-----------------+\n    |\n+-----------------+\n| LogLama Central |\n|    .env File    |\n+-----------------+\n    |\n+-----------------+\n|  Ollama API     |\n+-----------------+\n```\n\n---\n\n## Usage\n\n### Basic Usage\n\n```bash\n# Start interactive mode\ngetllm -i\n\n# List available models\ngetllm list\n\n# Install a model\ngetllm install codellama:7b\n\n# Set default model\ngetllm set-default codellama:7b\n\n# Search for models on Hugging Face\ngetllm --search bielik\n\n# Update models list from Hugging Face\ngetllm --update-hf\n```\n\n\n### Model Management\n\n```bash\n# List available models\ngetllm list\n\n# Install a model\ngetllm install codellama:7b\n\n# List installed models\ngetllm installed\n\n# Set default model\ngetllm set-default codellama:7b\n\n# Show default model\ngetllm default\n\n# Update models list from Ollama\ngetllm update\n```\n\n### Hugging Face Integration\n\nThe Hugging Face integration allows you to search for and install models directly from Hugging Face:\n\n```bash\n# Search for models on Hugging Face\ngetllm --search bielik\n\n# Update models list from Hugging Face\ngetllm --update-hf\n```\n\n## Testing\n\nFor comprehensive testing instructions, please see [TEST.md](TEST.md).\n\nThe TEST.md file includes:\n- Docker-based testing environment setup\n- Local testing instructions\n- Troubleshooting common test issues\n- Continuous integration information\n\n## Known Issues\n\n### Ollama Dependency Error\n\nWhen running `getllm --search` or other commands that interact with Ollama, you might encounter this error:\n\n```\nError installing model: [Errno 2] No such file or directory: 'ollama'\n```\n\nThis happens because getllm requires the Ollama binary to be installed and available in your PATH.\n\n#### Solutions:\n\n1. **Install Ollama**: Follow the instructions at [ollama.com](https://ollama.com) to install Ollama on your system.\n\n2. **Use Mock Mode**: If you can't install Ollama, use the mock mode:\n   ```bash\n   getllm --mock --search llama\n   ```\n\n3. **Use Docker Testing Environment**: Use our Docker testing environment which includes Ollama:\n   ```bash\n   make docker-test-with-ollama\n   ```\n\n### Other Known Issues\n\n- **Direct Code Generation**: The direct code generation functionality (e.g., `getllm \"create a function\"`) is currently experiencing timeout issues with the Ollama API. Use the interactive mode (`getllm -i`) for code generation in the meantime.\n\n- **Timeout Errors**: When using direct code generation, you might encounter timeout errors like `ReadTimeoutError: HTTPConnectionPool(host='localhost', port=11434): Read timed out`. This indicates that the Ollama server is not responding in time, which could be due to:\n  - The model is too large for your system's resources\n  - The Ollama server is busy with other requests\n  - The prompt requires too much processing time\n\n- **Workaround**: For now, the recommended approach is to use the interactive mode (`getllm -i`), which provides a more stable interface for code generation and model management.\n\n### Interactive Mode\n\n```bash\n# Start interactive mode\ngetllm -i\n\n# Start interactive mode with mock implementation\ngetllm -i --mock\n```\n\n---\n\n## Installation\n\n```bash\n# Create a virtual environment\npython -m venv venv\nsource venv/bin/activate  # On Windows: venv\\Scripts\\activate\n\n# Install the package in development mode\npip install -e .  # This is important! Always install in development mode before starting\n```\n\n\u003e **IMPORTANT**: Always run `pip install -e .` before starting the project to ensure all dependencies are properly installed and the package is available in development mode.\n\n---\n\n## Using the Makefile\n\nPyLLM includes a Makefile to simplify common development tasks:\n\n```bash\n# Set up the project (creates a virtual environment and installs dependencies)\nmake setup\n\n# Run the API server (default port 8001)\nmake run\n\n# Run the API server on a custom port\nmake run PORT=8080\n\n# The run-port command is also available for backward compatibility\nmake run-port PORT=8080\n\n# Run tests\nmake test\n\n# Format code with black\nmake format\n\n# Lint code with flake8\nmake lint\n\n# Clean up project (remove __pycache__, etc.)\nmake clean\n\n# Show all available commands\nmake help\n```\n\n---\n\n## Key Files\n\n- `getllm/cli.py` – main CLI\n- `getllm/interactive_cli.py` – interactive shell with menu and cursor selection\n- `getllm/models.py` – model logic, .env/env.example handling, Ollama integration\n- `.env`/`env.example` – environment config and default model\n\n---\n\n## Example Usage\n\nSearch polish moel bielik in huggingface\n```bash\ngetllm --search bielik\n```\nfrom huggingface \n\n```bash\nSearching for models matching 'bielik' on Hugging Face...\nSearching for models matching 'bielik' on Hugging Face...\n? Select a model to install: (Use arrow keys)\n » speakleash/Bielik-11B-v2.3-Instruct-FP8            Unknown    Downloads: 26,103 |\n   speakleash/Bielik-11B-v2.3-Instruct-GGUF           Unknown    Downloads: 2,203 |\n   speakleash/Bielik-4.5B-v3.0-Instruct-GGUF          Unknown    Downloads: 967 |\n   speakleash/Bielik-7B-Instruct-v0.1-GGUF            Unknown    Downloads: 712 |\n   speakleash/Bielik-1.5B-v3.0-Instruct-GGUF          Unknown    Downloads: 423 |\n   bartowski/Bielik-11B-v2.2-Instruct-GGUF            Unknown    Downloads: 382 |\n   gaianet/Bielik-4.5B-v3.0-Instruct-GGUF             Unknown    Downloads: 338 |\n   second-state/Bielik-1.5B-v3.0-Instruct-GGUF        Unknown    Downloads: 314 |\n   second-state/Bielik-4.5B-v3.0-Instruct-GGUF        Unknown    Downloads: 306 |\n   DevQuasar/speakleash.Bielik-4.5B-v3.0-Instruct-GGUF Unknown    Downloads: 219 |\n   DevQuasar/speakleash.Bielik-1.5B-v3.0-Instruct-GGUF Unknown    Downloads: 219 |\n   gaianet/Bielik-11B-v2.3-Instruct-GGUF              Unknown    Downloads: 173 |\n   tensorblock/Bielik-11B-v2.2-Instruct-GGUF          Unknown    Downloads: 168 |\n   speakleash/Bielik-11B-v2.2-Instruct-GGUF           Unknown    Downloads: 162 |\n   mradermacher/Bielik-11B-v2-i1-GGUF                 Unknown    Downloads: 147 |\n   gaianet/Bielik-1.5B-v3.0-Instruct-GGUF             Unknown    Downloads: 145 |\n   QuantFactory/Bielik-7B-v0.1-GGUF                   Unknown    Downloads: 135 |\n   second-state/Bielik-11B-v2.3-Instruct-GGUF         Unknown    Downloads: 125 |\n   RichardErkhov/speakleash_-_Bielik-11B-v2.1-Instruct-gguf Unknown    Downloads: 113 |\n   mradermacher/Bielik-7B-v0.1-GGUF                   Unknown    Downloads: 94 |\n   Cancel\n```\n\non local environment\n```bash\nSearching for models matching 'bielik' on Hugging Face...\nSearching for models matching 'bielik' on Hugging Face...\n? Select a model to install: speakleash/Bielik-1.5B-v3.0-Instruct-GGUF          Unknown    Downloads: 423 | \n? Do you want to install this model now? Yes\n\nDetected SpeakLeash Bielik model: speakleash/Bielik-1.5B-v3.0-Instruct-GGUF\nStarting special installation process...\n\nFound existing Bielik model installation: bielik-custom-1747866289:latest\nUsing existing model instead of downloading again.\nIncreased API timeout to 120 seconds for Bielik model.\nUpdated .env file with model settings: ~/getllm/.env\n```    \n\n### List available models\n```bash\ngetllm list\n```\n\n### Install a model\n```bash\ngetllm install deepseek-coder:6.7b\n```\n\n### Set default model\n```bash\ngetllm set-default deepseek-coder:6.7b\n```\n\n### Show default model\n```bash\ngetllm default\n```\n\n### Update model list from Ollama\n```bash\ngetllm update\n```\n\n### Run interactive mode (menu, cursor selection)\n```bash\ngetllm -i\n```\n\n---\n\n## set_default_model function flow (Mermaid)\n```mermaid\nflowchart TD\n    S[Start] --\u003e C{Does .env exist?}\n    C -- Yes --\u003e R[Update OLLAMA_MODEL in .env]\n    C -- No --\u003e K[Copy env.example to .env]\n    K --\u003e R\n    R --\u003e E[End]\n```\n\n---\n\n## Interactive mode - menu (ASCII)\n```\n+--------------------------------+\n|  getllm - interactive mode       |\n+--------------------------------+\n| \u003e List available models         |\n|   Show default model           |\n|   List installed models        |\n|   Install model                |\n|   Set default model            |\n|   Update model list            |\n|   Test default model           |\n|   Exit                         |\n+--------------------------------+\n  (navigation: arrow keys + Enter)\n```\n\n---\n\n## Installation\n\n```bash\npip install getllm\n```\n\n## Usage\n\n### Basic Model Management\n\n```python\nfrom getllm import get_models, get_default_model, set_default_model, install_model\n\n# Get available models\nmodels = get_models()\nfor model in models:\n    print(f\"{model['name']} - {model.get('desc', '')}\")\n\n# Get the current default model\ndefault_model = get_default_model()\nprint(f\"Current default model: {default_model}\")\n\n# Set a new default model\nset_default_model(\"codellama:7b\")\n\n# Install a model\ninstall_model(\"deepseek-coder:6.7b\")\n```\n\n### Direct Ollama Integration\n\n```python\nfrom getllm import OllamaServer\n\n# Start the Ollama server if it's not already running\nollama = OllamaServer()\n\n# Or create an OllamaServer instance with a specific model\nollama = OllamaServer(model=\"codellama:7b\")\n\n# Check if the model is available\nif ollama.check_model_availability():\n    print(f\"Model {ollama.model} is available\")\nelse:\n    print(f\"Model {ollama.model} is not available\")\n\n    # Install the model\n    if ollama.install_model(ollama.model):\n        print(f\"Successfully installed {ollama.model}\")\n\n# List installed models\ninstalled_models = ollama.list_installed_models()\nfor model in installed_models:\n    print(f\"Installed model: {model['name']}\")\n```\n\n## Ollama Installation Options\n\nGetLLM now offers multiple ways to install and use Ollama:\n\n### 1. Direct Installation (Recommended)\n\nInstalls Ollama directly on your system using the official installation script:\n\n```bash\n# When prompted during model search or installation\n$ getllm --search bielik\n# Select 'Install Ollama directly (recommended)' when prompted\n```\n\n### 2. Docker-based Installation\n\nInstalls and runs Ollama in a Docker container:\n\n```bash\n# When prompted during model search or installation\n$ getllm --search bielik\n# Select 'Install Ollama using Docker' when prompted\n```\n\nRequires Docker to be installed on your system.\n\n### 3. Bexy Sandbox\n\nRuns Ollama in a sandboxed environment using the bexy package:\n\n```bash\n# When prompted during model search or installation\n$ getllm --search bielik\n# Select 'Use bexy sandbox for testing' when prompted\n```\n\nRequires the bexy package to be available in your project.\n\n### 4. Mock Mode\n\nRun getllm without Ollama for testing and development:\n\n```bash\n# Use the --mock flag with any command\n$ getllm --mock --search bielik\n$ getllm --mock code 'Write a function to calculate factorial'\n\n# Or select 'Continue in mock mode' when prompted during installation\n```\n\n## Testing\n\nGetLLM includes several test suites to ensure all features work correctly:\n\n```bash\n# Run unit tests\nmake test\n\n# Test command-line functionality\nmake test-commands\n\n# Test installation options\nmake test-installation\n\n# Test model installation\nmake test-models\n\n# Run all tests\nmake test-all\n```\n\n## Environment Variables\n\nThe package uses the following environment variables for Ollama integration:\n\n- `OLLAMA_PATH`: Path to the Ollama executable (default: 'ollama')\n- `OLLAMA_MODEL`: Default model to use (default: 'codellama:7b')\n- `OLLAMA_FALLBACK_MODELS`: Comma-separated list of fallback models (default: 'codellama:7b,phi3:latest,tinyllama:latest')\n- `OLLAMA_AUTO_SELECT_MODEL`: Whether to automatically select an available model if the requested model is not found (default: 'true')\n- `OLLAMA_AUTO_INSTALL_MODEL`: Whether to automatically install a model when it's not found (default: 'true')\n- `OLLAMA_TIMEOUT`: API timeout in seconds (default: '30')\n\nThese variables can be set in a .env file in the project root directory or in the system environment.\n\n## License\nThis project is licensed under the Apache 2.0 License (see LICENSE file).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpy-lama%2Fgetllm","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpy-lama%2Fgetllm","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpy-lama%2Fgetllm/lists"}