{"id":30805564,"url":"https://github.com/aaronsb/ollama-docker","last_synced_at":"2025-09-06T00:59:12.699Z","repository":{"id":293477856,"uuid":"984161710","full_name":"aaronsb/ollama-docker","owner":"aaronsb","description":null,"archived":false,"fork":false,"pushed_at":"2025-05-15T15:08:47.000Z","size":9,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-05-15T15:24:08.700Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/aaronsb.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-05-15T13:45:09.000Z","updated_at":"2025-05-15T15:08:51.000Z","dependencies_parsed_at":"2025-05-15T15:24:17.952Z","dependency_job_id":"29b11ce6-97c3-4939-8d39-e821dbf3ad41","html_url":"https://github.com/aaronsb/ollama-docker","commit_stats":null,"previous_names":["aaronsb/ollama-docker"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/aaronsb/ollama-docker","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aaronsb%2Follama-docker","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aaronsb%2Follama-docker/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aaronsb%2Follama-docker/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aaronsb%2Follama-docker/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/aaronsb","download_url":"https://codeload.github.com/aaronsb/ollama-docker/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aaronsb%2Follama-docker/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":273842816,"owners_count":25177921,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-05T02:00:09.113Z","response_time":402,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-09-06T00:59:07.536Z","updated_at":"2025-09-06T00:59:12.674Z","avatar_url":"https://github.com/aaronsb.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Ollama Docker Setup with GPU Support\n\nThis repository contains a Docker Compose configuration for running Ollama with automatic GPU detection and acceleration. It supports:\n\n- NVIDIA GPUs (via NVIDIA Container Runtime)\n- Intel iGPUs (via OpenVINO)\n- CPU-only fallback mode\n\n## Prerequisites\n\nBefore you begin, ensure you have the following installed:\n\n- Docker Engine\n- Docker Compose (V2 recommended)\n\n### For NVIDIA GPU Support\n\n- NVIDIA GPU with appropriate drivers\n- [NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html)\n\n### For Intel iGPU Support\n\n- Intel GPU drivers (i915)\n- Intel Compute Runtime for OpenCL\n\n```bash\n# For Ubuntu/Debian\nsudo apt-get update\nsudo apt-get install -y intel-opencl-icd intel-level-zero-gpu level-zero \\\n  intel-media-va-driver-non-free libmfx1 libmfxgen1 libvpl2 \\\n  intel-gpu-tools\n  \n# Add your user to the 'video' group\nsudo usermod -a -G video $USER\n# Log out and log back in for changes to take effect\n```\n\n## Getting Started\n\n### Starting Ollama\n\nThe `start-ollama.sh` script automatically detects your available GPU hardware and configures the appropriate Docker Compose setup.\n\n```bash\n# Make the script executable if needed\nchmod +x start-ollama.sh\n\n# Run the setup script\n./start-ollama.sh\n```\n\nThis script will:\n1. Detect available GPU hardware (NVIDIA, Intel iGPU, or fallback to CPU)\n2. Create a symlink to the appropriate Docker Compose configuration\n3. Start the Ollama container with the correct GPU access\n4. Verify the Ollama API is responsive\n5. Offer to pull and run an initial model\n\n### Stopping Ollama\n\nTo stop the Ollama container:\n\n```bash\ndocker compose down\n```\n\n## Using Ollama\n\n### API Access\n\nThe Ollama API is available at:\n\n```\nhttp://localhost:11434\n```\n\n### Helper Script\n\nThis repository includes an `ollama` helper script that simplifies interactions with the containerized Ollama instance:\n\n```bash\n# Make the script executable\nchmod +x ollama\n\n# Use the script just like the regular ollama command\n./ollama list\n./ollama run llama3\n./ollama pull mistral\n```\n\nThe script automatically passes commands to the Docker container and handles interactive/non-interactive mode detection.\n\n### Running Models Directly\n\nYou can also run models directly using the Docker container:\n\n```bash\n# Run a model\ndocker exec -it ollama ollama run llama3\n\n# List available models\ndocker exec -it ollama ollama list\n\n# Pull a new model\ndocker exec -it ollama ollama pull mistral\n```\n\n### Using with Local Ollama Client\n\nIf you have the Ollama client installed locally, you can use it to interact with the Docker instance:\n\n```bash\nollama list\nollama run llama3\n```\n\nThe client will automatically connect to the API endpoint at localhost:11434.\n\n## Managing Models\n\nModels are stored in the Docker volume `ollama_data`. This ensures they persist even if the container is removed.\n\n```bash\n# List models\ndocker exec -it ollama ollama list\n\n# Pull a model\ndocker exec -it ollama ollama pull llama3\n\n# Remove a model\ndocker exec -it ollama ollama rm llama3\n```\n\n## Troubleshooting\n\n### Checking GPU Access\n\nTo verify that Ollama can access the GPU:\n\n#### For NVIDIA GPUs:\n\n```bash\n# Check if NVIDIA GPU is visible to the container\ndocker exec -it ollama nvidia-smi\n```\n\n#### For Intel GPUs:\n\n```bash\n# Check if Intel GPU devices are properly mounted\ndocker exec -it ollama ls -la /dev/dri/\n# You should see devices like card1, card2, renderD128, renderD129\n\n# Check OpenCL devices (install clinfo in the container if needed)\ndocker exec -it ollama apt-get update \u0026\u0026 apt-get install -y clinfo\ndocker exec -it ollama clinfo\n\n# Check if the container has access to the video group\ndocker exec -it ollama groups\n# Should include 'video' in the output\n```\n\n### Viewing Logs\n\nTo view the Ollama container logs:\n\n```bash\ndocker logs -f ollama\n```\n\n### Common Issues\n\n1. **NVIDIA GPU not detected**: Ensure NVIDIA Container Toolkit is properly installed and configured.\n\n2. **Intel GPU not detected**: Ensure Intel GPU drivers are properly installed and the /dev/dri devices are available.\n\n3. **Permission issues**: Make sure your user is in the 'video' group and the container has the proper device mappings.\n\n4. **OpenVINO errors**: If you see errors related to OpenVINO, ensure the Intel Compute Runtime is properly installed.\n\n5. **Network connectivity**: The container is configured to use host networking for optimal performance.\n\n## Advanced Configuration\n\n### Environment Variables\n\nYou can customize Ollama operation by editing the appropriate Docker Compose file and modifying the environment variables:\n\n```yaml\nservices:\n  ollama:\n    # existing configuration...\n    environment:\n      - OLLAMA_HOST=0.0.0.0\n      - OLLAMA_ORIGINS=chrome-extension://*  # For Chrome extension support\n      # Add other environment variables as needed\n```\n\n#### Chrome Extension Support\n\nTo use Ollama with Chrome extensions, you must set the `OLLAMA_ORIGINS` environment variable. There are two ways to do this:\n\n1. **Allow Specific Extension** (Recommended):\n   ```bash\n   # Replace [YOUR_EXTENSION_ID] with your Chrome extension ID\n   export OLLAMA_ORIGINS=\"chrome-extension://[YOUR_EXTENSION_ID]\"\n   ```\n   Find your extension ID in chrome://extensions/ (enable Developer mode).\n\n2. **Allow All Extensions** (Less Secure):\n   ```bash\n   export OLLAMA_ORIGINS=\"chrome-extension://*\"\n   ```\n\nMake this setting permanent by adding it to your shell profile (~/.bashrc, ~/.zshrc, etc.).\n\nThe included `start-ollama.sh` script will prompt you to configure this if not already set.\n\n### Custom Models\n\nTo use custom models, you can either:\n\n1. Pull them using the Ollama CLI\n2. Import them using the Ollama API\n3. Create a custom model with a modelfile:\n\n```bash\n# Create a custom model with specific parameters\ndocker exec -it ollama ollama create mymodel -f /path/to/modelfile\n```\n\nExample modelfile (see the included sample):\n```\nFROM qwen3:8b\nPARAMETER num_gpu 1\nPARAMETER num_thread 16\nPARAMETER num_ctx 131072  \nPARAMETER num_batch 1024\n```\n\n## License\n\nOllama is licensed under the MIT License. See the [Ollama GitHub repository](https://github.com/ollama/ollama) for more details.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faaronsb%2Follama-docker","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Faaronsb%2Follama-docker","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faaronsb%2Follama-docker/lists"}