{"id":13907440,"url":"https://github.com/ashleykleynhans/llava-docker","last_synced_at":"2025-12-24T10:35:23.096Z","repository":{"id":199585785,"uuid":"703259283","full_name":"ashleykleynhans/llava-docker","owner":"ashleykleynhans","description":"Docker image for LLaVA: Large Language and Vision Assistant","archived":false,"fork":false,"pushed_at":"2025-05-17T16:45:03.000Z","size":100,"stargazers_count":2,"open_issues_count":1,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-06-12T21:55:57.221Z","etag":null,"topics":["ai","chatbot","chatgpt","docker","docker-image","foundation-models","gpt-4","instruction-tuning","llama","llama-2","llama2","llava","llm","multimodal","runpod","vision-language-model","visual-language-learning"],"latest_commit_sha":null,"homepage":"","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ashleykleynhans.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null},"funding":{"github":"ashleykleynhans","buy_me_a_coffee":"ashleyk"}},"created_at":"2023-10-10T22:54:55.000Z","updated_at":"2025-06-07T11:28:27.000Z","dependencies_parsed_at":"2024-04-02T08:28:37.690Z","dependency_job_id":"14eb47b1-a760-4b50-beef-277a1c1d7349","html_url":"https://github.com/ashleykleynhans/llava-docker","commit_stats":null,"previous_names":["ashleykleynhans/llava-docker"],"tags_count":18,"template":false,"template_full_name":null,"purl":"pkg:github/ashleykleynhans/llava-docker","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ashleykleynhans%2Fllava-docker","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ashleykleynhans%2Fllava-docker/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ashleykleynhans%2Fllava-docker/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ashleykleynhans%2Fllava-docker/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ashleykleynhans","download_url":"https://codeload.github.com/ashleykleynhans/llava-docker/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ashleykleynhans%2Fllava-docker/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":261556493,"owners_count":23176673,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","chatbot","chatgpt","docker","docker-image","foundation-models","gpt-4","instruction-tuning","llama","llama-2","llama2","llava","llm","multimodal","runpod","vision-language-model","visual-language-learning"],"created_at":"2024-08-06T23:01:56.395Z","updated_at":"2025-12-24T10:35:23.014Z","avatar_url":"https://github.com/ashleykleynhans.png","language":"Shell","funding_links":["https://github.com/sponsors/ashleykleynhans","https://buymeacoffee.com/ashleyk","https://www.buymeacoffee.com/ashleyk"],"categories":["HarmonyOS"],"sub_categories":["Windows Manager"],"readme":"\u003cdiv align=\"center\"\u003e\n\n# Docker image for LLaVA: Large Language and Vision Assistant\n\n![Docker Pulls](https://img.shields.io/docker/pulls/ashleykza/llava?style=for-the-badge\u0026logo=docker\u0026label=Docker%20Pulls\u0026link=https%3A%2F%2Fhub.docker.com%2Frepository%2Fdocker%2Fashleykza%2Fllava%2Fgeneral)\n![Template Version](https://img.shields.io/github/v/tag/ashleykleynhans/llava-docker?style=for-the-badge\u0026logo=data%3Aimage%2Fsvg%2Bxml%3Bbase64%2CPD94bWwgdmVyc2lvbj0iMS4wIiBlbmNvZGluZz0idXRmLTgiPz4KPCEtLSBHZW5lcmF0b3I6IEFkb2JlIElsbHVzdHJhdG9yIDI2LjUuMywgU1ZHIEV4cG9ydCBQbHVnLUluIC4gU1ZHIFZlcnNpb246IDYuMDAgQnVpbGQgMCkgIC0tPgo8c3ZnIHZlcnNpb249IjEuMSIgaWQ9IkxheWVyXzEiIHhtbG5zPSJodHRwOi8vd3d3LnczLm9yZy8yMDAwL3N2ZyIgeG1sbnM6eGxpbms9Imh0dHA6Ly93d3cudzMub3JnLzE5OTkveGxpbmsiIHg9IjBweCIgeT0iMHB4IgoJIHZpZXdCb3g9IjAgMCAyMDAwIDIwMDAiIHN0eWxlPSJlbmFibGUtYmFja2dyb3VuZDpuZXcgMCAwIDIwMDAgMjAwMDsiIHhtbDpzcGFjZT0icHJlc2VydmUiPgo8c3R5bGUgdHlwZT0idGV4dC9jc3MiPgoJLnN0MHtmaWxsOiM2NzNBQjc7fQo8L3N0eWxlPgo8Zz4KCTxnPgoJCTxwYXRoIGNsYXNzPSJzdDAiIGQ9Ik0xMDE3Ljk1LDcxMS4wNGMtNC4yMiwyLjM2LTkuMTgsMy4wMS0xMy44NiwxLjgyTDM4Ni4xNyw1NTUuM2MtNDEuNzItMTAuNzYtODYuMDItMC42My0xMTYuNiwyOS43MwoJCQlsLTEuNCwxLjM5Yy0zNS45MiwzNS42NS0yNy41NSw5NS44LDE2Ljc0LDEyMC4zbDU4NC4zMiwzMjQuMjNjMzEuMzYsMTcuNCw1MC44Miw1MC40NSw1MC44Miw4Ni4zMnY4MDYuNzYKCQkJYzAsMzUuNDktMzguNDEsNTcuNjctNjkuMTUsMzkuOTRsLTcwMy4xNS00MDUuNjRjLTIzLjYtMTMuNjEtMzguMTMtMzguNzgtMzguMTMtNjYuMDJWNjY2LjYzYzAtODcuMjQsNDYuNDUtMTY3Ljg5LDEyMS45Mi0yMTEuNjYKCQkJTDkzMy44NSw0Mi4xNWMyMy40OC0xMy44LDUxLjQ3LTE3LjcsNzcuODMtMTAuODRsNzQ1LjcxLDE5NC4xYzMxLjUzLDguMjEsMzYuOTksNTAuNjUsOC41Niw2Ni41N0wxMDE3Ljk1LDcxMS4wNHoiLz4KCQk8cGF0aCBjbGFzcz0ic3QwIiBkPSJNMTUyNy43NSw1MzYuMzhsMTI4Ljg5LTc5LjYzbDE4OS45MiwxMDkuMTdjMjcuMjQsMTUuNjYsNDMuOTcsNDQuNzMsNDMuODIsNzYuMTVsLTQsODU3LjYKCQkJYy0wLjExLDI0LjM5LTEzLjE1LDQ2Ljg5LTM0LjI1LDU5LjExbC03MDEuNzUsNDA2LjYxYy0zMi4zLDE4LjcxLTcyLjc0LTQuNTktNzIuNzQtNDEuOTJ2LTc5Ny40MwoJCQljMC0zOC45OCwyMS4wNi03NC45MSw1NS4wNy05My45Nmw1OTAuMTctMzMwLjUzYzE4LjIzLTEwLjIxLDE4LjY1LTM2LjMsMC43NS00Ny4wOUwxNTI3Ljc1LDUzNi4zOHoiLz4KCQk8cGF0aCBjbGFzcz0ic3QwIiBkPSJNMTUyNC4wMSw2NjUuOTEiLz4KCTwvZz4KPC9nPgo8L3N2Zz4K\u0026logoColor=%23ffffff\u0026label=Template%20Version\u0026color=%23673ab7)\n\n\u003c/div\u003e\n\n## Installs\n\n* Ubuntu 22.04 LTS\n* CUDA 12.1\n* Python 3.10.12\n* [LLaVA](\n  https://github.com/haotian-liu/llava) v1.2.0 (LLaVA 1.6)\n* Torch 2.1.2\n* xformers 0.0.23.post1\n* [Jupyter Lab](https://github.com/jupyterlab/jupyterlab)\n* [code-server](https://github.com/coder/code-server)\n* [runpodctl](https://github.com/runpod/runpodctl)\n* [OhMyRunPod](https://github.com/kodxana/OhMyRunPod)\n* [RunPod File Uploader](https://github.com/kodxana/RunPod-FilleUploader)\n* [croc](https://github.com/schollz/croc)\n* [rclone](https://rclone.org/)\n* speedtest-cli\n* screen\n* tmux\n* llava-v1.6-mistral-7b model\n\n## Available on RunPod\n\nThis image is designed to work on [RunPod](https://runpod.io?ref=2xxro4sy).\nYou can use my custom [RunPod template](\nhttps://runpod.io/gsc?template=g7wd33iuwv\u0026ref=2xxro4sy)\nto launch it on RunPod.\n\n## Building the Docker image\n\n\u003e [!NOTE]\n\u003e You will need to edit the `docker-bake.hcl` file and update `USERNAME`,\n\u003e and `RELEASE`.  You can obviously edit the other values too, but these\n\u003e are the most important ones.\n\n```bash\n# Clone the repo\ngit clone https://github.com/ashleykleynhans/llava-docker.git\n\n# Log in to Docker Hub\ndocker login\n\n# Build the image, tag the image, and push the image to Docker Hub\ncd llava-docker\ndocker buildx bake -f docker-bake.hcl --push\n```\n\n## Running Locally\n\n### Install Nvidia CUDA Driver\n\n- [Linux](https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html)\n- [Windows](https://docs.nvidia.com/cuda/cuda-installation-guide-microsoft-windows/index.html)\n\n### Start the Docker container\n\n```bash\ndocker run -d \\\n  --gpus all \\\n  -v /workspace \\\n  -p 3000:3001 \\\n  -p 7777:7777 \\\n  -p 8888:8888 \\\n  -p 2999:2999 \\\n  ashleykza/llava:latest\n```\n\nYou can obviously substitute the image name and tag with your own.\n\n## Models\n\n\u003e [!IMPORTANT]\n\u003e If you select a 13B or larger model, CUDA will result in OOM errors\n\u003e with a GPU that has less than 48GB of VRAM, so A6000 or higher is\n\u003e recommended for 13B.\n\nYou can add an environment called `MODEL` to your Docker container to\nspecify the model that should be downloaded.  If the `MODEL` environment\nvariable is not set, the model will default to `liuhaotian/llava-v1.6-mistral-7b`.\n\n### LLaVA-v1.6\n\n| Model                                                                            | Environment Variable Value       | Version    | LLM           | Default |\n|----------------------------------------------------------------------------------|----------------------------------|------------|---------------|---------|\n| [llava-v1.6-vicuna-7b](https://huggingface.co/liuhaotian/llava-v1.6-vicuna-7b)   | liuhaotian/llava-v1.6-vicuna-7b  | LLaVA-1.6  | Vicuna-7B     | no      |\n| [llava-v1.6-vicuna-13b](https://huggingface.co/liuhaotian/llava-v1.6-vicuna-13b) | liuhaotian/llava-v1.6-vicuna-13b | LLaVA-1.6  | Vicuna-13B    | no      |\n| [llava-v1.6-mistral-7b](https://huggingface.co/liuhaotian/llava-v1.6-mistral-7b) | liuhaotian/llava-v1.6-mistral-7b | LLaVA-1.6  | Mistral-7B    | yes     |\n| [llava-v1.6-34b](https://huggingface.co/liuhaotian/llava-v1.6-34b)               | liuhaotian/llava-v1.6-34b        | LLaVA-1.6  | Hermes-Yi-34B | no      |\n\n### LLaVA-v1.5\n\n| Model                                                              | Environment Variable Value | Version   | Size | Default |\n|--------------------------------------------------------------------|----------------------------|-----------|------|---------|\n| [llava-v1.5-7b](https://huggingface.co/liuhaotian/llava-v1.5-7b)   | liuhaotian/llava-v1.5-7b   | LLaVA-1.5 | 7B   | no      |\n| [llava-v1.5-13b](https://huggingface.co/liuhaotian/llava-v1.5-13b) | liuhaotian/llava-v1.5-13b  | LLaVA-1.5 | 13B  | no      |\n| [BakLLaVA-1](https://huggingface.co/SkunkworksAI/BakLLaVA-1)       | SkunkworksAI/BakLLaVA-1    | LLaVA-1.5 | 7B   | no      |\n\n## Ports\n\n| Connect Port | Internal Port | Description          |\n|--------------|---------------|----------------------|\n| 3000         | 3001          | LLaVA                |\n| 7777         | 7777          | Code Server          |\n| 8888         | 8888          | Jupyter Lab          |\n| 2999         | 2999          | RunPod File Uploader |\n\n## Environment Variables\n\n| Variable             | Description                                      | Default                          |\n|----------------------|--------------------------------------------------|----------------------------------|\n| JUPYTER_LAB_PASSWORD | Set a password for Jupyter lab                   | not set - no password            |\n| DISABLE_AUTOLAUNCH   | Disable LLaVA from launching automatically       | (not set)                        |\n| DISABLE_SYNC         | Disable syncing if using a RunPod network volume | (not set)                        |\n| MODEL                | The path of the Huggingface model                | liuhaotian/llava-v1.6-mistral-7b |\n\n## Logs\n\nLLaVA creates log files, and you can tail the log files\ninstead of killing the services to view the logs.\n\n| Application   | Log file                         |\n|---------------|----------------------------------|\n| Controller    | /workspace/logs/controller.log   |\n| Webserver     | /workspace/logs/webserver.log    |\n| Model Worker  | /workspace/logs/model-worker.log |\n\nFor example:\n\n```bash\ntail -f /workspace/logs/webserver.log\n```\n\n## Flask API\n\n### Add port 5000\n\nIf you are running the RunPod template, edit your pod and add HTTP port 5000.\n\nIf you are running locally, add a port mapping for port 5000.\n\n### Starting the Flask API\n\n```bash\n# Stop model worker and controller to free up VRAM\nfuser -k 10000/tcp 40000/tcp\n\n# Install dependencies\nsource /venv/bin/activate\npip3 install flask protobuf\ncd /workspace/LLaVA\nexport HF_HOME=\"/workspace\"\npython -m llava.serve.api -H 0.0.0.0 -p 5000\n```\n\n### Sending requests to the Flask API\n\nYou can use the [test script](\nhttps://github.com/ashleykleynhans/LLaVA/blob/main/llava/serve/test_api.py)\nto test your API.\n\n## Acknowledgements\n\n1. Matthew Berman for giving me a demo on LLaVA, as well as his amazing\n   [YouTube videos](https://www.youtube.com/@matthew_berman/videos]).\n\n## Community and Contributing\n\nPull requests and issues on [GitHub](https://github.com/ashleykleynhans/llava-docker)\nare welcome. Bug fixes and new features are encouraged.\n\n## Appreciate my work?\n\n\u003ca href=\"https://www.buymeacoffee.com/ashleyk\" target=\"_blank\"\u003e\u003cimg src=\"https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png\" alt=\"Buy Me A Coffee\" style=\"height: 60px !important;width: 217px !important;\" \u003e\u003c/a\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fashleykleynhans%2Fllava-docker","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fashleykleynhans%2Fllava-docker","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fashleykleynhans%2Fllava-docker/lists"}