https://github.com/ashleykleynhans/textgen-docker
Docker image for the Text Generation Web UI: A Gradio web UI for Large Language Models with support for multiple inference backends.
https://github.com/ashleykleynhans/textgen-docker
api docker docker-image gradio llama llama-cpp llm runpod textgen webui
Last synced: 1 day ago
JSON representation
Docker image for the Text Generation Web UI: A Gradio web UI for Large Language Models with support for multiple inference backends.
- Host: GitHub
- URL: https://github.com/ashleykleynhans/textgen-docker
- Owner: ashleykleynhans
- License: gpl-3.0
- Created: 2023-06-11T15:35:26.000Z (about 3 years ago)
- Default Branch: main
- Last Pushed: 2026-05-21T08:50:30.000Z (27 days ago)
- Last Synced: 2026-06-07T09:05:04.325Z (10 days ago)
- Topics: api, docker, docker-image, gradio, llama, llama-cpp, llm, runpod, textgen, webui
- Language: Python
- Homepage:
- Size: 313 KB
- Stars: 4
- Watchers: 1
- Forks: 1
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Funding: .github/FUNDING.yml
- License: LICENSE
Awesome Lists containing this project
README
# Docker image for TextGen: The original local LLM interface. Text, vision, tool-calling, training. UI + API, 100% offline and private.
## Installs
* Ubuntu 22.04 LTS
* CUDA 12.8.1
* Python 3.13
* Torch 2.9.1
* [TextGen](
https://github.com/oobabooga/textgen) v4.9
* [Jupyter Lab](https://github.com/jupyterlab/jupyterlab)
* [code-server](https://github.com/coder/code-server)
* [runpodctl](https://github.com/runpod/runpodctl)
* [OhMyRunPod](https://github.com/kodxana/OhMyRunPod)
* [RunPod File Uploader](https://github.com/kodxana/RunPod-FilleUploader)
* [croc](https://github.com/schollz/croc)
* [rclone](https://rclone.org/)
* speedtest-cli
* screen
* tmux
## Available on RunPod
This image is designed to work on [RunPod](https://runpod.io?ref=2xxro4sy).
You can use my custom [RunPod template](
https://runpod.io/console/deploy?template=t9e1ycivcp&ref=2xxro4sy)
to launch it on RunPod.
## Building the Docker image
> [!NOTE]
> You will need to edit the `docker-bake.hcl` file and update `REGISTRY_USER`,
> and `RELEASE`. You can obviously edit the other values too, but these
> are the most important ones.
```bash
# Clone the repo
git clone https://github.com/ashleykleynhans/textgen-docker.git
# Log in to Docker Hub
docker login
# Build the image, tag the image, and push the image to Docker Hub
cd textgen-docker
docker buildx bake -f docker-bake.hcl --push
# Same as above but customize registry/user/release:
REGISTRY=ghcr.io REGISTRY_USER=myuser RELEASE=my-release docker buildx \
bake -f docker-bake.hcl --push
```
## Running Locally
### Install Nvidia CUDA Driver
- [Linux](https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html)
- [Windows](https://docs.nvidia.com/cuda/cuda-installation-guide-microsoft-windows/index.html)
### Start the Docker container
```bash
docker run -d \
--gpus all \
-v /workspace \
-p 3000:3001 \
-p 5000:5001 \
-p 7777:7777 \
-p 8888:8888 \
-p 2999:2999 \
-e VENV_PATH="/workspace/venvs/textgen" \
ashleykza/textgen:latest
```
You can obviously substitute the image name and tag with your own.
## Ports
| Connect Port | Internal Port | Description |
|--------------|---------------|----------------------------------|
| 3000 | 3001 | TextGen |
| 5000 | 5001 | Open AI/Anthropic Compatible API |
| 7777 | 7777 | Code Server |
| 8888 | 8888 | Jupyter Lab |
| 2999 | 2999 | Runpod File Uploader |
## Environment Variables
| Variable | Description | Default |
|----------------------|---------------------------------------------------------------|--------------------------|
| VENV_PATH | Set the path for the Python venv for the app | /workspace/venvs/textgen |
| JUPYTER_LAB_PASSWORD | Set a password for Jupyter lab | not set - no password |
| DISABLE_AUTOLAUNCH | Disable Web UI from launching automatically | (not set) |
| DISABLE_SYNC | Disable syncing if using a RunPod network volume | (not set) |
| HF_TOKEN | Hugging Face Hub token for models that require authentication | (not set) |
## Logs
TextGen creates a log file, and you can tail the log instead of
killing the service to view the logs.
| Application | Log file |
|-------------|-----------------------------|
| TextGen | /workspace/logs/textgen.log |
For example:
```bash
tail -f /workspace/logs/textgen.log
```
## Community and Contributing
Pull requests and issues on [GitHub](https://github.com/ashleykleynhans/textgen-docker)
are welcome. Bug fixes and new features are encouraged.
## Appreciate my work?
