{"id":28066944,"url":"https://github.com/nickytonline/secure-llm-pomerium","last_synced_at":"2025-10-06T13:16:25.359Z","repository":{"id":292191786,"uuid":"980101780","full_name":"nickytonline/secure-llm-pomerium","owner":"nickytonline","description":null,"archived":false,"fork":false,"pushed_at":"2025-05-08T15:35:48.000Z","size":4,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-05-08T16:34:20.028Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/nickytonline.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-05-08T15:05:50.000Z","updated_at":"2025-05-08T15:35:51.000Z","dependencies_parsed_at":"2025-05-08T16:35:31.811Z","dependency_job_id":"0597e325-4bda-40f8-a4ac-2e55a8e17906","html_url":"https://github.com/nickytonline/secure-llm-pomerium","commit_stats":null,"previous_names":["nickytonline/secure-llm-pomerium"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nickytonline%2Fsecure-llm-pomerium","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nickytonline%2Fsecure-llm-pomerium/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nickytonline%2Fsecure-llm-pomerium/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nickytonline%2Fsecure-llm-pomerium/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/nickytonline","download_url":"https://codeload.github.com/nickytonline/secure-llm-pomerium/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253770422,"owners_count":21961769,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-05-12T15:55:18.349Z","updated_at":"2025-10-06T13:16:20.340Z","avatar_url":"https://github.com/nickytonline.png","language":null,"funding_links":[],"categories":[],"sub_categories":[],"readme":"# Secure LLM with Pomerium\n\nThis project is based off the instructions in Pomerium's [Self-Hosted LLM Behind Pomerium](https://bit.ly/4376Wki) guide.\n\n## What's do you get?\n\n[Open WebUI](https://github.com/open-webui/open-webui) with your local LLMs, care of [Ollama](https://ollama.com), secured by [Pomerium](https://pomerium.io).\n\n## Prerequisites\n\n- [Pomerium Zero account](https://bit.ly/4k7RIm5)\n- [Docker](https://docs.docker.com/get-docker/) and [Docker Compose](https://docs.docker.com/compose/install/) or [Orbstack](https://orbstack.dev/)\n- Port 443 open on your router or wherever you host this project.\n\n## Configure Pomerium Zero\n\nWhen you create a Pomerium Zero account, you will be taken to the Pomerium Zero console. Here you can create a new route for Open WebUI. There will already be one for the verify route which is just a page to show your claims in your JWT when logged in.\n\nYou'll also have one policy out of the box which will allow access to only you via the email you used to sign up with.\n\n[Follow the steps in the docs](https://bit.ly/4376Wki#configure-pomerium-zero) to add a route for Open WebUI and a policy to allow access to it.\n\n\n## Environment Configuration\n\nTo configure the environment for this project, you need to set up the `.env` file with the following variables:\n\n```env\nPOMERIUM_ZERO_TOKEN=replace-with-your-pomerium-zero-token\nWEBUI_URL=https://llm.yourdomain.pomerium.app # or whatever you called your route for Open WebUI in the Pomerium Zero console.\nVERIFY_URL=https://verify.yourdomain.pomerium.app\nGPU_DEVICE=/dev/dri/your-device # e.g. /dev/dri/renderD128\n```\n\nMake sure to replace the placeholder values with your actual configuration details. This setup is crucial for the proper functioning of the Pomerium Zero integration and GPU acceleration if applicable.\n\n### GPU Acceleration (Optional)\n\nIf your machine has a GPU (e.g. AMD, Intel, or NVIDIA), you can enable GPU access for Open WebUI to accelerate model inference.To do so, uncomment the `devices` and `group_add` lines in the `docker-compose.yaml` and configure your GPU device in your `.env` file.\n\nFor example, on an AMD GPU (using `/dev/dri/renderD128`):\n\n```env\nGPU_DEVICE=/dev/dri/renderD128\n```\n\nMake sure your user has access to the GPU device (usually by being in the `video` group):\n\n```bash\ngetent group video\n```\n\n**Note:** GPU acceleration requires that Open WebUI and Ollama support your GPU type. Currently, support for AMD and Intel GPUs may be limited or experimental.\n\n## Adding Local Models\n\nTo add local models within the container, follow these steps:\n\n1. **Access the Container**:\n   - First, make sure your container is running. You can access it using:\n     ```bash\n     docker exec -it open-webui /bin/bash\n     ```\n\n2. **Download Local Models**:\n   - Once inside the container, you can download the desired local models using:\n     ```bash\n     ollama pull \u003cmodel_name\u003e\n     ```\n   - Replace `\u003cmodel_name\u003e` with the specific model you want to pull.\n\n3. **Verify the Model**:\n   - After pulling, verify that the model has been successfully added by listing the available models:\n     ```bash\n     ollama list\n     ```\n\nEnsure that your container has internet access to pull models successfully. If you encounter any issues, check your container's network settings or consult the Ollama documentation for troubleshooting tips.\n\n## Custom Domain\n\nYou can also use your own domain with Pomerium Zero. For more information, see the Pomerium Zero [documentation for custom domains](https://www.pomerium.com/docs/capabilities/custom-domains).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnickytonline%2Fsecure-llm-pomerium","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnickytonline%2Fsecure-llm-pomerium","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnickytonline%2Fsecure-llm-pomerium/lists"}