{"id":48588586,"url":"https://github.com/mitja/llamatunnel","last_synced_at":"2026-04-08T18:16:29.791Z","repository":{"id":226936406,"uuid":"769853287","full_name":"mitja/llamatunnel","owner":"mitja","description":"Publish local LLMs and LLM apps on the internet.","archived":false,"fork":false,"pushed_at":"2025-08-17T09:16:10.000Z","size":1860,"stargazers_count":27,"open_issues_count":0,"forks_count":4,"subscribers_count":1,"default_branch":"main","last_synced_at":"2026-01-26T17:50:41.814Z","etag":null,"topics":["caddy","cloudflare-tunnel","docker-compose","generativeai","llm","ollama","openwebui"],"latest_commit_sha":null,"homepage":"https://llamatunnel.com","language":"Jinja","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mitja.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-03-10T08:48:30.000Z","updated_at":"2025-11-01T05:17:10.000Z","dependencies_parsed_at":"2024-05-13T20:41:40.987Z","dependency_job_id":null,"html_url":"https://github.com/mitja/llamatunnel","commit_stats":null,"previous_names":["mitja/llamatunnel"],"tags_count":11,"template":false,"template_full_name":null,"purl":"pkg:github/mitja/llamatunnel","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mitja%2Fllamatunnel","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mitja%2Fllamatunnel/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mitja%2Fllamatunnel/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mitja%2Fllamatunnel/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mitja","download_url":"https://codeload.github.com/mitja/llamatunnel/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mitja%2Fllamatunnel/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31567766,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-08T14:31:17.711Z","status":"ssl_error","status_checked_at":"2026-04-08T14:31:17.202Z","response_time":54,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["caddy","cloudflare-tunnel","docker-compose","generativeai","llm","ollama","openwebui"],"created_at":"2026-04-08T18:16:24.231Z","updated_at":"2026-04-08T18:16:28.703Z","avatar_url":"https://github.com/mitja.png","language":"Jinja","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Llama Tunnel\n\n**Publish Local LLMs and Apps on the Internet.**\n\nWith Llama Tunnel, you can publish your local LLM APIs and apps on the internet so that you can use your local LLMs and LLM apps on the go and share them with friends.\n\nThe services are also published locally with TLS certificates for the same domain name. When you configure a local DNS server to resolve the domain name to the local IP address of the machine running the services, you can use the services at home within your local network without changing the domain name.\n\nLearn how to set it up on Youtube: [Publish Ollama and OpenWebUI on the Internet with Cloudflare Tunnels](https://www.youtube.com/watch?v=-kmrfrL8W2Q). Note: This video is still based on pipx, instead of uv.\n\n**It's a Docker Compose Stack.**\n\nIf you don't see a `copier.yml` here, then this is the actual Docker Compose stack that\npublishes your local LLM services on the internet. This project was\ncreated with the [llamatunnel](https://github.com/mitja/llamatunnel)\n[copier](https://copier.readthedocs.io/en/stable/) template.\n\nTo learn how to set up the stack, please read the [Installation](#installation) section below.\n\n## Installation\n\n### Prerequisites\n\n- [Docker Desktop](https://www.docker.com/products/docker-desktop/)\n- [Ollama](https://ollama.com) installed on your machine and running on http://localhost:11434\n- A [Cloudflare](https://www.cloudflare.com/) account.\n- A domain mangaged with Cloudflare DNS.\n- [cloudflared](https://developers.cloudflare.com/cloudflare-one/connections/connect-networks/downloads/) installed on your machine. You can install it with `brew install cloudflare/cloudflare/cloudflared` on macOS or `winget install --id Cloudflare.cloudflared` on Windows. For linux, use the package manager of your distribution.\n- [Python](https://www.python.org/downloads/) 3.8 or newer\n- [Git](https://git-scm.com) 2.27 or newer\n\n### Install Copier\n\nInstall [uv](https://docs.astral.sh/uv/getting-started/installation/) first, then install copier:\n\n```bash\nuv tool install copier\n```\n\nThis works on all platforms (Linux, macOS, Windows) and is much simpler than the previous pipx approach.\n\n### Create a Cloudflare API Token\n\nCreate a Cloudflare API token with the permissions to modify the DNS zone. This is needed for the Caddy DNS challenge. Go to the Cloudflare dashboard and create a token as follows:\n\n`My Profile` \u003e `API Tokens` \u003e `Create Token` \u003e `Edit Zone DNS` \u003e `Zone:DNS:Edit` \u003e `Include:Specific Zone:\u003cDOMAIN_NAME\u003e` \u003e `Continue to Summary` \u003e `Create Token`\n\n### Login with cloudflared on Your Local Machine\n\nOn you local machine, login with cloudflared:\n\n```bash\ncloudflared tunnel login\n```\n\n### Create a Folder for the Docker Stack\n\nIf you keep all you docker stacks in a directory like `$HOME/docker-stacks`, you can use the following commands to create the Llama Tunnel stack in a new sub-directory.\n\nmacOS and Linux:\n\n```bash\nSTACK_DIR=\"$HOME/docker-stacks/llamatunnel\"\nmkdir -p $STACK_DIR\ncd $STACK_DIR\n```\n\nWindows:\n\n```powershell\n$STACK_DIR=\"$HOME\\docker-stacks\\llamatunnel\"\nmkdir -p $STACK_DIR\ncd $STACK_DIR\n```\n\n### Create a Data Directory for cloudflared\n\n```bash\nmkdir -p ./data/cloudflared\n```\n\n### Create a Tunnel and DNS Routes\n\nCreate a tunnel and DNS routes for the services. `cloudflared tunnel create` returns a tunnel id. Please note it as you will need it later. This assumes, you keep the data directory at the default location which is `./data` in the project directory. If you want to change location of the data directory, you need to create the directory structure and adjust the path accordingly.\n\nmacOS and Linux:\n\n```bash\nTUNNEL_NAME=\"llamatunnel\"\nDOMAIN_NAME=\"example.com\"\nAPI_SUBDOMAIN=\"api\"\nAPP_SUBDOMAIN=\"app\"\ncloudflared tunnel create --credentials-file ./data/cloudflared/credentials.json $TUNNEL_NAME\ncloudflared tunnel route dns $TUNNEL_NAME $API_SUBDOMAIN.$DOMAIN_NAME\ncloudflared tunnel route dns $TUNNEL_NAME $APP_SUBDOMAIN.$DOMAIN_NAME\n```\n\nWindows:\n\n```powershell\n$TUNNEL_NAME=\"llamatunnel\" `\n$DOMAIN_NAME=\"example.com\" `\n$API_SUBDOMAIN=\"api\" `\n$APP_SUBDOMAIN=\"app\" `\ncloudflared tunnel create --credentials-file .\\data\\cloudflared\\credentials.json $TUNNEL_NAME `\ncloudflared tunnel route dns $TUNNEL_NAME $API_SUBDOMAIN.$DOMAIN_NAME `\ncloudflared tunnel route dns $TUNNEL_NAME $APP_SUBDOMAIN.$DOMAIN_NAME\n```\n\nNote that you cannot manage this tunnel on the Cloudflare Dashboard. Instead, you need to use the `cloudflared` CLI tool to manage the tunnel and the DNS routing, otherwise you won't get the `credentials.json` file which is required to authenticate the `cloudflared` service.\n\n### Create the Project from the Template\n\nCreate the project from the template with copier and answer the questions. If you forgot the tunnel id, you can find it in the `data/cloudflared/credentials.json` file or see it with `cloudflared tunnel list`.\n\n```bash\ncopier copy gh:mitja/llamatunnel .\n```\n\nThis will create a new directory in the `STACK_DIR` with the all the files necessary to run the tunnel.\n\n### Start the Services\n\nChange into the project directory and start the services in the foreground:\n\n```bash\ndocker-compose up --build\n```\n\nNow you can use Ollama at `https://api.example.com` and OpenWebUI `https://app.example.com` (assuming you use app and api as subdomains). You can find the API key for Ollama in the `.env` file.\n\n### Commit the Project to Git\n\nYou need to keep this project in git if you want to use the [update feature of copier](https://copier.readthedocs.io/en/stable/updating/). See below for more info abut this. This is how you can do it locally:\n\n```bash\ngit init\ngit add .\ngit commit -m \"Initial commit\"\n```\n\n## Usage\n\n### Start in Detached Mode\n\n```bash\ndocker-compose up --build -d\n```\n\n### Stop\n\n```bash\ndocker-compose down\n```\n\n### Update or Change the Configuration\n\nYou can update the project with copier. This way, you can get new features,\nupdated docker images, or bug fixes in the template and you can also\nchange the configuration of the services.\n\n1. Change into the project directory.\n2. Stop the services, for example with `docker-compose down`.\n3. Take a backup, for example with `zip -r -X \"../llamatunnel.zip\" .`\n4. Gather the existing secret configurations (you can find them in the `.env` file).\n5. Run `copier update` in the project directory.\n6. Resolve any conflicts, if necessary.\n7. Review the changes with `git diff` and commit them with `git commit -am \"Update\"`.\n8. Restart the services with `docker-compose up --build -d`.\n\n**Caution**: Make sure to provide the same answers for the secrets, as before\nif you want to keep the secrets. Again, you can find the secrets in the `.env` file.\n\nYou can just acknowledge the other, non-secret options if you don't want to change anything.\n\nIf you just want to change the configuration, while keeping the current version,\nget the `_commit` value from the `.copier-answers.yml` file, and use it as the\nvalue of the `--vcs-ref` option. Usually, this is a git tag, in which case you can\ncall `copier update` for example like this:\n\n```bash\ncopier update --vcs-ref tags/0.1.7\n```\n\n## Security Notes\n\nThe `.env` file contains secrets, namely the `CLOUDFLARE_API_TOKEN`, the `OLLAMA_API_KEY`,\nand the `WEBUI_SECRET_KEY`. Don't commit the `.env` file to version control and store the secrets\nsomewhere secure, so that you can enter them again on updates or reinstalls.\n\nThe `data` directory also contains secrets, for example the `credentials.json` file and the private key\nof the TLS certificate. Take backups of the data directory, and don't commit it to version control.\n\nIf you think you've found a security issue, please follow GitHub's general instructions to [privately report a security vulnerability](https://docs.github.com/en/code-security/security-advisories/guidance-on-reporting-and-writing-information-about-vulnerabilities/privately-reporting-a-security-vulnerability#privately-reporting-a-security-vulnerability).\n\n## Solution Overview\n\nThis project uses **Docker Compose** to start cloudflared, Caddy, and OpenWebUI and **Cloudflare Tunnels** to route traffic from certain domains on the internet to Ollama and OpenWebUI on your local machine.\n\n### DNS\n\n- api.example.com points to your local Ollama (which should already be installed on your machine)\n- app.example.com points to your local OpenWebUI\n\n### Cloudflared\n\n- Creates an outbound-only connection to Cloudflare’s global network.\n- Forwards traffic only to the local Caddy service. Ollama and OpenWebUI are not directly exposed to the internet.\n\n### Caddy\n\n- Acts as a reverse proxy.\n- Protects the Ollama API with a configurable API key.\n- Serves on https with SSL certificates for the same domain name. Thus, you can use a local DNS server to access the services from your local network without going over the internet.\n\n### OpenWebUI\n\n- Is a chat app that uses the OpenAI API to create a chat experience.\n- Directly talks to the local Ollama, without going over the internet or Caddy.\n\n### Ollama\n\n- Uses llama.cpp to run large language models on your local machine and expose them with an OpenAI compatible API.\n- Should already be installed directly on your machine and listen on http://localhost:11434 (the default configuration).\n\n### Docker Compose\n\n- Manages multiple Docker containers in a single yaml file andå with a single command.\n- Encapsulates the services in a user-defined network, so that they can communicate with each other over an internal DNS.\n- Maps the ports of the Caddy and OpenWebUI services to the host system, so that you can also access the services from your local machine or your local network.\n- Requires that Docker Desktop or something similar is installed on your machine.\n\n## FAQ\n\n### What are the Benefits of Using Cloudflare Tunnels to publish my local LLM Services on the Internet?\n\nIn principle, you can publish your local services on the internet in three different ways:\n\n| Method | Pros | Cons |\n| --- | --- | --- |\n| VPN with Dynamic DNS | Secure, private, and encrypted connection | gives access to your network, you need to open the tunnel to use the services |\n| Open ports with Dynamic DNS | Easy to set up, no need for a VPN | you need to open ports on your router, needs additional security measures in your network, could be blocked by your internet service provider |\n| Outbound tunnel to a gateway service on the internet | no need to open ports on your home network, can be shared with friends without handing out VPN access to your network, you can authenticate/authorize outside of your network | you need a service on the internet |\n\nAn outbound tunnel is a good compromise between security, convenience, and shareability.\n\n[Cloudflare Tunnel](https://developers.cloudflare.com/cloudflare-one/connections/connect-networks/) is just one of many services you can use for outbound tunnels. The benefit is that it's free if you already have a domain you can use for this purpose. The main drawbacks of Cloudflare Tunnels are the complex setup, and some limitations of the service.\n\nIt would be more secure to perform pre-authentication of API clients and users outside of your network. Cloudflare supports this with the Zero Trust Access service, but this can get quite expensive.\n\n### Why Do You Install cloudflared as CLI Tool?\n\nI use cloudflared as cli tool for administrative tasks and don't run them in the docker container defined in docker-compose.yaml. This way, I don't need to make my Cloudflare admin credentials available to the cloudflared service.\n\n### What if I have a Firewall?\n\nIf you have a firewall, you need to allow outbound connections to the Cloudflare network form the machine you run the solution on. You can find the DNS names and ports in the Cloudflare [Tunnel with firewall](https://developers.cloudflare.com/cloudflare-one/connections/connect-networks/deploy-tunnels/tunnel-with-firewall/) documentation.\n\n### Why and How Do You Build a Custom Docker Image for Caddy?\n\nThe custom Caddy image adds the Cloudflare module for the DNS challenge to Caddy which is not built into Caddy by default. \n\nCaddy uses the DNS challenge to create TLS certificates with Let's Encrypt for the public domains. Trusted TLS certificates for this domain are now both on Cloudflare and Caddy. This way, you can use the services on your local network at same domain name. You just need to configure your local DNS server to point them to the machine Caddy is running on. \n\nCreating a custom Caddy binary and package it in an image can be done with a Docker [Multi-stage build](https://docs.docker.com/build/building/multi-stage/):\n\n- The first stage uses Caddy's builder image and runs the `xcaddy` command to build caddy with the Cloudflare module.\n- The second stage create a new image and copies the resulting binary from the builder into it.\n\nHere is the Dockerfile, you can find it in  `./images/caddy/Dockerfile`:\n\n```Dockerfile\nFROM caddy:2.10.0-builder AS builder\nRUN xcaddy build --with github.com/caddy-dns/cloudflare\nFROM caddy:2.10.0\nCOPY --from=builder /usr/bin/caddy /usr/bin/caddy\n```\n\nIf you want to learn more about this, You can find a description of the procedure in the [Caddy Image Overview](https://hub.docker.com/_/caddy) on Docker Hub (scroll down to the \"Adding custom Caddy modules\" section).\n\n### What Do You Use to Edit the Caddyfile?\n\nI use Visual Studio Code with the [Caddyfile Support](https://marketplace.visualstudio.com/items?itemName=matthewpi.caddyfile-support) extension for Visual Studio Code to get syntax highlighting, suggestions, and documentation hints for Caddyfiles.\n\n### Can I Publish the Services on Deeper Level Subdomain like ollama.home.example.com?\n\nThis is possible, but you need a paid feature from Cloudflare. If you want to proxy a wildcard DNS record on a deeper level like `*.local.example.com` you can subscribe to [Cloudflare Advanced Certificate Manager](https://developers.cloudflare.com/ssl/edge-certificates/advanced-certificate-manager/). For more information, see the Cloudflare Blog post [Wildcard proxy for everyone](https://blog.cloudflare.com/wildcard-proxy-for-everyone) by Hannes Gerhart.\n\n### How Can I Add Monitoring and Logging?\n\nYou can start all services with a monitoring endpoint that exposes Prometheus metrics. You can use the Prometheus and Grafana Docker images to monitor the services. Here are some links to get started:\n\n- [Cloudflare Tunnel Metrics](https://developers.cloudflare.com/cloudflare-one/connections/connect-networks/monitor-tunnels/metrics/)\n- [Monitoring Caddy with Prometheus Metrics](https://caddyserver.com/docs/metrics)\n- Thibault Debatty: [Use Loki to monitor the logs of your docker compose application](https://cylab.be/blog/241/use-loki-to-monitor-the-logs-of-your-docker-compose-application)\n\nFor `cloudflared`, you can use the `--loglevel debug` option to get more detailed logs.\n\nYou can also use the Cloudflare Tunnel logs to monitor the tunnel and the DNS routing.\n\n### Why Don't You Use DNSSEC?\n\nI don't use DNSSEC in this setup because I want to use the same domain name for local and public access with a local DNS server that resolves the domain to the local IP address. While this is possible to implement with DNSSEC, I have not tried it and I believe it is also quite hard to do.\n\nIf you don't want to use the services locally with the same domain name, you can probably activate DNSSEC on your domain. If you do, I would like to hear about your experience, as I haven't tried it yet.\n\n### Can I Control the Geographic Region of the Cloudflare Tunnel?\n\nWhile you can theoretically use the `--region` option to specify the region to which the tunnel connects, the only available value is `us` which routes all connections through data centers in the United States. \n\nWhen you don't select this option or leave it empty you will connect to Cloudflare's global region. Thus, in effect, you cannot control in which region your tunnel will run, except for the global region or the us region.\n\nReference: [Cloudflare Tunnel Run Parameters](https://developers.cloudflare.com/cloudflare-one/connections/connect-networks/configure-tunnels/tunnel-run-parameters/#region)\n\n### How Can I Troubleshoot the Cloudflare Tunnel?\n\nYou can set a verbose log level and run tunnels with test commands. See the Cloudflare docs for [Locally Managed Tunnels](https://developers.cloudflare.com/cloudflare-one/connections/connect-networks/configure-tunnels/local-management/) for more information.\n\n### How Can I Contribute or Give Feedback?\n\nIf you have any questions, suggestions, or improvements, feel free to open an issue or a pull request.\n\n## License\n\nThis project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmitja%2Fllamatunnel","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmitja%2Fllamatunnel","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmitja%2Fllamatunnel/lists"}