{"id":20697904,"url":"https://github.com/fly-apps/self-host-ollama","last_synced_at":"2026-03-10T13:35:19.119Z","repository":{"id":255238583,"uuid":"834311610","full_name":"fly-apps/self-host-ollama","owner":"fly-apps","description":"A simple repo for deploying Ollama on Fly.io.","archived":false,"fork":false,"pushed_at":"2024-09-09T22:14:25.000Z","size":5,"stargazers_count":26,"open_issues_count":2,"forks_count":5,"subscribers_count":6,"default_branch":"main","last_synced_at":"2025-01-17T18:53:34.286Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/fly-apps.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-07-26T23:05:18.000Z","updated_at":"2025-01-13T23:51:30.000Z","dependencies_parsed_at":null,"dependency_job_id":"2dd687c7-40d1-4295-a098-182d32d6a58a","html_url":"https://github.com/fly-apps/self-host-ollama","commit_stats":null,"previous_names":["fly-apps/self-host-ollama"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fly-apps%2Fself-host-ollama","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fly-apps%2Fself-host-ollama/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fly-apps%2Fself-host-ollama/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fly-apps%2Fself-host-ollama/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/fly-apps","download_url":"https://codeload.github.com/fly-apps/self-host-ollama/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":242961745,"owners_count":20213315,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-17T00:20:35.690Z","updated_at":"2026-03-10T13:35:14.070Z","avatar_url":"https://github.com/fly-apps.png","language":null,"funding_links":[],"categories":[],"sub_categories":[],"readme":"# Self-hosted Ollama app on Fly.io\n\nA simple repo for deploying Ollama on Fly.io.\n\n## Getting started\n\nIf you don't already have a Fly.io account, do that first (https://fly.io/app/sign-up), and make sure you have the Fly CLI installed (https://fly.io/docs/flyctl/install/).\n\nFirst, clone this repository and `cd` into the repo directory: \n\n```bash\n$ git clone https://github.com/fly-apps/self-host-ollama\n$ cd self-host-ollama\n```\n\nFeel free to change the `app` name in the `fly.toml`, and then launch it as a new Fly App with:\n\n```bash\n$ fly launch --flycast\n```\n\nWhen it asks **if you'd like to copy the configuration, say YES.** The `fly.toml` file in this repo is all the configuration you need to deploy this Ollama app.\n\nThe `--flycast` flag will make your application private. To access your application, you'll use the address `http://\u003cyour-app\u003e.flycast` and ensure you're connected over your Wireguard VPN. This will work in both production and local development. More on this later.\n\nNow that your app is launched, let's download some Ollama models. First, SSH into one of your Fly Machines like so:\n\n```bash\n$ fly ssh console\n```\n\nNext, set the value of your `OLLAMA_HOST`, so our `ollama` commands know what to use:\n\n```bash\n$ export OLLAMA_HOST=\u003cyour-app\u003e.flycast\n```\n\nFinally, pull down the model of your choice:\n\n```bash\n$ ollama pull llava\n```\n\nAnd you're done! Your Ollama app is now available for use.\n\n## Connecting to your app\n\nThis Ollama app will exist separate from whatever app you're building. Since our app is private (we don't randos eating up our Fly GPU usage), we'll need to connect to it over a secure Wireguard connection. When developing locally, the easiest method is to run:\n\n```bash\n# you don't need -a if you're in the Ollama app directory\n$ fly proxy 11434:80 -a \u003cyour-app\u003e\n```\n\nThis command proxies requests from a local port (`11434`) to port `80` on your Ollama Fly Machine, over a secure Wireguard tunnel.\n\nWhen using Ollama in your app **locally**, you'll set the host to `http://localhost:11434`. Note that while `11434` is the standard port used by Ollama, since this is just a proxy, that number can really be anything.\n\nIn **production**, you'll use the host `http://\u003cyour-app\u003e.flycast` instead.\n\n## Examples\n\nThe following code would live in an app separate from your Ollama app; This allows you to auto-start and stop your Ollama app, so you're not paying for GPUs when not in use.\n\n**JavaScript (npm `ollama` package)**\n\n```typescript\nimport { Ollama } from \"ollama\";\n\nconst ollama = new Ollama({ \n  host: process.env.OLLAMA_APP_URL // either http://localhost:11434 or http://\u003cyour-app\u003e.flycast on production\n});\n\nconst response = await ollama.generate({\n  model: 'llama3.1',\n  prompt: \"Give me a week's worth of healthy vegetarian meal ideas and their recipes.\",\n  stream: false,\n})\n```\n\n**JavaScript (basic fetch request)**\n\n```typescript\nconst params = {\n  model: 'llama3.1',\n  prompt: \"Give me a week's worth of healthy vegetarian meal ideas and their recipes.\",\n  stream: false,\n};\n\nlet resp = await fetch(\"http://sparkling-violet-709.flycast/api/generate\", {\n  method: \"POST\",\n  body: JSON.stringify(params),\n});\n```\n\n## Watch the video\n\nCheck out the accompanying video here: https://youtu.be/xkWcGmbhZRQ","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffly-apps%2Fself-host-ollama","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffly-apps%2Fself-host-ollama","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffly-apps%2Fself-host-ollama/lists"}