{"id":17862859,"url":"https://github.com/dheavy/ollama-secure-proxy","last_synced_at":"2025-08-14T14:31:24.206Z","repository":{"id":237234688,"uuid":"794091500","full_name":"dheavy/ollama-secure-proxy","owner":"dheavy","description":"Proxy for the Ollama inference server, enhancing the standard API with additional security features suitable for web-based LLM applications.","archived":false,"fork":false,"pushed_at":"2024-04-30T12:46:31.000Z","size":15,"stargazers_count":4,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2024-10-28T09:11:13.260Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dheavy.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-04-30T12:46:15.000Z","updated_at":"2024-08-11T09:38:28.000Z","dependencies_parsed_at":"2024-04-30T14:17:10.386Z","dependency_job_id":"1ef44752-5af2-4427-919d-5f8f308fd966","html_url":"https://github.com/dheavy/ollama-secure-proxy","commit_stats":null,"previous_names":["dheavy/ollama-secure-proxy"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dheavy%2Follama-secure-proxy","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dheavy%2Follama-secure-proxy/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dheavy%2Follama-secure-proxy/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dheavy%2Follama-secure-proxy/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dheavy","download_url":"https://codeload.github.com/dheavy/ollama-secure-proxy/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":229834919,"owners_count":18131527,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-28T08:56:35.286Z","updated_at":"2024-12-15T15:18:48.857Z","avatar_url":"https://github.com/dheavy.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# OSP - Ollama Secure Proxy\n\nOSP acts as a secure proxy for the Ollama inference server, enhancing the standard API with additional security features suitable for web-based LLM applications.\n\n## About\n\nWhile Ollama's default inference server (`api/generate`) is openly accessible, OSP adds crucial security layers to it, including CORS policy management, IP allow-listing, and access tokens. OSP handles API requests for generating completions, chatting, embedding generation, and displaying model information. Direct model manipulation (push, pull, delete) through OSP is not supported to enhance security.\n\n## Installation\n\nFirst, create a `.env` file based on the provided template:\n\n```bash\ncp .env.example .env\n```\n\nUpdate the `.env` file with the URL of your Ollama server:\n\n```plaintext\nOLLAMA_URL=http://localhost:11434\n```\n\nInstall dependencies:\n\n```bash\nnpm install\n```\n\nExecute tests:\n\n```bash\nnpm run test\n```\n\nBuild for deployment or start the development server:\n\n```bash\nnpm run build    # Compiles to ./dist\nnpm run dev      # Starts development server\n```\n\n## Usage\n\n### Setting Up Ollama\n\nStart the Ollama server using:\n\n```bash\nollama run \u003cmodel\u003e\n```\n\n### Configuring and Running OSP\n\n1. **Environment Setup:**\n   Set `OLLAMA_URL` to point to your running Ollama server. Configure the `TOKEN` for securing requests with an access token.\n\n2. **Secure Requests:**\n   Use the `x-osp-token` header for secure access:\n\n```bash\ncurl -X POST http://localhost:3456/api/generate \\\n-H \"Content-Type: application/json\" \\\n-H \"x-osp-token: secret\" \\\n-d '{\"model\": \"mistral:7b\", \"prompt\": \"Why is the sky blue?\"}'\n```\n\n3. **IP Restriction:**\n   Limit access by setting `ALLOWED_IPS` with a single IP or a list of IPs.\n\n4. **CORS Configuration:**\n   Restrict cross-origin requests by specifying allowed origins in `ALLOWED_CORS_ORIGINS`.\n\n5. **Running OSP:**\n   After configuration, build and run OSP to start handling requests securely.\n\n### Advanced Options\n\n- **Streaming Responses:**\n  Toggle response streaming with `IS_STREAM`.\n\n- **Model Enforcement:**\n  Set a default model and version with `DEFAULT_MODEL` and `DEFAULT_MODEL_VERSION`, and enforce them using `FORCE_MODEL`.\n\n## Contributing\n\nThank you for your interest in contributing to OSP! Here's how you can help:\n\n1. **Issue Reporting:** Identify bugs or propose new features by creating an issue in our repository.\n\n2. **Pull Requests:** Submit pull requests with bug fixes or new functionality. Ensure you adhere to our coding standards and include tests where applicable.\n\n3. **Code Reviews:** Participate in code reviews to discuss and improve the codebase.\n\n4. **Documentation:** Help us improve the documentation by suggesting changes or writing additional content.\n\nPlease read the CONTRIBUTING.md file for more details on our code of conduct and the process for submitting pull requests to us.\n\n## License\n\nMIT\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdheavy%2Follama-secure-proxy","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdheavy%2Follama-secure-proxy","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdheavy%2Follama-secure-proxy/lists"}