{"id":30105706,"url":"https://github.com/fairdataihub/prompt-gateway","last_synced_at":"2025-08-10T00:19:06.645Z","repository":{"id":299913587,"uuid":"1004552275","full_name":"fairdataihub/prompt-gateway","owner":"fairdataihub","description":"A Flask-based API gateway that provides secure, authenticated access to local Ollama LLM models with Docker support.","archived":false,"fork":false,"pushed_at":"2025-06-19T00:07:43.000Z","size":33,"stargazers_count":1,"open_issues_count":2,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-06-19T00:25:07.012Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://pg.fairdataihub.org","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/fairdataihub.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-06-18T20:06:55.000Z","updated_at":"2025-06-19T00:07:48.000Z","dependencies_parsed_at":"2025-06-19T00:25:20.637Z","dependency_job_id":"64c7ad90-257e-416b-8d13-1f0d8b0e95f4","html_url":"https://github.com/fairdataihub/prompt-gateway","commit_stats":null,"previous_names":["fairdataihub/prompt-gateway"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/fairdataihub/prompt-gateway","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fairdataihub%2Fprompt-gateway","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fairdataihub%2Fprompt-gateway/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fairdataihub%2Fprompt-gateway/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fairdataihub%2Fprompt-gateway/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/fairdataihub","download_url":"https://codeload.github.com/fairdataihub/prompt-gateway/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fairdataihub%2Fprompt-gateway/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":269657011,"owners_count":24454757,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-09T02:00:10.424Z","response_time":111,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-08-10T00:19:01.375Z","updated_at":"2025-08-10T00:19:06.620Z","avatar_url":"https://github.com/fairdataihub.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# prompt-gateway\n\n## Getting started\n\n### Prerequisites/Dependencies\n\nYou will need the following installed on your system:\n\n- Python 3.8+\n- [Pip](https://pip.pypa.io/en/stable/)\n- [Docker](https://www.docker.com/)\n- [Ollama](https://ollama.ai/) - Required for local LLM inference\n\n### Ollama Setup\n\nThis application is designed to run in a Docker container while Ollama runs on the host machine. The application will automatically retry connecting to Ollama up to 10 times during startup.\n\n1. **Install Ollama** on the host machine (if not already installed):\n\n   - Download from [https://ollama.ai/download](https://ollama.ai/download)\n   - Follow the installation instructions for your platform\n\n2. **Start Ollama** on the host machine:\n\n   ```bash\n   ollama serve\n   ```\n\n3. **Verify Ollama is accessible** (optional but recommended):\n\n   ```bash\n   python startup_check.py\n   ```\n\n   This script will check if Ollama is accessible from within a container context.\n\n4. **Pull required models** (if needed):\n\n   ```bash\n   ollama pull llama3:8b\n   ```\n\n### API Key Authentication\n\nThis application uses token-based authentication to secure access to the LLM endpoints. API tokens are stored as environment variables in JSON format and must be included in request headers using the standard Bearer token format.\n\n#### Setting Up API Tokens\n\n1. **Generate API tokens** using the provided utility:\n\n   ```bash\n   python generate_api_key.py\n   ```\n\n   This will generate secure API tokens and provide setup instructions.\n\n2. **Configure API tokens** using one of these methods:\n\n   **Option A: Using a .env file**\n\n   ```bash\n   # Create a .env file in the project root\n   echo 'API_KEYS=[{\"appname\":\"APP1\",\"key\":\"your-token-here\"},{\"appname\":\"WEBAPP\",\"key\":\"another-token-here\"}]' \u003e .env\n   ```\n\n   **Option B: Using system environment variables**\n\n   ```bash\n   export API_KEYS='[{\"appname\":\"APP1\",\"key\":\"your-token-here\"},{\"appname\":\"WEBAPP\",\"key\":\"another-token-here\"}]'\n   ```\n\n   The JSON format is designed to work with external environment variable management tools. Each object in the array contains an `appname` and `key` property.\n\n3. **Using API tokens in requests**:\n\n   Include the token in the `Authorization` header using Bearer format:\n\n   ```bash\n   curl -X POST \"http://host.docker.internal:5000/query\" \\\n     -H \"Content-Type: application/json\" \\\n     -H \"Authorization: Bearer your-token-here\" \\\n     -d '{\"query\": \"Hello, how are you?\"}'\n   ```\n\n#### Security Notes\n\n- Keep your API tokens secure and don't share them\n- Use different tokens for different applications\n- Rotate tokens periodically for better security\n- Never commit API tokens to version control\n- The health check endpoints (`/echo`, `/up`, `/health/ollama`) do not require authentication\n\n### Setup\n\nIf you would like to update the api, please follow the instructions below.\n\n1. Create a local virtual environment and activate it:\n\n   ```bash\n   python -m venv .venv\n   source .venv/bin/activate # on Windows use .venv\\Scripts\\activate\n   ```\n\n   If you are using Anaconda, you can create a virtual environment with:\n\n   ```bash\n   conda create -n prompt-gateway-dev-env python=3.13\n   conda activate prompt-gateway-dev-env\n   ```\n\n2. Install the dependencies for this package.\n\n   ```bash\n   pip install -r requirements.txt\n   ```\n\n## Running\n\n**Important**: Ensure Ollama is running on the host machine before starting the application. The app will automatically retry connecting to Ollama up to 10 times during startup.\n\nFor developer mode:\n\n```bash\npython app.py --host $HOST --port $PORT\n```\n\nor\n\n```bash\nflask run --debug\n```\n\nFor production mode:\n\n```bash\npython3 app.py --host $HOST --port $PORT\n```\n\n## Docker Deployment\n\nWhen running in a Docker container, ensure that:\n\n1. **Ollama is running on the host machine** (not in the container)\n2. **The container can access host.docker.internal:11434** (Ollama's default port)\n3. **Docker network configuration allows host access**\n4. **API tokens are properly configured** as environment variables\n\nExample Docker run command:\n\n```bash\ndocker run -p 5000:5000 \\\n  -e API_KEY_APP1=your-token-here \\\n  -e API_KEY_WEBAPP=another-token-here \\\n  --network host your-app-image\n```\n\n## Health Checks\n\nThe application provides several health check endpoints:\n\n- `/up` - Basic application health check\n- `/health/ollama` - Check if Ollama is running and accessible\n- `/query` - Test the LLM functionality (requires Ollama to be running and API token authentication)\n\n## Troubleshooting\n\nIf you encounter issues with Ollama connectivity:\n\n1. **Check if Ollama is running on the host**:\n\n   ```bash\n   curl http://host.docker.internal:11434/api/tags\n   ```\n\n2. **Verify container can access host**:\n\n   ```bash\n   python startup_check.py\n   ```\n\n3. **Check application logs** for detailed error messages about Ollama connectivity issues.\n\n4. **Ensure the required model is available**:\n\n   ```bash\n   ollama list\n   ```\n\n5. **Docker network issues**: If the container cannot access host.docker.internal:11434, try:\n   - Using `--network host` flag\n   - Or mapping the port: `-p 11434:11434`\n\nIf you encounter authentication issues:\n\n1. **Verify API tokens are configured**:\n\n   ```bash\n   python -c \"from config import get_api_keys; print(get_api_keys())\"\n   ```\n\n2. **Check that the Authorization header is included** in your requests\n\n3. **Ensure the token matches** one of the configured environment variables\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffairdataihub%2Fprompt-gateway","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffairdataihub%2Fprompt-gateway","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffairdataihub%2Fprompt-gateway/lists"}