{"id":49657688,"url":"https://github.com/raaihank/llm-sentinel","last_synced_at":"2026-05-06T10:47:00.431Z","repository":{"id":316054766,"uuid":"1047974558","full_name":"raaihank/llm-sentinel","owner":"raaihank","description":"Privacy-first proxy that automatically detects and masks sensitive data before it reaches AI models without compromising latency or SDK capabilities!","archived":false,"fork":false,"pushed_at":"2025-09-30T13:54:30.000Z","size":842,"stargazers_count":0,"open_issues_count":10,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-09-30T15:30:33.399Z","etag":null,"topics":["llm","llm-guardrails","privacy","security"],"latest_commit_sha":null,"homepage":"https://github.com/raaihank/llm-sentinel","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/raaihank.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"docs/CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-08-31T16:38:34.000Z","updated_at":"2025-09-30T13:54:34.000Z","dependencies_parsed_at":null,"dependency_job_id":"8785bedf-17cb-4437-b82f-f24e6d7f80b0","html_url":"https://github.com/raaihank/llm-sentinel","commit_stats":null,"previous_names":["raaihank/llm-sentinel"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/raaihank/llm-sentinel","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/raaihank%2Fllm-sentinel","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/raaihank%2Fllm-sentinel/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/raaihank%2Fllm-sentinel/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/raaihank%2Fllm-sentinel/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/raaihank","download_url":"https://codeload.github.com/raaihank/llm-sentinel/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/raaihank%2Fllm-sentinel/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32690377,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-06T08:33:17.875Z","status":"ssl_error","status_checked_at":"2026-05-06T08:33:17.221Z","response_time":117,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["llm","llm-guardrails","privacy","security"],"created_at":"2026-05-06T10:46:59.812Z","updated_at":"2026-05-06T10:47:00.419Z","avatar_url":"https://github.com/raaihank.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# LLM-Sentinel\n\n\u003e A Go-based proxy server that sits between your application and LLM APIs to detect PII and block prompt injection attempts in real-time.\n\n## Features\n\n- **PII Detection \u0026 Masking**: Automatically detects and masks 80+ types of sensitive data (emails, SSNs, credit cards, API keys, etc.)\n- **Prompt Injection Protection**: Blocks malicious prompts using advanced pattern matching with fuzzy detection\n- **Real-time Dashboard**: WebSocket-powered monitoring with live security events and response time tracking\n- **Multi-Provider Support**: Works with OpenAI, Anthropic, Ollama, and other LLM APIs\n- **Zero Configuration**: Works out of the box with Docker Compose\n- **Production Ready**: Preserves authentication headers, configurable security modes, comprehensive logging\n\n## Quick Start\n\n```bash\ngit clone https://github.com/raaihank/llm-sentinel\ncd llm-sentinel\ndocker-compose up --build\n```\n\nThen visit `http://localhost:8080` for the dashboard.\n\n## Usage in Your Application\n\n### Using Provider SDKs (Recommended)\n\nMost LLM provider SDKs support custom base URLs. Simply change the base URL to route through LLM-Sentinel:\n\n#### OpenAI SDK\n\n```python\nimport openai\n\n# Just change the base URL - everything else stays the same\nclient = openai.OpenAI(\n    api_key=\"your-openai-api-key\",\n    base_url=\"http://localhost:8080/openai/v1\"  # Add LLM-Sentinel proxy\n)\n\nresponse = client.chat.completions.create(\n    model=\"gpt-4\",\n    messages=[{\"role\": \"user\", \"content\": \"My email is user@company.com\"}]\n)\n```\n\n#### Ollama SDK\n\n```python\nfrom ollama import Client\n\nclient = Client(host=\"http://localhost:8080/ollama\")\nresponse = client.chat(\n  model=\"llama3.1:8b\", \n  messages=[\n    {\"role\": \"user\", \"content\": \"Explain HTTP in simple terms\"}\n  ]\n)\n```\n\n#### Anthropic SDK\n\n```python\nimport anthropic\n\n# Just change the base URL\nclient = anthropic.Anthropic(\n    api_key=\"your-anthropic-api-key\",\n    base_url=\"http://localhost:8080/anthropic\"  # Add LLM-Sentinel proxy\n)\n\nresponse = client.messages.create(\n    model=\"claude-3-sonnet-20240229\",\n    max_tokens=1024,\n    messages=[{\"role\": \"user\", \"content\": \"Hello\"}]\n)\n```\n\n#### LangChain Integration\n\n```python\nfrom langchain.llms import OpenAI\nfrom langchain.chat_models import ChatOpenAI\n\n# For OpenAI models through LLM-Sentinel\nllm = ChatOpenAI(\n    openai_api_key=\"your-openai-api-key\",\n    openai_api_base=\"http://localhost:8080/openai/v1\"  # Route through proxy\n)\n\n# For Ollama models through LLM-Sentinel  \nfrom langchain.llms import Ollama\n\nllm = Ollama(\n    model=\"llama3.1:8b\",\n    base_url=\"http://localhost:8080/ollama\"  # Route through proxy\n)\n```\n\n#### Environment Variables (Universal)\n\n```bash\n# Set these environment variables to route all SDK calls through LLM-Sentinel\nexport OPENAI_API_BASE=\"http://localhost:8080/openai/v1\"\nexport ANTHROPIC_BASE_URL=\"http://localhost:8080/anthropic\"\n\n# Your existing code will automatically use the proxy\npython your_existing_app.py\n```\n\n### Direct API Calls (Alternative)\n\nIf you prefer direct HTTP calls or your language doesn't have an official SDK:\n\n```bash\n# OpenAI API call\ncurl http://localhost:8080/openai/v1/chat/completions \\\n  -H \"Authorization: Bearer $OPENAI_API_KEY\" \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\"model\":\"gpt-4\",\"messages\":[{\"role\":\"user\",\"content\":\"Hello\"}]}'\n\n# Anthropic API call  \ncurl http://localhost:8080/anthropic/v1/messages \\\n  -H \"x-api-key: $ANTHROPIC_API_KEY\" \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\"model\":\"claude-3-sonnet-20240229\",\"max_tokens\":1024,\"messages\":[{\"role\":\"user\",\"content\":\"Hello\"}]}'\n```\n\n## Configuration\n\nCreate or edit `configs/default.yaml`:\n\n```yaml\nserver:\n  port: 8080\n\nprivacy:\n  enabled: true\n  header_scrubbing:\n    enabled: true\n    preserve_upstream_auth: true  # Keep auth headers for upstream APIs\n\nsecurity:\n  enabled: true\n  mode: block  # \"block\", \"log\", or \"passthrough\"\n  vector_security:\n    enabled: true\n    block_threshold: 0.70  # 70% confidence threshold\n    embedding:\n      service_type: \"pattern\"  # Use pattern matching (production ready)\n\nupstream:\n  openai: https://api.openai.com\n  ollama: http://localhost:11434\n  anthropic: https://api.anthropic.com\n\nwebsocket:\n  events:\n    broadcast_pii_detections: true\n    broadcast_vector_security: true\n    broadcast_system: true\n    broadcast_connections: true\n```\n\n## Performance Benchmarks\n\n| Dataset | Samples | Threshold | Balanced Accuracy | Precision | Recall | Mean Latency |\n|---------|---------|-----------|-------------------|-----------|--------|--------------|\n| **Gandalf** | 222 (111 attacks) | 0.70 | **73.9%** | **100.0%** | 47.7% | 10.7ms |\n| **Qualifire** | 9,992 (4,996 attacks) | 0.70 | **57.8%** | **100.0%** | 15.6% | 17.5ms |\n\n*Latency measured for blocked requests only (security processing time)*\n\n## Docker Compose Integration\n\n```yaml\nservices:\n  your-app:\n    build: .\n    environment:\n      - OPENAI_API_BASE=http://llm-sentinel:8080/openai\n      - OPENAI_API_KEY=${OPENAI_API_KEY}\n    depends_on:\n      - llm-sentinel\n  \n  llm-sentinel:\n    image: llm-sentinel:latest\n    ports:\n      - \"8080:8080\"\n    volumes:\n      - ./configs:/app/configs\n```\n\n## What Gets Protected\n\n### PII Detection\n\n- Email addresses → `[EMAIL_MASKED]`\n- SSNs → `[SSN_MASKED]`  \n- Credit cards → `[CREDIT_CARD_MASKED]`\n- API keys → `[API_KEY_MASKED]`\n- Phone numbers → `[PHONE_MASKED]`\n- File paths → `[PATH_MASKED]`\n- 80+ other sensitive data patterns\n\n### Prompt Injection Blocking\n\n- Instruction manipulation: \"ignore all previous instructions\"\n- Jailbreak attempts: \"pretend you are not an AI\"\n- Information extraction: \"reveal your system prompt\"\n- Obfuscation techniques: \"ignor all previus instructons\"\n- Role manipulation: \"you are now a different AI\"\n\n## Monitoring \u0026 Observability\n\n### Real-time Dashboard\n\n- Visit `http://localhost:8080` for live monitoring\n- WebSocket-powered real-time updates\n- Security alerts, PII detections, response times\n- Request activity logs with status codes\n\n### Structured Logging\n\n```json\n{\n  \"level\": \"info\",\n  \"timestamp\": \"2025-09-29T14:50:20.444Z\",\n  \"caller\": \"proxy/middleware.go:102\",\n  \"msg\": \"PII detected in request\",\n  \"component\": \"proxy\",\n  \"request_id\": \"1759157420441888750\",\n  \"findings_count\": 2,\n  \"findings\": [\n    {\"entityType\": \"email\", \"masked\": \"[EMAIL_MASKED]\", \"count\": 1},\n    {\"entityType\": \"userPath\", \"masked\": \"[PATH_MASKED]\", \"count\": 1}\n  ]\n}\n```\n\n## Production Deployment\n\n### Docker (Recommended)\n\n```bash\n# Use the ONNX-enabled version for better performance\ndocker-compose -f docker-compose.onnx.yml up -d\n```\n\n### Binary Deployment\n\n```bash\n# Build for production\ngo build -o llm-sentinel ./cmd/sentinel\n\n# Run with custom config\n./llm-sentinel --config /etc/llm-sentinel/config.yaml\n```\n\n### Environment Variables\n\n```bash\nexport LLM_SENTINEL_PORT=8080\nexport LLM_SENTINEL_CONFIG_PATH=/etc/llm-sentinel/config.yaml\nexport OPENAI_API_KEY=your-key-here\n```\n\n## Changelog\n\n### 2025-09-29 - Advanced Security Features\n\n- **Fuzzy Pattern Matching**: Detects obfuscated attacks like \"ignor all previus instructons\"\n- **Enhanced Prompt Injection**: Blocks instruction manipulation, jailbreaks, and role hijacking\n- **Attack Pattern Recognition**: 90%+ confidence detection with zero false positives\n- **Security Benchmarks**: 73.9% accuracy on Gandalf, 57.8% on Qualifire datasets\n\n### 2025-09-28 - Multi-Provider \u0026 Authentication\n\n- **Anthropic Claude Support**: Full API integration with proper header handling\n- **Authentication Preservation**: Upstream API keys properly forwarded\n- **WebSocket Security**: Removed auth barriers for dashboard access\n- **Production Configuration**: Configurable security modes and thresholds\n\n### 2025-09-27 - Real-time Monitoring\n\n- **Live Dashboard**: WebSocket-powered monitoring at `http://localhost:8080`\n- **Security Alerts**: Real-time PII detections and threat blocking\n- **Response Time Tracking**: Accurate latency monitoring for blocked requests\n- **Activity Logging**: Request tracking with status codes and processing times\n\n### 2025-09-26 - PII Protection\n\n- **Comprehensive PII Detection**: 80+ patterns for sensitive data\n- **Automatic Masking**: Emails, SSNs, credit cards, API keys, file paths\n- **Request Sanitization**: PII removed before forwarding to LLM APIs\n- **Privacy Compliance**: GDPR/CCPA-ready data protection\n\n### 2025-09-25 - Proxy Infrastructure\n\n- **Multi-Provider Routing**: OpenAI, Ollama, and Anthropic API support\n- **Docker Deployment**: Complete containerized setup\n- **Configuration Management**: YAML-based settings with environment overrides\n- **Structured Logging**: JSON logs with request IDs and component tracking\n\n### 2025-09-24 - Core Platform\n\n- **HTTP Middleware Pipeline**: Rate limiting, logging, and security layers\n- **Vector Store Integration**: PostgreSQL with pgvector for embeddings\n- **Redis Caching**: High-performance embedding cache with binary storage\n- **ETL Pipeline**: Dataset processing and security pattern training\n\n\n## License\n\nMIT - Use it however you want.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fraaihank%2Fllm-sentinel","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fraaihank%2Fllm-sentinel","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fraaihank%2Fllm-sentinel/lists"}