{"id":44368997,"url":"https://github.com/epappas/llmtrace","last_synced_at":"2026-02-19T22:10:15.898Z","repository":{"id":335748486,"uuid":"1146904914","full_name":"epappas/llmtrace","owner":"epappas","description":"Zero-code LLM security \u0026 observability proxy. Real-time prompt injection detection, PII scanning, and cost control for OpenAI-compatible APIs. Built in Rust.","archived":false,"fork":false,"pushed_at":"2026-02-17T11:18:30.000Z","size":40046,"stargazers_count":6,"open_issues_count":0,"forks_count":1,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-02-17T11:33:33.555Z","etag":null,"topics":["agentic","ai-agents","ai-infrastructure","ai-security","aiops","chatgpt","llm-inference","llm-monitoring","llm-security","llm-security-compliance-prompt-injection","llmops","mlops","observability","openai","pii-detection","prompt-injection","proxy","rust","security"],"latest_commit_sha":null,"homepage":"https://llmtrace.io","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/epappas.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":"AGENTS.md","dco":null,"cla":null}},"created_at":"2026-01-31T21:44:13.000Z","updated_at":"2026-02-17T11:18:33.000Z","dependencies_parsed_at":null,"dependency_job_id":"36773df3-f166-478b-a3a5-d789abdde0cb","html_url":"https://github.com/epappas/llmtrace","commit_stats":null,"previous_names":["epappas/llmtrace"],"tags_count":4,"template":false,"template_full_name":null,"purl":"pkg:github/epappas/llmtrace","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/epappas%2Fllmtrace","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/epappas%2Fllmtrace/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/epappas%2Fllmtrace/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/epappas%2Fllmtrace/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/epappas","download_url":"https://codeload.github.com/epappas/llmtrace/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/epappas%2Fllmtrace/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29634839,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-19T18:02:07.722Z","status":"ssl_error","status_checked_at":"2026-02-19T18:01:46.144Z","response_time":117,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["agentic","ai-agents","ai-infrastructure","ai-security","aiops","chatgpt","llm-inference","llm-monitoring","llm-security","llm-security-compliance-prompt-injection","llmops","mlops","observability","openai","pii-detection","prompt-injection","proxy","rust","security"],"created_at":"2026-02-11T19:10:55.296Z","updated_at":"2026-02-19T22:10:15.882Z","avatar_url":"https://github.com/epappas.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# LLMTrace\n\n[![CI](https://github.com/epappas/llmtrace/actions/workflows/ci.yml/badge.svg)](https://github.com/epappas/llmtrace/actions/workflows/ci.yml)\n[![Security Audit](https://github.com/epappas/llmtrace/actions/workflows/security.yml/badge.svg)](https://github.com/epappas/llmtrace/actions/workflows/security.yml)\n[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE)\n[![Rust](https://img.shields.io/badge/rust-1.75%2B-orange.svg)](https://www.rust-lang.org/)\n[![GitHub Stars](https://img.shields.io/github/stars/epappas/llmtrace)](https://github.com/epappas/llmtrace/stargazers)\n\n**Zero-code LLM observability and security for production.**\n\nLLMTrace is a transparent proxy that captures, analyzes, and secures your LLM interactions in real-time. Drop it between your app and any OpenAI-compatible API to get instant visibility into prompt injection attacks, PII leaks, cost overruns, and performance bottlenecks — without changing a single line of code.\n\n## Why LLMTrace?\n\nProduction LLM applications face three critical blind spots:\n\n- **Security vulnerabilities** — Prompt injection, data leakage, PII exposure\n- **Cost runaway** — Uncontrolled API spend, inefficient token usage\n- **Performance opacity** — No visibility into latency, failure rates, or user behavior\n\nLLMTrace solves this by sitting transparently between your application and LLM providers, giving you complete observability and control.\n\n## Key Features\n\n- **Transparent Proxy** — Drop-in replacement for any OpenAI-compatible API\n- **ML Ensemble Detection** — Multi-detector majority voting (regex, DeBERTa, InjecGuard, PIGuard)\n- **Real-time Security** — Prompt injection detection, PII scanning, data leakage prevention\n- **Performance Monitoring** — Latency, token usage, streaming metrics (TTFT), error tracking\n- **Cost Control** — Per-agent budgets, rate limits, anomaly detection\n- **Multi-tenant Ready** — Isolated per API key or custom tenant headers\n- **High Performance** — Built in Rust, handles streaming responses, circuit breaker protection\n\n## Security Performance\n\n| Metric    | Value |\n|-----------|-------|\n| Accuracy  | 87.6% |\n| Precision | 95.5% |\n| F1 Score  | 86.9% |\n| Recall    | 79.7% |\n\nTested on a 153-sample adversarial corpus across 12 attack categories including CyberSecEval2, BIPIA, TensorTrust, and InjecAgent. See [benchmarks/](benchmarks/) for methodology and full results.\n\n## Quick Start\n\n### 1. Run with Docker Compose (Recommended)\n\n```bash\n# Clone and start\ngit clone https://github.com/epappas/llmtrace\ncd llmtrace\ndocker compose up -d\n\n# The proxy is now running on localhost:8080\n```\n\n### 2. Try it with your existing code\n\n```python\nimport openai\n\n# Before: Point to OpenAI directly\nclient = openai.OpenAI()\n\n# After: Point to LLMTrace proxy (that's it!)\nclient = openai.OpenAI(base_url=\"http://localhost:8080/v1\")\n\n# Your code stays exactly the same\nresponse = client.chat.completions.create(\n    model=\"gpt-4\",\n    messages=[{\"role\": \"user\", \"content\": \"Hello!\"}]\n)\n```\n\n### 3. See your traces\n\n```bash\n# View recent activity\ncurl http://localhost:8080/api/v1/traces | jq '.[0]'\n\n# Check security findings\ncurl http://localhost:8080/api/v1/security/findings | jq\n\n# Monitor costs\ncurl http://localhost:8080/api/v1/costs/current | jq\n```\n\n**That's it!** You now have full observability into your LLM interactions.\n\n## Architecture\n\n```mermaid\ngraph LR\n    A[Your Application] --\u003e|HTTP| B[LLMTrace Proxy]\n    B --\u003e|HTTP| C[OpenAI/LLM Provider]\n    B --\u003e|Async| D[Security Engine]\n    B --\u003e|Async| E[Storage Engine]\n\n    D --\u003e F[SQLite/PostgreSQL]\n    E --\u003e F\n    D --\u003e G[Real-time Alerts]\n\n    H[Dashboard] --\u003e|REST API| B\n    I[Monitoring] --\u003e|Metrics API| B\n\n    style B fill:#e1f5fe\n    style D fill:#fff3e0\n    style E fill:#f3e5f5\n```\n\n**How it works:**\n1. **Transparent Proxy** — Your app sends requests to LLMTrace instead of OpenAI\n2. **Pass-through** — LLMTrace forwards requests to the real LLM provider\n3. **Background Analysis** — Security analysis and trace capture happen asynchronously\n4. **Zero Impact** — Your application never waits for analysis, even if something fails\n\n## Integration Examples\n\n### OpenAI Python SDK\n```python\nimport openai\n\n# Just change the base_url\nclient = openai.OpenAI(\n    base_url=\"http://localhost:8080/v1\",\n    api_key=\"your-openai-key\"\n)\n```\n\n### OpenAI Node.js SDK\n```javascript\nimport OpenAI from 'openai';\n\nconst openai = new OpenAI({\n  baseURL: 'http://localhost:8080/v1',\n  apiKey: 'your-openai-key'\n});\n```\n\n### LangChain\n```python\nfrom langchain_openai import ChatOpenAI\n\nllm = ChatOpenAI(\n    base_url=\"http://localhost:8080/v1\",\n    api_key=\"your-openai-key\"\n)\n```\n\n### curl\n```bash\ncurl http://localhost:8080/v1/chat/completions \\\n  -H \"Authorization: Bearer $OPENAI_API_KEY\" \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\"model\": \"gpt-4\", \"messages\": [{\"role\": \"user\", \"content\": \"Hello!\"}]}'\n```\n\n**[View all integration guides -\u003e](docs/README.md#guides)**\n\n## Dashboard \u0026 Monitoring\n\nLLMTrace includes a built-in dashboard for visualizing traces, security findings, and costs:\n\n```bash\n# Access the dashboard\nopen http://localhost:3000\n\n# Or use the REST API\ncurl http://localhost:8080/api/v1/traces\ncurl http://localhost:8080/api/v1/security/findings\ncurl http://localhost:8080/api/v1/costs/current\n```\n\n**Dashboard features:**\n- Real-time trace visualization\n- Security incident timeline\n- Cost breakdown by model/agent\n- Performance metrics and alerts\n\n## Configuration\n\n### Minimal Configuration\n```yaml\n# config.yaml\nupstream_url: \"https://api.openai.com\"\nlisten_addr: \"0.0.0.0:8080\"\n\nstorage:\n  profile: \"lite\"  # SQLite for simple deployments\n\nsecurity:\n  enable_prompt_injection_detection: true\n  enable_pii_detection: true\n```\n\n### Production Configuration\n```yaml\n# config.yaml\nupstream_url: \"https://api.openai.com\"\nlisten_addr: \"0.0.0.0:8080\"\n\nstorage:\n  profile: \"production\"\n  postgres_url: \"postgresql://user:pass@localhost/llmtrace\"\n  clickhouse_url: \"http://localhost:8123\"\n  redis_url: \"redis://localhost:6379\"\n\nsecurity:\n  enable_prompt_injection_detection: true\n  enable_pii_detection: true\n  enable_streaming_analysis: true\n\ncost_control:\n  daily_budget_usd: 1000\n  per_agent_daily_budget_usd: 100\n\nalerts:\n  slack:\n    webhook_url: \"https://hooks.slack.com/...\"\n\nrate_limiting:\n  requests_per_minute: 1000\n  burst_capacity: 2000\n```\n\n**[Full configuration guide -\u003e](docs/getting-started/configuration.md)**\n\n## API Reference\n\n| Endpoint | Description |\n|----------|-------------|\n| `GET /api/v1/traces` | List recent traces |\n| `GET /api/v1/traces/{id}` | Get specific trace details |\n| `GET /api/v1/security/findings` | List security incidents |\n| `GET /api/v1/costs/current` | Cost breakdown and usage |\n| `GET /health` | Health check and circuit breaker status |\n| `POST /policies/validate` | Validate custom security policies |\n\n**[Full API documentation -\u003e](docs/guides/API.md)**\n\n## Installation\n\n### Docker (Recommended)\n```bash\ndocker run -p 8080:8080 epappas/llmtrace:latest\n```\n\n### Docker Compose with Dependencies\n```bash\ncurl -o compose.yaml https://raw.githubusercontent.com/epappas/llmtrace/main/compose.yaml\ndocker compose up -d\n```\n\n### Kubernetes\n```bash\nkubectl apply -f https://raw.githubusercontent.com/epappas/llmtrace/main/deployments/kubernetes/\n```\n\n### From Source\n```bash\ngit clone https://github.com/epappas/llmtrace\ncd llmtrace\ncargo build --release --features ml\n./target/release/llmtrace-proxy --config config.yaml\n```\n\n**[Installation guide with all methods -\u003e](docs/getting-started/installation.md)**\n\n## Production Deployment\n\n### High-Availability Setup\n- **Load Balancer** -\u003e Multiple LLMTrace instances\n- **PostgreSQL** for persistent trace storage\n- **ClickHouse** for high-volume analytics\n- **Redis** for caching and rate limiting\n\n### Security Best Practices\n- API key validation and tenant isolation\n- TLS termination at load balancer\n- Network segmentation between components\n- Regular security policy updates\n\n### Monitoring \u0026 Alerting\n- Prometheus metrics export\n- Grafana dashboards\n- PagerDuty/Slack integration\n- OWASP LLM Top 10 compliance reporting\n\n**[Production deployment guide -\u003e](docs/deployment/kubernetes.md)**\n\n## Contributing\n\nWe welcome contributions! Please see our [Contributing Guide](CONTRIBUTING.md) for details.\n\n### Development Setup\n```bash\ngit clone https://github.com/epappas/llmtrace\ncd llmtrace\ncargo build --workspace\ncargo test --workspace\n```\n\n### Project Structure\n| Crate | Purpose |\n|-------|---------|\n| `llmtrace-core` | Shared types and traits |\n| `llmtrace-proxy` | HTTP proxy server |\n| `llmtrace-security` | Security analysis engine (regex + DeBERTa + InjecGuard + PIGuard ensemble) |\n| `llmtrace-storage` | Storage backends (SQLite, PostgreSQL, ClickHouse, Redis) |\n| `llmtrace-python` | Python bindings |\n\n**[Development guide -\u003e](CONTRIBUTING.md)**\n\n## License\n\n[MIT](LICENSE) - Free for commercial and personal use.\n\n---\n\n**Star this repo** if LLMTrace helps secure your LLM applications!\n\n**Found a bug?** [Open an issue](https://github.com/epappas/llmtrace/issues)\n\n**Questions?** [Start a discussion](https://github.com/epappas/llmtrace/discussions)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fepappas%2Fllmtrace","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fepappas%2Fllmtrace","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fepappas%2Fllmtrace/lists"}