{"id":23408476,"url":"https://github.com/teilomillet/hapax","last_synced_at":"2025-04-09T01:32:49.267Z","repository":{"id":269295510,"uuid":"905714918","full_name":"teilomillet/hapax","owner":"teilomillet","description":"The reliability layer between your code and LLM providers.","archived":false,"fork":false,"pushed_at":"2025-01-06T12:17:06.000Z","size":11547,"stargazers_count":15,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-02-14T19:51:47.986Z","etag":null,"topics":["anthropic","infrastructure","infrastructure-as-code","llm","llm-gateway","llm-observability","llmops","monitoring","observability","open-source","openai"],"latest_commit_sha":null,"homepage":"https://teilomillet.github.io/hapax","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/teilomillet.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"docs/security.md","support":null,"governance":null,"roadmap":"ROADMAP.md","authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-12-19T11:31:36.000Z","updated_at":"2025-02-04T16:43:15.000Z","dependencies_parsed_at":null,"dependency_job_id":"16a6c7bc-8034-426c-b6c5-ef2354035b10","html_url":"https://github.com/teilomillet/hapax","commit_stats":null,"previous_names":["teilomillet/hapax"],"tags_count":34,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/teilomillet%2Fhapax","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/teilomillet%2Fhapax/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/teilomillet%2Fhapax/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/teilomillet%2Fhapax/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/teilomillet","download_url":"https://codeload.github.com/teilomillet/hapax/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247956777,"owners_count":21024602,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["anthropic","infrastructure","infrastructure-as-code","llm","llm-gateway","llm-observability","llmops","monitoring","observability","open-source","openai"],"created_at":"2024-12-22T15:15:07.505Z","updated_at":"2025-04-09T01:32:49.246Z","avatar_url":"https://github.com/teilomillet.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Hapax: AI Infrastructure\n\nHapax is a production-ready AI infrastructure layer that ensures uninterrupted AI operations through intelligent provider management and automatic failover. Named after the Greek word ἅπαξ (meaning \"once\"), it embodies our core promise: configure once, then let it seamlessly manage your AI infrastructure.\n\n## Common AI Infrastructure Challenges\n\nOrganizations face several critical challenges in managing their AI infrastructure. Service disruptions from AI provider outages create direct revenue impacts, while engineering teams dedicate significant resources to managing multiple AI providers. Teams struggle with limited visibility into AI usage across departments, compounded by complex integration requirements spanning different AI providers.\n\n## Core Capabilities\n\nHapax delivers a robust infrastructure layer through three core capabilities:\n\n### Intelligent Provider Management\nThe system ensures continuous service through real-time health monitoring with configurable timeouts and check intervals. Automatic failover between providers maintains zero downtime, while a sophisticated three-state circuit breaker (closed, half-open, open) with configurable thresholds prevents cascade failures. Request deduplication using the singleflight pattern optimizes resource utilization.\n\n### Production-Ready Architecture\nThe architecture prioritizes reliability through high-performance request routing and load balancing. Comprehensive error handling and request validation ensure data integrity, while structured logging with request tracing enables detailed debugging. Configurable timeout and rate limiting mechanisms protect system resources.\n\n### Security \u0026 Monitoring\nSecurity is foundational, implemented through API key-based authentication and comprehensive request validation and sanitization. The monitoring system provides granular usage tracking per endpoint and detailed request logging for operational visibility.\n\n## Usage Tracking \u0026 Monitoring\n\nHapax provides built-in monitoring capabilities through Prometheus integration, offering comprehensive visibility into your AI infrastructure:\n\n### Request Tracking\nMonitor API usage through versioned endpoints:\n```bash\n# Standard endpoint structure\n/v1/completions\n/health          # Global system health status\n/v1/health       # Versioned API health status\n/metrics\n```\n\n### Prometheus Integration\nThe monitoring system tracks essential metrics including request counts and status by endpoint, request latencies, active request volume, error rates by provider, and circuit breaker states. Health check performance metrics and request deduplication statistics provide deep insights into system efficiency.\n\nEach metric is designed for operational visibility:\n- `hapax_http_requests_total` tracks request volume by endpoint and status\n- `hapax_http_request_duration_seconds` measures request latency\n- `hapax_http_active_requests` shows current load by endpoint\n- `hapax_errors_total` monitors error rates by type\n- `circuit_breaker_state` indicates provider health status\n- `hapax_health_check_duration_seconds` validates provider responsiveness\n- `hapax_deduplicated_requests_total` confirms request efficiency\n- `hapax_rate_limit_hits_total` tracks rate limiting by client\n\n### Access Management\nSecurity is enforced through API key-based authentication, with per-endpoint rate limiting and comprehensive request validation and sanitization.\n\n## Technical Implementation\n\n```json\n// Example: Completion Request\n{\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a customer service assistant.\"},\n        {\"role\": \"user\", \"content\": \"I need help with my order #12345\"}\n    ]\n}\n```\n\nWhen your primary provider experiences issues, Hapax:\n1. Detects the failure through continuous health checks (1-minute intervals)\n2. Activates the circuit breaker after 3 consecutive failures\n3. Routes traffic to healthy backup providers in preference order\n4. Maintains detailed metrics for operational visibility\n\n## Deployment Options\n\nDeploy Hapax in minutes with our production-ready container:\n\n```bash\ndocker run -p 8080:8080 \\\n  -e OPENAI_API_KEY=your_key \\\n  -e ANTHROPIC_API_KEY=your_key \\\n  -e CONFIG_PATH=/app/config.yaml \\\n  teilomillet/hapax:latest\n```\n\nDefault configuration is provided but can be customized via `config.yaml`:\n```yaml\nserver:\n  port: 8080\n  read_timeout: 30s\n  write_timeout: 45s\n  max_header_bytes: 2097152  # 2MB\n  shutdown_timeout: 30s\n  http3:  # Optional HTTP/3 support\n    enabled: true\n    port: 443  # Default HTTPS/QUIC port\n    tls_cert_file: \"/path/to/cert.pem\"\n    tls_key_file: \"/path/to/key.pem\"\n    idle_timeout: 30s\n    max_bi_streams_concurrent: 100\n    max_uni_streams_concurrent: 100\n    max_stream_receive_window: 6291456      # 6MB\n    max_connection_receive_window: 15728640  # 15MB\n\ncircuitBreaker:\n  maxRequests: 100\n  interval: 30s\n  timeout: 10s\n  failureThreshold: 5\n\nproviderPreference:\n  - ollama\n  - anthropic\n  - openai\n```\n\n## Integration Architecture\n\nHapax provides comprehensive integration capabilities through multiple components:\n\n### REST API with Versioned Endpoints\nThe API architecture provides dedicated endpoints for core functionalities: \n- `/v1/completions` handles AI completions, \n- `/v1/health` provides versioned API health monitoring, \n- `/health` offers global system health status. \n- `/metrics` exposes Prometheus metrics for comprehensive monitoring.\n\n### Comprehensive Monitoring\nThe monitoring infrastructure integrates Prometheus metrics across all critical components, enabling detailed tracking of request latencies, circuit breaker states, provider health status, and request deduplication. This comprehensive approach ensures complete operational visibility.\n\n### Health Checks\nThe health monitoring system operates with enterprise-grade configurability. Check intervals default to one minute with adjustable timeouts, while failure thresholds are tuned to prevent false positives. Health monitoring extends from individual providers to Docker container status, with granular per-provider health tracking.\n\n### Production Safeguards\nSystem integrity is maintained through multiple safeguards: request deduplication prevents redundant processing, automatic failover ensures continuous operation, circuit breaker patterns protect against cascade failures, and structured JSON logging with correlation IDs enables thorough debugging.\n\n### Protocol Support\nThe server supports both HTTP/1.1 and HTTP/3 (QUIC) protocols:\n- HTTP/1.1 for universal compatibility\n- HTTP/3 for improved performance:\n  - Reduced latency through 0-RTT connections\n  - Better multiplexing with independent streams\n  - Improved congestion control\n  - Automatic connection migration\n  - Built-in TLS 1.3 encryption\n\n## Technical Requirements\n\nRunning Hapax requires:\n- Docker-compatible environment with network access to AI providers\n- 1GB RAM minimum (4GB recommended for production)\n- TLS certificates for HTTP/3 support (if enabled)\n- Access credentials (API keys) for supported providers: OpenAI, Anthropic, etc.\n\n## Documentation\n\nComprehensive documentation is available through multiple resources. The [Quick Start Guide](https://github.com/teilomillet/hapax/wiki) provides initial setup instructions, while detailed information about the API and security measures can be found in the [API Documentation](docs/api.md) and [Security Overview](docs/security.md). For operational insights, consult the [Monitoring Guide](docs/monitoring.md).\n\n## License\n\nLicensed under Apache 2.0. See [LICENSE](LICENSE) for details.\n\n---\n\nFor detailed technical specifications, visit our [Technical Documentation](docs/technical.md).","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fteilomillet%2Fhapax","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fteilomillet%2Fhapax","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fteilomillet%2Fhapax/lists"}