{"id":49877893,"url":"https://github.com/riipandi/radium","last_synced_at":"2026-05-15T13:07:28.098Z","repository":{"id":299411765,"uuid":"1002917983","full_name":"riipandi/radium","owner":"riipandi","description":"[WIP] Radium is an open-source LLM proxy gateway built for resource efficiency and high performance.","archived":false,"fork":false,"pushed_at":"2026-03-30T09:17:20.000Z","size":662,"stargazers_count":4,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-04-26T17:34:03.173Z","etag":null,"topics":["ai","ai-gateway","antrophic","gateway","llm","llm-gateway","llm-proxy","llmops","mcp-gateway","openai","openai-compatible","openai-proxy"],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/riipandi.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":"AGENTS.md","dco":null,"cla":null}},"created_at":"2025-06-16T10:46:50.000Z","updated_at":"2026-03-30T09:17:24.000Z","dependencies_parsed_at":"2025-06-17T15:49:45.134Z","dependency_job_id":null,"html_url":"https://github.com/riipandi/radium","commit_stats":null,"previous_names":["riipandi/swift-relay","riipandi/sorai","riipandi/radium"],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/riipandi/radium","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/riipandi%2Fradium","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/riipandi%2Fradium/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/riipandi%2Fradium/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/riipandi%2Fradium/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/riipandi","download_url":"https://codeload.github.com/riipandi/radium/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/riipandi%2Fradium/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33067520,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-15T11:35:32.926Z","status":"ssl_error","status_checked_at":"2026-05-15T11:35:31.362Z","response_time":103,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","ai-gateway","antrophic","gateway","llm","llm-gateway","llm-proxy","llmops","mcp-gateway","openai","openai-compatible","openai-proxy"],"created_at":"2026-05-15T13:07:26.747Z","updated_at":"2026-05-15T13:07:28.089Z","avatar_url":"https://github.com/riipandi.png","language":"TypeScript","funding_links":["https://github.com/sponsors/riipandi"],"categories":[],"sub_categories":[],"readme":"# Radium LLM Proxy Gateway\n\n[![GitHub Release](https://img.shields.io/github/v/release/riipandi/radium?logo=docker)](https://github.com/riipandi/radium/releases)\n[![MSRV](https://img.shields.io/badge/rust-v1.93+-orange.svg?logo=rust\u0026label=Rust)](https://www.rust-lang.org)\n[![Dependencies](https://deps.rs/repo/github/riipandi/radium/status.svg)](https://deps.rs/repo/github/riipandi/radium)\n![License Apache-2.0](https://img.shields.io/badge/License-Apache%202.0-blue.svg)\n[![Contribution](https://img.shields.io/badge/Contribute-GitHub-brightgreen)](https://github.com/riipandi/radium/graphs/contributors)\n\u003c!-- ![Build Status](https://img.shields.io/github/actions/workflow/status/riipandi/radium/ci.yml?branch=main) --\u003e\n\nRadium is an open-source, unified HTTP API gateway for accessing multiple AI model providers. Built in Rust\nas a resource-efficient, high-performance open-source LLM proxy, Radium acts as a central orchestrator,\nhandling every request and response with precision. It delivers uniform endpoints for text and chat\ncompletions, intelligent fallback logic, and complete observability, transforming client-to-LLM interactions\ninto a seamless, elegantly managed experience.\n\n## Key Features\n\n- **🚀 High Performance**: Leverages Rust's speed and memory safety for low-latency, high-throughput proxying.\n- **🔌 Multi-Provider Support**: Seamlessly connects to OpenAI, Anthropic, AWS Bedrock, Cohere, etc.\n- **⚡ Flexible Integration**: Minimal configuration required for various LLM backends.\n- **📊 Built-in Monitoring**: Prometheus metrics and comprehensive observability.\n- **🛠️ Developer-Friendly**: Simple setup, clear documentation, and extensible design.\n- **🔄 Fallback Support**: Automatic failover between providers for reliability.\n- **🌐 CORS Support**: Configurable Cross-Origin Resource Sharing.\n- **📝 Structured Logging**: Configurable logging with rotation and timestamps.\n- **🐳 Docker Ready**: Container support with multi-platform builds.\n- **📈 Scalable Architecture**: Connection pooling and request timeout handling.\n- **📝 Open Source**: Licensed under Apache 2.0.\n\n## Supported Providers\n\n| Provider         | Key         | Configuration Section | Status |\n|------------------|-------------|-----------------------|--------|\n| OpenAI           | `openai`    | `[openai]`            | ✅      |\n| Anthropic        | `anthropic` | `[anthropic]`         | ⏳      |\n| Azure OpenAI     | `azure`     | `[azure_openai]`      | ⏳      |\n| AWS Bedrock      | `bedrock`   | `[bedrock]`           | ⏳      |\n| Cohere           | `cohere`    | `[cohere]`            | ⏳      |\n| Google Vertex AI | `vertex`    | `[vertext]`           | ⏳      |\n\n## Getting Started\n\n### Prerequisites\n\n- **Rust**: Ensure you have Rust installed (version 1.93 or later). Install via [rustup](https://rustup.rs/)\n- **Git**: Required to clone the repository\n- **API Keys**: Valid API keys for your chosen LLM providers\n- **Optional**: Docker for containerized deployment\n\n### Installation\n\n1. Clone the repository:\n```sh\ngit clone https://github.com/riipandi/radium.git \u0026\u0026 cd radium\n```\n\n2. Build the project:\n```sh\n# Using cargo directly\ncargo build --release\n\n# Or using just (recommended)\njust build\n```\n\n3. Set up configuration:\n```sh\n# Copy example configuration\ncp config.toml.example config.toml\n\n# Edit with your API keys and settings\nnano config.toml\n```\n\n### Configuration\n\nCreate your `config.toml` file based on the [`config.toml.example`](./config.toml.example)\n\n### Running the Server\n\n```sh\n# Using cargo\ncargo run -- serve\n\n# Using just (with auto-reload for development)\njust dev\n\n# Using built binary\n./target/release/radium serve\n\n# With custom config path\n./target/release/radium serve -config /path/to/config.toml\n```\n\n## API Endpoints\n\nRadium provides OpenAI-compatible API endpoints:\n\n- `POST /v1/chat/completions` - Chat completions with conversation context\n- `POST /v1/text/completions` - Simple text completions\n- `GET /metrics` - Prometheus metrics for monitoring\n\n### Base URL\n\n```\nhttp://localhost:8000\n```\n\n## Performance Benchmarks\n\nHere are example benchmarks using [bombardier](https://github.com/codesenberg/bombardier) HTTP benchmarking tool:\n\n### Health Check Endpoint Performance\n\n**Test Configuration:**\n- **Concurrent connections**: 125\n- **Number of requests**: 100,000\n- **Target endpoint**: `GET /healthz`\n- **Environment**: local development server\n\n```sh\nbombardier -c 125 -n 100000 -m GET http://localhost:8000/healthz\n```\n\n**Example Results:**\n```\nStatistics        Avg      Stdev        Max\n  Reqs/sec     35160.01    8333.65   46008.85\n  Latency        3.55ms     2.04ms    44.74ms\n  HTTP codes:\n    1xx - 0, 2xx - 100000, 3xx - 0, 4xx - 0, 5xx - 0\n    others - 0\n  Throughput:    16.64MB/s\n```\n\n### Load Testing Different Scenarios\n\n```sh\n# Light load - 50 concurrent connections, 1000 requests\nbombardier -n 1000 -c 50 http://localhost:8000/healthz\n\n# Medium load - 100 concurrent connections, 2500 requests with 10s duration\nbombardier -d 10s -n 2500 -c 100 http://localhost:8000/healthz\n\n# Heavy load - 500 concurrent connections, 10000 requests\nbombardier -d 10s -n 10000 -c 500 http://localhost:8000/healthz\n\n# Sustained load test - 30 seconds duration\nbombardier -c 100 -d 30s http://localhost:8000/healthz\n```\n\n### API Endpoint Benchmarks\n\nFor testing actual LLM proxy endpoints:\n\n```sh\n# Test chat completions endpoint (requires valid API key)\nbombardier -n 100 -c 10 -m POST \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer your-api-key\" \\\n  -b '{\"model\":\"gpt-4o-mini\",\"messages\":[{\"role\":\"user\",\"content\":\"Hello\"}]}' \\\n  http://localhost:8000/v1/chat/completions\n```\n\n**Performance Notes:**\n- Actual LLM endpoints performance depends on upstream provider latency\n- Connection pooling and keep-alive significantly improve throughput\n- Memory usage remains stable under high concurrent load\n\n## Documentation\n\nFor detailed documentation, see:\n- **[HTTP Transport Documentation](./docs/http-transport.md)** - Complete API reference\n- **[OpenAPI Specification](./docs/openapi.json)** - Machine-readable API spec\n- **[Example Requests](./specs)** - Sample requests using [httl](https://httl.dev)\n\n## Monitoring\n\nRadium provides comprehensive monitoring through Prometheus metrics at `/metrics` endpoint, including:\n- Request counts by provider, model, and status\n- Request latency histograms\n- Token usage statistics\n- Error rates and types\n- Connection pool statistics\n\n## Docker Support\n\nRadium includes full Docker support with multi-platform builds:\n\n```sh\n# Build Docker image\njust docker-build\n\n# Run with Docker\njust docker-run serve\n\n# Using Docker Compose\njust compose-up\n```\n\n## Contributing\n\nWe welcome contributions to make Radium even better!\n\n- Read our **[Contributing Guidelines](./CONTRIBUTING.md)** for detailed guidelines\n- Fork the repository and create a feature branch\n- Submit a pull request with a clear title and description\n- Join the discussion on [GitHub Issues](https://github.com/riipandi/radium/issues)\n\nJoin the flow. Amplify your your AI-powered applications with Radium! 🚀\n\n## License\n\nRadium is licensed under the [Apache License 2.0](https://choosealicense.com/licenses/apache-2.0).\nSee the [LICENSE](./LICENSE) file for more information.\n\n\u003e Unless you explicitly state otherwise, any contribution intentionally submitted\n\u003e for inclusion in this project by you shall be licensed under the Apache License 2.0,\n\u003e without any additional terms or conditions.\n\nCopyrights in this project are retained by their contributors.\n\n---\n\n\u003csub\u003e🤫 Psst! If you like my work you can support me via [GitHub sponsors](https://github.com/sponsors/riipandi).\u003c/sub\u003e\n\n[![Made by](https://badgen.net/badge/icon/Made%20by%20Aris%20Ripandi?icon=cocoapods\u0026label\u0026color=black\u0026labelColor=black)](https://x.com/intent/follow?screen_name=riipandi)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Friipandi%2Fradium","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Friipandi%2Fradium","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Friipandi%2Fradium/lists"}