{"id":13633263,"url":"https://github.com/missingstudio/gateway","last_synced_at":"2026-05-22T20:01:30.695Z","repository":{"id":217791220,"uuid":"744802120","full_name":"missingstudio/gateway","owner":"missingstudio","description":"[deprecated] AI Gateway - core infrastructure stack for building production-ready AI Applications","archived":false,"fork":false,"pushed_at":"2024-04-08T04:02:05.000Z","size":4338,"stargazers_count":151,"open_issues_count":10,"forks_count":16,"subscribers_count":7,"default_branch":"main","last_synced_at":"2024-08-23T17:31:07.201Z","etag":null,"topics":["ai","aistudio","api","gateway","generative-ai","grpc","inference","large-language-models","llmops","llms","protobuf","workforce"],"latest_commit_sha":null,"homepage":"https://www.missing.studio","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/missingstudio.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":".github/CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2024-01-18T03:04:52.000Z","updated_at":"2024-08-23T14:19:21.000Z","dependencies_parsed_at":"2024-04-08T05:22:06.039Z","dependency_job_id":"ac71c6bb-535b-4891-8e50-1af50653217f","html_url":"https://github.com/missingstudio/gateway","commit_stats":null,"previous_names":["missingstudio/studio","missingstudio/aistudio","missingstudio/ai"],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/missingstudio%2Fgateway","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/missingstudio%2Fgateway/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/missingstudio%2Fgateway/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/missingstudio%2Fgateway/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/missingstudio","download_url":"https://codeload.github.com/missingstudio/gateway/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":223779615,"owners_count":17201209,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","aistudio","api","gateway","generative-ai","grpc","inference","large-language-models","llmops","llms","protobuf","workforce"],"created_at":"2024-08-01T23:00:32.123Z","updated_at":"2026-01-12T03:06:21.232Z","avatar_url":"https://github.com/missingstudio.png","language":"Go","funding_links":[],"categories":["LLMOps","HarmonyOS"],"sub_categories":["Observability","Windows Manager"],"readme":"![Gateway](/.github/gateway.png)\n\n\u003cdiv align=\"center\"\u003e\n\n### Core infrastructure stack for building production-ready AI Applications\n\n![Lint](https://github.com/missingstudio/gateway/actions/workflows/lint.yml/badge.svg)\n![Release](https://github.com/missingstudio/gateway/actions/workflows/release.yml/badge.svg)\n[![License](https://img.shields.io/github/license/missingstudio/gateway)](./LICENSE)\n[![Version](https://img.shields.io/github/v/release/missingstudio/gateway?logo=semantic-release)](Version)\n[![Discord](https://img.shields.io/discord/1136647481128661082)](https://discord.gg/yxV58ydzV7)\n[![Twitter](https://img.shields.io/twitter/follow/_missingstudio)](https://twitter.com/_missingstudio)\n\u003c/div\u003e\n\u003cbr\u003e\u003cbr\u003e\n\n## Introduction\n🌈 A Robust cloud-native [AI Gateway](https://www.missing.studio) - core LLMOps infrastructure stack for building production-ready AI Applications. It provides an Universal API for inferencing 100+ LLMs(OpenAI, Azure, Cohere, Anthropic, HuggingFace, Replicate, Stable Diffusion).\n\n## 🚀 Key Features\n\n✅\u0026nbsp; Seamless Integration with **Universal API** \u003cbr\u003e\n✅\u0026nbsp; Reliable LLM Routing with **AI Router** \u003cbr\u003e\n✅\u0026nbsp; **Load balance** across multiple models and providers \u003cbr\u003e\n✅\u0026nbsp; **Automatic Retries** with exponential fallbacks \u003cbr\u003e\n✅\u0026nbsp; High availability and resiliency using **production-ready LLMOps** \u003cbr\u003e\n🚧\u0026nbsp; Detailed **Usage Analytics** \u003cbr\u003e\n🚧\u0026nbsp; **PII detection and masking**\u003cbr\u003e\n🚧\u0026nbsp; **Simple \u0026 Semantic Caching** for cost reduction \u003cbr\u003e\n🚧\u0026nbsp; **No Vendor lock-in Observability** - Logging, monitoring and tracing \u003cbr\u003e\n✅\u0026nbsp; **Enterprise-ready** with enhanced security, reliability, and scale with custom deployments support.\n\n## Supported Providers\n|                                               |   Provider     |   Provider Name  |   Support   |   Supported Endpoints    |\n|-----------------------------------------------|----------------|    :---:         |    :---:    |--------------------------|\n|\u003cimg src=\"assets/openai.png\" width=16\u003e         | OpenAI         |     openai       |      ✅     |  `/chat/completions`, `/chat/completions:stream`     |\n|\u003cimg src=\"assets/groq.svg\" width=16\u003e           | Groq           |     groq         |      ✅     |  `/chat/completions`, `/chat/completions:stream`     |\n|\u003cimg src=\"assets/anyscale.png\" width=16\u003e       | Anyscale       |    anyscale      |      ✅     |  `/chat/completions`     |\n|\u003cimg src=\"assets/deepinfra.jpeg\" width=16\u003e     | Deepinfra      |    deepinfra     |      ✅     |  `/chat/completions`     |\n|\u003cimg src=\"assets/togetherai.svg\" width=16\u003e     | Together AI\t   |    togetherai    |      ✅     |  `/chat/completions`     |\n\n\u003e Not supported (yet): images, audio, files, fine-tunes, moderations\n\n\n## Installation\nAI gateway can be intall on macOS, Windows, Linux, OpenBSD, FreeBSD, and on any machine\n\n### Binary (Cross-platform)\n\nDownload the appropriate version for your platform from [releases](https://github.com/missingstudio/gateway/releases) page. Once downloaded, the binary can be run from anywhere. Ideally, you should install it somewhere in your PATH for easy use. `/usr/local/bin` is the most probable location.\n\n### MacOS\n\n`gateway` is available via a Homebrew Tap, and as downloadable binary from the [releases](https://github.com/missingstudio/gateway/releases/latest) page:\n\n```sh\nbrew install missingstudio/tap/gateway\n```\n\nTo upgrade to the latest version:\n\n```sh\nbrew upgrade gateway\n```\n\n### Linux\n\n`gateway` is available as downloadable binaries from the [releases](https://github.com/missingstudio/gateway/releases/latest) page. Download the `.deb` or `.rpm` from the releases page and install with `sudo dpkg -i` and `sudo rpm -i` respectively.\n\n### Windows\n\n`gateway` is available via [scoop](https://scoop.sh/), and as a downloadable binary from the [releases](https://github.com/missingstudio/gateway/releases/latest) page:\n\n```sh\nscoop bucket add gateway https://github.com/missingstudio/scoop-bucket.git\n```\n\nTo upgrade to the latest version:\n\n```sh\nscoop update gateway\n```\n\n### Docker\n\nWe provide ready to use Docker container images. To pull the latest image:\n\n```sh\ndocker pull missingstudio/gateway:latest\n```\n\nTo pull a specific version:\n\n```sh\ndocker pull missingstudio/gateway:v0.0.1\n```\n\n### Docker compose\n\nTo start missing studio AI gateway, simply run the following command:\n\n```sh\nmake up\n```\n\n\u003e Your AI Gateway is now running on http://localhost:8080 💥\n\n## Usage\nLet's make a chat completion request to OpenAI through the AI Gateway using both REST and gRPC protocols\n\n### Send a request using curl\n```sh\ncurl \\\n--header \"Content-Type: application/json\" \\\n--header \"x-ms-provider: openai\" \\\n--header \"Authorization: Bearer {{OPENAI_API_KEY}}\" \\\n--data '{\"model\":\"gpt-3.5-turbo\",\"messages\":[{\"role\":\"user\",\"content\":\"who are you?\"}]}' \\\nhttp://localhost:8080/v1/chat/completions\n```\n\n### Send a request using grpcurl\n\n```sh\ngrpcurl \\\n-d '{\"model\":\"gpt-3.5-turbo\",\"messages\":[{\"role\":\"user\",\"content\":\"hi\"}]}' \\\n-H 'x-ms-provider: openai' \\\n-H 'Authorization: Bearer {{OPENAI_API_KEY}}' \\\n-plaintext  localhost:8080  llm.v1.LLMService.ChatCompletions\n```\n\n## 🫶 Contributions\nAI studio is an open-source project, and  contributions are welcome. If you want to contribute, you can create new features, fix bugs, or improve the infrastructure. \n\nIt's still very early days for this so your mileage will vary here and lots of things will break. But almost any contribution will be beneficial at this point. Check the [current Issues](https://github.com/missingstudio/ai/issues) to see where you can jump in!\n\nIf you've got an improvement, just send in a pull request!\n\n1. Fork it\n2. Create your feature branch (`git checkout -b my-new-feature`)\n3. Commit your changes (`git commit -am 'feat(module): add some feature'`)\n4. Push to the branch (`git push origin my-new-feature`)\n5. Create new Pull Request\n\nIf you've got feature ideas, simply [open a new issues](https://github.com/missingstudio/ai/issues/new)!\n\nPlease refer to the [CONTRIBUTING.md](https://github.com/missingstudio/ai/blob/main/.github/CONTRIBUTING.md) file in the repository for more information on how to contribute.\n\n\u003ca href=\"https://github.com/missingstudio/ai/graphs/contributors\"\u003e\n  \u003cimg src=\"https://contrib.rocks/image?repo=missingstudio/studio\" /\u003e\n\u003c/a\u003e\n\n## License\nAI Studio is [Apache 2.0](https://github.com/missingstudio/ai/blob/main/LICENSE) licensed.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmissingstudio%2Fgateway","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmissingstudio%2Fgateway","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmissingstudio%2Fgateway/lists"}