{"id":13614370,"url":"https://github.com/fluxninja/aperture","last_synced_at":"2025-04-08T09:08:18.706Z","repository":{"id":57905424,"uuid":"496566928","full_name":"fluxninja/aperture","owner":"fluxninja","description":"Rate limiting, caching, and request prioritization for modern workloads","archived":false,"fork":false,"pushed_at":"2024-06-25T22:33:15.000Z","size":212447,"stargazers_count":654,"open_issues_count":100,"forks_count":26,"subscribers_count":6,"default_branch":"main","last_synced_at":"2025-04-01T07:49:49.695Z","etag":null,"topics":["caching","cloud-native","concurrency-limiter","kubernetes","llm","microservices","observability","rate-limiter","scheduler"],"latest_commit_sha":null,"homepage":"https://docs.fluxninja.com","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/fluxninja.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-05-26T09:58:42.000Z","updated_at":"2025-03-18T13:47:25.000Z","dependencies_parsed_at":"2024-01-20T20:24:17.807Z","dependency_job_id":"f849d634-b403-40b5-9309-eac1d8d6bd22","html_url":"https://github.com/fluxninja/aperture","commit_stats":{"total_commits":2199,"total_committers":32,"mean_commits":68.71875,"dds":0.8021828103683493,"last_synced_commit":"d8f0e26a36efa258b0796daf5629db78355470cf"},"previous_names":[],"tags_count":2340,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fluxninja%2Faperture","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fluxninja%2Faperture/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fluxninja%2Faperture/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fluxninja%2Faperture/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/fluxninja","download_url":"https://codeload.github.com/fluxninja/aperture/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247809962,"owners_count":20999816,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["caching","cloud-native","concurrency-limiter","kubernetes","llm","microservices","observability","rate-limiter","scheduler"],"created_at":"2024-08-01T20:01:00.745Z","updated_at":"2025-04-08T09:08:18.680Z","avatar_url":"https://github.com/fluxninja.png","language":"Go","funding_links":[],"categories":["Go"],"sub_categories":[],"readme":"\u003c!-- vale off --\u003e\n\u003c!-- markdownlint-disable --\u003e\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"docs/content/assets/img/aperture_logo.png\" alt=\"FluxNinja Aperture\" width=\"75%\"\u003e\n  \u003cbr/\u003e\n  \u003ca href=\"https://docs.fluxninja.com\"\u003e\n    \u003cimg alt=\"Documentation Reference\" src=\"https://img.shields.io/badge/documentation-reference-brightgreen?style=for-the-badge\u0026logo=data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAACgAAAAoCAYAAACM/rhtAAAABGdBTUEAALGPC/xhBQAAACBjSFJNAAB6JgAAgIQAAPoAAACA6AAAdTAAAOpgAAA6mAAAF3CculE8AAAABmJLR0QA/wD/AP+gvaeTAAAACXBIWXMAABYlAAAWJQFJUiTwAAAAB3RJTUUH5gkaEAsvan7AVwAAB5FJREFUWMPtmFtsHOUVgL/zz+zFe/Oud23n4thOQgJJsXOnBEUoKlHoLVABKn0CVe0DVdUKVWqf2qilV4SqSkhVhaq+0NJGpQq9ASWlkQoEGpqQC5ALJMGOndjxbdf27nq9O/OfPswmaYPtBEFbIfmMRjvSzJz5/nP5zzkL8zIv8/LhFvkglOTHRkE1qiIrARHVtxCZyjRl37du8wHBNajId0T1eVH7NxV5BNVUfmz0/2vB/NgoGqi5S/za4+7+3XGsj3fzPRV1w18EfUKA92NJ95qtFCzGRTWuIkmgUUUWI2YzcL85cyTu7n8KrI9t7oz6q7b8EFilaveNjY32AeOiOolICfAAvRbwOS1Yd5+rIl0Y53aQjaDtqDaJ9VJ4taSU8lHzzmHc/buR8WEQ0EQT3kfvwi5bj8Yz04TCk2qcccTkQfpBD2D950T1CCK1uUDlKnBJFfM1UfslGR9qleFepDCIFMeQUgGZGMEUBpCJETSawNt0Bzgu7qu/R8rjaDKHzSxEUzk0noFEBptegOba0XTrsIrzC1H7MCKF2SBlNjgNbn1DquXvuwf+7DpH/4qMDyHWosYBx4VILIBYtBJ/1RZs2ypAkPMncU/sw/QfRyaGYboMvoeoj4pBkzn87m14N91pNRx7SNR+FxE7E+QcgCxCzB730F8+EnruZ2gqh12+EdvcgSYy0JBCG1JovBEi8QBabV2rAbVIpYSUC8jUJJQnkFIeM3IWc/ogUhikdtsX8DbdcQrVbQK9MwHOkSTSJl61zTlzENwQte0P4F+38Yo1KWg9j60PUr9nfVQEjcbRhsS77OCsOEx49w9wzryGv/bjC9UNd4L2zkQxO6AQxvqGqQmIJtBcGwBGLYKg9c9aQFUvw0FwrYqIYBS0HjCKYkXQ9AIIx6BaAbUuEJ0NY3ZARRETuC8URY2LABemJjg21k9husSieBNd2SXE3AhaP4K1CUaEil/jjdE++ktjJENRVmXaWBjPoI6LhmMQjoIYDVwxm51mj8GlwFNmqGeNVIpo22pOjA/y67dfZnhqgmQoyrRf48ZsO3d0rqc11ojUraiqjFQm+VPPaxwa7iHqhCh50zSGY3xuxS10Ny1G+k+goQh2wXXHRP27G761tUCyaTUwBLwBEH34xTkAjYP43jo1zo8EtherZX76+h4GywV2dK7HU8u0V+OlwbcIG4fubAcdyRwi0Dc5ytHRs5S8aT7VsY6q7zHlVXlx4ATpSJyvdN9OYySBiux13n71x5EnH9oI3I3IGWAn8PpFwBldnGnKBvugyCFR+6BBnh0qj3f0F8dY39yJAs/2HqEz2cyabDt7+99ksFzAEQcAX32sKlsXr6ZmfZ45e5hlqRa6su3sv3CKgVKeTCRxnncOPRTZtfMBHHcFIo8CfwBGLsLNHYOXJQKEtB6WRgzHx87hWZ90JIYRCaoH4KutJ8PlODyeP4dVSzoSv5Tx9R9HG5IGtAg0AD1XwgE4s7kYVQPcqsY8gsiNrhjeHOunrzjKmlwHy1MtZKNJ9l84RcQJsaFlKVsWXk9XbgnpcJyyN83w1CRrcx0sa2wlE4nz6tBpGsMxti3pIupGEprMrvSv3/xL5+jze8XaAiJ9AN/bd/aakuQG4GmTH1jGdBlal/HaSC+73n6ZslclHYlRrFVYlmrlzqUbWJLI4pige/Ot5Xwpzx97DvJWYYBEKMp4tUzUCXPvdZvZ0NyJDPVAKIxtWnwW4+wAjjal0+9hm0Fy4teaQ39/HBk9R/Xeb7O2eSnZaIIjI72MV6dYFM+wqWU5qXADiuLboJKICG2JLPddfysHhs/QPzlKKtxAd66D9mQOSgXCzzyKppqpfubrrYosFPToTBRzbdRgfSgWkEoxqKUCHclmOpLNlyJNUaxaROTSNgNg1RIPRdi6aPWlZ7l45VWDZiIUBWtnCbRAZu+oFQ/jWKJxmC4j40OoGKzI5fPiB+XdkSISVBsL//GOGgPFPNQq4IZBxAeqs2HM4WI9p27ogt/R3eic+iehPY9hbrgFm2uHeAaNJoI6G4mj4XqlqqfnxbIntWmoFJH6SamAGenDOfkKUiliO7pQN3Jhtjo8J6Co9qvILr/rtm/WJkeN+8ZeQi/+JrCA40IoGkA2tmDbVuOvvBnb0hm4ZbgX5+Q/MOeOIfnBAK5WQXwPrEVjKbwNn8Zbux2E34nVXmTm1vRqDWtGjdkpvvd5GelrNIOnkfx5pJQPGtbJUWRiBJmaQFPN1DbfA8bFfeW3mMJQfQHNaCKLxtNoIoNmFmBbl6O59qK6oV+JtTsRGX5PDesVkBEVsxljdgDrUF0k1k/je3FqlZhMjhjn1AHcg09DpRhoDMfw1n8Sf8VNaCpnCTWUcdyyGqeAyABwCGufFrX7rjaeXtNUd2loCsbLNJACySHSjsgtqN7jHnuhNfTMo2AttU98Ga/rY8OI2Y3qS6j2go4AE6JaQGQKsO97aLomcN8TdUP3SW36MfeFJyLi16htvb+q4ehXxfd+jnGuCeS/AngJUrVRRX4ivvdZUFEn9KSoPjjXMPQ/A/w3yKSKWQuIqD2sIhNNH8BfH/MyL/PyYZd/ASXTkEaNSEXqAAAAJXRFWHRkYXRlOmNyZWF0ZQAyMDIyLTA4LTMxVDEzOjA5OjA2KzAwOjAwaZ/rqgAAACV0RVh0ZGF0ZTptb2RpZnkAMjAyMi0wOC0zMVQxMzowOTowNiswMDowMBjCUxYAAAAASUVORK5CYII=\"\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://join.slack.com/t/fluxninja-aperture/shared_invite/zt-1eunlrkhh-10P1HUkmBfVJX3qrSLRk~g\"\u003e\n    \u003cimg alt=\"Slack Community\" src=\"https://img.shields.io/badge/Join%20Our%20Community-Slack-brightgreen?style=for-the-badge\u0026logo=slack\"\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://dl.circleci.com/status-badge/img/gh/fluxninja/aperture/tree/main.svg?style=svg\"\u003e\n    \u003cimg alt=\"Build Status\" src=\"https://img.shields.io/circleci/build/github/fluxninja/aperture/main?style=for-the-badge\u0026logo=circleci\"\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://goreportcard.com/report/github.com/fluxninja/aperture\"\u003e\n    \u003cimg alt=\"Go Report Card\" src=\"https://goreportcard.com/badge/github.com/fluxninja/aperture?style=for-the-badge\"\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://pkg.go.dev/github.com/fluxninja/aperture\"\u003e\n    \u003cimg alt=\"Godoc Reference\" src=\"https://img.shields.io/badge/godoc-reference-brightgreen?style=for-the-badge\u0026logo=go\u0026logoColor=white\"\u003e\n  \u003c/a\u003e\n\u003c/p\u003e\n\n\u003c!-- markdownlint-enable --\u003e\n\u003c!-- vale on --\u003e\n\n# 🥷 FluxNinja Aperture\n\nAperture is a distributed load management platform designed for rate limiting,\ncaching, and prioritizing requests in cloud applications. Built upon a\nfoundation of distributed counters, observability, and a global control plane,\nit provides a comprehensive suite of load management capabilities. These\ncapabilities enhance the reliability and performance of cloud applications,\nwhile also optimizing cost and resource utilization.\n\n![Unified Load Management](./docs/content/assets/img/unified-load-management-light.svg#gh-light-mode-only)\n![Unified Load Management](./docs/content/assets/img/unified-load-management-dark.svg#gh-dark-mode-only)\n\nIntegrating Aperture in your application through SDKs is a simple 3-step\nprocess:\n\n- **Define labels**: Define labels to identify users, entities, or features\n  within your application. For example, you can define labels to identify\n  individual users, features, or API endpoints.\n\n\u003c!-- vale off --\u003e\n\u003c!-- markdownlint-disable --\u003e\n\n  \u003cdetails\u003e\n  \u003csummary\u003eExample\u003c/summary\u003e\n\n```typescript\n// Tailor policies to get deeper insights into your workload with labels that\n// capture business context.\nconst labels = {\n  // You can rate limit each user individually.\n  user: \"jack\",\n  // And have different rate limits for different tiers of users.\n  tier: \"premium\",\n  // You can also provide the tokens for each request.\n  // Tokens are flexible: LLM AI tokens in a prompt, complexity of a request,\n  // number of sub-actions, etc.\n  tokens: \"200\",\n  // When peak load exceeds external quotas or infrastructure capacity,\n  // requests can be throttled and prioritized.\n  priority: HIGH,\n  // Get deep insights into your workload. You can slice and dice performance\n  // metrics by any label.\n  workload: \"/chat\",\n};\n```\n\n\u003c!-- markdownlint-enable --\u003e\n\u003c!-- vale on --\u003e\n\n  \u003c/details\u003e\n\n- **Wrap your workload**: Wrap your workload with `startFlow` and `endFlow`\n  calls to establish control points around specific features or code sections\n  inside your application. For example, you can wrap your API endpoints with\n  Aperture SDKs to limit the number of requests per user or feature.\n\n\u003c!-- vale off --\u003e\n\u003c!-- markdownlint-disable --\u003e\n\n  \u003cdetails\u003e\n  \u003csummary\u003eExample\u003c/summary\u003e\n\n```typescript\n// Wrap your workload with startFlow and endFlow calls, passing in the\n// labels you defined earlier.\nconst flow = await apertureClient.startFlow(\"your_workload\", {\n  labels: labels,\n  // Lookup result cache key to retrieve a cached result.\n  resultCacheKey: queryParams,\n});\n\n// If rate or quota limit is not exceeded, the workload is executed.\nif (flow.shouldRun()) {\n  // Return a cached result or execute the workload.\n  const cachedResult = flow.resultCache();\n  const result = await yourWorkload(cachedResult);\n  flow.setResultCache({\n    value: result,\n    ttl: { seconds: 86400, nanos: 0 },\n  });\n}\n//\n```\n\n  \u003c/details\u003e\n\n\u003c!-- markdownlint-enable --\u003e\n\u003c!-- vale on --\u003e\n\n- **Configure \u0026 monitor policies**: Configure policies to control the rate,\n  concurrency, and priority of requests.\n\n\u003c!-- vale off --\u003e\n\u003c!-- markdownlint-disable --\u003e\n\n  \u003cdetails\u003e\n  \u003csummary\u003ePolicy YAML\u003c/summary\u003e\n\n```yaml\nblueprint: rate-limiting/base\nuri: github.com/fluxninja/aperture/blueprints@latest\npolicy:\n  policy_name: rate_limit\n  rate_limiter:\n    bucket_capacity: 60\n    fill_amount: 60\n    parameters:\n      interval: 3600s\n      limit_by_label_key: user\n    selectors:\n      - control_point: your_workload\n        label_matcher:\n          match_list:\n            - key: tier\n              operator: In\n              values:\n                - premium\n```\n\n  \u003c/details\u003e\n\n\u003c!-- markdownlint-enable --\u003e\n\u003c!-- vale on --\u003e\n\n![Rate Limiter Blueprint](./docs/content/get-started/assets/rate-limiter-blueprint-dark.png#gh-dark-mode-only)\n![Rate Limiter Blueprint](./docs/content/get-started/assets/rate-limiter-blueprint-light.png#gh-light-mode-only)\n![Rate Limiter Dashboard](./docs/content/get-started/assets/rate-limiter-dashboard-dark.png#gh-dark-mode-only)\n![Rate Limiter Dashboard](./docs/content/get-started/assets/rate-limiter-dashboard-light.png#gh-light-mode-only)\n\nIn addition to language SDKs, Aperture also integrates with existing control\npoints such as API gateways, service meshes, and application middlewares.\n\n## ⚙️ Load management capabilities\n\n- ⏱️\n  [**Global Rate and Concurrency Limiting**](https://docs.fluxninja.com/concepts/rate-limiter):\n  Safeguard APIs and features against excessive usage with Aperture's\n  high-performance, distributed rate limiter. Identify individual users or\n  entities by fine-grained labels. Create precise rate limiters controlling\n  burst-capacity and fill-rate tailored to business-specific labels. Limit per\n  user or global concurrency of in-flight requests. Refer to the\n  [Rate Limiting](https://docs.fluxninja.com/guides/per-user-rate-limiting) and\n  [Concurrency Limiting](https://docs.fluxninja.com/guides/per-user-concurrency-limiting)\n  guides for more details.\n- 📊\n  [**API Quota Management**](https://docs.fluxninja.com/concepts/request-prioritization/quota-scheduler):\n  Maintain compliance with external API quotas with a global token bucket and\n  smart request queuing. This feature regulates requests aimed at external\n  services, ensuring that the usage remains within prescribed rate limits and\n  avoids penalties or additional costs. Refer to the\n  [API Quota Management](https://docs.fluxninja.com/guides/api-quota-management/)\n  guide for more details.\n- 🚦\n  [**Concurrency Control and Prioritization**](https://docs.fluxninja.com/concepts/request-prioritization/concurrency-scheduler):\n  Safeguard against abrupt service overloads by limiting the number of\n  concurrent in-flight requests. Any requests beyond this limit are queued and\n  let in based on their priority as capacity becomes available. Refer to the\n  [Concurrency Control and Prioritization](https://docs.fluxninja.com/development/guides/concurrency-control-and-prioritization/)\n  guide for more details.\n- 🎯\n  [**Workload Prioritization**](https://docs.fluxninja.com/concepts/scheduler/):\n  Safeguard crucial user experience pathways and ensure prioritized access to\n  external APIs by strategically prioritizing workloads. With\n  [weighted fair queuing](https://en.wikipedia.org/wiki/Weighted_fair_queueing),\n  Aperture aligns resource distribution with business value and urgency of\n  requests. Workload prioritization applies to API Quota Management and\n  Concurrency Control and Prioritization use cases.\n- 💾 [**Caching**](https://docs.fluxninja.com/concepts/cache): Boost application\n  performance and reduce costs by caching costly operations, preventing\n  duplicate requests to pay-per-use services, and easing the load on constrained\n  services. Refer to the [Caching](https://docs.fluxninja.com/guides/caching)\n  guide for more details.\n\n## 🏁 Getting Started\n\n### ☁️ Aperture Cloud\n\n\u003c!-- vale off --\u003e\n\n\u003e [!NOTE]\n\u003e\n\u003e FluxNinja has been acquired by [CodeRabbit](https://coderabbit.ai). New\n\u003e sign-ups are temporarily disabled. Existing users can continue to use Aperture\n\u003e Cloud by signing in to their accounts.\n\n\u003c!-- vale on --\u003e\n\nThe easiest way to try Aperture is to\n[sign up](https://app.fluxninja.com/sign-up) for a free Aperture Cloud account.\nAperture Cloud is a fully managed service by FluxNinja. With Aperture Cloud,\nthere's no need to manage any infrastructure, and you can integrate your\napplication with Aperture using SDKs. For more information, refer to the\n[get started](https://docs.fluxninja.com/get-started/) guide.\n\n![Quota Management Dashboard](./docs/content/guides/assets/openai/priorities-light.png#gh-light-mode-only)\n![Quota Management Dashboard](./docs/content/guides/assets/openai/priorities-dark.png#gh-dark-mode-only)\n_Prioritization Metrics for gpt-4_\n![Flow Analytics](./docs/content/guides/assets/openai/flow-analytics-light.png#gh-light-mode-only)\n![Flow Analytics](./docs/content/guides/assets/openai/flow-analytics-dark.png#gh-dark-mode-only)\n_Performance Metrics for OpenAI Models_\n\n### 🎮 Local Kubernetes Playground\n\nTo try Aperture in a local Kubernetes environment, refer to\n[Playground docs](./playground/README.md).\n\n### 📖 Learn More\n\n- [Concepts](https://docs.fluxninja.com/concepts) section in Aperture\n  documentation provides an overview of Aperture's features and architecture.\n- [Guides](https://docs.fluxninja.com/guides) section in Aperture documentation.\n- [Observability-driven Load Management](https://docs.google.com/presentation/d/1Funig5u63w96G91ZjHV7ftgLOl1c5z5yDeuoVRH1QSs)\n  deck covers the overview and use-cases of Aperture.\n\n## 🎥 Videos\n\n- [IstioCon 2023 | Achieving Fault Tolerance in Istio with Observability-Driven Load Management](https://www.youtube.com/watch?v=eiSroeyACAg)\n- [SREcon'23 APAC | Mastering Chaos: Achieving Fault Tolerance with Observability-Driven Prioritized Load Shedding](https://www.youtube.com/watch?v=ws9__JjaJsE)\n- [Chaos Carnival 2023 | Graceful Degradation: Keeping The Lights On When Everything Goes Wrong](https://www.youtube.com/watch?v=yHKPXsZOc5I)\n- [Conf42 Chaos Engineering 2023 | Graceful Degradation: When All Goes Wrong](https://www.youtube.com/watch?v=nm62d2gYqNk)\n- [Demo | How Concurrency Limits Help Protect Against Cascading Failures](https://youtu.be/m070bAvrDHM)\n- [Explainer | Build Indestructible Applications with Aperture Flow Control](https://www.youtube.com/watch?v=sEl4SMo3KNo)\n\n## 👷 Contributing\n\n[Reporting bugs](https://github.com/fluxninja/aperture/issues/new?assignees=\u0026labels=bug\u0026template=bug_report.md\u0026title=)\nhelps us improve Aperture to be more reliable and user-friendly. Include all the\nrequired information to reproduce and understand the bug you are reporting.\nFollow helper questions in the bug report template to make it easier. If you see\na way to improve Aperture, use the\n[feature request](https://github.com/fluxninja/aperture/issues/new?assignees=\u0026labels=feature+request\u0026template=feature_request.md\u0026title=)\ntemplate to create an issue.\n\nTo contribute code, read the [Contribution guide](CONTRIBUTING.md).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffluxninja%2Faperture","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffluxninja%2Faperture","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffluxninja%2Faperture/lists"}