{"id":17108451,"url":"https://github.com/g-andrade/aequitas","last_synced_at":"2025-04-13T02:48:10.509Z","repository":{"id":51354020,"uuid":"128668470","full_name":"g-andrade/aequitas","owner":"g-andrade","description":"Fairness regulator and rate limiter","archived":false,"fork":false,"pushed_at":"2022-04-09T19:02:26.000Z","size":200,"stargazers_count":49,"open_issues_count":1,"forks_count":2,"subscribers_count":7,"default_branch":"master","last_synced_at":"2025-04-13T02:47:49.717Z","etag":null,"topics":["fairness","outlier-detection","rate-limiting"],"latest_commit_sha":null,"homepage":"https://hexdocs.pm/aequitas/","language":"Erlang","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/g-andrade.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2018-04-08T18:09:04.000Z","updated_at":"2022-01-03T23:08:26.000Z","dependencies_parsed_at":"2022-09-26T21:41:01.776Z","dependency_job_id":null,"html_url":"https://github.com/g-andrade/aequitas","commit_stats":null,"previous_names":[],"tags_count":9,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/g-andrade%2Faequitas","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/g-andrade%2Faequitas/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/g-andrade%2Faequitas/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/g-andrade%2Faequitas/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/g-andrade","download_url":"https://codeload.github.com/g-andrade/aequitas/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248657869,"owners_count":21140844,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["fairness","outlier-detection","rate-limiting"],"created_at":"2024-10-14T16:05:24.359Z","updated_at":"2025-04-13T02:48:10.492Z","avatar_url":"https://github.com/g-andrade.png","language":"Erlang","funding_links":[],"categories":[],"sub_categories":[],"readme":"# aequitas\n\n**This library is not under active maintenance; if you'd like to perform\nmaintenance yourself, feel free to open an issue requesting access.**\n\n[![](https://img.shields.io/hexpm/v/aequitas.svg?style=flat)](https://hex.pm/packages/aequitas)\n[![](https://github.com/g-andrade/aequitas/workflows/build/badge.svg)](https://github.com/g-andrade/aequitas/actions?query=workflow%3Abuild)\n\n`aequitas` is a fairness regulator for Erlang/OTP and Elixir, with\noptional rate limiting capabilities.\n\nIt intends on allowing fair access to limited external resources, like\ndatabases and web services, amongst distinct actors.\n\nIt does so by attempting to detect outliers in ordinary workload\ndistributions.\n\n#### Example\n\nThere's a web server handling HTTP requests. We want to ensure\nmisbehaving IP addresses don't steal (too much) system capacity from\nbenevolent clients.\n\nWe'll name our category `http_requests` and start its handler.\nCategories and actors can be represented by any term.\n\n``` erlang\nok = aequitas:start(http_requests).\n```\n\nNow, before we handle each HTTP request we ask `aequitas` whether an IP\naddress can be granted work. We'll get a reply that's based on the\nstatistical distribution of recent work allocations.\n\n``` erlang\ncase aequitas:ask(http_requests, IPAddress) of\n    accepted -\u003e\n        Reply = handle_request(...),\n        {200, Reply};\n    {rejected, _Reason} -\u003e\n        % too many requests!\n        {429, \u003c\u003c\u003e\u003e}\nend.\n```\n\nSome more definitions of the `:ask` function exist:\n\n  - `:ask(Category, ActorId, Opts)` - for\n    [tweaking](#work-request-tweaking)\n  - `:async_ask(Category, ActorId)` - analogous to `:ask/2` but replies\n    asynchronously\n  - `:async_ask(Category, ActorId, Opts)` - analogous to `:ask/3` but\n    replies asynchronously\n\n#### Documentation and Reference\n\nDocumentation and reference are hosted on\n[HexDocs](https://hexdocs.pm/aequitas/).\n\n#### Tested setup\n\n  - Erlang/OTP 22 or higher\n  - rebar3\n\n#### Category Tweaking\n\nThe following options can be used to tweak categories, both through\n[static](#static-configuration) and [dynamic\nconfiguration](#dynamic-configuration).\n\n  - `{max_window_size, _}`\n      - Enforces a ceiling on how many of the last work acceptances will\n        be [tracked](#work-tracking)\n      - Default is 10000; value must be a positive integer or infinity\n  - `{max_window_duration, _}`\n      - Enforces an eventual expiration of tracked work acceptances\n      - Default is `{seconds,5}`; value must be of type\n        `aequitas_time_interval:t()` or infinity (see [type\n        reference](#documentation-and-reference))\n  - `{min_actor_count, _}`\n      - Establishes the requirement of a minimum amount of tracked\n        actors before [outlier detection](#outlier-detection) is\n        performed\n      - Default is 30; value must be a positive integer\n  - `{iqr_factor, _}`\n      - IQR factor used to detect outlying actors among the workload\n        distribution\n      - Default is 1.5; value must be a non-negative number\n  - `{max_collective_rate, _}`\n      - Enforces a [collective rate limit](#collective-rate-limiting),\n        per second, on work acceptances\n      - Default is infinity; value must be a non-negative integer\n\n#### Work Request Tweaking\n\nThe following options can be used to tweak individual work requests,\ni.e. in calls to either the `:ask/3` or `:async_ask/3` functions.\n\n  - `{weight, _}`\n      - [Relative weight](#work-weighing) of the work request\n      - Default is 1; value must be a positive integer\n  - `{min_actor_count, _}`\n      - Overrides the `min_actor_count` configured globally for the\n        category; its meaning remains but it only applies to the work\n        request that overrode it\n  - `{iqr_factor, _}`\n      - Overrides the `iqr_factor` configured globally for the category;\n        its meaning remains but it only applies to the work request that\n        overrode it\n  - `return_stats`\n      - Return the work stats used to detect outliers together with\n        acceptance status (see [API\n        reference](#documentation-and-reference))\n\n#### Collective Rate Limiting\n\nCollective rate limiting can be enabled in order to limit the total\namount of work performed, per second, within a category.\n\nOnce the configured limit is reached, its enforcement should tend to\nhomogenize the number of accepted work requests per actor, even in case\nof very unbalanced workloads - within the reach of the configured\noutlier detection, that is.\n\nIf the number of distinct actors tracked by the window is rather larger\nthan the rate limit, and if `iqr_factor` is set to a very strict value,\nit should ultimately result in every actor performing at most one\nrequest within the period covered by the sliding window.\n\nThis contrasts with impartial rate limiting which is the scope of many\nother libraries, whereby the acceptance/rejection ratios per actor tend\nto be the same, independently of how much work each actor is effectively\nperforming.\n\n[Work weighing](#work-weighing) is taken into account when rate\nlimiting.\n\n#### Outlier Detection\n\nThe outlier detection algorithm is based on [John Tukey's\nfences](http://sphweb.bumc.bu.edu/otlt/MPH-Modules/BS/BS704_SummarizingData/BS704_SummarizingData7.html).\n\nThe [IQR](https://en.wikipedia.org/wiki/Interquartile_range)\n(interquartile range) of the work shares per actor, encompassed by the\nwork tracker, is updated continuously (although asynchronously). Based\non this statistical measure, whenever there's a new request for work\nexecution we determine whether the actor's present work share is an\noutlier to the right.\n\nThe default threshold of outlier categorization is set to `Q3 + (1.5 x\nIQR)`, with IQR being `Q3 - Q1` and Q1 and Q3 being the median values of\nthe lower and higher halves of the samples, respectively.\n\nThe IQR factor can be customized, both per\n[category](#category-tweaking) and per [work\nrequest](#work-request-tweaking), using the `iqr_factor` setting, as\ndetailed previously.\n\nLower values will result in greater intolerance of high work-share\noutliers; higher values, the opposite.\n\nThe reason for picking `1.5` as the default is rather down to convention\nand might not be appropriate for your specific workloads. If necessary:\nmeasure, adjust, repeat.\n\nIt's possible you'll conclude the IQR technique is not adequate to solve\nyour problem; a score of other approaches exist, with many of them being\ncomputationally (much) more expensive, - it's a trade off between\ncorrectness and availability.\n\n#### Work Weighing\n\nWork requests can be weighted by specifying the `weight` option when\nasking permission to execute. It must be a positive integer; the default\nvalue is `1`.\n\nPicking the web server example above, if we were to weight our requests\nbased on their body size, it could become something similar to this:\n\n``` erlang\nReqBodySize = req_body_size(...),\nWorkWeight = 1 + ReqBodySize,\ncase aequitas:ask(http_requests, IPAddress, [{weight, Weight}]) of\n% [...]\nend.\n```\n\nThis way, the work share of an IP address performing a few large\nrequests could of similar magnitude to the work share of an IP address\nperforming many small requests.\n\n#### Work Tracking\n\nEach category is backed by a process that keeps a sliding window; this\nsliding window helps keep track of how many work units were attributed\nto each actor within the last `N` accepted requests;\n\nThe size of the sliding window, `N`, is determined by constraints\nderived from [category settings](#category-tweaking). Whenever it gets\nexcessively large, old events will be dropped until it is no longer so.\n\n#### Static Configuration\n\nThe configuration of foreknown categories can be tweaked in `app.config`\n/ `sys.config` by declaring the overriden settings, per category, in the\nfollowing fashion:\n\n``` erlang\n% ...\n {aequitas,\n  [{categories,\n   #{ http_requests =\u003e\n        [{max_window_duration, {seconds,10}}] % Override default 5s to 10s\n\n      rare_ftp_requests =\u003e\n        [{max_window_size, 100}] % Only track up to the last 100 acceptances\n   }}\n  ]}\n% ...\n```\n\nThese categories will start on boot if the configuration is valid.\n\nProper app. configuration reloads that result in calls to the\napplication's internal `:config_change/3` callback will trigger a reload\nof settings in each of the relevant category processes.\n\n#### Dynamic Configuration\n\nReconfiguration of running categories can be performed by calling\n`aequitas:reconfigure/2`, e.g.:\n\n``` erlang\nok = aequitas:reconfigure(http_requests,\n                          [{max_window_duration, {seconds,10}}]).\n```\n\n(Re)configuration performed this way will override the static category\nconfiguration present in `app.config`, if any.\n\nIt will also trigger a reload of settings in the relevant category\nprocess.\n\n#### License\n\nMIT License\n\nCopyright (c) 2018-2022 Guilherme Andrade\n\nPermission is hereby granted, free of charge, to any person obtaining a\ncopy of this software and associated documentation files (the\n\"Software\"), to deal in the Software without restriction, including\nwithout limitation the rights to use, copy, modify, merge, publish,\ndistribute, sublicense, and/or sell copies of the Software, and to\npermit persons to whom the Software is furnished to do so, subject to\nthe following conditions:\n\nThe above copyright notice and this permission notice shall be included\nin all copies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS\nOR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF\nMERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.\nIN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY\nCLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,\nTORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE\nSOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.\n\n-----\n\n*Generated by EDoc*\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fg-andrade%2Faequitas","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fg-andrade%2Faequitas","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fg-andrade%2Faequitas/lists"}