{"id":15285817,"url":"https://github.com/g-andrade/deigma","last_synced_at":"2025-07-15T19:06:46.032Z","repository":{"id":51353696,"uuid":"121907224","full_name":"g-andrade/deigma","owner":"g-andrade","description":"Event sampler","archived":false,"fork":false,"pushed_at":"2022-04-09T19:08:58.000Z","size":142,"stargazers_count":24,"open_issues_count":0,"forks_count":2,"subscribers_count":5,"default_branch":"master","last_synced_at":"2025-04-13T02:38:09.899Z","etag":null,"topics":["elixir","erlang","event-sampler","rate-limiting","sampling"],"latest_commit_sha":null,"homepage":"https://hexdocs.pm/deigma/","language":"Erlang","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/g-andrade.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2018-02-18T01:14:01.000Z","updated_at":"2022-01-03T23:10:01.000Z","dependencies_parsed_at":"2022-09-26T21:40:57.782Z","dependency_job_id":null,"html_url":"https://github.com/g-andrade/deigma","commit_stats":null,"previous_names":[],"tags_count":7,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/g-andrade%2Fdeigma","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/g-andrade%2Fdeigma/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/g-andrade%2Fdeigma/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/g-andrade%2Fdeigma/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/g-andrade","download_url":"https://codeload.github.com/g-andrade/deigma/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248657853,"owners_count":21140843,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["elixir","erlang","event-sampler","rate-limiting","sampling"],"created_at":"2024-09-30T15:07:44.282Z","updated_at":"2025-04-13T02:38:31.111Z","avatar_url":"https://github.com/g-andrade.png","language":"Erlang","funding_links":[],"categories":[],"sub_categories":[],"readme":"# deigma\n\n**This library is not under active maintenance; if you'd like to perform\nmaintenance yourself, feel free to open an issue requesting access.**\n\n[![](https://img.shields.io/hexpm/v/deigma.svg?style=flat)](https://hex.pm/packages/deigma)\n[![](https://github.com/g-andrade/deigma/workflows/build/badge.svg)](https://github.com/g-andrade/deigma/actions?query=workflow%3Abuild)\n\n`deigma` is an event sampler for Erlang/OTP and Elixir.\n\nIt performs sampling of reported events within continuous one second\nwindows\\[\\*\\].\n\nThe sampling percentage is steadily adjusted so that the events that\nseep through are representative of what's happening in the system while\nnot exceeding specified rate limits.\n\nThe sampling percentage is also exposed in the context of each reported\nevent, so that whichever other component that later receives the samples\ncan perform reasonable guesses of the original population properties\nwith limited information.\n\n\\[\\*\\] As far as the [monotonic\nclock](http://erlang.org/doc/apps/erts/time_correction.html#Erlang_Monotonic_Time)\nresolution goes.\n\n#### Example (Erlang)\n\nThere's a heavy duty web service; we want to report metrics on inbound\nHTTP requests to a [StatsD](https://github.com/etsy/statsd) service over\nUDP while minimising the risk of dropped datagrams due to an excessive\namount of them.\n\nFor this, we can downsample the reported metrics while determining the\nreal sampling percentage using `deigma`.\n\n##### 1\\. Start a deigma instance\n\n``` erlang\nCategory = metrics,\n{ok, _Pid} = deigma:start(Category).\n```\n\n##### 2\\. Sample events\n\n``` erlang\nCategory = metrics,\nEventType = http_request,\n\ncase deigma:ask(Category, EventType) of\n    {sample, SamplingPercentage} -\u003e\n        your_metrics:report(counter, EventType, +1, SamplingPercentage);\n    {drop, _SamplingPercentage} -\u003e\n        ok\nend.\n```\n\n  - [`Category`](#categories) must be an atom\n  - [`EventType`](#event-windows) can be any term\n  - `SamplingPercentage` is a floating point number between 0.0 and 1.0\n    representing the percentage of events that were sampled during the\n    last 1000 milliseconds, **including** the event reported just now.\n  - The rate limit defaults to 100 `EventType` occurences per second\n    within a `Category`; it can be [overridden](#rate-limiting).\n  - The function invoked each time an event gets registered can also be\n    [customized](#custom-event-functions-and-serializability).\n\n#### Example (Elixir)\n\nSame scenario as in the Erlang example.\n\n##### 1\\. Start a deigma instance\n\n``` elixir\ncategory = :metrics\n{:ok, _pid} = :deigma.start(category)\n```\n\n##### 2\\. Sample events\n\n``` elixir\ncategory = :metrics\nevent_type = :http_request\n\ncase :deigma.ask(category, event_type) do\n    {:sample, sampling_percentage} -\u003e\n        YourMetrics.report(:counter, event_type, +1, sampling_percentage)\n    {:drop, _sampling_percentage} -\u003e\n        :ok\nend\n```\n\n#### Documentation and Reference\n\nDocumentation and reference are hosted on\n[HexDocs](https://hexdocs.pm/deigma/).\n\n#### Tested setup\n\n  - Erlang/OTP 22 or higher\n  - rebar3\n\n#### Categories\n\nEach `Category` represents an independent group of events and is managed\nseparately; categories can be launched under your own supervision tree\n(using `:child_spec/1` or `:start_link/1`) as well as under the `deigma`\napplication (using `:start/1`).\n\nCategories launched under `deigma` can be stopped using `:stop/1`.\n\n#### Event windows\n\nWithin the context of each `Category`, each distinct `EventType` will be\nhandled under dedicated event windows that are owned by independent\nprocesses.\n\nThese processes are created on demand as new `EventType` values get\nsampled, and stopped after 1000 milliseconds of inactivity.\n\n#### Rate limiting\n\nEach time a new event is reported, the rate limit is applied according\nto how many events were sampled so far during the previous 1000\nmilliseconds; if the limit has been or is about to be exceeded, the\nevent gets dropped.\n\nThe default rate limit is set to 100 `EventType` occurrences per second\nwithin a `Category`. It can be overridden using `:ask` options:\n\n``` erlang\nCategory = metrics,\nEventType = http_request,\nMaxRate = 50,\n\ncase deigma:ask(Category, EventType, [{max_rate, MaxRate}]) of\n    {sample, SamplingPercentage} -\u003e\n        your_metrics:report(counter, EventType, +1, SamplingPercentage);\n    {drop, _SamplingPercentage} -\u003e\n        ok\nend.\n```\n\n#### Custom event functions and serializability\n\nThe function invoked upon an event getting registered, within an event\nwindow, can be customized.\n\nThis is useful if you need serializability when handling sampling\ndecisions and percentages, at the expense of increasing the risk of the\nevent window becoming a performance bottleneck.\n\n``` erlang\nCategory = metrics,\nEventType = http_request,\n\ndeigma:ask(\n    Category, EventType,\n    fun (Timestamp, sample, SamplingPercentage) -\u003e\n            your_metrics:report(counter, EventType, +1, SamplingPercentage);\n        (_Timestamp, drop, _SamplingPercentage) -\u003e\n            ok\n    end).\n```\n\n  - `Timestamp` is the [monotonic\n    timestamp](http://erlang.org/doc/man/erlang.html#monotonic_time-0),\n    in native units, at which the event was registered\n\nIn this scenario, whatever your function returns (or throws) will be\nwhat `deigma:ask` returns (or throws.)\n\n#### License\n\nMIT License\n\nCopyright (c) 2018-2022 Guilherme Andrade\n\nPermission is hereby granted, free of charge, to any person obtaining a\ncopy of this software and associated documentation files (the\n\"Software\"), to deal in the Software without restriction, including\nwithout limitation the rights to use, copy, modify, merge, publish,\ndistribute, sublicense, and/or sell copies of the Software, and to\npermit persons to whom the Software is furnished to do so, subject to\nthe following conditions:\n\nThe above copyright notice and this permission notice shall be included\nin all copies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS\nOR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF\nMERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.\nIN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY\nCLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,\nTORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE\nSOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fg-andrade%2Fdeigma","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fg-andrade%2Fdeigma","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fg-andrade%2Fdeigma/lists"}