{"id":21346879,"url":"https://github.com/queer/emit","last_synced_at":"2025-07-12T17:31:24.517Z","repository":{"id":41247372,"uuid":"491252526","full_name":"queer/emit","owner":"queer","description":"Powerful metadata-backed pubsub for Elixir.","archived":false,"fork":false,"pushed_at":"2024-01-10T22:19:59.000Z","size":41,"stargazers_count":13,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"mistress","last_synced_at":"2024-09-14T13:13:58.428Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Elixir","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/queer.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2022-05-11T19:49:32.000Z","updated_at":"2024-07-22T05:14:48.000Z","dependencies_parsed_at":"2024-01-10T01:06:02.513Z","dependency_job_id":null,"html_url":"https://github.com/queer/emit","commit_stats":{"total_commits":9,"total_committers":1,"mean_commits":9.0,"dds":0.0,"last_synced_commit":"ec8bff4076e9e7210f8a47f9a8727ff9d46e053e"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/queer%2Femit","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/queer%2Femit/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/queer%2Femit/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/queer%2Femit/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/queer","download_url":"https://codeload.github.com/queer/emit/tar.gz/refs/heads/mistress","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":225072069,"owners_count":17416482,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-22T02:11:49.912Z","updated_at":"2024-11-22T02:11:50.440Z","avatar_url":"https://github.com/queer.png","language":"Elixir","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Emit\n\n\u003e *Powerful metadata-backed pubsub for Elixir.*\n\nEmit is a pubsub library that allows controlling the receivers of messages\nthrough granular metadata queries. pids that subscribe to Emit messages can\nset metadata about itself -- basically just a map with any keys/values the user\nwants -- and messages can be sent to pids matching certain metadata.\n\nFor example, you could use this to send a message to all pids that have been\nwaiting more than 30 seconds:\n\n```elixir\nrequire Lethe\n\nquery =\n  Emit.query()\n  |\u003e Lethe.where(map_get(:wait_time, :metadata) \u003e= 30_000)\n\nEmit.pub {:my_message, \"some data\"}, query\n```\n\nOr you could push a message to all pids for a specific user:\n\n```elixir\nrequire Lethe\n\nuser_id = ...\n\nquery =\n  Emit.query()\n  |\u003e Lethe.where(map_get(:user_id, :metadata) == ^user_id)\n\nEmit.pub :disconnect, query\n```\n\nEmit will always be slower than a basic pubsub system, due to the fact that it\nhas to query an in-memory database on each message send.\n\nEmit works transparently across nodes.\n\n## Usage\n\n1. [Get it from Hex](https://hex.pm/packages/emit) and add to mix.exs\n2. Add `config :emit, :task_scheduler, MyApp.Emit.TaskScheduler` to config.exs\n3. Add `{Task.Supervisor, name: MyApp.Emit.TaskScheduler}` towards the top of\n   your supervision tree\n4. Add `Emit.Cluster` to your supervision tree, after `libcluster` or similar\n5. Add `Emit.DB` to your supervision tree, after `Emit.Cluster`\n\n## API\n\n### Subscribe to messages, setting some metadata about the pid\n\n```elixir\nEmit.sub %{key: \"value\"}\n```\n\n### Unsubscribe this pid from Emit messages\n\n```elixir\nEmit.unsub()\n```\n\n### Automatically unsubscribe from Emit messages when this pid stops\n\n```elixir\nEmit.unsub_auto()\n```\n\n### Push messages\n\n```elixir\n# Broadcast a message to all clients\nEmit.pub :hello_world, Emit.query()\n\n# Broadcast a message to all clients that have `key: \"value\"` in their metadata\n# Emit queries are powered by Lethe: https://github.com/queer/lethe\nrequire Lethe\n\nquery =\n  Emit.query()\n  |\u003e Lethe.where(map_get(:key, :metadata) == \"value\")\n\nEmit.pub :hello_world, query\n```\n\n### Complex metadata\n\n```elixir\nrequire Lethe\n\n# Set some deeply-nested metadata\nEmit.sub %{key: %{key2: %{\"value\"}}\n\nquery =\n  Emit.query()\n  |\u003e Lethe.where(map_get(:key2, map_get(:key, :metadata)) == \"value\")\n\nEmit.pub :hello_world, query\n```\n\n# Benchmarks\n\n### Please run your own benchmarks to determine if Emit is suitable for your use-case!!!\n\nThe machine this was benchmarked on is quite high-end relative to most\ndeveloper workstations etc.\n\n\u003cdetails\u003e\n   \u003csummary\u003eBenchmarks (1k clients)\u003c/summary\u003e\n   \u003cpre\u003e\u003ccode\u003e\ngit:(mistress) 10 | ▶  mix bench.1k\n\n11:15:07.892 [debug] [EMIT] [CLUSTER] boot: node: monitor up\n\n11:15:07.896 [notice] Application mnesia exited: :stopped\nOperating System: Linux\nCPU Information: AMD Ryzen Threadripper 3960X 24-Core Processor\nNumber of Available Cores: 48\nAvailable memory: 251.62 GB\nElixir 1.13.4\nErlang 24.3.4\n\nBenchmark suite executing with the following configuration:\nwarmup: 2 s\ntime: 5 s\nmemory time: 0 ns\nreduction time: 0 ns\nparallel: 8\ninputs: none specified\nEstimated total run time: 28 s\n\nBenchmarking sending messages to all 1000 clients ...\nBenchmarking sending messages to only 100 clients ...\nBenchmarking sending messages to only 50 clients ...\nBenchmarking sending messages to only 500 clients ...\n\nName                                           ips        average  deviation         median         99th %\nsending messages to only 100 clients        3.11 K      321.96 μs    ±38.37%      298.17 μs      618.06 μs\nsending messages to only 500 clients        2.82 K      354.12 μs    ±25.38%      343.55 μs      618.60 μs\nsending messages to only 50 clients         2.78 K      359.52 μs    ±26.43%      347.99 μs      633.01 μs\nsending messages to all 1000 clients        1.24 K      805.97 μs    ±28.84%      758.06 μs     1491.08 μs\n\nComparison:\nsending messages to only 100 clients        3.11 K\nsending messages to only 500 clients        2.82 K - 1.10x slower +32.16 μs\nsending messages to only 50 clients         2.78 K - 1.12x slower +37.56 μs\nsending messages to all 1000 clients        1.24 K - 2.50x slower +484.02 μs\ngit:(mistress) 10 | ▶\n   \u003c/code\u003e\u003c/pre\u003e\n\u003c/details\u003e\n\n\u003cdetails\u003e\n   \u003csummary\u003eBenchmarks (10k clients)\u003c/summary\u003e\n   \u003cpre\u003e\u003ccode\u003e\ngit:(mistress) 10 | ▶  mix bench.10k\n\n11:15:52.647 [debug] [EMIT] [CLUSTER] boot: node: monitor up\n\n11:15:52.650 [notice] Application mnesia exited: :stopped\nOperating System: Linux\nCPU Information: AMD Ryzen Threadripper 3960X 24-Core Processor\nNumber of Available Cores: 48\nAvailable memory: 251.62 GB\nElixir 1.13.4\nErlang 24.3.4\n\nBenchmark suite executing with the following configuration:\nwarmup: 2 s\ntime: 5 s\nmemory time: 0 ns\nreduction time: 0 ns\nparallel: 8\ninputs: none specified\nEstimated total run time: 28 s\n\nBenchmarking sending messages to all 10k clients ...\nBenchmarking sending messages to only 1k clients ...\nBenchmarking sending messages to only 500 clients ...\nBenchmarking sending messages to only 5k clients ...\n\nName                                           ips        average  deviation         median         99th %\nsending messages to only 500 clients        967.03        1.03 ms    ±17.09%        1.00 ms        1.64 ms\nsending messages to only 5k clients         946.15        1.06 ms    ±17.49%        1.02 ms        1.71 ms\nsending messages to only 1k clients         933.83        1.07 ms    ±23.72%        1.03 ms        1.75 ms\nsending messages to all 10k clients         174.44        5.73 ms    ±22.20%        5.52 ms        9.47 ms\n\nComparison:\nsending messages to only 500 clients        967.03\nsending messages to only 5k clients         946.15 - 1.02x slower +0.0228 ms\nsending messages to only 1k clients         933.83 - 1.04x slower +0.0368 ms\nsending messages to all 10k clients         174.44 - 5.54x slower +4.70 ms\ngit:(mistress) 10 | ▶\n   \u003c/code\u003e\u003c/pre\u003e\n\u003c/details\u003e\n\n\u003cdetails\u003e\n   \u003csummary\u003eBenchmarks (100k clients)\u003c/summary\u003e\n   \u003cpre\u003e\u003ccode\u003e\ngit:(mistress) 10 | ▶  mix bench.100k\n\n11:16:36.182 [debug] [EMIT] [CLUSTER] boot: node: monitor up\n\n11:16:36.186 [notice] Application mnesia exited: :stopped\nOperating System: Linux\nCPU Information: AMD Ryzen Threadripper 3960X 24-Core Processor\nNumber of Available Cores: 48\nAvailable memory: 251.62 GB\nElixir 1.13.4\nErlang 24.3.4\n\nBenchmark suite executing with the following configuration:\nwarmup: 2 s\ntime: 5 s\nmemory time: 0 ns\nreduction time: 0 ns\nparallel: 8\ninputs: none specified\nEstimated total run time: 28 s\n\nBenchmarking sending messages to all 100k clients ...\nBenchmarking sending messages to only 10k clients ...\nBenchmarking sending messages to only 50k clients ...\nBenchmarking sending messages to only 5k clients ...\n\nName                                           ips        average  deviation         median         99th %\nsending messages to only 5k clients          58.56       17.08 ms     ±8.50%       17.04 ms       20.20 ms\nsending messages to only 50k clients         57.75       17.32 ms    ±10.10%       17.26 ms       21.96 ms\nsending messages to only 10k clients         56.91       17.57 ms    ±17.55%       17.37 ms       26.96 ms\nsending messages to all 100k clients         10.98       91.09 ms     ±9.96%       90.61 ms      112.20 ms\n\nComparison:\nsending messages to only 5k clients          58.56\nsending messages to only 50k clients         57.75 - 1.01x slower +0.24 ms\nsending messages to only 10k clients         56.91 - 1.03x slower +0.50 ms\nsending messages to all 100k clients         10.98 - 5.33x slower +74.02 ms\ngit:(mistress) 10 | ▶\n   \u003c/code\u003e\u003c/pre\u003e\n\u003c/details\u003e\n\n## Key points\n\n- It works! :D\n- Performance scaling is about one order of magnitude of time per order of\n  magnitude of number of clients. For example, if 10 pids takes 10ms, 100 pids\n  takes 100ms, 1000 pids takes 1000ms, etc.\n- Performace is great at low numbers of pids, and is acceptable even when\n  pushing messages to up to 50k out of 100k pids.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fqueer%2Femit","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fqueer%2Femit","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fqueer%2Femit/lists"}