{"id":17056782,"url":"https://github.com/halturin/mx","last_synced_at":"2025-04-12T17:33:57.612Z","repository":{"id":36813254,"uuid":"41120127","full_name":"halturin/mx","owner":"halturin","description":"OTP Message Broker (pub/sub, workers queue, priority/deferring delivery)","archived":false,"fork":false,"pushed_at":"2019-04-28T19:58:24.000Z","size":2221,"stargazers_count":17,"open_issues_count":0,"forks_count":5,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-03-26T11:51:11.603Z","etag":null,"topics":["erlang","otp","otp-message-broker","pubsub-messages","pubsubbroker"],"latest_commit_sha":null,"homepage":"","language":"Erlang","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/halturin.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2015-08-20T21:40:36.000Z","updated_at":"2024-07-10T10:58:51.000Z","dependencies_parsed_at":"2022-07-20T10:32:07.327Z","dependency_job_id":null,"html_url":"https://github.com/halturin/mx","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/halturin%2Fmx","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/halturin%2Fmx/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/halturin%2Fmx/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/halturin%2Fmx/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/halturin","download_url":"https://codeload.github.com/halturin/mx/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248605112,"owners_count":21132112,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["erlang","otp","otp-message-broker","pubsub-messages","pubsubbroker"],"created_at":"2024-10-14T10:25:35.164Z","updated_at":"2025-04-12T17:33:57.586Z","avatar_url":"https://github.com/halturin.png","language":"Erlang","funding_links":[],"categories":[],"sub_categories":[],"readme":"# OTP Message Broker\n\n## Overview\n\nUniversal OTP message broker features:\n* create channels (pub/sub)\n* pools (workers queue)\n* mixing it (pool of channels, channel of pools... etc.)\n* send messages with specify priority of delivering messages (range: 1..10)\n* pool has 3 balance methods: rr(round robin), hash (by erlang:phash(Message, lenth(Pool))), random\n* defer message delivering in case of\n    - exceed the queue limit (10000) and receiver has the 'true' in 'defer' option.\n    - client has 'offline' state and the 'defer' option is set to 'true'\n* 'async' Client option allows you control the delivery process. Default value of this option is 'true'.\n\n## Install\nFor **dev** purposes simple run:\n```\ngit clone https://github.com/halturin/mx \u0026\u0026 cd mx \u0026\u0026 make compile \u0026\u0026 echo \"MX is installed in $(pwd)\"\n```\nFor **production** use:\n```\n% rebar.config -- add mx in deps section\n{deps, [\n       {mx, {git, \"git://github.com/halturin/mx.git\", {branch, \"master\"}}}\n]}.\n%% and add mx to the relx section\n{relx, [{release, {your_rel, \"0.0.1\"},\n         [...,\n          mx]}\n]}.\n```\n\nYou can specifying a **queue limits** in sys.config:\n```\n[\n    {mx, [\n       {queue_length_limit, 100000},\n       {queue_low_threshold, 0.6},   % 60%\n       {queue_high_threshold, 0.8}   % 80%\n    ]}\n]\n```\n\n## Run\n\n```\nmake run\n```\n\n## Distributed mode\n\nRun the first node:\n```erlang\nmake demo_run node_name='mxnode01@127.0.0.1'\n\n(mxnode01@127.0.0.1)1\u003e application:start(mx).\n```\n\nand the second one:\n\n```erlang\nmake demo_run node_name='mxnode02@127.0.0.1'\n\n(mxnode02@127.0.0.1)1\u003e application:load(mx).\n%% Set via environment value of **'master'** to run it in slave mode.\n(mxnode02@127.0.0.1)2\u003e application:set_env(mx, master, 'mxnode01@127.0.0.1').\n(mxnode02@127.0.0.1)3\u003e application:start(mx).\n```\n\nCall **mx:nodes()** to get the list of mx cluster nodes.\n\n```erlang\n(mxnode01@127.0.0.1)2\u003e mx:nodes().\n['mxnode01@127.0.0.1','mxnode02@127.0.0.1']\n```\n\n## Mnesia custom directory (optional)\n\nCreate dir, for example, **/usr/local/var/lib/mx/mnesia/** with correct (Read/Write) permissions.\n\nSet option `mnesia_base_dir` with this **directory** in `sys.config`:\n\n```\n    {mx, [\n       {mnesia_base_dir,      \"/usr/local/var/lib/mx/mnesia/\"}\n    ]},\n```\n\nOr set the value of configuration parameter `mnesia_base_dir` for **mx**:\n\n```erlang\nmake demo_run node_name='mxnode01@127.0.0.1'\n\n(mxnode01@127.0.0.1)1\u003e application:load(mx).\n(mxnode01@127.0.0.1)1\u003e application:set_env(mx, mnesia_base_dir, \"/usr/local/var/lib/mx/mnesia/\").\n(mxnode01@127.0.0.1)1\u003e application:start(mx).\n```\n\nSo, mnesia data will be located:\n```\n/usr/local/var/lib/mx/mnesia/mxnode01@127.0.0.1  %% node name\n```\n\n## Examples\n\n```erlang\n% Client has higest priority by default (priority = 1)\n{clientkey, Client1Key} = mx:register(client, \"Client1\"),\n{clientkey, Client2Key} = mx:register(client, \"Client2\", [{priority, 8}]),\n{clientkey, Client3Key} = mx:register(client, \"Client3\", [{async, false}, {defer, true}]),\n{clientkey, Client4Key} = mx:register(client, \"Client4\"),\n\n% register channel with default priority (5)\n{channelkey, Channel1Key} = mx:register(channel, \"Channel1\", Client4Key),\nok = mx:subscribe(Client1Key, Channel1Key),\n% just for example try to subscribe one more time\n{already_subscribed, Client1Key} = mx:subscribe(Client1Key, Channel1Key),\n\nok = mx:subscribe(Client2Key, Channel1Key),\nok = mx:subscribe(Client3Key, Channel1Key),\n\nmx:send(Channel1Key, \"Hello, Channel1!\").\n\n% register pool with default balance method is 'rr' - round robin\n%             default priority (5)\n{poolkey, Pool1Key} = mx:register(pool, \"Pool1\", Client4Key),\nmx:join(Client1Key, Pool1Key),\nmx:join(Client2Key, Pool1Key),\nmx:join(Client3Key, Pool1Key),\n\n% create lowest priority channel and pool by Client2\n{channelkey, LPCh1} = mx:register(channel, \"LP Channel1\", Client2Key, [{priority, 10}]),\n{poolkey, LPPl1}    = mx:register(pool, \"LP Pool1\", Client2Key, [{priority, 10}]),\n\n% create highest priority channel and pool by Client3\n{channelkey, HPCh1} = mx:register(channel, \"HP Channel1\", Client2Key, [{priority, 1}]),\n{poolkey, HPPl1}    = mx:register(pool, \"HP Pool1\", Client2Key, [{priority, 1}]),\n\n% high priority pool with 'hash' balance\n{poolkey, HP_Hash_Pl}    = mx:register(pool, \"HP Pool (hash)\", Client2Key, [{priority, 1}, {balance, hash}]),\n\n% pool with random balance\n{poolkey, Rand_Pl}    = mx:register(pool, \"Pool (random)\", Client2Key, [{balance, random}]),\n\n```\n\n## API\n\n### local usage\n\n* Create client/channel/pool\n\n    ```erlang\n    mx:register(client, Name)\n    mx:register(client, Name, Opts)\n    ```\n       Name - list or binary\n       Opts - proplists\n\n    returns: `{clientkey, Key} | {duplicate, Key}`\n\n       Key - binary\n\n    ```erlang\n    mx:register(channel, Name, ClientKey)\n    mx:register(channel, Name, ClientKey, Opts)\n    ```\n       Name - list or binary\n       Opts - proplists\n       ClientKey - binary\n\n    returns: `{channelkey, Key} | {duplicate, Key}`\n\n       Key - binary\n\n    ```erlang\n    mx:register(pool, Name, ClientKey)**\n    mx:register(pool, Name, ClientKey, Opts)**\n    ```\n       Name - list or binary\n       Opts - proplists\n       ClientKey - binary\n\n    returns: `{poolkey, Key} | {duplicate, Key}`\n\n       Key - binary\n\n* Delete client/channel/pool\n    ```erlang\n    mx:unregister(Key)\n    ```\n\n* Set online/offline state\n    ```erlang\n    mx:online(ClientKey, Pid)\n    mx:offline(ClientKey)\n    ```\n\n* Work with channel/pool\n    ```erlang\n    mx:subscribe(Key, Channel)\n    mx:unsubscribe(Key, Channel)\n    ```\n       Key - binary (ClientKey, ChannelKey, PoolKey)\n       Channel - channel name or channel key\n\n    ```erlang\n    mx:join(Key, Pool)\n    mx:leave(Key, Pool)\n    ```\n       Key - binary (ClientKey, ChannelKey, PoolKey)\n       Pool - pool name or pool key\n\n* Set options for client/channel/pool\n    ```erlang\n    mx:set(Key, Opts)\n    ```\n       Key - binary (ClientKey, ChannelKey, PoolKey)\n       Opts - proplists\n\n* Sending message\n    ```erlang\n    mx:send(ClientKey, Message)\n    mx:send(ChannelKey, Message)\n    mx:send(PoolKey, Message)\n    ```\n\n* Owning Pool/Channel\n    ```erlang\n    mx:own(Key, ClientKey)\n    ```\n       Key - binary (ChannelKey, PoolKey)\n\n    orphan Pool/Channel will unregister automaticaly\n\n    ```erlang\n    mx:abandon(Key, ClientKey)\n    ```\n       Key - binary (ChannelKey, PoolKey)\n\n\n* Clear deferred messages\n    ```erlang\n    mx:flush(Key)\n    ```\n       Key - binary (ClientKey, ChannelKey, PoolKey)\n       Key = all - truncate the 'deferred' table\n\n* Clear all MX tables\n    ```erlang\n    mx:clear_all_tables()\n    ```\n\n* Info\n\n    show MX cluster nodes\n    ```erlang\n    mx:nodes()\n    ```\n\n    show full information about the client\n    ```erlang\n    mx:info(Key)\n    ```\n       Key - binary (ClientKey, ChannelKey, PoolKey)\n\n    show the only `Name` property from the information list about the client\n    ```erlang\n    mx:info(Key, Name)\n    ```\n       Key - binary (ClientKey, ChannelKey, PoolKey)\n       Name - field name\n\n    show the list of Clients are subscribed/joined to.\n    ```erlang\n    mx:relation(Key)\n    ```\n       Key - binary (ChannelKey, PoolKey)\n\n\n### remote usage\n\n```erlang\ngen_server:call(MX, Message)\n```\nwhere the `Message` is one of the listed values below:\n\n- `{register_client, Client}`\n- `{register_client, Client, Opts}`\n- `{register_channel, ChannelName, ClientKey}`\n- `{register_channel, ChannelName, ClientKey, Opts}`\n- `{register_pool, PoolName, ClientKey}`\n- `{register_pool, PoolName, ClientKey, Opts}`\n- `{unregister, Key}`\n- `{online, ClientKey, Pid}`\n- `{offline, ClientKey}`\n- `{subscribe, Client, To}`\n- `{unsubscribe, Client, From}`\n- `{join, Client, To}`\n- `{leave, Client, From}`\n- `{send, To, Message}`\n- `{own, Key, ClientKey}`\n- `{abandon, Key, ClientKey}`\n- `{info, Key}`\n- `{info, {Key, Name}}`\n- `{relation, Key}`\n- `{set, Key, Opts}`\n- `nodes`\n- `clear_all_tables`\n\n```erlang\n\u003e (mxnode02@127.0.0.1)2\u003e gen_server:call({mx, 'mxnode01@127.0.0.1'}, nodes).\n['mxnode02@127.0.0.1','mxnode01@127.0.0.1']\n```\n\n## Testing\n\nThere are only common tests (CT) are implemented with some limited set of cases\n\n### Direct sending tests.\nSequentualy running:\n\n  * [x] 1 reciever - 1 message\n  * [x] 1 reciever - 1000 messages\n  * [x] 1000 recievers - 1 messages\n  * [x] 1000 recievers - 100 messages\n\nParallel:\n\n  * [x] 1 reciever - 1000 messages (3 processes)\n  * [x] 1000 recievers - 1 messages (3 processes)\n\n\n### Channel tests (pub/sub).\n\nSequentualy running:\n\n  * [x] 1 subscriber (subscriber) recieves 1 messages\n  * [x] 1 subscriber - 1000 messages\n  * [x] 1000 subscribers - 1 messages\n  * [x] 1000 subscribers - 10 messages\n\nParallel:\n\n  * [ ] 1000 subscribers - 10 messages (10 processes)\n\nPriority delivering:\n  * 1 subscriber recieves 1000 messages with 10 different priorities:\n    * [ ] 100 messages with priority 1\n    * [ ] 100 messages with priority 2\n    * [ ] ...\n    * [ ] 100 messages with priority 10\n\n\n### Pool tests (worker queue)\n\nSequentualy running:\n\n  * [ ] 1 client sends 1 messages to 1 worker\n  * [ ] 1 client - 1000 messages - 2 workers (round robin)\n  * [ ] 1 client - 1000 messages - 2 workers (hash)\n  * [ ] 1 client - 1000 messages - 2 workers (random)\n  * [ ] 1000 clients - 1 messages - 4 workers\n  * [ ] 1000 clients - 1000 messages - 4 workers\n\nParallel: _(not implemented)_\n  * [ ] 1000 clients - 10 messages (10 processes)\n\n### Run the testing\n1. Run MX application as standalone application\n\n```shell\n$ make run\n```\n\n2. Run \"Common Tests\"\n\n```shell\n$ make ct\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhalturin%2Fmx","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhalturin%2Fmx","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhalturin%2Fmx/lists"}