{"id":13631689,"url":"https://github.com/discourse/message_bus","last_synced_at":"2025-05-11T03:50:25.771Z","repository":{"id":8471575,"uuid":"10071490","full_name":"discourse/message_bus","owner":"discourse","description":"A reliable and robust messaging bus for Ruby and Rack","archived":false,"fork":false,"pushed_at":"2025-03-26T08:53:13.000Z","size":1820,"stargazers_count":1656,"open_issues_count":21,"forks_count":135,"subscribers_count":54,"default_branch":"main","last_synced_at":"2025-05-08T17:17:13.498Z","etag":null,"topics":["rubygem"],"latest_commit_sha":null,"homepage":"","language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/discourse.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2013-05-15T05:19:31.000Z","updated_at":"2025-04-29T09:08:31.000Z","dependencies_parsed_at":"2024-05-02T02:36:22.322Z","dependency_job_id":"5de95375-8081-4d7b-bc2e-249ae178a2bd","html_url":"https://github.com/discourse/message_bus","commit_stats":{"total_commits":566,"total_committers":53,"mean_commits":"10.679245283018869","dds":0.568904593639576,"last_synced_commit":"7c7ecee0ea7b6a81fc8b280ef5fd728b0cbe4c40"},"previous_names":["samsaffron/message_bus"],"tags_count":80,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/discourse%2Fmessage_bus","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/discourse%2Fmessage_bus/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/discourse%2Fmessage_bus/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/discourse%2Fmessage_bus/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/discourse","download_url":"https://codeload.github.com/discourse/message_bus/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253514567,"owners_count":21920334,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["rubygem"],"created_at":"2024-08-01T22:02:34.497Z","updated_at":"2025-05-11T03:50:25.752Z","avatar_url":"https://github.com/discourse.png","language":"Ruby","readme":"# MessageBus\n\nA reliable, robust messaging bus for Ruby processes and web clients.\n\nMessageBus implements a Server to Server channel based protocol and Server to Web Client protocol (using polling, long-polling or long-polling + streaming)\n\nSince long-polling is implemented using Rack Hijack and Thin::Async, all common Ruby web servers (Thin, Puma, Unicorn, Passenger) can run MessageBus and handle a large number of concurrent connections that wait on messages.\n\nMessageBus is implemented as Rack middleware and can be used by any Rails / Sinatra or pure Rack application.\n\nRead the generated docs: \u003chttps://www.rubydoc.info/gems/message_bus\u003e\n\n## Ruby version support\n\nMessageBus only support officially supported versions of Ruby; as of [2025-03-14](https://www.ruby-lang.org/en/downloads/branches/) this means we only support Ruby version 3.2 and up.\n\n## Can you handle concurrent requests?\n\n**Yes**, MessageBus uses Rack Hijack and this interface allows us to take control of the underlying socket. MessageBus can handle thousands of concurrent long polls on all popular Ruby webservers. MessageBus runs as middleware in your Rack (or by extension Rails) application and does not require a dedicated server. Background work is minimized to ensure it does not interfere with existing non-MessageBus traffic.\n\n## Is this used in production at scale?\n\n**Yes**, MessageBus was extracted out of [Discourse](http://www.discourse.org/) and is used in thousands of production Discourse sites at scale.\n\n## Installation\n\nAdd this line to your application's Gemfile:\n\n```ruby\ngem 'message_bus'\n```\n\nAnd then execute:\n\n```shell\n$ bundle\n```\n\nOr install it yourself as:\n\n```shell\n$ gem install message_bus\n```\n\n## Usage\n\nServer to Server messaging\n\n```ruby\nmessage_id = MessageBus.publish \"/channel\", \"message\"\n\n# in another process / spot\n\nMessageBus.subscribe \"/channel\" do |msg|\n  # block called in a background thread when message is received\nend\n\n# subscribe to channel and receive the entire backlog\nMessageBus.subscribe \"/channel\", 0 do |msg|\n  # block called in a background thread when message is received\nend\n\n# subscribe to channel and receive the backlog starting at message 6\nMessageBus.subscribe \"/channel\", 5 do |msg|\n  # block called in a background thread when message is received\nend\n```\n\n```ruby\n# get the ID of the last message on a channel\nid = MessageBus.last_id(\"/channel\")\n\n# returns all messages after some id\nMessageBus.backlog \"/channel\", id\n```\n\n### Targeted messages\n\nMessages can be targeted to particular clients by supplying the `client_ids` option when publishing a message.\n\n```ruby\nMessageBus.publish \"/channel\", \"hello\", client_ids: [\"XXX\", \"YYY\"] # (using MessageBus.clientId)\n```\n\nBy configuring the `user_id_lookup` and `group_ids_lookup` options with a Proc or Lambda which will be called with a [Rack specification environment](https://github.com/rack/rack/blob/master/SPEC.rdoc#the-environment-), messages can be targeted to particular clients users or groups by supplying either the `user_ids` or `group_ids` options when publishing a message.\n\n```ruby\nMessageBus.configure(user_id_lookup: proc do |env|\n  # this lookup occurs on JS-client polling, so that server can retrieve backlog\n  # for the client considering/matching/filtering user_ids set on published messages\n  # if user_id is not set on publish time, any user_id returned here will receive the message\n  # return the user id here\nend)\n\n# Target user_ids when publishing a message\nMessageBus.publish \"/channel\", \"hello\", user_ids: [1, 2, 3]\n\nMessageBus.configure(group_ids_lookup: proc do |env|\n  # return the group ids the user belongs to\n  # can be nil or []\nend)\n\n# Target group_ids when publishing a message\nMessageBus.publish \"/channel\", \"hello\", group_ids: [1, 2, 3]\n\n# example of MessageBus to set user_ids from an initializer in Rails and Devise:\n# config/initializers/message_bus.rb\nMessageBus.user_id_lookup do |env|\n  req = Rack::Request.new(env)\n\n  if req.session \u0026\u0026 req.session[\"warden.user.user.key\"] \u0026\u0026 req.session[\"warden.user.user.key\"][0][0]\n    user = User.find(req.session[\"warden.user.user.key\"][0][0])\n    user.id\n  end\nend\n```\n\nIf both `user_ids` and `group_ids` options are supplied when publishing a message, the message will be targeted at clients with lookup return values that matches on either the `user_ids` **or** the `group_ids` options.\n\n```ruby\nMessageBus.publish \"/channel\", \"hello\", user_ids: [1, 2, 3], group_ids: [1, 2, 3]\n```\n\nIf the `client_ids` option is supplied with either the `user_ids` or `group_ids` options when publishing a message, the `client_ids` option will be applied unconditionally and messages will be filtered further using `user_id` or `group_id` clauses.\n\n```ruby\nMessageBus.publish \"/channel\", \"hello\", client_ids: [\"XXX\", \"YYY\"], user_ids: [1, 2, 3], group_ids: [1, 2, 3]\n```\n\nPassing `nil` or `[]` to either `client_ids`, `user_ids` or `group_ids` is equivalent to allowing all values on each option.\n\n### Filtering Client Messages\n\nCustom client message filters can be registered via `MessageBus#register_client_message_filter`. This can be useful for filtering away messages from the client based on the message's payload.\n\nFor example, ensuring that only messages seen by the server in the last 20 seconds are published to the client:\n\n```ruby\nMessageBus.register_client_message_filter('/test') do |message|\n  (Time.now.to_i - message.data[:published_at]) \u003c= 20\nend\n\nMessageBus.publish('/test/5', { data: \"somedata\", published_at: Time.now.to_i })\n```\n\n### Error handling\n\n```ruby\nMessageBus.configure(on_middleware_error: proc do |env, e|\n   # If you wish to add special handling based on error\n   # return a rack result array: [status, headers, body]\n   # If you just want to pass it on return nil\nend)\n```\n\n#### Disabling message_bus\n\nIn certain cases, it is undesirable for message_bus to start up on application start, for example in a Rails application during the `db:create` rake task when using the Postgres backend (which will error trying to connect to the non-existent database to subscribe). You can invoke `MessageBus.off` before the middleware chain is loaded in order to prevent subscriptions and publications from happening; in a Rails app you might do this in an initializer based on some environment variable or some other conditional means. If you want to just disable subscribing to the bus but want to continue to allow publications to be made, you can do `MessageBus.off(disable_publish: false)`.\n\n### Debugging\n\nWhen setting up MessageBus, it's useful to manually inspect channels before integrating a client application.\n\nYou can `curl` MessageBus; this is helpful when trying to debug what may be going wrong. This example uses https://chat.samsaffron.com:\n\n```\ncurl -H \"Content-Type: application/x-www-form-urlencoded\" -X POST --data \"/message=0\" https://chat.samsaffron.com/message-bus/client-id/poll\\?dlp\\=t\n```\n\nYou should see a reply with the messages of that channel you requested (in this case `/message`) starting at the message ID you requested (`0`). The URL parameter `dlp=t` disables long-polling: we do not want this request to stay open.\n\n### Transport\n\nMessageBus ships with 3 transport mechanisms.\n\n1. Long Polling with chunked encoding (streaming)\n2. Long Polling\n3. Polling\n\nLong Polling with chunked encoding allows a single connection to stream multiple messages to a client, and this requires HTTP/1.1.\n\nChunked encoding provides all the benefits of [EventSource](https://developer.mozilla.org/en-US/docs/Web/API/EventSource) with greater browser support (as it works on IE10 and up as well)\n\nTo setup NGINX to proxy to your app correctly be sure to enable HTTP1.1 and disable buffering:\n\n```\nlocation /message-bus/ {\n  ...\n  proxy_http_version 1.1;\n  proxy_buffering off;\n  ...\n}\n```\n\n**NOTE**: do not set proxy_buffering off globally, it may have unintended consequences.\n\nIn order to disable chunked encoding for a specific client in Javascript:\n\n```javascript\nMessageBus.enableChunkedEncoding = false;\n```\n\nor as a server-side policy in Ruby for all clients:\n\n```ruby\nMessageBus.configure(chunked_encoding_enabled: false)\n```\n\nLong Polling requires no special setup; as soon as new data arrives on the channel the server delivers the data and closes the connection.\n\nPolling also requires no special setup; MessageBus will fallback to polling after a tab becomes inactive and remains inactive for a period.\n\n### Multisite support\n\nMessageBus can be used in an environment that hosts multiple sites by multiplexing channels. To use this mode:\n\n```ruby\n# define a site_id lookup method, which is executed\n# when `MessageBus.publish` is called\nMessageBus.configure(site_id_lookup: proc do\n  some_method_that_returns_site_id_string\nend)\n\n# you may post messages just to this site\nMessageBus.publish \"/channel\", \"some message\"\n\n# you can also choose to pass the `:site_id`.\n# This takes precedence over whatever `site_id_lookup`\n# returns\nMessageBus.publish \"/channel\", \"some message\", site_id: \"site-id\"\n\n# you may publish messages to ALL sites using the /global/ prefix\nMessageBus.publish \"/global/channel\", \"will go to all sites\"\n```\n\n### Client support\n\n#### JavaScript Client\n\nMessageBus ships a simple ~300 line JavaScript library which provides an API to interact with the server.\n\nJavaScript clients can listen on any channel and receive messages via polling or long polling. You may simply include the source file (located in `assets/` within the message_bus source code):\n\n```html\n\u003cscript src=\"message-bus.js\" type=\"text/javascript\"\u003e\u003c/script\u003e\n```\n\nor when used in a Rails application, import it through the asset pipeline:\n\n```javascript\n//= require message-bus\n```\n\nIn your application Javascript, you can then subscribe to particular channels and define callback functions to be executed when messages are received:\n\n```javascript\nMessageBus.start(); // call once at startup\n\n// how often do you want the callback to fire in ms\nMessageBus.callbackInterval = 500;\n\n// you will get all new messages sent to channel\nMessageBus.subscribe(\"/channel\", function (data) {\n  // data shipped from server\n});\n\n// you will get all new messages sent to channel (-1 is implicit)\nMessageBus.subscribe(\"/channel\", function(data){\n  // data shipped from server\n}, -1);\n\n// all messages AFTER message id 7 AND all new messages\nMessageBus.subscribe(\"/channel\", function(data){\n  // data shipped from server\n}, 7);\n\n// last 2 messages in channel AND all new messages\nMessageBus.subscribe(\"/channel\", function(data){\n  // data shipped from server\n}, -3);\n\n// you will get the entire backlog\nMessageBus.subscribe(\"/channel\", function(data){\n  // data shipped from server\n}, 0);\n```\n\n#### JavaScript Client settings\n\nAll client settings are settable via `MessageBus.OPTION`\n\nSetting|Default|Info\n----|---|---|\nenableLongPolling|true|Allow long-polling (provided it is enabled by the server)\ncallbackInterval|15000|Safeguard to ensure background polling does not exceed this interval (in milliseconds)\nbackgroundCallbackInterval|60000|Interval to poll when long polling is disabled (either explicitly or due to browser being in background)\nminPollInterval|100|When polling requests succeed, this is the minimum amount of time to wait before making the next request.\nmaxPollInterval|180000|If request to the server start failing, MessageBus will backoff, this is the upper limit of the backoff.\nalwaysLongPoll|false|For debugging you may want to disable the \"is browser in background\" check and always long-poll\nshouldLongPollCallback|undefined|A callback returning true or false that determines if we should long-poll or not, if unset ignore and simply depend on window visibility.\nbaseUrl|/|If message bus is mounted at a sub-path or different domain, you may configure it to perform requests there.  See `MessageBus.base_route=` on how to configure the MessageBus server to listen on a sub-path.\najax|$.ajax falling back to XMLHttpRequest|MessageBus will first attempt to use jQuery and then fallback to a plain XMLHttpRequest version that's contained in the `message-bus-ajax.js` file. `message-bus-ajax.js` must be loaded after `message-bus.js` for it to be used. You may override this option with a function that implements an ajax request by some other means\nheaders|{}|Extra headers to be include with requests. Properties and values of object must be valid values for HTTP Headers, i.e. no spaces or control characters.\nminHiddenPollInterval|1500|Time to wait between poll requests performed by background or hidden tabs and windows, shared state via localStorage\nenableChunkedEncoding|true|Allows streaming of message bus data over the HTTP connection without closing the connection after each message.\n\n#### Javascript Client API\n\n`MessageBus.start()` : Starts up the MessageBus poller\n\n`MessageBus.subscribe(channel,func,lastId)` : Subscribes to a channel. You may optionally specify the id of the last message you received in the channel. The callback receives three arguments on message delivery: `func(payload, globalId, messageId)`. You may save globalId or messageId of received messages and use then at a later time when client needs to subscribe, receiving the backlog since that id.\n\n`MessageBus.unsubscribe(channel,func)` : Removes a subscription from a particular channel that was defined with a particular callback function (optional).\n\n`MessageBus.pause()` : Pauses all MessageBus activity\n\n`MessageBus.resume()` : Resumes MessageBus activity\n\n`MessageBus.stop()` : Stops all MessageBus activity\n\n`MessageBus.status()` : Returns status (started, paused, stopped)\n\n`MessageBus.diagnostics()` : Returns a log that may be used for diagnostics on the status of message bus.\n\n#### Ruby\n\nThe gem ships with a Ruby implementation of the client library available with an\nAPI very similar to that of the JavaScript client. It was inspired by\nhttps://github.com/lowjoel/message_bus-client.\n\n```ruby\n# Creates a client with the default configuration\nclient = MessageBus::HTTPClient.new('http://some.test.com')\n\n# Listen for the latest messages\nclient.subscribe(\"/channel\") { |data| puts data }\n\n# Listen for all messages after id 7\nclient.subscribe(\"/channel\", last_message_id: 7) { |data| puts data }\n\n# Listen for last message and all new messages\nclient.subscribe(\"/channel\", last_message_id: -2) { |data| puts data }\n\n# Unsubscribe from a channel\nclient.unsubscribe(\"/channel\")\n\n# Unsubscribe a particular callback from a channel\ncallback = -\u003e { |data| puts data }\nclient.subscribe(\"/channel\", \u0026callback)\nclient.unsubscribe(\"/channel\", \u0026callback)\n```\n\n#### Ruby Client Settings\n\nSetting|Default|Info\n----|---|---|\nenable_long_polling|true|Allow long-polling (provided it is enabled by the server)\nbackground_callback_interval|60s|Interval to poll when long polling is disabled\nmin_poll_interval|0.1s|When polling requests succeed, this is the minimum amount of time to wait before making the next request.\nmax_poll_interval|180s|If request to the server start failing, MessageBus will backoff, this is the upper limit of the backoff.\nenable_chunked_encoding|true|Allows streaming of message bus data over the HTTP connection without closing the connection after each message.\nheaders|{}|Extra headers to be include with requests. Properties and values of object must be valid values for HTTP Headers, i.e. no spaces or control characters.\n\n\n## Configuration\n\nmessage_bus can be configured to use one of several available storage backends, and each has its own configuration options.\n\n### Keepalive\n\nTo ensure correct operation of message_bus, every 60 seconds a message is broadcast to itself. If for any reason the message is not consumed by the same process within 3 keepalive intervals a warning log message is raised.\n\nTo control keepalive interval use\n\n```ruby\nMessageBus.configure(keepalive_interval: 60)\n```\n\n### Redis\n\nmessage_bus supports using Redis as a storage backend, and you can configure message_bus to use redis in `config/initializers/message_bus.rb`, like so:\n\n```ruby\nMessageBus.configure(backend: :redis, redis_config: { \n  url: \"redis://:p4ssw0rd@10.0.1.1:6380/15\"\n})\n```\n\nThe redis client message_bus uses is [redis-rb](https://github.com/redis/redis-rb), so you can visit it's repo to see what other options you can pass besides a `url`.\n\n#### Data Retention\n\nOut of the box Redis keeps track of 2000 messages in the global backlog and 1000 messages in a per-channel backlog. Per-channel backlogs get\ncleared automatically after 7 days of inactivity. By default, the backlog will be pruned on every message publication. If exact backlog\nlength limiting is not required, the `clear_every` parameter can be set higher to improve performance.\n\nThis is configurable via accessors on the Backend instance.\n\n```ruby\n# only store 100 messages per channel\nMessageBus.backend_instance.max_backlog_size = 100\n\n# only store 100 global messages\nMessageBus.backend_instance.max_global_backlog_size = 100\n\n# flush per-channel backlog after 100 seconds of inactivity\nMessageBus.backend_instance.max_backlog_age = 100\n\n# clear the backlog every 50 messages\nMessageBus.backend_instance.clear_every = 50\n```\n\n### PostgreSQL\n\nmessage_bus also supports PostgreSQL as a backend, and can be configured like so:\n\n```ruby\nMessageBus.configure(backend: :postgres, backend_options: {user: 'message_bus', dbname: 'message_bus'})\n```\n\nThe PostgreSQL client message_bus uses is [ruby-pg](https://github.com/ged/ruby-pg), so you can visit it's repo to see what options you can include in `:backend_options`.\n\nA `:clear_every` option is also supported, which limits backlog trimming frequency to the specified number of publications. If you set `clear_every: 100`, the backlog will only be cleared every 100 publications. This can improve performance in cases where exact backlog length limiting is not required.\n\n### Memory\n\nmessage_bus also supports an in-memory backend. This can be used for testing or simple single-process environments that do not require persistence or horizontal scalability.\n\n```ruby\nMessageBus.configure(backend: :memory)\n```\n\nThe `:clear_every` option is supported in the same way as the PostgreSQL backend.\n\n### Transport codecs\n\nBy default MessageBus serializes messages to the backend using JSON. Under most situation this performs extremely well.\n\nIn some exceptional cases you may consider a different transport codec. To configure a custom codec use:\n\n```ruby\nMessageBus.configure(transport_codec: codec)\n```\n\nA codec class must implement MessageBus::Codec::Base. Specifically an `encode` and `decode` method.\n\nSee the `bench` directory for examples where the default JSON codec can perform poorly. A specific examples may be\nattempting to distribute a message to a restricted list of thousands of users. In cases like this you may consider\nusing a packed string encoder.\n\nKeep in mind, much of MessageBus internals and supporting tools expect data to be converted to JSON and back, if you use a naive (and fast) `Marshal` based codec you may need to limit the features you use. Specifically the Postgresql backend expects the codec never to return a string with `\\u0000`, additionally some classes like DistributedCache expect keys to be converted to Strings.\n\nAnother example may be very large and complicated messages where Oj in compatibility mode outperforms JSON. To opt for the Oj codec use:\n\n```ruby\nMessageBus.configure(transport_codec: MessageBus::Codec::Oj.new)\n```\n\n### Forking/threading app servers\n\nIf you're using a forking or threading app server and you're not getting immediate delivery of published messages, you might need to configure your web server to re-connect to the message_bus backend\n\n#### Passenger\n\n```ruby\n# Rails: config/application.rb or config.ru\nif defined?(PhusionPassenger)\n  PhusionPassenger.on_event(:starting_worker_process) do |forked|\n    if forked\n      # We're in smart spawning mode.\n      MessageBus.after_fork\n    else\n      # We're in conservative spawning mode. We don't need to do anything.\n    end\n  end\nend\n```\n\nMessageBus uses long polling which needs to be configured in Passenger\n\nFor passenger version \u003c 5.0.21, add the following to `application.rb`:\n\n```ruby\nPhusionPassenger.advertised_concurrency_level = 0\n```\n\nFor passenger version \u003e 5.0.21, add the following to `nginx.conf`:\n\n```\nlocation /message-bus {\n  passenger_app_group_name foo_websocket;\n  passenger_force_max_concurrent_requests_per_process 0;\n}\n```\n\nFor more information see the [Passenger documentation](https://www.phusionpassenger.com/library/config/nginx/tuning_sse_and_websockets/) on long-polling.\n\n#### Puma\n\n```ruby\n# path/to/your/config/puma.rb\non_worker_boot do\n  MessageBus.after_fork\nend\n```\n\n#### Unicorn\n\n```ruby\n# path/to/your/config/unicorn.rb\nafter_fork do |server, worker|\n  MessageBus.after_fork\nend\n```\n\n### Middleware stack in Rails\n\nMessageBus middleware has to show up after the session middleware, but depending on how the Rails app is configured that might be either `ActionDispatch::Session::CookieStore` or `ActionDispatch::Session::ActiveRecordStore`. To handle both cases, the middleware is inserted before `ActionDispatch::Flash`.\n\nFor APIs or apps that have `ActionDispatch::Flash` deleted from the stack the middleware is pushed to the bottom.\n\nShould you wish to manipulate the default behavior please refer to [Rails MiddlewareStackProxy documentation](http://api.rubyonrails.org/classes/Rails/Configuration/MiddlewareStackProxy.html) and alter the order of the middlewares in stack in `app/config/initializers/message_bus.rb`\n\n```ruby\n# config/initializers/message_bus.rb\nRails.application.config do |config|\n  # do anything you wish with config.middleware here\nend\n```\n\nSpecifically, if you use a Rack middleware-based authentication solution (such as Warden) in a Rails application and wish to use it for authenticating message_bus requests, you must ensure that the MessageBus middleware comes after it in the stack.\n\n```ruby\n# config/initializers/message_bus.rb\nRails.application.config.middleware.move_after(Warden::Manager, MessageBus::Rack::Middleware)\n```\n\n### A Distributed Cache\n\nMessageBus ships with an optional `DistributedCache` API which provides a simple and efficient way of synchronizing a cache between processes, based on the core of message_bus:\n\n```ruby\nrequire 'message_bus/distributed_cache'\n\n# process 1\ncache = MessageBus::DistributedCache.new(\"animals\")\n\n# process 2\ncache = MessageBus::DistributedCache.new(\"animals\")\n\n# process 1\ncache[\"frogs\"] = 5\n\n# process 2\nputs cache[\"frogs\"]\n# =\u003e 5\n\ncache[\"frogs\"] = nil\n\n# process 1\nputs cache[\"frogs\"]\n# =\u003e nil\n```\n\nYou can automatically expire the cache on application code changes by scoping the cache to a specific version of the application:\n\n```ruby\ncache = MessageBus::DistributedCache.new(\"cache name\", app_version: \"12.1.7.ABDEB\")\ncache[\"a\"] = 77\n\ncache = MessageBus::DistributedCache.new(\"cache name\", app_version: \"12.1.7.ABDEF\")\n\nputs cache[\"a\"]\n# =\u003e nil\n```\n\n#### Error Handling\n\nThe internet is a chaotic environment and clients can drop off for a variety of reasons. If this happens while message_bus is trying to write a message to the client you may see something like this in your logs:\n\n```\nErrno::EPIPE: Broken pipe\n  from message_bus/client.rb:159:in `write'\n  from message_bus/client.rb:159:in `write_headers'\n  from message_bus/client.rb:178:in `write_chunk'\n  from message_bus/client.rb:49:in `ensure_first_chunk_sent'\n  from message_bus/rack/middleware.rb:150:in `block in call'\n  from message_bus/client.rb:21:in `block in synchronize'\n  from message_bus/client.rb:21:in `synchronize'\n  from message_bus/client.rb:21:in `synchronize'\n  from message_bus/rack/middleware.rb:147:in `call'\n  ...\n```\n\nThe user doesn't see anything, but depending on your traffic you may acquire quite a few of these in your logs or exception tracking tool.\n\nYou can rescue from errors that occur in MessageBus's middleware stack by adding a config option:\n\n```ruby\nMessageBus.configure(on_middleware_error: proc do |env, e|\n  # env contains the Rack environment at the time of error\n  # e contains the exception that was raised\n  if Errno::EPIPE === e\n    [422, {}, [\"\"]]\n  else\n    raise e\n  end\nend)\n```\n\n### Adding extra response headers\n\nIn e.g. `config/initializers/message_bus.rb`:\n\n```ruby\nMessageBus.extra_response_headers_lookup do |env|\n  [\n    [\"Access-Control-Allow-Origin\", \"http://example.com:3000\"],\n  ]\nend\n```\n\n## How it works\n\nMessageBus provides durable messaging following the publish-subscribe (pubsub) pattern to subscribers who track their own subscriptions. Durability is by virtue of the persistence of messages in backlogs stored in the selected backend implementation (Redis, Postgres, etc) which can be queried up until a configurable expiry. Subscribers must keep track of the ID of the last message they processed, and request only more-recent messages in subsequent connections.\n\nThe MessageBus implementation consists of several key parts:\n\n* Backend implementations - these provide a consistent API over a variety of options for persisting published messages. The API they present is around the publication to and reading of messages from those backlogs in a manner consistent with message_bus' philosophy. Each of these inherits from `MessageBus::Backends::Base` and implements the interface it documents.\n* `MessageBus::Rack::Middleware` - which accepts requests from subscribers, validates and authenticates them, delivers existing messages from the backlog and informs a `MessageBus::ConnectionManager` of a connection which is remaining open.\n* `MessageBus::ConnectionManager` - manages a set of subscribers with active connections to the server, such that messages which are published during the connection may be dispatched.\n* `MessageBus::Client` - represents a connected subscriber and delivers published messages over its connected socket.\n* `MessageBus::Message` - represents a published message and its encoding for persistence.\n\nThe public API is all defined on the `MessageBus` module itself.\n\n### Subscriber protocol\n\nThe message_bus protocol for subscribing clients is based on HTTP, optionally with long-polling and chunked encoding, as specified by the HTTP/1.1 spec in RFC7230 and RFC7231.\n\nThe protocol consists of a single HTTP end-point at `/message-bus/[client_id]/poll`, which responds to `POST` and `OPTIONS`. In the course of a `POST` request, the client must indicate the channels from which messages are desired, along with the last message ID the client received for each channel, and an incrementing integer sequence number for each request (used to detect out of order requests and close those with the same client ID and lower sequence numbers).\n\nClients' specification of requested channels can be submitted in either JSON format (with a `Content-Type` of `application/json`) or as HTML form data (using `application/x-www-form-urlencoded`). An example request might look like:\n\n```\nPOST /message-bus/3314c3f12b1e45b4b1fdf1a6e42ba826/poll HTTP/1.1\nHost: foo.com\nContent-Type: application/json\nContent-Length: 37\n\n{\"/foo/bar\":3,\"/doo/dah\":0,\"__seq\":7}\n```\n\nIf there are messages more recent than the client-specified IDs in any of the requested channels, those messages will be immediately delivered to the client. If the server is configured for long-polling, the client has not requested to disable it (by specifying the `dlp=t` query parameter), and no new messages are available, the connection will remain open for the configured long-polling interval (25 seconds by default); if a message becomes available in that time, it will be delivered, else the connection will close. If chunked encoding is enabled, message delivery will not automatically end the connection, and messages will be continuously delivered during the life of the connection, separated by `\"\\r\\n|\\r\\n\"`.\n\nThe format for delivered messages is a JSON array of message objects like so:\n\n```json\n[\n  {\n    \"global_id\": 12,\n    \"message_id\": 1,\n    \"channel\": \"/some/channel/name\",\n    \"data\": [the message as published]\n  }\n]\n```\n\nThe `global_id` field here indicates the ID of the message in the global backlog, while the `message_id` is the ID of the message in the channel-specific backlog. The ID used for subscriptions is always the channel-specific one.\n\nIn certain conditions, a status message will be delivered and look like this:\n\n```json\n{\n  \"global_id\": -1,\n  \"message_id\": -1,\n  \"channel\": \"/__status\",\n  \"data\": {\n    \"/some/channel\": 5,\n    \"/other/channel\": 9\n  }\n}\n```\n\nThis message indicates the last ID in the backlog for each channel that the client subscribed to. It is sent in the following circumstances:\n\n* When the client subscribes to a channel starting from `-1`. When long-polling, this message will be delivered immediately.\n* When the client subscribes to a channel starting from a message ID that is beyond the last message on that channel.\n* When delivery of messages to a client is skipped because the message is filtered to other users/groups.\n\nThe values provided in this status message can be used by the client to skip requesting messages it will never receive and move forward in polling.\n\n### Publishing to MessageBus from outside of MessageBus\n\nIt may be necessary or desired for integration with existing systems to publish messages from outside the Ruby app where MessageBus is running. @tgodfrey has an example of how to do that, using the Redis backend, from Elixir here: https://gist.github.com/tgodfrey/1a67753d51cb202ca8eb04b933cec924.\n\n## Contributing\n\nIf you are looking to contribute to this project here are some ideas\n\n- MAKE THIS README BETTER!\n- Build backends for other providers (zeromq, rabbitmq, disque) - currently we support pg and redis.\n- Improve and properly document admin dashboard (add opt-in stats, better diagnostics into queues)\n- Improve general documentation (Add examples, refine existing examples)\n- Make MessageBus a nice website\n- Add optional transports for websocket and shared web workers\n\nWhen submitting a PR, please be sure to include notes on it in the `Unreleased` section of the changelog, but do not bump the version number.\n\n### Running tests\n\nTo run tests you need both Postgres and Redis installed. By default on Redis the tests connect to `localhost:6379` and on Postgres connect the database `localhost:5432/message_bus_test` with the system username; if you wish to override this, you can set alternative values:\n\n```shell\nPGUSER=some_user PGDATABASE=some_db bundle exec rake\n```\n\nWe include a Docker Compose configuration to run test suite in isolation, or if you do not have Redis or Postgres installed natively. To execute it, do `docker-compose run tests`.\n\n### Generating the documentation\n\nRun `rake yard` (or `docker-compose run docs rake yard`) in order to generate the implementation's API docs in HTML format, and `open doc/index.html` to view them.\n\nWhile working on documentation, it is useful to automatically re-build it as you make changes. You can do `yard server --reload` (or `docker-compose up docs`) and `open http://localhost:8808` to browse live-built docs as you edit them.\n\n### Benchmarks\n\nSome simple benchmarks are implemented in `spec/performance` and can be executed using `rake performance` (or `docker-compose run tests rake performance`). You should run these before and after your changes to avoid introducing performance regressions.\n","funding_links":[],"categories":["Ruby"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdiscourse%2Fmessage_bus","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdiscourse%2Fmessage_bus","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdiscourse%2Fmessage_bus/lists"}