{"id":21192173,"url":"https://github.com/mu-semtech/mu-dispatcher","last_synced_at":"2025-08-22T18:14:37.748Z","repository":{"id":30244652,"uuid":"33795862","full_name":"mu-semtech/mu-dispatcher","owner":"mu-semtech","description":"Core microservice for dispatching requests to the preferred microservice","archived":false,"fork":false,"pushed_at":"2025-07-09T08:21:33.000Z","size":76,"stargazers_count":5,"open_issues_count":8,"forks_count":9,"subscribers_count":5,"default_branch":"master","last_synced_at":"2025-07-09T09:33:06.404Z","etag":null,"topics":["dispatcher","elixir","mu-project","musemtech"],"latest_commit_sha":null,"homepage":null,"language":"Elixir","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mu-semtech.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2015-04-11T23:08:53.000Z","updated_at":"2025-07-09T08:21:37.000Z","dependencies_parsed_at":"2025-07-09T09:28:03.514Z","dependency_job_id":"abc5a774-5621-4147-abf2-df77434cd43d","html_url":"https://github.com/mu-semtech/mu-dispatcher","commit_stats":null,"previous_names":[],"tags_count":10,"template":false,"template_full_name":null,"purl":"pkg:github/mu-semtech/mu-dispatcher","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mu-semtech%2Fmu-dispatcher","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mu-semtech%2Fmu-dispatcher/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mu-semtech%2Fmu-dispatcher/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mu-semtech%2Fmu-dispatcher/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mu-semtech","download_url":"https://codeload.github.com/mu-semtech/mu-dispatcher/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mu-semtech%2Fmu-dispatcher/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":271680370,"owners_count":24802074,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-22T02:00:08.480Z","response_time":65,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["dispatcher","elixir","mu-project","musemtech"],"created_at":"2024-11-20T19:07:48.752Z","updated_at":"2025-08-22T18:14:37.725Z","avatar_url":"https://github.com/mu-semtech.png","language":"Elixir","funding_links":[],"categories":[],"sub_categories":[],"readme":"# mu-dispatcher\n\nCore microservice for dispatching requests to the preferred microservice.\n\nThe mu-dispatcher is one of the core elements in the mu.semte.ch architecture.  This service will dispatch requests to other microservices based on the incoming request path.  You can run the service through docker, but you probably want to configure it using [mu-project](http://github.com/mu-semtech/mu-project) so it uses your own configuration.\n\nThe Dispatcher runs as an application in which the `Dispatcher` module is overridden.  It expects a dispatcher.ex file to exist on `/config/dispatcher.ex` when the dispatcher boots up.\n\n# Table of Contents\n1. [Configuration](#Configuration)\n2. [Supported API](#Supported-API)\n    1. [Matcher](#Use-Matcher)\n    2. [Http Verbs](#Http-verbs)\n    3. [define_accept_types](#define_accept_types)\n3. [forwarding requests](#Forwarding-requests)\n    1. [Basic forwarding](#Basic-forwarding)\n    2. [Forwarding paths](#Forwarding-paths)\n    3. [Matching on verb](#Matching-on-verb)\n    4. [Matching on host](#Matching-on-host)\n    5. [Matching Accept headers](#Matching-Accept-headers)\n    6. [Layers](#Layers)\n4. [Fallback routes and 404 pages](#Fallback-routes-and-404-pages)\n5. [Manipulating responses](#Manipulating-responses)\n6. [How-to / Extra information](#Extra-information)\n    1. [Host an EmberJS app](#Host-an-EmberJS-app)\n    2. [External API CORS headers](#External-API-CORS-headers)\n    3. [Provide 404 pages](#Provide-404-pages)\n7. [Architecture](#Architecture)\n    1. [forwarding connections with plug_mint_proxy](#Forwarding-Connections)\n    2. [Wiring with Plug](#Wiring-with-Plug)\n    3. [Header manipulation](#Header-manipulation)\n    4. [Matcher](#Matcher)\n\n### Configuration\n\nThe disptacher is configured using the dispatcher.ex file in a [mu-project](https://github.com/mu-semtech/mu-project).\n\nThe basic (default) configuration of the mu-dispatcher is an Elixir module named `Dispatcher` which uses the `Matcher` functionality.  \nAn empty set of accept types is required (`define_accept_types []`).\n\n```elixir\ndefmodule Dispatcher do\n  use Matcher\n  define_accept_types []\n\n  ...\nend\n```\n\n### Supported API\n\n#### Use Matcher\n\nThe using Matcher macro sets up the matcher.  It imports Matcher, send_resp and forward.\n\n### Http verbs\n### `get`, `put`, `post`, `delete`, `patch`, `head`, `options`, `match`\n\nImplements a specific matcher on the http verb with the corresponding name.  The `match` macro matches all verbs.\n\n```Accepts:```\n\n  - path: A string which is deconstructed into variables.\n  - options: The options hash containing options to match on for this call:\n    - accept: hash with all required accept shortforms analyzed through `define_accept_types`\n    - last_call: set to true when searching for a fallback solution (for sending a clean 404)\n  - block: Code block for processing and sending the request\n\n```Supplies:```\n\n  - conn: Plug connection to be forwarded or responded to\n  - path: Often the `path` as input is set as `\"/something/*path\"` in which case the `path` variable contains unused path segments\n\n### define_accept_types\n\nProvides a way to match the accept types to more readable terms so matching can happen in an easy and consistent manner.  Receives a property array describing each of the keys that will be used and their corresponding accept headers.  Wildcards are allowed in this specification.\n\n\n## Forwarding requests\n\n### Basic forwarding\n\nYou can proxy one path to another path using this service.\nIn order to forward requests coming in on `/sessions` to `http://sessionsservice/login`, we can add the following.\n\n```elixir\ndefmodule Dispatcher do\n  use Matcher\n  define_accept_types []\n\n  match \"/sessions\", _ do\n    forward conn, [], \"http://sessionsservice/login\"\n  end\nend\n```\n\nThis uses the match macro to match any verb, it ignores any extra info, and it forwards the connection to the sessionsservice.  The body is just Elixir code, hence we can add any extra logic in here if need be.  An example would be to log the conn we receive when forwarding.\n\n```elixir\ndefmodule Dispatcher do\n  use Matcher\n  define_accept_types []\n\n  match \"/sessions\", _ do\n    IO.inspect( conn, label: \"conn for /sessions\" )\n    forward conn, [], \"http://sessionsservice/login\"\n  end\nend\n```\n\n### Forwarding paths\n\nIn many cases it is desired to verbatim forward all subroutes of a route.  A common case would be to dispatch the handling of some resource through mu-cl-resources.  We can forward any call to widgets this way.\n\n```elixir\ndefmodule Dispatcher do\n  use Matcher\n  define_accept_types []\n\n  match \"/widgets/*path\", _ do\n    forward conn, path, \"http://resource/widgets\"\n  end\nend\n```\n\nThis match will forward any verb on any path that begins with `/widgets` to the [resource](http://github.com/mu-semtech/mu-cl-resources) microservice.\n\n\n### Matching on verb\n\nIt is possible to explicitly match on an HTTP verb.  Supported verbs are GET, PUT, POST, DELETE, HEAD, OPTIONS.  Use the downcased name of the verb as a matching construct.  In order to only forward POST requests, we would update our sample to the following:\n\n```elixir\ndefmodule Dispatcher do\n  use Matcher\n  define_accept_types []\n\n  post \"/sessions\", _ do\n    forward conn, [], \"http://sessionsservice/login\"\n  end\nend\n```\n\n\n### Matching Accept headers\n\nIt is important to only dispatch to services which can formulate an acceptable response.  This ensures the best response for the user is selected.\n\nClients can request content in many formats.  These formats are expressed using MIME types.  Common mime types on the web are are `application/json`, `text/html`, `image/jpeg` and (for {JSON:API}) `application/vnd.api+json`.  Star patterns are allowed in mime types, allowing for `image/*` or just `*/*` to be requested.  A browser can supply many MIME types for a single request with differing preferences, allowing it to state \"I would like to have an image, but a web page will do if you don't have the image.\"\n\nWhen responding to a request, we should respond with applicable content.  If the browser requests an image, we should yield an image rather than a json representation of that image.  A non-functional example could look like this.\n\n```elixir\n# THIS DOES NOT WORK!\ndefmodule Dispatcher do\n  use Matcher\n  define_accept_types []\n\n  match \"/images/*path\", %{ accept: \"image/jpeg\" } do\n    forward conn, path, \"http://images/\"\n  end\n\n  match \"/images/*path\", %{ accept: \"application/json\" } do\n    forward conn, path, \"http://resource/images/\"\n  end\nend\n```\n\nIn practice we tend to build abstractions on these MIME types.  Resources responds with `application/vnd.api+json` which matches the specification for `application/json`.  Hence, we'd want that service to respond to both of these.  Abstractions of these settings are made using accept types abstractions as shown in the following example.\n\n```elixir\ndefmodule Dispatcher do\n  use Matcher\n\n  define_accept_types [\n    json: [ \"application/json\", \"application/vnd.api+json\" ]\n  ]\n\n  match \"/images/*path\", %{ accept: %{ json: true } } do\n    forward conn, path, \"http://resource/images\"\n  end\nend\n```\n\nA more convoluted example contains the hosting of images.  We may have an image hosting/scaling/conversion service that supports JPEG and PNG, another service for handling GIFs.  A full example would then look like the following:\n\n```elixir\ndefmodule Dispatcher do\n  use Matcher\n\n  define_accept_types [\n    json: [ \"application/json\", \"application/vnd.api+json\" ],\n    img: [ \"image/jpg\", \"image/jpeg\", \"image/png\" ],\n    gif: [ \"image/gif\" ]\n  ]\n\n  get \"/images/*path\", %{ accept: %{ json: true } } do\n    forward conn, path, \"http://resource/images\"\n  end\n\n  get \"/images/*path\", %{ accept: %{ img: true } } do\n    forward conn, path, \"http://images/images\"\n  end\n\n  get \"/images/*path\", %{ accept: %{ gif: true } } do\n    forward conn, path, \"http://gifs/images\"\n  end\nend\n```\n\nIn this configuration the first case that matches wins.  If the user prefers json that's what they'll get, the same for gif or jpeg.\n\n### Matching on host\n\nDispatching may occur based on a hostname.  Both an array-format as well as a string-format are supported to match on a host.  The string-format currently supports matching only, the array format also allows extraction.\n\nIn order to reply only to calls coming in for `api.redpencil.io`, you can use the rule:\n\n```elixir\n  get \"/employees\", %{ host: [\"io\", \"redpencil\", \"api\"] } do\n    ...\n  end\n```\n\nOr, for simple matches like this, you can use a simplified API like:\n\n```elixir\n  get \"/employees\", %{ host: \"api.redpencil.io\" } do\n    ...\n  end\n```\n\nThis simplified syntax is internally converted into an array match.  Wildcards are supported too:\n\n```elixir\n  get \"/employees\", %{ host: \"*.redpencil.io\" } do\n    ...\n  end\n```\n\nThis wildcard will match `redpencil.io`, `api.redpencil.io`, `dev.api.redpencil.io`, etc.\n\nIf you need to access a part of the API, revert back to the array syntax and define a variable:\n\n```elixir\n  get \"/employees\", %{ host: [\"io\", \"redpencil\", subdomain | subsubdomains] }\n    IO.inspect( subdomain, \"First subdomain\" )\n    IO.inspect( subsubdomains, \"Array of subdomains under subdomain\" )\n    ...\n  end\n```\n\nThis specific implementation does require at least one subdomain and it will thus not match `redpencil.io`.\n\n\n### Fallback routes and 404 pages\n\nWhen no response can be given, a 404 page should be provided.  This 404 page should only be offered when no other service could answer the request.  Hence we should only serve the 404 page when all other services have had their go.  The format of the 404 page follows the same accept header rules as the actual content so the same flow holds.\n\nIn order to provide a 404 page or other fallback, the `last_call` option is supplied on which you can filter.  Extending our previous example with json, html, a 404 page in all sorts of images gives us the following result.\n\n```elixir\ndefmodule Dispatcher do\n  use Matcher\n\n  define_accept_types [\n    text: [ \"text/*\" ],\n    html: [ \"text/html\", \"application/xhtml+html\" ],\n    json: [ \"application/json\", \"application/vnd.api+json\" ],\n    img: [ \"image/jpg\", \"image/jpeg\", \"image/png\" ],\n    gif: [ \"image/gif\" ],\n    image: [ \"image/*\" ]\n  ]\n\n  get \"/images/*path\", %{ accept: %{ json: true } } do\n    forward conn, path, \"http://resource/images\"\n  end\n\n  get \"/images/*path\", %{ accept: %{ img: true } } do\n    forward conn, path, \"http://images/images\"\n  end\n\n  get \"/images/*path\", %{ accept: %{ gif: true } } do\n    forward conn, path, \"http://gifs/images\"\n  end\n\n  get \"/*_\", %{ last_call: true, accept: %{ json: true } } do\n    send_resp( conn, 404, \"{ \\\"error\\\": { \\\"code\\\": 404, \\\"message\\\": \\\"Route not found.  See config/dispatcher.ex\\\" } }\" )\n  end\n\n  get \"/*_\", %{ last_call: true, accept: %{ image: true } } do\n    forward conn, [], \"http://images/404\"\n  end\n\n  get \"/*_\", %{ last_call: true, accept: %{ text: true } } do\n    send_resp( conn, 404, \"404 - page not found\\n\\nSee config/dispatcher.ex\" )\n  end\n\n  get \"/*_\", %{ last_call: true, accept: %{ html: true } } do\n    send_resp( conn, 404, \"\u003chtml\u003e\u003chead\u003e\u003ctitle\u003e404 - Not Found\u003c/title\u003e\u003c/head\u003e\u003cbody\u003e\u003ch1\u003e404 - Page not found\u003c/h1\u003e\u003c/body\u003e\u003c/html\u003e\" )\n  end\nend\n```\n\n### Manipulating responses\n\nThe dispatcher is just code.  As you start reusing the same properties more often, you may want to supply default values to clean things up.  You can also add conditional logging, or manipulate the request before forwarding it to the client.\n\nAlthough not considered a public API, it is possible to manipulate the request or to draft responses manually.\n\n```elixir\ndefmodule Dispatcher do\n  use Matcher\n\n  define_accept_types [\n    json: [ \"application/json\", \"application/vnd.api+json\" ]\n  ]\n\n  @json %{ accept: %{ json: true } }\n\n  match \"/sessions/*path\", @json do\n    IO.inspect( conn, label: \"Connection for sessions service.\" )\n    forward conn, path, \"http://sessions/login\"\n  end\n\n  match \"/images/*path\", @json do\n    forward conn, path, \"http://resource/images\"\n  end\n\n  match \"/*_\", %{ last_call: true, accept: %{ json: true } } do\n    send_resp( conn, 404, \"{ \\\"error\\\": { \\\"code\\\": 404, \\\"message\\\": \\\"Route not found.  See config/dispatcher.ex\\\" } }\" )\n  end\nend\n```\n\n### Layers\n\nLayers help with organizing your forwarding rules.\n\nYou can define them like this:\n`define_layers [ :api, :frontend ]`!!!Order matters!!!\n\nLets look at an example:\n```elixir\ndefine_layers [ :api, :frontend ]\n\ndefine_accept_types [\n    html: [\"text/html\", \"application/xhtml+html\"],\n    json: [\"application/json\", \"application/vnd.api+json\"],\n]\n\nmatch \"/*path\", %{ accept: %{html: true}, layer: :api} do\n    Proxy.forward conn, path, \"http://frontend\"\nend\n\nmatch \"/*path\", %{ accept: %{json: true}, layer: :api} do\n    Proxy.forward conn, path, \"http://api\"\nend\n\n```\n\nLets say a request comes in from a browser looking for the index page. First the dispatcher will traverse the `api` layer (since its the first one defined).  When nothing is found it will move on to the next layer (the `html` layer) where it will find a match\n\n## Extra information\n\nThis section contains various recipes to implement specific behaviour with basic explanation as to why it works.\n\n### Host an EmberJS app\n\nThe Ember application should be served on most calls when an HTML page is requested.  The assets and styles should be served on the respective paths whenever they are requested.\n\n```elixir\ndefmodule Dispatcher do\n  use Matcher\n\n  define_accept_types [\n    json: [ \"application/json\", \"application/vnd.api+json\" ],\n    html: [ \"text/html\", \"application/xhtml+html\" ],\n    any: [ \"*/*\" ]\n  ]\n\n  @html %{ accept: %{ html: true } }\n  @json %{ accept: %{ json: true } }\n  @any %{ accept: %{ any: true } }\n\n  ... # your other rules belong here\n\n  match \"/assets/*path\", @any do\n    forward conn, path, \"http://frontend/assets/\"\n  end\n\n  match \"/*_path\", @html do\n    # *_path allows a path to be supplied, but will not yield\n    # an error that we don't use the path variable.\n    forward conn, [], \"http://frontend/index.html\"\n  end\n\n  match \"/*_\", %{ last_call: true, accept: %{ json: true } } do\n    send_resp( conn, 404, \"{ \\\"error\\\": { \\\"code\\\": 404, \\\"message\\\": \\\"Route not found.  See config/dispatcher.ex\\\" } }\" )\n  end\nend\n```\n\n### External API CORS headers\n\nWhen using the dispatcher with a frontend running on another domain, browsers need to know what headers they can pass to your service.  In order to verify what headers are allowed, they send an options call.  We can hook into this options call to set the necessary headers.  Any 200 response makes the browsers accept those headers.\n\n```elixir\ndefmodule Dispatcher do\n  use Matcher\n  define_accept_types []\n\n  options \"*path\", _ do\n    conn\n    |\u003e Plug.Conn.put_resp_header( \"access-control-allow-headers\", \"content-type,accept\" )\n    |\u003e Plug.Conn.put_resp_header( \"access-control-allow-methods\", \"*\" )\n    |\u003e send_resp( 200, \"{ \\\"message\\\": \\\"ok\\\" }\" )\n  end\nend\n```\n\n### Provide 404 pages\n\nYou can provide many types of 404 pages.  If you use a Single Page Application, you may want to default to the 404 page using the single page app.  You may want to provide a 404 json response in any case.  The same holds for other formats.\n\nThe following example hosts 404 pages in various types assuming there is a `static` microservice that hosts static assets for you.  Note that the HTML 404 option may be served by your backend instead.\n\n```elixir\ndefmodule Dispatcher do\n  use Matcher\n\n  define_accept_types [\n    text: [ \"text/*\" ],\n    html: [ \"text/html\", \"application/xhtml+html\" ],\n    json: [ \"application/json\", \"application/vnd.api+json\" ],\n    jpeg: [ \"image/jpg\", \"image/jpeg\" ],\n    png: [ \"image/png\" ],\n    gif: [ \"image/gif\" ],\n  ]\n\n  ... # other calls here\n\n  get \"/*_\", %{ last_call: true, accept: %{ json: true } } do\n    send_resp( conn, 404, \"{ \\\"error\\\": { \\\"code\\\": 404, \\\"message\\\": \\\"Route not found.  See config/dispatcher.ex\\\" } }\" )\n  end\n\n  get \"/*_\", %{ last_call: true, accept: %{ text: true } } do\n    send_resp( conn, 404, \"404 - page not found\\n\\nSee config/dispatcher.ex\" )\n  end\n\n  get \"/*_\", %{ last_call: true, accept: %{ html: true } } do\n    send_resp( conn, 404, \"\u003chtml\u003e\u003chead\u003e\u003ctitle\u003e404 - Not Found\u003c/title\u003e\u003c/head\u003e\u003cbody\u003e\u003ch1\u003e404 - Page not found\u003c/h1\u003e\u003c/body\u003e\u003c/html\u003e\" )\n  end\n\n  get \"/*_\", %{ last_call: true, accept: %{ jpeg: true } } do\n    forward conn, [], \"http://static/404.jpeg\"\n  end\n\n  get \"/*_\", %{ last_call: true, accept: %{ png: true } } do\n    forward conn, [], \"http://static/404.png\"\n  end\n\n  get \"/*_\", %{ last_call: true, accept: %{ gif: true } } do\n    forward conn, [], \"http://static/404.gif\"\n  end\nend\n```\n\n## Architecture\n\nThe Dispatcher offers support for forwarding connections and for dispatching connections.\n\n### Forwarding Connections\n\nForwarding connections is built on top of `plug_mint_proxy` which uses the Mint library for efficient creation of requests.  Request accepting is based on Cowboy 2 which allows for http/2 support.\n\n### Wiring with Plug\n[Plug](https://github.com/elixir-plug/plug) expects call to be matched using its own matcher and dispatcher.\nThis library provides some extra support.  \nAlthough tying this in within Plug might be simple, the request is dispatched to our own matcher in [plug_router_dispatcher.ex](./lib/plug_router_dispatcher.ex).\n\n### Header manipulation\n\nThe dispatcher knows about certain header manipulations to smoothen out configuration.  These are configured using `plug_mint_proxy`'s manipulators as seen in [the Proxy module](./lib/proxy.ex)\n\n  - [Manipulators.AddXRewriteUrlHeader](./lib/manipulators/add_x_rewrite_url_header.ex): Sets the x-rewrite-url header on the incoming request so backend services can figure out tho original request if needed.\n  - [Manipulators.RemoveAcceptEncodingHeader](./lib/manipulators/remove_accept_encoding_header.ex): Removes the `accept_encoding` header from the request as encryption is handled by the identifier and should not be hanlded by backend services.\n  - [Manipulators.AddVaryHeader](./lib/add_vary_header.ex): Adds the `vary` header with value `\"accept, cookie\"` so both of these are taken into account during incidental caching in between links or the user's browser.\n\n### Matcher\n\n[The Matcher module](./lib/matcher.ex) contains the bulk of the logic in this component.  It parses the request Accept header and parses the accept types inside of it.  It also parses the supplied accept types and searches for an optimal solution to dispatch to.\n\nHigh-level the dispatching works as follows:\n\n1. Parse the accept header\n2. Group each score of the accept header (A)\n3. Match each (A) with the set of `define_accept_types` noting that a `*` wildmatch may occur in both, thus resulting in (B).\n4. For each (B) try to find a matched solution\n5. If a solution is found, return it\n6. If no solution is found, try to find a matched solution with the `last_call` option set to true\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmu-semtech%2Fmu-dispatcher","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmu-semtech%2Fmu-dispatcher","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmu-semtech%2Fmu-dispatcher/lists"}