{"id":15880078,"url":"https://github.com/crowdhailer/gen_browser","last_synced_at":"2025-03-17T12:31:18.195Z","repository":{"id":54274230,"uuid":"149796842","full_name":"CrowdHailer/gen_browser","owner":"CrowdHailer","description":"Transparent bi-directional communication for clients, servers and more","archived":false,"fork":false,"pushed_at":"2018-11-12T16:25:15.000Z","size":341,"stargazers_count":68,"open_issues_count":0,"forks_count":1,"subscribers_count":10,"default_branch":"master","last_synced_at":"2024-10-07T03:07:14.417Z","etag":null,"topics":["actor-model","actors","browser","distributed-systems","elixir","javascript"],"latest_commit_sha":null,"homepage":"https://hex.pm/packages/gen_browser","language":"Elixir","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/CrowdHailer.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2018-09-21T17:35:15.000Z","updated_at":"2024-05-09T09:15:39.000Z","dependencies_parsed_at":"2022-08-13T10:40:37.759Z","dependency_job_id":null,"html_url":"https://github.com/CrowdHailer/gen_browser","commit_stats":null,"previous_names":[],"tags_count":4,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CrowdHailer%2Fgen_browser","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CrowdHailer%2Fgen_browser/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CrowdHailer%2Fgen_browser/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CrowdHailer%2Fgen_browser/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/CrowdHailer","download_url":"https://codeload.github.com/CrowdHailer/gen_browser/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":221678126,"owners_count":16862368,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["actor-model","actors","browser","distributed-systems","elixir","javascript"],"created_at":"2024-10-06T03:07:27.385Z","updated_at":"2024-10-27T13:05:36.333Z","avatar_url":"https://github.com/CrowdHailer.png","language":"Elixir","funding_links":[],"categories":[],"sub_categories":[],"readme":"# GenBrowser\n\n**Transparent bi-directional communication for clients, servers and more**\n\n GenBrowser gives every client a unique identifier that can be used to send messages to/from/between clients AND server processes.\n\n*Project in active development, it is advised to read the [Notes](#notes) section.*\n\n## Example Client\n\nTo experiment with a similar example follow the instructions to set up the [Playground](#playground).\n\n```js\n// ponger.html\nconst client = await GenBrowser.start('http://localhost:8080')\n\nconsole.log(client.address)\n\nclient.mailbox.setHandler((message) =\u003e {\n  console.log('received message:', message)\n  client.send(message.from, {type: 'pong'})\n})\n```\n\n```js\n// pinger.html\nconst client = await GenBrowser.start('http://localhost:8080')\n\nvar peer = prompt(\"Enter a ponger address.\")\nif (peer == '') {\n  peer = client.communal.logger\n}\nclient.send(peer, {type: 'ping', from: client.address})\n\nawait client.mailbox.receive({timeout: 5000})\nconsole.log(\"Pong received\")\n```\n\nOnce started `gen-browser` has four things.\n\n1. An address that other clients, or the server, use it to send messages to this browser.\n2. A mailbox to receive messages. The mailbox can be used in one of two ways, but not both.\n  - `mailbox.receive()` will return a promise that will resolve on the next message received by the browser.\n  - `mailbox.handle(callback)` will call the callback with the contents of a message each time one is received.\n3. A function to send messages, that takes the target address and messages as arguments.\n  The message must be serialisable to JSON.\n4. Communal information from server, this can be anything including addresses for processes on the backend,\n  such as the logger process in the examples above.\n\n## Embed in a Phoenix (or Plug) application\n\nIn Phoenix add the plug to your `endpoint.ex`.\n**NOTE:** it must be added after the Parser Plugs.\n\n```elixir\ndefmodule MyAppWeb.Endpoint do\n  use Phoenix.Endpoint, otp_app: :my_app\n\n  # other plugs ...\n\n  plug(Plug.Parsers,\n    parsers: [:urlencoded, :multipart, :json],\n    pass: [\"*/*\"],\n    json_decoder: Poison\n  )\n\n  plug(GenBrowser.Plug, communal: %{myProcess: GenBrowser.Address.new(MyProcess)})\n  plug(Plug.MethodOverride)\n  plug(Plug.Head)\n\n  # more plugs\nend\n```\n\n*Any server process can be added to the communal information.*\n\nAdd the following script tags to the relevant pages.\n\n```html\n\u003cscript type=\"text/javascript\" src=\"/_gb/client.js\"\u003e\u003c/script\u003e\n\u003cscript type=\"text/javascript\"\u003e\n  const {address, mailbox, send, communal} = await GenBrowser.start()\n\u003c/script\u003e\n```\n\n## Playground\n\nRun a standalone backend that shares a process for name registration and a processes for server logging in the communal information.\nThis backend is used for the ping/pong examples in `/examples`\n\n```\ngit clone git@github.com:CrowdHailer/gen_browser.git\ncd gen_browser\n```\n\nMake sure the JavaScript bundle is built.\n\n```\nnpm install\nnpm run build\n```\n\nTo start the backend use Docker as follows.\n\n```\ndocker build -t gen-browser .\ndocker run -it -e SECRET=s3cr3t -p 8080:8080 gen-browser iex -S mix run examples/playground.exs\n```\n\nOr, if you have Elixir installed but not docker, mix can be used directly\n\n```\nSECRET=s3cr3t iex -S mix run examples/playground.exs\n```\n\nOpen `examples/pinger.html` and `examples/ponger.html` in your browser.\n\n### Send a message to a client\n\nFirst fetch the address of a running ponger browser.\n\nIn the server console\n\n```sh\niex\u003e address = \"g2gCZAAGZ2xvYmFsaAJkABlFbGl4aXIuR2VuQnJvd3Nlci5NYWlsYm94bQAAAAxWR0hFWFkwZWExUEg=--qp7BCZMlqtGpO7nUDwQmZC4CkA-tPZE56uVISq6xEcU=\"\niex\u003e {:ok, ponger} = GenBrowser.decode_address(address)\n# {:ok, {:global, {GenBrowser.Mailbox, \"VGHEXY0ea1PH\"}}}\n\niex\u003e GenBrowser.send(ponger, %{\"type\" =\u003e \"ping\", \"from\" =\u003e GenBrowser.Address.new(self)})\n# {:ok, :sent}\niex\u003e flush\n# %{\n#   \"from\" =\u003e \"g2gCZAAGZ2xvYmFsaAJkABlFbGl4aXIuR2VuQnJvd3Nlci5NYWlsYm94bQAAAAxWR0hFWFkwZWExUEg=--qp7BCZMlqtGpO7nUDwQmZC4CkA-tPZE56uVISq6xEcU=\",\n#   \"type\" =\u003e \"pong\"\n# }\n# :ok\n```\n\n## Contributing\n\nThis project requires both Elixir and node (+ npm) to be installed.\n\nTo fetch dependencies.\n\n```\nmix deps.get\n(cd client ; npm install)\n```\n\nTo run tests use `mix test`, The elixir tests include running the JavaScript test suite.\n\nIt would be good if, using `mix hex.publish` is aliased to run the tests, which includes the build step.\nCurrently it is required to remember to build before pushing to hex.\n\n## Notes\n\n### So experimental right now\n\nBest thing to do is play with examples.\n\nCheck the [Roadmap](#Roadmap) for what I'm working on next.\n\n### Rational\n\nClient/Server is just another type of distributed system.\nWhat if the whole system can be treated as a group of processes that send messages to each other.\n\n#### Goal 1\n\nMake it just as easy to send messages client to server and even client to client.\n\n#### Goal 2\n\nModel all communication as message passing so the system can be verified using session types or similar\n\n### Whats in a name?\n\nThe name `GenBrowser` comes from the erlang/Elixir terms for generalised servers; `GenServer`.\nOriginally this project was aiming to model interations with the browser the same as any other erlang process.\nHowever at this point it is clear this is just a general communication layer and does not need to be tied to any erlang/OTP terminology.\n\nOther names\n// cloud sounds like backup. instead redux-swarm, redux-flock, redux-cluster, redux-comms\n// conveyance, transmission,\n\n### Redux is the JavaScript name for Actor\n\nRedux doesn't have a way to spawn new processes, so the analogy is not perfect.\nHowever it is probably the closed many developers have come.\nThis might be a useful place to start the conversation.\n\n### `EventSource` is not implemented in IE\n\nPolyfills are available and some of these work in a node environment.\n\n### There is no alert for disconnects\n\nThe browser disconnecting from the internet might not be important.\nMessages for the browser will be kept on the server until it reconnects.\nBecause this is a distributed system, being connected to the server might still not mean you are connected to the peer you need.\n\nTo check connections send a message and await a reply or timeout.\n\nIf it is necessary to check connection with the server in general a message can be sent from the browser to it's own address.\nThis will involve a server round trip.\n\n### Clients are temporary\n\nThe client created from `GenBrowser.start` is deliberatly temporary, it is analogous to a process on the server side.\nIt is perhaps closer to the concept of a session.\n\nIf the user wants to access some persistent data they should send a message, probably with authentication.\nThis message should indicate the persistent data to access and the current session to forward it to.\n\n### Messages that are not understood\n\nMessages can always not be understood by the server.\nWhen using `send` the response is 202 accepted, this just means the server in general was happy the target process might not know how to handle the message.\n\nThe target process might not even know who the message came from, and so be unable to return an error.\nIn this case the system has to rely on timeouts and crashlogging.\n\nTo tackle this problem a standard message container could be provided,\nit might include a standard format for sender, and perhaps tracing.\nSuch a standard format could easily be built on top of this library.\n\n### Client outbound queue\n\nThe client is able to handle network interuptions on the receive side.\nHowever if sending during a network interuption then the send will fail.\n\nMessages could be automatically retried in case of 5xx response (but not 4xx),\nThis would require knowledge about the idempotency of the message in question.\n`dispatch(address, message, retry)` where retry is true/false or function taking error.\n\n### Ordering between two clients\n\nThe order of messages in the server mailbox is kept when streaming these to the client.\nThis does not guarantee ordering of messages sent from a client to a server process.\nThere is no correlation between requests sent.\n\nThe client could queue messages and only process the next once it has received an ack that the first is in the mailbox.\nThis does not need to be inefficient as it could send a list of messages in the case that it has multiple messages in the queue.\n\nSimply queuing before sending would not guarantee order, the HTTP handling process on the backend would be different for each process.\n\n### Addresses are large\n\nAddress encoding just makes use of `:erlang.term_to_binary` + `Base.url_encode64`.\nWith the addition of signing this makes for very large address strings.\n\nIf the encode function knew about all possible things that could be encoded then encoding could be super effiecient.\nSay a single byte for which module will following bytes just handled by that module.\n\nThis would need some top level information about what spec the address was created with,\nonce things are sent to the client they can survive redeploys.\n\nConnection would need to communicate that spec version `/mailbox/?spec=v1.1`\nThe secret for signing could be the spec id, however that probably just increases the risk of leaking the secret.\n\n### Every event involves a signature\n\nAs it stands every event involves calculating a digest.\nIt is probably safe to sign just the mailbox_id and append the count number to the signed mailbox_id.\nCurrently the count number is appended to the mailbox_id and that combined cursor signed\n\n### GenCall\n\nA call function could exist, both where the server can call the client and block on response and vica versa.\n\nThis would require the send function to have a reference to the mailbox.\nIt would also require a selective receive, other push messages from the server might have arrived.\nA standard call format would go someway to providing a standard message container as discussed in messages not understood.\n\n### Security\n\nA client should only be able to send message to addresses it has been provided with.\ni.e. it should not be possible to guess the address of processes.\n\nThis is the Object capability security model https://en.wikipedia.org/wiki/Object-capability_model\n\nBy signing the event-id this method can be used to secure an individual clients reconnections\n\n- If id generation secure enough don't need to sign those addresses,\n  However will always need to sign encoded tuples, don't want those to be generated\n- Reconnection id is the security mechanism to act as someone.\n  In what cases does that id get sent again? How can nefarious actors get to that information,\n  EventSource doesn't let you add custom headers.\n\n### Redux swarm where you send every browser you are connected to\n\n### Single language\n\nThis project aims to unify the programming model accross both client and server.\nIt does not yet unify language.\nThe goal to unify language would be an argument for a server implemented in node, however this is not the best host for thinging of the system using the Actor model.\nIt might be possible to use ElixirScript for the front end.\n\n### Is this crazy?\n\nMaybe.\n\nLet me know what you think.\n\n## Roadmap\n\n### Setup Docker image that can be pulled\n\ninstall node on Docker\nhave docker build the npm packages\nserver the bundle under `/_gb`\nJavaScript get your own source\n\n### Develop a general purpose address structure\n\nMy suggesion for this is probably `{:comms, module, term}`,\nwhere the module implementes the `:comms_address` behaviour.\nWhe the message is sent it should call `module.dispatch(term, message)`\n\nThis tuple could also wrap other kinds of address e.g. `{:comms, pid}` and `{:comms, {:via, module, term}}`.\nThe value of this structure is that When mapping through data to send this can be matched on an securely serialized.\n\nBy this point we are very close to having `{mod, func, args}` as the structure of an address.\nI guess the above alows a behaviour to have more than one function that can be used in different cases.\n\nGeneral purpose address structure will need information on starting processes.\n\nNeed to blog about how create new process is not a thing.\n\nActor model only needs two rules.\nAnalogy is employers are hired, send message to the recruitment deparatment, not spawned.\n\nAddress is abstract key for linearizability.\nCreate a `Comms.HTTP` for sending messages to an endpoint, or even a session?\nCreate a `Comm.CallReply` for replying to a call.\n\n```elixir\ndef handle({:comms_call, from, :x}, state) do\n  {[{from, :y}], state}\nend\n```\n\nThe other end will need to handle a Down/Timeout message, they should probably be the same form with different reason\n```elixir\ndef handle({:comms_fail, reason, refence}, state) do\n  {[{from, :y}], state}\nend\n```\n\nIn most cases send to a destination.\ne.g.\n```\nsend(email_address, message)\n# better than\nsend(mailer, message+address)\n```\nThe fact it goes through a mailer process is just an implementation detail.\n\nlinearizability should be enforcable by making a call, MyModule.dispatch could return conflict.\n\n### Better encoding decoding\n\n- Is it possible to encode and sign with a secret.\n  The encode protocol would need to be able to pass custom options.\n- What is the best way to iterate through an object before it gets jsoned\n\nIf not then `GenBrowser.Address` does not need to exist as a struct, because the protocol for json encoding will not be used.\n\nMove the iterate through JSON function to web.\n\n### Work out how to provide a validation layer for messages as they are received by server\n\nAddresses should not need to know how to decode messages, there might be more than one transport format for messages.\nHowever it is possible that addresses could also opt in to a second behaviour such as `GenBrowser.JSONTransport`\nShould return error argument error.\nI'm not sure it even makes sense to have addresses have a binary format, given general choice they would not choose to be url safe\n\n`send` already returns a promise, awaiting on this could at least validate message was legit.\n\nDo an example with Ecto as the form validation.\n\n### Clear the server mailbox\n\nClient should be able to send the ability to ack messages and clear up the contents of the server mailbox.\n\nThe server mailbox process should also have a timeout after which a reconnection is not possible.\n\nCan the id of a reconnect be taken as an ack, or should that be sent separatly.\n\nThere should also be a timeout after which the mailbox process dies.\nFinally we need to signal to the client in cases when a reconnect to a dead mailbox is attempted.\n\n`Clients will reconnect if the connection is closed; a client can be told to stop reconnecting using the HTTP 204 No Content response code.`\nThis might not trigger an onerror, it might be best to on error on the client so sending a 4xx could be better\n\n### Redux middleware\n\nThis is the comms goal, i.e. don't call send, just return a list of addresses and messages thus making the whole thing pure.\n\n### Demonstrate with Redux\n\nThis could just be the redux swarm below.\nThere are ways to start new browser windows, this might be fun\n\n### Support a multi-node backend\n\nTo support a multi node backend it is required to guarantee that only on server mailbox can exist for each client(mailbox_id)\n`:global` can suffer split brain and so might not be an adequate solution.\nHowever because we know when to create a new mailbox, and they are always started with a random id, it might be the case that a split brain is not a problem.\nThe effect of a split brain would be sometimes thinking a server mailbox had died when it was infact in the other partition.\nThis would require setting up a new mailbox, but this eventuallity already has to be accounted for.\n\n### Composition of Effectful code\n\nhttps://elixirforum.com/t/functionaly-pure-message-passing/11173\n\n Finally once the above is proved extract addresses and make a Comms.Monad and a general Comms.Worker.\n This work should be done after discovering if a general equivalent to `:gen.call` and `Process.monitor` are needed.\n\n Pure messages with comms means no untestable side effects EVER.\n How things are modelled through session types. By this point we might need Rust for affine types.\n\n```elixir\nComms.for(message, state) do\n  :ok \u003c- send(a, :b)\n  :ok \u003c- send(foo, :bar)\nafter\n  state\nend\n```\n\n*Macro can rewrite address, to proxy affine types. However it's always possible to just assign variable earlier*\n`{:ok, a} \u003c- send(a, :b)`\n\nQ? - how to expose error case if allowing error cases?\n\nA good proof of concept of this should be in `Raxx.SimpleServer`\n\nNeeds a process that never needs starting. Is just transparently started,\nTry Comms.Worker started locally, generated with id in place.\nUses global and relies on generated id really being random.\nCan run the risk of having a hot node.\n\n### Typing message sending\n\nPing pong typed example using dialyzer.\n\nLook at existing work in comms/pachyderm.\n\nTo share this will be a recipe for working with dialyzer because lack of generics prevent solution being exported from a library.\n\n### (Type/Pure)Script Actor\n\nJSAP - JavaScript Actor Platform\n\nNamed actor with reference to Dom\n- create mailbox\u003cT\u003e\n- receive fn -\u003e Promise\u003cResult\u003cT\u003e\u003e\n- receive with a select function T -\u003e A returns a result of A\n- monitor needs to accept a Mailbox where T includes Down message.\n\n### Session type interactions\n\nThis probably requires pure interactions.\nAutowriting is a pretend for affine types.\nNeed multiparty to handle timout.\n\nThis is the point I am likely to seriously revist Rust\n\n### RGBA Framework\n\nRaft - GenBrowser - Ace\n\nFull stack Framework, use Raxx.Kit\nIs it possible to have a Raft setup which just manages an optimistic concurrency key,\nthen the client can just write allowed values.\n\n### Typed transport\n\nLimited state types.\n\nThis builds on the fact that all the even numbers can be represented by droping the last bit,\nand ensuring your type knows that they are even numbers.\n\nIf the language primative is binaries or maybe integers, both of those must exist in a range.\nso instead of JSON where you can have any integer you have a spec which always includes an upper limit on the integer\n\nWorks for all countable sets (not transdentals) but I thnk this is a limit using computers has.\n\nThis is useful for sending things over the wire.\nOpening a protocol probably has a hash of the protocol spec.\n\nAlso type operations can always be shown to go to a decreasing number of outcomes, and therefore total.\n\nOnce comms monad exists having totallity in a language is desirable.\n\nThe only language I know with this trait is idris.\n\n### Work out if client can generate own address\n\nThen no need to await at startup\n\nClient can pass mailbox as an argument to the start gen browser function.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcrowdhailer%2Fgen_browser","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcrowdhailer%2Fgen_browser","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcrowdhailer%2Fgen_browser/lists"}