{"id":17248026,"url":"https://github.com/eugeneia/erlangen","last_synced_at":"2026-01-07T03:04:36.141Z","repository":{"id":78177401,"uuid":"68979050","full_name":"eugeneia/erlangen","owner":"eugeneia","description":"Distributed, asychronous message passing system for Clozure Common Lisp","archived":false,"fork":false,"pushed_at":"2018-02-08T09:10:03.000Z","size":194,"stargazers_count":66,"open_issues_count":2,"forks_count":9,"subscribers_count":9,"default_branch":"master","last_synced_at":"2025-01-31T07:22:26.548Z","etag":null,"topics":["actors","common-lisp","distributed","message-passing","supervision"],"latest_commit_sha":null,"homepage":"","language":"Common Lisp","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/eugeneia.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"COPYING","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2016-09-23T02:00:44.000Z","updated_at":"2024-12-21T12:35:03.000Z","dependencies_parsed_at":"2023-03-18T06:30:44.505Z","dependency_job_id":null,"html_url":"https://github.com/eugeneia/erlangen","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eugeneia%2Ferlangen","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eugeneia%2Ferlangen/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eugeneia%2Ferlangen/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eugeneia%2Ferlangen/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/eugeneia","download_url":"https://codeload.github.com/eugeneia/erlangen/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245598311,"owners_count":20641884,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["actors","common-lisp","distributed","message-passing","supervision"],"created_at":"2024-10-15T06:39:48.890Z","updated_at":"2026-01-07T03:04:31.108Z","avatar_url":"https://github.com/eugeneia.png","language":"Common Lisp","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Erlangen\n\nDistributed, asynchronous message passing system for Clozure Common Lisp.\n\n_Warning: Erlangen is immature, experimental software, subject to bugs and\nchanges._\n\n## Documentation\n\n * [Manual and API Documentation](http://mr.gy/software/erlangen/api.html)\n * [Erlangen: introduction](http://mr.gy/blog/erlangen-intro.html)\n\n## Getting Started\n\nFirst, make sure you have [Clozure Common Lisp](http://ccl.clozure.com/) and\n[Quicklisp](https://www.quicklisp.org/) installed (if you use a Debian based OS\nyou can install CCL via this [package](http://mr.gy/blog/clozure-cl-deb.html)).\nNext clone Erlangen into `quicklisp/local-projects`.\n\nYou can now start Clozure Common Lisp and load Erlangen:\n\n```\n(ql:quickload :erlangen)\n(use-package :erlangen)\n```\n\nAlternatively, you can build Erlangen as a stand-alone executable that boots\ninto a REPL that uses the packages `erlangen` and `erlangen.management`:\n\n```\ncd ~/quicklisp/local-projects/erlangen\nmake bin/erlangen-kernel \u0026\u0026 bin/erlangen-kernel\n```\n\n### Programming a Parallelized Map with Asynchronous Agents and Message Passing\n\nLet’s jump straight into a practical example. The Erlangen repository contains\nthe `erlangen.example` package which implements `parallel-map`, a parallelized\n*map* function. Its like Common Lisp’s `map` over a single vector, only that it\nspreads the work across multiple concurrent *agents* (fancy processes/native\nthreads).\n\n`parallel-map` returns the results of calling a *function* on the elements in a\n*vector* in a new vector. It uses up to *level* agents to parallelize the\nworkload. Optionally, a *result-type* can be specified as the element type of\nthe result vector. Here is roughly how `parallel-map` works:\n\n  1. it spawns some worker agents and attaches to them in *monitor* mode, so\n     that it will receive an exit notification for each worker\n  2. it sends each worker agent a message with a chunk of work\n  3. it waits for and receives the exit notification of each worker, which\n     contains the chunk’s result, and inserts them into the final result vector\n\nThe worker agents initially do nothing at all. They each just wait to receive a\nfunction to execute, and quit when they are done. Note that we use the [Trivia](https://github.com/guicho271828/trivia)\npattern matching library to match received messages.\n\n```\n(defun worker ()\n  (ematch (receive)\n    ((and (type function) function)\n     (funcall function))))\n```\n\nEventually, each worker will receive a `map-thunk` closure, which maps the\nmapping *function* over a chunk of the *vector* bounded by *start* and *end*.\n\n```\n(defun map-chunk (function vector start end result-type)\n  (lambda ()\n    (let ((results (make-array (- end start) :element-type result-type)))\n      (loop for i from start below end do\n           (setf (aref results (- i start))\n                 (funcall function (aref vector i))))\n      (values start end results))))\n```\n\nNow let’s look at `parallel-map`. To distribute the work, it computes\n\n - *length*—the length of our input *vector*\n - *n-chunks*—the number of chunks we will divide the work up into\n - *chunk-size*—the minimum length (in elements) of a chunk\n\nand spawns a worker agent for each chunk.\n\n```\n(defun parallel-map (function vector \u0026key (level 2) (result-type t))\n  (let* ((length (length vector))\n         (n-chunks (min level length))\n         (chunk-size (floor length n-chunks))\n         (workers (loop for i from 1 to n-chunks collect\n                       (spawn 'worker :attach :monitor))))\n```\n\nNext it sends each worker a closure for the chunk it should process. It\ndivides the work into *n-chunks* intervals of at least *chunk-size* length,\nthat fully cover the *vector*.\n\n```\n    (loop for worker in workers\n          for chunk from 1\n          for start from 0 by chunk-size\n          for end = (if (\u003c chunk n-chunks)\n                        (+ start chunk-size)\n                        length)\n       do (send (map-chunk function vector start end result-type) worker))\n```\n\nFinally it allocates a vector to store the results in, and waits to receive\neach chunk result. If any worker exits unexpectedly, `parallel-map` exits with\nthat workers exit reason. Again, because we attached to the worker agents in\n*monitor* mode, all remaining workers will also receive the exit signal and\nshut down.\n\n```\n    (loop with results = (make-array length :element-type result-type)\n          for worker in workers do\n         (ematch (receive)\n           ((list (type agent) :ok start end chunk-result)\n            (replace results chunk-result :start1 start :end1 end))\n           ((list (type agent) :exit reason)\n            (exit reason)))\n       finally (return results))))\n```\n\nNow we can spawn `parallel-map` agents like this\n\n```\n(spawn '(parallel-map 1+ #(2 4 6 8 10 12 14) :level 3) :attach :monitor)\n(receive)\n→ (#\u003cAGENT #x302002A191ED\u003e :OK #(3 5 7 9 11 13 15))\n```\n\n### Getting Distributed\n\nWhat fun are agents if they aren’t distributed over a network? Erlangen comes\nwith support for distribution via TCP/IP built in. Each instance of Erlangen\ncan act as a *node*, and talk to other Erlangen nodes. In order to facilitate\nport discovery of of remote nodes, there needs to be a *port mapper* running on\nthe host. To build and run the Erlangen port mapper, execute the commands\n\n```\nmake bin/erlangen-port-mapper\nbin/erlangen-port-mapper localhost \u0026\n```\n\nin a shell in the root of the Erlangen repository. Now build the Erlangen\nkernel in the same way to conveniently run additional Erlangen instances, and\nuse it to start a node named *map-node*.\n\n```\nmake bin/erlangen-kernel\nbin/erlangen-kernel -n -e '(node :host \"localhost\" :name \"map-node\")'\n```\n\nHint: if you use Emacs, you can start a new Erlangen instance with Slime via\n`C-u M-x slime RET /path/to/erlangen-kernel`.\n\nFinally, we can also make our initial Erlangen instance a node, and offload\nsome work to *map-node*:\n\n```\n(spawn '(node :host \"localhost\"))\n(spawn '(erlangen.examples:parallel-map 1+ #(2 4 6 8 10 12 14))\n       :attach :monitor\n       :node \"map-node\")\n(receive)\n→ (\"localhost/map-node/0\" :OK #(3 5 7 9 11 13 15))\n```\n\nWhat happened? We spawned an agent on the remote *map-node* instance to run\n`parallel-map`, and received its exit notification transparently over the\nnetwork.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Feugeneia%2Ferlangen","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Feugeneia%2Ferlangen","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Feugeneia%2Ferlangen/lists"}