{"id":17964787,"url":"https://github.com/okeuday/varpool","last_synced_at":"2025-03-25T06:31:00.750Z","repository":{"id":30211840,"uuid":"33762821","full_name":"okeuday/varpool","owner":"okeuday","description":"Erlang Process Pools as a Local Variable","archived":false,"fork":false,"pushed_at":"2023-10-26T19:08:06.000Z","size":83,"stargazers_count":19,"open_issues_count":0,"forks_count":4,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-03-19T09:21:08.909Z","etag":null,"topics":["data-structures","erlang","erlang-process-pool"],"latest_commit_sha":null,"homepage":null,"language":"Erlang","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/okeuday.png","metadata":{"files":{"readme":"README.markdown","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2015-04-11T04:39:49.000Z","updated_at":"2023-07-25T13:55:47.000Z","dependencies_parsed_at":"2022-09-01T15:25:12.182Z","dependency_job_id":"9f9be494-762f-43cd-b91d-0475e287f1f0","html_url":"https://github.com/okeuday/varpool","commit_stats":{"total_commits":40,"total_committers":2,"mean_commits":20.0,"dds":0.375,"last_synced_commit":"c57982b1ed50d76b2974c011ddb1325380e5c04d"},"previous_names":[],"tags_count":22,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/okeuday%2Fvarpool","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/okeuday%2Fvarpool/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/okeuday%2Fvarpool/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/okeuday%2Fvarpool/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/okeuday","download_url":"https://codeload.github.com/okeuday/varpool/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245413567,"owners_count":20611350,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["data-structures","erlang","erlang-process-pool"],"created_at":"2024-10-29T12:08:39.645Z","updated_at":"2025-03-25T06:31:00.490Z","avatar_url":"https://github.com/okeuday.png","language":"Erlang","funding_links":[],"categories":[],"sub_categories":[],"readme":"varpool: Erlang Process Pools as a Local Variable\n=================================================\n\n\"Let messages ride together to minimize computational costs\"\n\nPurpose\n-------\n\nThere are two main approaches to process pooling in Erlang:\n\n1. Treat the pool as shared data to operate on the pool in ways commonly used\n   with imperative programming languages:  A process is removed from the pool,\n   the process is used and the process is added back to the same pool\n   (interacting with the pool as mutable data).\n2. Use the pool with Flow-Based-Programming (FBP) to use Erlang in a simpler\n   way:  The pool returns a process without modifying the pool and a\n   message is sent to the process to utilize the process.  Either an\n   asynchronous or synchronous message is used with the process based on\n   the requirements of the process and the effect of the request rate on the\n   process message queue.\n\nThe #1 approach is unfortunately very common in Erlang source code and is\nnormally (erroneously) regarded as the correct approach when writing Erlang\nsource code.  A major negative aspect of the #1 approach is the impact on\nfault-tolerance.  How is the process fault-tolerance handled when the process\nis removed from the pool (to be used)?  If the process is still linked to the\npool in a way where the pool will restart the process if it crashes while not\nbeing within the pool, it is impossible to isolate the cause of the crash\nin the implementation.  The crash could have occurred due to the external\nusage of the process or it could have occurred due to an internal error\nwithin the pool source code.  While you can easily argue that a stacktrace\nwill show you the cause of the error, you will be unable to easily determine\nif the stacktrace occurred during or after the error on one side\nor the other.  The ambiguity and complexity in the #1 approach makes it\nerror-prone.\n\nA second major negative aspect of the #1 approach is the scalability impact\nwhen constantly mutating data to get a process from a pool.  As the request\nrate increases, the latency associated with altering the pool becomes\nmore significant.  The extra latency helps to put artifical limits on any\nsource code that depends on pooling with the #1 approach.\n\nThe #2 approach is a more natural fit to the fault-tolerance and\nconcurrency Erlang provides (the pool handles the process fault-tolerance\nand relying on the process message queue utilizes Erlang process concurrency).\nThe pool does not become a scalability bottleneck while it remains\nimmutable data.  An additional benefit is that the implementation is simpler\nand easier to understand.\n\nThe most common example of the #1 approach is probably\n[poolboy](https://github.com/devinus/poolboy).  The varpool library\nprovides the #2 approach as local variable data (for efficient access).\n\nBuild\n-----\n\n    rebar compile\n\nTest\n----\n\n    rebar eunit\n\nAuthor\n------\n\nMichael Truog (mjtruog at protonmail dot com)\n\nLicense\n-------\n\nMIT License\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fokeuday%2Fvarpool","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fokeuday%2Fvarpool","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fokeuday%2Fvarpool/lists"}