{"id":13579761,"url":"https://github.com/shdown/lua-shm-state-poc","last_synced_at":"2025-04-05T23:32:19.055Z","repository":{"id":115821037,"uuid":"138317183","full_name":"shdown/lua-shm-state-poc","owner":"shdown","description":"Lua state in shared memory: a proof of concept","archived":false,"fork":false,"pushed_at":"2020-02-26T04:50:22.000Z","size":12,"stargazers_count":23,"open_issues_count":1,"forks_count":1,"subscribers_count":3,"default_branch":"master","last_synced_at":"2024-10-29T18:47:35.367Z","etag":null,"topics":["linux","lua","posix","shared-memory","sharedmemory"],"latest_commit_sha":null,"homepage":null,"language":"C","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/shdown.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2018-06-22T15:13:07.000Z","updated_at":"2023-09-23T21:24:47.000Z","dependencies_parsed_at":"2023-06-28T09:57:21.637Z","dependency_job_id":null,"html_url":"https://github.com/shdown/lua-shm-state-poc","commit_stats":{"total_commits":10,"total_committers":1,"mean_commits":10.0,"dds":0.0,"last_synced_commit":"1a3f58b22cbdd64f0d83ec39a5a3dfdc1c537606"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shdown%2Flua-shm-state-poc","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shdown%2Flua-shm-state-poc/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shdown%2Flua-shm-state-poc/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shdown%2Flua-shm-state-poc/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/shdown","download_url":"https://codeload.github.com/shdown/lua-shm-state-poc/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247142572,"owners_count":20890748,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["linux","lua","posix","shared-memory","sharedmemory"],"created_at":"2024-08-01T15:01:42.777Z","updated_at":"2025-04-05T23:32:19.019Z","avatar_url":"https://github.com/shdown.png","language":"C","funding_links":[],"categories":["C"],"sub_categories":[],"readme":"PoC of sharing Lua interpreter memory between multiple processes.\n\nIt features two processes that take turns calling functions `f()` and `g()` *in the same Lua state* (loaded from `demo.lua`).\n\n![Screenshot](https://user-images.githubusercontent.com/5462697/41799127-e976f7a6-7678-11e8-8b06-fb6b4ee1d005.png)\n\nHow?\n===\nThere’s nothing new in Lua being flexible: it allows to override all the memory management by passing a custom allocator to [`lua_newstate`](https://www.lua.org/manual/5.3/manual.html#lua_newstate).\nThe only things left are to create a shared memory mapping and implement an allocator working in it.\n\nThis is more or less safe as long as Lua knows nothing about parallelism. Some things in the standard library do know, though; see the “Caveats” section.\n\nOn systems other that Linux, shared memory mappings have to be of fixed size; by default, this PoC allocates 16 Mb on these systems.\n\nIn fact, Linux shared memory mappings also have to be of fixed “size” — of fixed *virtual size*:\nthanks to the [overcommit feature](https://www.kernel.org/doc/Documentation/vm/overcommit-accounting) and the [`MAP_NORESERVE`](http://man7.org/linux/man-pages/man2/mmap.2.html) `mmap` flag,\nwhich tells Linux to only allocate physical pages on demand (this does *not* depend on which overcommit policy your system is configured to use), we can only pay for what we use,\nand not a page more. And with [`madvise(..., MADV_REMOVE)`](http://man7.org/linux/man-pages/man2/madvise.2.html), we can reclaim the pages we don’t need anymore. Yay!\n\nOn Linux, this PoC goes on to allocate 16 Gb of *virtual* memory *(1 Gb on 32-bit systems)*. Because this ought to be enough for anybody.\nIf your virtual memory is, in some reason, limited, run it as `./main -p`, which forces it to fall back to a portable 16 Mb mapping.\n\nCaveats\n===\n* Some functions in Lua’s standard library *really* don’t expect being called with such a setup.\nThese functions are `os.execute`, `io.popen`, `os.exit` and `os.setlocale`.\n\n* All actions on files (including opening and closing) are local to the current process.\n\n* My memory allocator sucks. Implementing a better one — possibly leveraging the aforementioned Linux features — is left as an exercise for the reader.\n\n* Does not work with LuaJIT.\n\n* Be careful with external modules.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fshdown%2Flua-shm-state-poc","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fshdown%2Flua-shm-state-poc","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fshdown%2Flua-shm-state-poc/lists"}