{"id":13580131,"url":"https://github.com/facebookexperimental/hermit","last_synced_at":"2025-05-14T15:11:45.614Z","repository":{"id":62956115,"uuid":"553679728","full_name":"facebookexperimental/hermit","owner":"facebookexperimental","description":"Hermit launches linux x86_64 programs in a special, hermetically isolated sandbox to control their execution. Hermit translates normal, nondeterministic behavior, into deterministic, repeatable behavior. This can be used for various applications, including replay-debugging, reproducible artifacts, chaos mode concurrency testing and bug analysis.","archived":false,"fork":false,"pushed_at":"2025-05-08T19:10:31.000Z","size":2106,"stargazers_count":1253,"open_issues_count":22,"forks_count":34,"subscribers_count":14,"default_branch":"main","last_synced_at":"2025-05-08T20:24:37.760Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/facebookexperimental.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2022-10-18T15:37:38.000Z","updated_at":"2025-05-08T19:10:35.000Z","dependencies_parsed_at":"2023-10-05T04:30:48.912Z","dependency_job_id":"614b2838-c239-485f-9368-d6c1872eae8a","html_url":"https://github.com/facebookexperimental/hermit","commit_stats":{"total_commits":261,"total_committers":47,"mean_commits":5.553191489361702,"dds":0.5287356321839081,"last_synced_commit":"cd573917ffb8d1faea4986c59831cb11af462e79"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/facebookexperimental%2Fhermit","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/facebookexperimental%2Fhermit/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/facebookexperimental%2Fhermit/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/facebookexperimental%2Fhermit/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/facebookexperimental","download_url":"https://codeload.github.com/facebookexperimental/hermit/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254170059,"owners_count":22026219,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-01T15:01:47.819Z","updated_at":"2025-05-14T15:11:45.577Z","avatar_url":"https://github.com/facebookexperimental.png","language":"Rust","funding_links":[],"categories":["Rust","Continuous integration","Projects","others"],"sub_categories":["Shell into containers"],"readme":"# Hermit: A reproducible container\n\nHermit forces deterministic execution of arbitrary programs and acts like a\nreproducible container. That is, it _hermetically_ isolates the program from\nsources of non-determinism such as time, thread interleavings, random number\ngeneration, etc. Guaranteed determinism is a powerful tool and it serves as a\nbasis for a number of applications, including concurrency stress testing,\nrecord/replay, reproducible builds, and automatic diagnosis of concurrency bugs,\nand more.\n\nHermit cannot isolate the guest program from sources of non-determinism such as\nfile system changes or external network responses. Instead, in order to provide\ncomplete determinism, the user should provide a fixed file system base image\n(e.g., with Docker) and disable external networking.\n\n\u003e [!WARNING]\n\u003e\n\u003e Hermit is no longer under active development within Meta and is in maintenance\n\u003e mode. There is a long tail of unsupported system calls that may cause your\n\u003e program to fail while running under Hermit. Unfortunately, we (the team behind\n\u003e this project) don't have the resources to triage issues, fix major bugs, or\n\u003e add features at this point in time.\n\u003e\n\u003e If you are interested in this project and want to make contributions, please\n\u003e submit a pull request. We will prioritize merging these over anything else.\n\n# How it works\n\nHermit sits between the guest process and the OS intercepting system calls made\nby the guest (using [Reverie][]). In some cases, it can completely replace the\nfunctionality of the kernel and suppress the original system call. In other\ncases, it forwards the system call to the kernel and sanitizes the response such\nthat it is made deterministic.\n\nAs a concrete example, lets say we have a program that reads random bytes from\n`/dev/urandom`. Hermit will see that the guest opened this file (a known source\nof non-determinism) and intercept subsequent reads to this file. Instead of\nletting the OS fill a buffer with random bytes, Hermit uses a deterministic\npseudorandom number generator with a fixed seed to fill in the buffer. The\ncontents of the buffer are then guaranteed to be the same upon every execution\nof the program.\n\nThe most complex source of non-determinism is in the thread scheduler. The way\nthreads are scheduled by the kernel depends on many external factors, including\nthe number of physical CPUs or other threads running on the system that require\nCPU time. To ensure that the threads of the guest process (and all of its child\nprocesses) are scheduled in a repeatable way, we first make sure that all thread\nexecutions are serialized so that there is effectively only one CPU. Then, we\ndeterministically pick which thread is allowed to run next. In order to only\nallow a thread to run for a fixed number of instructions, we use the CPU's\nPerformance Monitoring Unit (PMU) to stop execution after a fixed number of\nretired conditional branches (RCBs).\n\nRead below about how to build Hermit, and you can get an idea of what it does\nfrom running examples in the [./examples](./examples) folder.\n\n[Reverie]: https://github.com/facebookexperimental/reverie\n\n# Building and running Hermit\n\nHermit is built with the standard Rust cargo tool.\n\n```bash\ncargo build\n```\n\nThis builds the whole cargo workspace. The actual binary is located in target\ndirectory (`target/debug/hermit`).\n\nThen, once you've built Hermit, all you need to run your program\ndeterministically is:\n\n```bash\nhermit run \u003cprog\u003e\n```\n\nAfter that you can try running it in a concurrency stress testing (chaos) mode,\nor varying other parameters of the configuration such as the speed at which\nvirtual time passes inside the container, or the random number generation seed:\n\n```bash\nhermit run --chaos --sched-seed=3 \u003cprog\u003e\n```\n\nYou can use hermit as a replay-debugger as well, either recording a\nnon-deterministic execution (real time, real randomness, etc), or repeatedly\nrunning a controlled, deterministic one (virtual time, pseudo-randomness, etc).\n\n```bash\nhermit record start \u003cprog\u003e\nhermit replay\n```\n\n# Example programs\n\nSee the [the examples folder](./examples/README.md) for example programs and\ninstructions on how to run them. These showcase different sources of\nnondeterminism, and how hermit eliminates or controls them.\n\nIn order to explore more advanced examples, you can look at some of the\nintegration tests built from [./tests/](./tests/) or\n[./flaky-tests/](./flaky-tests/). For example, using the commands below you can\nrun a racey example multiple times to see its nondeterminism. Then run it under\nhermit to watch that nondeterminism disappear. Then run it under hermit\n`--chaos` to bring that nondeterminism back, but in a controlled way that can be\nreproduced based on the input seed.\n\n```bash\ncargo build\nfor ((i=0; i\u003c20; i++)); do ./target/debug/hello_race; done\n\nfor ((i=0; i\u003c20; i++)); do hermit run ./target/debug/hello_race; done\n\nfor ((i=0; i\u003c20; i++)); do\n  hermit run --chaos --seed-from=SystemRandom ./target/debug/hello_race;\ndone\n```\n\n# The state of CI and testing\n\nAt Meta, this repository is built using buck. We have over 700 integration tests\nthat run under this setup. But as of this initial release (2022-11-21), we have\nnot ported these tests to an external build system yet.\n\nA few unit tests run under `cargo test`, but the integration tests are more\ncomplicated because they combine various run modes with each of the test\nbinaries (which are built from `tests/`, `flaky-tests/`, and the rr test suite\ntoo).\n\nWe plan to get the internal Buck configuration files building externally with\nbuck or bazel.\n\n# Applications\n\nHermit translates normal, non-deterministic Linux behavior, into deterministic,\nrepeatable behavior. This can be used for various applications, including:\nrecord/replay debugging, simple reproducibility, \"chaos mode\" to expose\nconcurrency bugs in a controlled and repeatable way. Generally, Hermit makes\nimplicit inputs into explicit ones, and so enables searching over possible\nexecutions by explicitly varying these inputs and study the changes in outcomes.\nThis can be used for either searching for bugs or trying to narrow down their\ncauses.\n\n## Diagnosing concurrency bugs\n\nOne experimental application that Hermit has built-in support for is diagnosing\nconcurrency bugs, in terms of identifying the stack traces of racing critical\noperations which, if their order is flipped in the schedule, cause the program\nto crash (also called an \"order violation\" bug). Hermit can detect these even if\nthe racing instructions are in different processes and written in different\nprogramming languages.\n\nYou can kick off analyze with any program invocation, and tell it to search for\nfailing (and passing) executions, and then diagnose the difference between them.\n\n```\nhermit analyze --search -- \u003crun_args\u003e \u003cprog\u003e\n```\n\n# License\n\nHermit is licensed under a BSD-3 clause license, included in the `LICENSE` file\nin this directory.\n\n# Support\n\nHermit currently supports x86_64 Linux. Aarch64 support is a work in progress.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffacebookexperimental%2Fhermit","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffacebookexperimental%2Fhermit","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffacebookexperimental%2Fhermit/lists"}