{"id":22901057,"url":"https://github.com/mpdn/unthread","last_synced_at":"2025-05-08T01:44:33.551Z","repository":{"id":67595495,"uuid":"271648432","full_name":"mpdn/unthread","owner":"mpdn","description":"A deterministic, fuzzable pthread implementation","archived":false,"fork":false,"pushed_at":"2023-05-07T17:03:01.000Z","size":1920,"stargazers_count":37,"open_issues_count":1,"forks_count":7,"subscribers_count":6,"default_branch":"master","last_synced_at":"2025-05-08T01:44:27.635Z","etag":null,"topics":["concurrency","fuzzing","pthreads"],"latest_commit_sha":null,"homepage":"","language":"C","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mpdn.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-06-11T21:18:00.000Z","updated_at":"2025-03-02T06:46:41.000Z","dependencies_parsed_at":"2023-02-21T18:15:32.962Z","dependency_job_id":null,"html_url":"https://github.com/mpdn/unthread","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mpdn%2Funthread","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mpdn%2Funthread/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mpdn%2Funthread/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mpdn%2Funthread/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mpdn","download_url":"https://codeload.github.com/mpdn/unthread/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252983757,"owners_count":21835758,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["concurrency","fuzzing","pthreads"],"created_at":"2024-12-14T01:31:29.102Z","updated_at":"2025-05-08T01:44:33.440Z","avatar_url":"https://github.com/mpdn.png","language":"C","funding_links":[],"categories":[],"sub_categories":[],"readme":"Unthread\n========\n\nUnthread is an implmentation of POSIX threads designed for fuzzing and debugging of concurrent\nprograms. Unthread is:\n\n- **deterministic**: Identical seeds will result in identical threading schedules.\n- **fuzzable**: Unthread can be used for instrumented fuzzing of threaded code, although the\n  coverage-based heuristics used by most fuzzers are likely inefficient for this.\n- **strict**: Unthread only implementes the minimum required by the spec.\n- **instant**: Timed locks do not wait for wall-time to pass before a lock is acquired/times out.\n\nWith some caveats:\n\n- **Not parallel**: Unthread does not use any actual hardware threading. All threads run on a single\n  core.\n- **No preemption**: Control is only transfered at \"yield points\", which includes most pthread\n  functions. This means that code cannot depend on other threads making progress by eg. waiting in a\n  loop until some condition is met. `pthread_yield` can be inserted in these cases to provide a\n  yield point.\n- **Non-conformance**: Not being a \"real\" pthreads implementation that integrates with the system\n  means some parts of the POSIX spec are difficult/impossible to implement:\n  - Only `pthread_cond_timedwait`, `pthread_cond_wait`, `pthread_join`, and `pthread_testcancel` are\n    cancellation points.\n  - Process shared objects are not supported (`PTHREAD_PROCESS_SHARED`).\n  - Signal handling is not supported.\n- **Glibc only**: Only Glibc is supported at the moment. PR's welcome!\n- **Incomplete**: Unthread is a work in progress and some pthread features have not been implemented\n  yet:\n  - Robust mutexes.\n  - Semaphores.\n\nExample\n-------\n\nHere is a example from the Unthread test suite of a program with a race condition:\n\n```c\n#include \u003cstdio.h\u003e\n#include \u003cpthread.h\u003e\n#include \u003cstdlib.h\u003e\n\nstatic int val = 0;\nstatic pthread_mutex_t mutex = PTHREAD_MUTEX_INITIALIZER;\n\nvoid* incr(void* arg) {\n    int v;\n    \n    pthread_mutex_lock(\u0026mutex);\n    v = val;\n    pthread_mutex_unlock(\u0026mutex);\n\n    pthread_mutex_lock(\u0026mutex);\n    val = v + 1;\n    pthread_mutex_unlock(\u0026mutex);\n\n    return NULL;\n}\n\nint main() {\n    pthread_t a,b;\n    pthread_create(\u0026a, NULL, incr, NULL);\n    pthread_create(\u0026b, NULL, incr, NULL);\n    pthread_join(a, NULL);\n    pthread_join(b, NULL);\n    printf(\"%d\", val);\n    \n    return 0;\n}\n```\n\nThis program has no data-races as all access to `val` is protected behind a mutex. But it *does*\nhave a scheduling-based race condition as the final value it outputs depends on the scheduling\norder. For this particular example and my particular machine/OS, this non-determinism does occur\nunder standard pthreads given enough executions:\n\n```console\n$ make bin/mutex-pthread\n$ seq 1000 | xargs -i bin/mutex-pthread\n2222222222222222222222222222222222222212222222222222222122222222222222222222222222222222222222222222\n2222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222\n2222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222\n2222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222\n2222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222\n2222222222212222222222222222222222222222222222222222222222222222222222222122222222222222222222222222\n2222222221222222222222222222222222222222222222222222222222222222222222222222222222222222222221222222\n2222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222\n2222222222222222222222222222221222222222222222222222222222222222221222222222222222222222222222222222\n2222222222222222222222222222222222222222222222222222222222222222222222222222222122222222222222222222\n```\n\nThis is not true in the general case. It might be that a race condition only shows under some\nspecial conditions, like high load, a certain os, or certain hardware. To make matters worse,\ndebuggers can affect timings such that race conditions disappear. Unthread allows us to test this in\na structured way instead of just hoping that we are able to trigger non-deterministic behavior.\n\nLet us say that the expected value is 2 in the above example is 2 and the buggy value is 1. We can\ntest this using the `unthread-fuzz` utility. By default, this program just executes the program with\ndifferent seeds until it returns with a non-zero return code, in which case the failing seed is\nprinted to standard out.\n\n```console\n$ make bin/mutex\n$ make bin/unthread-fuzz\n$ bin/unthread-fuzz sh -c '[ $(bin/mutex) -eq \"2\" ]'\nf503e38ee865d082f5773198fb24df71\n```\n\nIt found a failing seed! We can execute the program with the given seed and get the exact same\nresult each time:\n\n```console\n$ UNTHREAD_SEED=f503e38ee865d082f5773198fb24df71 bin/mutex\n1\n```\n\nThe program can then be debugged with standard debuggers like GDB or similar.\n\nUsage\n-----\n\nBuild Unthread by running `make` in this dir (or add it to your build). Then link to Unthread\ninstead of pthread and use the Unthread include dir. Eg. instead of `cc main.c -lpthread` do\n`cc main.c -Lunthread/bin/unthread.o -I unthread/include`. Optionally also include the\n`include-util` dir.\n\nThe thread schedule (i.e. the sequence in which threads yield to each other) can be defined in the\nfollowing ways:\n- By setting the environment variable `UNTHREAD_SEED` to a 32-character hexadecimal string. Unthread\n  will use this to seed a PRNG that then picks the thread schedule.\n- By setting the environment variable `UNTHREAD_FILE` to point to a file that will be used as\n  entropy source for the thread schedule. This could for example be random noise to get a random\n  execution (much like setting `UNTHREAD_SEED`) or it could be the result of a instrumented fuzzing\n  process. When using file schedule input, Unthread will exit if the entropy file is depleted.\n- By setting neither, in which case the standard input is used as an entropy source like it would be\n  from a file.\n\nUsing Unthread with libraries that themselves link to pthread will need to be rebuilt with Unthread.\nGood luck. :)\n\nConfiguration\n-------------\n\nUnthread can be configured by setting the following environment variables:\n\n| Environment variable | Use                              |\n|----------------------|----------------------------------|\n| UNTHREAD_SEED        | Seed for thread schedule         |\n| UNTHREAD_FILE        | Entropy file for thread schedule |\n| UNTHREAD_VERBOSE     | `true` for verbose logging       |\n| UNTHREAD_RET_OFFSET  | Return code offset               |\n\nReturn codes\n------------\n\nUnthread may stop the application under a number of conditions:\n\n| Retcode |                                                |\n|---------|------------------------------------------------|\n| 40      | The thread schedule entropy was exhausted      |\n| 41      | All threads have become deadlocked             |\n| 42      | An illegal operation was performed             |\n| 43      | Allocation failure                             |\n| 44      | Misc. failure (usually if a system call fails) |\n| 45      | Failure while reading from IO                  |\n| 46      | An unsupported operation was performed         |\n\nIn case that the retcode clashes with the retcode of the application, the default retcode offset of\n40 can be changed by setting the `UNTHREAD_RET_OFFSET` environment variable.\n\nTesting\n-------\n\nThe Unthread test runner requires Python 3 to be installed.\n\nUnthread is tested by the Unthread tests in the `test-src` dir and a set of tests from the POSIX\nTest Suite project in the `posixtestsuite`. See the\n[POSIX Test Suite project site](https://sourceforge.net/projects/posixtest) for more information on\nthe POSIX Test Suite.\n\nUnthread tests in the `test-src` directory are examples of non-deterministic pthread programs that\nare deterministic in Unthread with a given seed. The tests contains a spec containing all the\npossible outcomes of running the test, and a list of seeds in a seperate `.seeds` file. These seeds\n*must* cover all the possible outcomes or the test will fail. When emodifying Unthread, it may be\nnecessary to generate new seeds by executing `UNTHREAD_GEN=true make test`.\n\nThe above will only generate the minimum number of seeds to cover the outcomes. Additional fuzzing\ncan be done by executing `UNTHREAD_ITER=N make unthread-fuzz` where `N` is the number of iterations\nto fuzz.\n\nSee also\n--------\n\n[rr](https://rr-project.org/), a gdb debugger with deterministic replay.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmpdn%2Funthread","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmpdn%2Funthread","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmpdn%2Funthread/lists"}