{"id":44883581,"url":"https://github.com/radrow/ddtrace","last_synced_at":"2026-02-17T17:00:51.071Z","repository":{"id":324245930,"uuid":"1096519514","full_name":"radrow/ddtrace","owner":"radrow","description":"Distributed deadlock detection via asynchronous (pitch)black-box monitors","archived":false,"fork":false,"pushed_at":"2026-02-09T13:32:23.000Z","size":3082,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-02-09T16:32:56.079Z","etag":null,"topics":["deadlock-detection","erlang","gen-server","monitoring"],"latest_commit_sha":null,"homepage":"","language":"Erlang","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/radrow.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-11-14T14:45:47.000Z","updated_at":"2026-02-04T16:49:24.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/radrow/ddtrace","commit_stats":null,"previous_names":["radrow/ddtrace"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/radrow/ddtrace","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/radrow%2Fddtrace","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/radrow%2Fddtrace/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/radrow%2Fddtrace/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/radrow%2Fddtrace/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/radrow","download_url":"https://codeload.github.com/radrow/ddtrace/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/radrow%2Fddtrace/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29550800,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-17T14:33:00.708Z","status":"ssl_error","status_checked_at":"2026-02-17T14:32:58.657Z","response_time":100,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deadlock-detection","erlang","gen-server","monitoring"],"created_at":"2026-02-17T17:00:50.445Z","updated_at":"2026-02-17T17:00:51.065Z","avatar_url":"https://github.com/radrow.png","language":"Erlang","funding_links":[],"categories":[],"sub_categories":[],"readme":"# DDTrace\n\nDDTrace is a tool for asynchronous distributed deadlock detection in\n`gen_server`-based systems.\n\n## Repository layout\n\nThe three top-level applications are:\n\n- `apps/ddtrace` – the main DDTrace library.\n- `apps/model` – the scenario generator, tracer tooling, and Elixir CLI used to\n  exercise the library.\n- `apps/microchip_factory` – an example `gen_server`-based Elixir application\n  which shows DDTrace in a slightly more realistic setup. Refer to its README\n  for more details.\n- `elephant_patrol` — an example *distributed* `gen_server`-based Elixir app. \n\nTo build the tooling with the testing models run:\n\n```\nmix deps.get\nmix escript.build\n```\n\nThe scenario testing escript is written to `./ddtrace`. Example usage:\n\n```\n./ddtrace apps/model/priv/scenarios/deadlock.conf\n```\n\nThe microchip factory example can be run as follows:\n\n``` \nmix run -e \"MicrochipFactory.start_two(true)\"\n```\n\n\n## Prerequisites\n\n- Erlang/OTP 26\n- Elixir 1.14\n\n## Application requirements\n\nThe monitored system must entirely consist of `gen_server` instances. Moreover,\neach server must adhere to _Single-threaded Remote Procedure Call_ (SRPC), which\nin practice means that it may only use `gen_server:call` and `gen_server:cast`\nfor communication. To calls, they must always reply via `{reply, _Reply,\n_State}` (i.e. no accumulation of the `From` argument and returning `{noreply,\n_State}`). Multi-calls* through `gen_server:multi_call` and manual request\nhandling via `gen_server:send_request`/`gen_server:reply` is also forbidden. In\norder for deadlock detection to work properly, every generic server must be\nmonitored.\n\nTODO: there is a chance that `gen_server:multi_call` would work, but this is to be investigated.\n\n### Tracing limitations\n\n`ddtrace` monitors employ the `trace` facility to oversee their `gen_server` instances. \nBecause Erlang allows at most one tracer for each process, this effectively prevents using\n`trace` to debug systems monitored by `ddtrace`. \n\n## Instrumenting generic servers with DDTrace\n\nA monitor is started via `ddtrace:start` or `ddtrace:start_link`. The PID of the\nmonitored `gen_server` is passed as a parameter.\n\nMonitors recognise each other via a *monitor registry* which maps generic\nservers' PIDs to their monitors. The registry is implemented in the `mon_reg`\nmodule using `pg` process groups. Monitors take care of registering themselves\nin the registry automatically.\n\nIn order to receive a deadlock notification, the user needs to register itself\nas a subscriber to a particular monitor. One would normally subscribe to a\nmonitor immediately after making a call, and unsubscribe upon receiving a\nresponse or deadlock notification. To subscribe to deadlocks, use the\n`ddtrace:subscribe_deadlocks` function (use `ddtrace:unsubscribe_deadlocks` to opt out). The\nsubscribtion function returns a request identifier that can be used in generic\nserver's `reqid` or listened to directly via `gen_server:wait_response`.\n\nThe following snippet exemplifies how to monitor a single generic\nserver with DDTrace:\n\n``` erlang\n%% Start the service\n{ok, P} = gen_server:start(my_gen_server_module, []),\n\n%% Start the monitor\n{ok, M} = ddtrace:start_link(P),\n\n%% Subscribe to deadlocks\nReqM = ddtrace:subscribe_deadlocks(M),\n\n%% Call the service\nReqP = gen_server:send_request(P, request)\n\n%% Set up request ID collection\nReqIds0 = gen_server:reqids_new(),\nReqIds1 = gen_server:reqids_add(ReqP, process, ReqIds0),\nReqIds2 = gen_server:reqids_add(MonP, monitor, ReqIds1),\n\ncase gen_statem:receive_response(ReqIds2, infinity, true) of\n  {{reply, R}, process, _ReqIds} -\u003e %% Handle reply\n  {{reply, {deadlock, Cycle}}, monitor, _ReqIds} -\u003e %% Handle deadlock\nend.\n```\n**IMPORTANT:** Self-inflicted deadlocks (e.g. `gen_server:call(self(), lol)`)\nare handled by `gen_server` and cause the process to crash without sending a\ncall message. DDTrace will handle this case as well, but the end user might a\nreceive crash result before the deadlock notification from DDTrace. Note that\nsimply waiting for `{error, {calling_self, _}, _Label, _ReqIds}` is not\nsufficient, as this may happen in a nested call. Therefore, some additional\nrecursion might be needed to distinguish such a deadlock from a regular error.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fradrow%2Fddtrace","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fradrow%2Fddtrace","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fradrow%2Fddtrace/lists"}