{"id":17267467,"url":"https://github.com/lemmy/blockingqueue","last_synced_at":"2025-05-16T02:08:15.732Z","repository":{"id":45884281,"uuid":"220093229","full_name":"lemmy/BlockingQueue","owner":"lemmy","description":"Tutorial \"Weeks of debugging can save you hours of TLA+\".  Each git commit introduces a new concept =\u003e check the git history!","archived":false,"fork":false,"pushed_at":"2024-10-27T18:32:57.000Z","size":19139,"stargazers_count":492,"open_issues_count":3,"forks_count":23,"subscribers_count":11,"default_branch":"main","last_synced_at":"2025-05-16T02:08:08.292Z","etag":null,"topics":["fifo","fifo-queue","java","model-checking","specification","tla","tlaplus","tlaps"],"latest_commit_sha":null,"homepage":"https://youtu.be/wjsI0lTSjIo","language":"TLA","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/lemmy.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-11-06T21:26:06.000Z","updated_at":"2025-04-13T21:07:19.000Z","dependencies_parsed_at":"2023-01-20T10:15:53.489Z","dependency_job_id":"406a5579-5c4f-4352-a9f8-7d1a66ec92ac","html_url":"https://github.com/lemmy/BlockingQueue","commit_stats":{"total_commits":41,"total_committers":2,"mean_commits":20.5,"dds":0.07317073170731703,"last_synced_commit":"d2e0dc12c72fa5d5721cf08bbc1e3b9e7cde309f"},"previous_names":[],"tags_count":32,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lemmy%2FBlockingQueue","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lemmy%2FBlockingQueue/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lemmy%2FBlockingQueue/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lemmy%2FBlockingQueue/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/lemmy","download_url":"https://codeload.github.com/lemmy/BlockingQueue/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254453655,"owners_count":22073617,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["fifo","fifo-queue","java","model-checking","specification","tla","tlaplus","tlaps"],"created_at":"2024-10-15T08:10:44.110Z","updated_at":"2025-05-16T02:08:10.722Z","avatar_url":"https://github.com/lemmy.png","language":"TLA","readme":"# BlockingQueue\nTutorial-style talk \"Weeks of debugging can save you hours of TLA+\".  The inspiration  for this tutorial and definitive background reading material (with spoilers) is [\"An Example of Debugging Java with a Model Checker\n\"](http://www.cs.unh.edu/~charpov/programming-tlabuffer.html) by [Michel Charpentier](http://www.cs.unh.edu/~charpov/).  I believe it all goes back to [Challenge 14](http://wiki.c2.com/?ExtremeProgrammingChallengeFourteen) of the c2 wiki.\n\nEach [git commit](https://github.com/lemmy/BlockingQueue/commits/tutorial) introduces a new TLA+ concept.  Go back to the very first commit to follow along!  Please note - especially when you create PRs -that the git history will be rewritten frequently to stay linear.\n\nClick either one of the buttons to launch a zero-install IDE to give the TLA+ specification language a try:\n\n[![Open TLA+ BlockingQueue in Visual Studio Codespaces](https://img.shields.io/badge/TLA+-in--VSCodespaces-grey?labelColor=ee4e14\u0026style=for-the-badge\u0026logo=data:image/svg+xml;base64,PHN2ZyBmaWxsPSIjNjY2NjY2IiByb2xlPSJpbWciIHZpZXdCb3g9IjAgMCAyNCAyNCIgeG1sbnM9Imh0dHA6Ly93d3cudzMub3JnLzIwMDAvc3ZnIj48dGl0bGU+TWljcm9zb2Z0IGljb248L3RpdGxlPjxwYXRoIGQ9Ik0xMS40IDI0SDBWMTIuNmgxMS40VjI0ek0yNCAyNEgxMi42VjEyLjZIMjRWMjR6TTExLjQgMTEuNEgwVjBoMTEuNHYxMS40em0xMi42IDBIMTIuNlYwSDI0djExLjR6Ii8+PC9zdmc+)](https://github.com/codespaces/new?hide_repo_select=true\u0026ref=main\u0026repo=220093229)\n[![Open TLA+ BlockingQueue in Gitpod Ready-to-Code](https://img.shields.io/badge/TLA+-in--Gitpod-grey?labelColor=ee4e14\u0026style=for-the-badge\u0026logo=gitpod)](https://gitpod.io/#https://github.com/lemmy/BlockingQueue)\n\nThis tutorial is work in progress. More chapters will be added in the future. In the meantime, feel free to open issues with questions, clarifications, and recommendations. You can also reach out to me on [twitter](https://twitter.com/lemmster).  Basic TLA+ learning material can be found over at [Lamport's TLA+ page](http://lamport.azurewebsites.net/tla/learning.html).\n\n--------------------------------------------------------------------------\n\n### v37 (Termination): Gradually terminate Consumers when Producers shutdown by sending Poison Apples.\n\nPoison Pill variant where each producer sends `N` slices of a Poison Apple to `N` consumers. A consumer dies (terminates) when it has eaten `M` apple slices, where `M` is the number of producers.\n    \nThe advantage over the Poison Pill approach is that no extra synchronization primitive is required to detect global producer termination that triggers sending the poison pills. Also, consumers down-scale gradually following the producer termination.\n\n### v36 (Termination): Check BlockingQueuePoisonPill for all subsets of the constants by mimicking Apalache's ConstInit feature.\n\n### v35 (Termination): Check refinement of BlockingQueue by BlockingQueuePoisonPill.\n\n### v34 (Termination): Terminate Consumers when Producers are done by sending a poison pill in a termination stage.\n\n### v33 (Refinement Fair): Prove BlockingQueueFair implements BlockingQueueSplit.\n\nA proof showing that ```BlockingQueueSplit``` implements ```BlockingQueueSplit```.   Two lemmas show that it is sometimes necessary to prove simpler facts first to prove the main theorem.\n\nCompared to the previous proofs, this one is far more involved and more realistically reflects the efforts require to write proofs; the refinement proof took approximately three weeks.  Three weeks is quite an investment considering that the correctness of the refinement seems straight forward in the first place.  However, three weeks also includes the work to detect and help with a fix for a [regression in TLAPS 1.4.4](https://github.com/tlaplus/tlapm/releases/tag/v1.4.5), identifying [two](https://github.com/tlaplus/tlaplus/issues/434) Toolbox [issues](https://github.com/tlaplus/tlaplus/issues/433), and familiarizing myself with the TLAPS [build](https://github.com/tlaplus/tlapm/blob/master/tools/installer/tlaps-release.sh.in) and [development](https://www.youtube.com/watch?v=Eu46FmhpR_0\u0026feature=youtu.be) process.\n\n### v32 (Refinement Fair): Prove TypeInv is an inductive invariant of BlockingQueueFair.\n\nAs a first step of proving that ```BlockingQueueFair``` refines ```BlockingQueueSplit```, we once again prove inductiveness of ```TypeInv```.\n\n### v31 (Refinement Fair): Refine BlockingQueue with BlockingQueueFair spec.\n\n```BlockingQueueFair``` refines ```BlockingQueueSplit``` by notifying the longest-waiting producer or consumer thread instead of any thread in ```waitC``` or ```waitP```. For that, it uses (ordered) sequences in place of the (unordered) sets.  The refinement mapping is straight forward: ```BlockingQueueSplit!waitC``` and ```waitP``` are refined by the sequences ```waitSeqC``` and ```waitSeqP``` respectively.  TLC checked the refinement mapping for the finite model with the configuration p4c3b3.\n\n### v30 (Starvation): Liveness-check multiple configurations.\n\nThe trick of adding auxiliary variables to check sets of constants values as in [v06](#v06-continue-convert-constants-into-variables) does not work if we verify liveness properties (`Producer` and `Consumer` in the fairness constraint become [state-level](http://lamport.azurewebsites.net/tla/newmodule.html)). Instead, we come up with another trick that lets us check sets of constant values: By writing a new spec `BlockingQueueMC.tla` extending `BlockingQueue` that redefines the three constants `P`, `C`, and `B` with expressions that read (concrete) values from the process' environment variables (which is what the `IOEnv` operator from the [`IOUtils`](https://github.com/tlaplus/CommunityModules/blob/master/modules/IOUtils.tla) CommunityModules is for).\nAt this point, we could write a shell script in today's favorite scripting language to repeatedly invoke TLC for various configurations (TLC wrappers such as [tlacli](https://github.com/hwayne/tlacli) make this easier). However, I prefer to keep things simple and also use the expressiveness of TLA+. Thus, again with the help of the [`IOUtils` module](https://github.com/tlaplus/CommunityModules/blob/master/modules/IOUtils.tla#L26-L32), a TLA+ one-liner will check a set of constants. When evaluated with the REPL, it equals a set of results that can be processed (filtered, converted, printed, ...) further.\n\n```tla\n$ tlcrepl\nWelcome to the TLA+ REPL!\nTLC2 Version 2.16 of Day Month 20??\nEnter a constant-level TLA+ expression.\n(tla+) LET TLC==\u003c\u003c\"java\", \"-jar\", \"/opt/toolbox/tla2tools.jar\", \"-config\", \"BlockingQueueMC.tla\", \"-workers\", \"auto\", \"-noTE\", \"BlockingQueueMC\"\u003e\u003e IN { \u003c\u003cConf, IOEnvExec(Conf, TLC).exitValue\u003e\u003e : Conf \\in [ {\"P\",\"C\",\"B\"} -\u003e 1..4 ] }\n{\u003c\u003c[B |-\u003e 1, P |-\u003e 1, C |-\u003e 1], 0\u003e\u003e, \u003c\u003c[B |-\u003e 1, P |-\u003e 1, C |-\u003e 2], 0\u003e\u003e, \u003c\u003c[B |-\u003e 1, P |-\u003e 1, C |-\u003e 3], 0\u003e\u003e, \u003c\u003c[B |-\u003e 1, P |-\u003e 1, C |-\u003e 4], 0\u003e\u003e, \u003c\u003c[B |-\u003e 1, P |-\u003e 2, C |-\u003e 1], 0\u003e\u003e, \u003c\u003c[B |-\u003e 1, P |-\u003e 2, C |-\u003e 2], 0\u003e\u003e, \u003c\u003c[B |-\u003e 1, P |-\u003e 2, C |-\u003e 3], 0\u003e\u003e, \u003c\u003c[B |-\u003e 1, P |-\u003e 2, C |-\u003e 4], 0\u003e\u003e, \u003c\u003c[B |-\u003e 1, P |-\u003e 3, C |-\u003e 1], 0\u003e\u003e, \u003c\u003c[B |-\u003e 1, P |-\u003e 3, C |-\u003e 2], 0\u003e\u003e, \u003c\u003c[B |-\u003e 1, P |-\u003e 3, C |-\u003e 3], 0\u003e\u003e, \u003c\u003c[B |-\u003e 1, P |-\u003e 3, C |-\u003e 4], 0\u003e\u003e, \u003c\u003c[B |-\u003e 1, P |-\u003e 4, C |-\u003e 1], 0\u003e\u003e, \u003c\u003c[B |-\u003e 1, P |-\u003e 4, C |-\u003e 2], 0\u003e\u003e, \u003c\u003c[B |-\u003e 1, P |-\u003e 4, C |-\u003e 3], 0\u003e\u003e, \u003c\u003c[B |-\u003e 1, P |-\u003e 4, C |-\u003e 4], 0\u003e\u003e, \u003c\u003c[B |-\u003e 2, P |-\u003e 1, C |-\u003e 1], 0\u003e\u003e, \u003c\u003c[B |-\u003e 2, P |-\u003e 1, C |-\u003e 2], 0\u003e\u003e, \u003c\u003c[B |-\u003e 2, P |-\u003e 1, C |-\u003e 3], 0\u003e\u003e, \u003c\u003c[B |-\u003e 2, P |-\u003e 1, C |-\u003e 4], 0\u003e\u003e, \u003c\u003c[B |-\u003e 2, P |-\u003e 2, C |-\u003e 1], 0\u003e\u003e, \u003c\u003c[B |-\u003e 2, P |-\u003e 2, C |-\u003e 2], 0\u003e\u003e, \u003c\u003c[B |-\u003e 2, P |-\u003e 2, C |-\u003e 3], 0\u003e\u003e, \u003c\u003c[B |-\u003e 2, P |-\u003e 2, C |-\u003e 4], 0\u003e\u003e, \u003c\u003c[B |-\u003e 2, P |-\u003e 3, C |-\u003e 1], 0\u003e\u003e, \u003c\u003c[B |-\u003e 2, P |-\u003e 3, C |-\u003e 2], 0\u003e\u003e, \u003c\u003c[B |-\u003e 2, P |-\u003e 3, C |-\u003e 3], 0\u003e\u003e, \u003c\u003c[B |-\u003e 2, P |-\u003e 3, C |-\u003e 4], 0\u003e\u003e, \u003c\u003c[B |-\u003e 2, P |-\u003e 4, C |-\u003e 1], 0\u003e\u003e, \u003c\u003c[B |-\u003e 2, P |-\u003e 4, C |-\u003e 2], 0\u003e\u003e, \u003c\u003c[B |-\u003e 2, P |-\u003e 4, C |-\u003e 3], 0\u003e\u003e, \u003c\u003c[B |-\u003e 2, P |-\u003e 4, C |-\u003e 4], 0\u003e\u003e, \u003c\u003c[B |-\u003e 3, P |-\u003e 1, C |-\u003e 1], 0\u003e\u003e, \u003c\u003c[B |-\u003e 3, P |-\u003e 1, C |-\u003e 2], 0\u003e\u003e, \u003c\u003c[B |-\u003e 3, P |-\u003e 1, C |-\u003e 3], 0\u003e\u003e, \u003c\u003c[B |-\u003e 3, P |-\u003e 1, C |-\u003e 4], 0\u003e\u003e, \u003c\u003c[B |-\u003e 3, P |-\u003e 2, C |-\u003e 1], 0\u003e\u003e, \u003c\u003c[B |-\u003e 3, P |-\u003e 2, C |-\u003e 2], 0\u003e\u003e, \u003c\u003c[B |-\u003e 3, P |-\u003e 2, C |-\u003e 3], 0\u003e\u003e, \u003c\u003c[B |-\u003e 3, P |-\u003e 2, C |-\u003e 4], 0\u003e\u003e, \u003c\u003c[B |-\u003e 3, P |-\u003e 3, C |-\u003e 1], 0\u003e\u003e, \u003c\u003c[B |-\u003e 3, P |-\u003e 3, C |-\u003e 2], 0\u003e\u003e, \u003c\u003c[B |-\u003e 3, P |-\u003e 3, C |-\u003e 3], 0\u003e\u003e, \u003c\u003c[B |-\u003e 3, P |-\u003e 3, C |-\u003e 4], 0\u003e\u003e, \u003c\u003c[B |-\u003e 3, P |-\u003e 4, C |-\u003e 1], 0\u003e\u003e, \u003c\u003c[B |-\u003e 3, P |-\u003e 4, C |-\u003e 2], 0\u003e\u003e, \u003c\u003c[B |-\u003e 3, P |-\u003e 4, C |-\u003e 3], 0\u003e\u003e, \u003c\u003c[B |-\u003e 3, P |-\u003e 4, C |-\u003e 4], 0\u003e\u003e, \u003c\u003c[B |-\u003e 4, P |-\u003e 1, C |-\u003e 1], 0\u003e\u003e, \u003c\u003c[B |-\u003e 4, P |-\u003e 1, C |-\u003e 2], 0\u003e\u003e, \u003c\u003c[B |-\u003e 4, P |-\u003e 1, C |-\u003e 3], 0\u003e\u003e, \u003c\u003c[B |-\u003e 4, P |-\u003e 1, C |-\u003e 4], 0\u003e\u003e, \u003c\u003c[B |-\u003e 4, P |-\u003e 2, C |-\u003e 1], 0\u003e\u003e, \u003c\u003c[B |-\u003e 4, P |-\u003e 2, C |-\u003e 2], 0\u003e\u003e, \u003c\u003c[B |-\u003e 4, P |-\u003e 2, C |-\u003e 3], 0\u003e\u003e, \u003c\u003c[B |-\u003e 4, P |-\u003e 2, C |-\u003e 4], 0\u003e\u003e, \u003c\u003c[B |-\u003e 4, P |-\u003e 3, C |-\u003e 1], 0\u003e\u003e, \u003c\u003c[B |-\u003e 4, P |-\u003e 3, C |-\u003e 2], 0\u003e\u003e, \u003c\u003c[B |-\u003e 4, P |-\u003e 3, C |-\u003e 3], 0\u003e\u003e, \u003c\u003c[B |-\u003e 4, P |-\u003e 3, C |-\u003e 4], 0\u003e\u003e, \u003c\u003c[B |-\u003e 4, P |-\u003e 4, C |-\u003e 1], 0\u003e\u003e, \u003c\u003c[B |-\u003e 4, P |-\u003e 4, C |-\u003e 2], 0\u003e\u003e, \u003c\u003c[B |-\u003e 4, P |-\u003e 4, C |-\u003e 3], 0\u003e\u003e, \u003c\u003c[B |-\u003e 4, P |-\u003e 4, C |-\u003e 4], 0\u003e\u003e}\n```\n\nAnother nice property of the `IOEnv` trickery is that one can check spec variants without relying on sed, awk, ..., and the likes.  For example, the `ConditionalFairSpec` formula below could re-defining `FairSpec` and then checked for all configurations in `{ [P|-\u003ec[1],C|-\u003ec[2],B|-\u003ec[3],F|-\u003ec[4]] : c \\in ((1..3) \\X (1..3) \\X (1..2) \\X {\"A\",\"B\",\"C\"}) }`.\n\n```tla\nConditionalFairSpec == \n/\\ Spec\n/\\ LET f == IOEnv.F IN\n   IF \"A\" = f\n   THEN SomeFairnessCandidate\n   ELSE IF \"B\" = f\n        THEN AnotherFairnessCandidate\n        ELSE YetAnotherFairnessCandidate\n```\n\n### v29 (Starvation): Advanced fairness ruling out starvation entirely.\n\nStipulates that `Get` actions (consumers!) will eventually notify *all* waiting producers. In other words, given repeated  `Get` actions (we don't care which consumer, thus, existential quantification), all waiting producers will eventually be notified. Because  `Get`  actions are not continuously enabled (the buffer might be empty), weak fairness is not strong enough. Obviously, no real system scheduler implements such an inefficient \"policy\".\nThis fairness constraint was initially proposed by Leslie Lamport (with the minor typo \"in\" instead of \"notin\", which happens to hold for configurations with at most two producers).\n\n### v28 (Starvation): Weak fairness defined for Put.\n\nDefining ```Next``` to be (weakly) [fair](https://pron.github.io/posts/tlaplus_part3#machine-closure-and-fairness) makes only sure that a Next-step is (eventually) taken.  However, ```Next``` is a disjunct of the ```Put``` and ```Get``` sub-actions and fairness does not distribute.  Since, we want all producers to eventually take steps, we specify (weak) fairness at the level of the ```Put``` sub-actions.  Unfortunately, producers can still starve:\n\n```tla\nChecking temporal properties for the complete state space with 1600 total distinct states at (2020-01-18 15:28:06)\nError: Temporal properties were violated.\n\nError: The following behavior constitutes a counter-example:\n\nState 1: \u003cInitial predicate\u003e\n/\\ buffer = \u003c\u003c\u003e\u003e\n/\\ waitSet = {}\n\nState 2: \u003cNext line 54, col 9 to line 57, col 45 of module BlockingQueue\u003e\n/\\ buffer = \u003c\u003cp2\u003e\u003e\n/\\ waitSet = {}\n\nState 3: \u003cNext line 54, col 9 to line 57, col 45 of module BlockingQueue\u003e\n/\\ buffer = \u003c\u003cp2, p3\u003e\u003e\n/\\ waitSet = {}\n\nState 4: \u003cNext line 54, col 9 to line 57, col 45 of module BlockingQueue\u003e\n/\\ buffer = \u003c\u003cp2, p3, p2\u003e\u003e\n/\\ waitSet = {}\n\nState 5: \u003cNext line 54, col 9 to line 57, col 45 of module BlockingQueue\u003e\n/\\ buffer = \u003c\u003cp2, p3, p2\u003e\u003e\n/\\ waitSet = {p1}\n\nState 6: \u003cNext line 54, col 9 to line 57, col 45 of module BlockingQueue\u003e\n/\\ buffer = \u003c\u003cp2, p3, p2\u003e\u003e\n/\\ waitSet = {p1, p4}\n\nState 7: \u003cNext line 54, col 9 to line 57, col 45 of module BlockingQueue\u003e\n/\\ buffer = \u003c\u003cp2, p3, p2\u003e\u003e\n/\\ waitSet = {p1, p2, p4}\n\nState 8: \u003cNext line 54, col 9 to line 57, col 45 of module BlockingQueue\u003e\n/\\ buffer = \u003c\u003cp2, p3, p2\u003e\u003e\n/\\ waitSet = {p1, p2, p3, p4}\n\nState 9: \u003cNext line 54, col 9 to line 57, col 45 of module BlockingQueue\u003e\n/\\ buffer = \u003c\u003cp3, p2\u003e\u003e\n/\\ waitSet = {p1, p2, p3}\n\nState 10: \u003cNext line 54, col 9 to line 57, col 45 of module BlockingQueue\u003e\n/\\ buffer = \u003c\u003cp2\u003e\u003e\n/\\ waitSet = {p1, p2}\n\nState 11: \u003cNext line 54, col 9 to line 57, col 45 of module BlockingQueue\u003e\n/\\ buffer = \u003c\u003c\u003e\u003e\n/\\ waitSet = {p1}\n\nState 12: \u003cNext line 54, col 9 to line 57, col 45 of module BlockingQueue\u003e\n/\\ buffer = \u003c\u003cp2\u003e\u003e\n/\\ waitSet = {p1}\n\nState 13: \u003cNext line 54, col 9 to line 57, col 45 of module BlockingQueue\u003e\n/\\ buffer = \u003c\u003cp2, p3\u003e\u003e\n/\\ waitSet = {p1}\n\nBack to state 5: \u003cNext line 54, col 9 to line 57, col 45 of module BlockingQueue\u003e\n\nFinished checking temporal properties in 00s at 2020-01-18 15:28:06\n12313 states generated, 1600 distinct states found, 0 states left on queue.\n```\n\n### v27 (Starvation): Starvation of individual producers.\n\nIndividual Producer or Consumer threads can starve because we haven't specified fair (except for weak fairness of Next to avoid the trivial counter-example of stuttering after Init).\n    \nThis steps introduces the concept of liveness and fairness and how they are expressed in TLA+.\n\n```tla\nChecking temporal properties for the complete state space with 1600 total distinct states at (2020-01-17 16:04:37)\nError: Temporal properties were violated.\n\nError: The following behavior constitutes a counter-example:\n\nState 1: \u003cInitial predicate\u003e\n/\\ buffer = \u003c\u003c\u003e\u003e\n/\\ waitSet = {}\n\nState 2: \u003cNext line 54, col 9 to line 57, col 45 of module BlockingQueue\u003e\n/\\ buffer = \u003c\u003c\u003e\u003e\n/\\ waitSet = {c1}\n\nState 3: \u003cNext line 54, col 9 to line 57, col 45 of module BlockingQueue\u003e\n/\\ buffer = \u003c\u003cp4\u003e\u003e\n/\\ waitSet = {}\n\nBack to state 1: \u003cNext line 54, col 9 to line 57, col 45 of module BlockingQueue\u003e\n```\n\n### v26 (Starvation): Refactor specification to move action enabling condition into Put and Get.\n\nSo far, the spec was written in a way that the ```Put``` and ```Get``` sub-actions of ```Next``` were permanently enabled, i.e. ```\\A p \\in Producers : ENABLED Put(p,p)``` was an invariant of ```Spec``` (vice versa for Consumers). The next-state relation ```\\E t \\in RunningThreads: t \\in Producers /\\ ...``` took care of scheduling \"enabled\" producers only. Here, we refactor the next-state relation and \"push\" the enabling condition into the sub-actions. With this change, ```\\A p \\in Producers : ENABLED Put(p,p)``` is no longer invariant (see trace below). Subsequent steps will show the reason why we refactored the spec. Note however, that this refactoring does not change the set of behaviors defined by ```Spec```.\n\n```tla\nInvariant PutEnabled is violated.\nThe behavior up to this point is:\n1: \u003cInitial predicate\u003e\n/\\ buffer = \u003c\u003c\u003e\u003e\n/\\ waitSet = {}\n2: \u003cPut line 34, col 1 to line 39, col 16 of module BlockingQueue\u003e\n/\\ buffer = \u003c\u003cp1\u003e\u003e\n/\\ waitSet = {}\n3: \u003cPut line 34, col 1 to line 39, col 16 of module BlockingQueue\u003e\n/\\ buffer = \u003c\u003cp1\u003e\u003e\n/\\ waitSet = {p1}\n```\n--------------------------------------------------------------------------\n\n### v25 (Refinement): Implement BlockingQueueSplit spec in Java and C.\n\nKnowing that ```BlockingQueueSplit``` refines ```BlockingQueue``` and thus is deadlock-free, we shift our attention to the Java (and C) program.  Instead of Java's ```synchronized``` statement, we implement BlockingQueueSplit with the help of the low-level synchronization primitive ```java.util.concurrent.locks.ReentrantLock``` and ```java.util.concurrent.locks.Condition```.  Executing the new program for a couple of hours with configuration p2c1b1 reveals no deadlock (the broken version of the program deadlocked within seconds).  This, and the fact that Java's own [```ArrayBlockingQueue```](https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/util/concurrent/ArrayBlockingQueue.html) has a similar implementation, should silence our concerns to put the program into production.\n\n### v24 (Refinement): Prove refinement mapping of BlockingQueueSplit.\n\nBelow, TLC checked the refinement mapping for a finite model/particular configuration, which gives for ```BlockingQueueSplit``` sufficient confidence that the refinement mapping is correct.  The fact that the refinement mapping is straight forward indicates that a TLAPS prove is likely straight forward too. So let's give in to the academic in us and prove the correctness of the refinement mapping.  To prove that ```BlockingQueueSplit``` implements ```BlockingQueue```, we first prove ```TypeInv``` inductive with the now know invariance proof rule.  Once we have proven this LEMMA, we reuse it and the [proof rule for refinement (section 4.2)](https://members.loria.fr/SMerz/papers/tla+logic2008.pdf) to prove ```THEOREM Implements == Spec =\u003e A!Spec```.\n\n### v23 (Refinement): Refine BlockingQueue with BlockingQueueSplit.\n\nThe specs ```BlockingQueueSplit.tla``` and ```BlockingQueue.tla``` look almost identical, except ```BlockingQueueSplit.tla``` has one more variable and slightly different definitions for ```Put```, ```Get```, ```Wait```, and ```NotifyOther```.  However, there is no (logical) connection between the two specs, albeit spec ```BlockingQueueSplit``` can be considered an implementation of ```BlockingQueue```. \n\nThis is where TLA's [secret power](https://news.ycombinator.com/item?id=21669689) comes into play. In an earlier step, we wrote ```THEOREM DeadlockFreedom == Spec =\u003e []Invariant``` to state that ```BlockingQueue``` satisfies ```[]Invariant``` where ```[]Invariant``` was a (safety) property.  However, TLA does not distinguish between properties and (machine) specs; both are \"just\" formulas.  This power means that we may also say that the spec defined in ```BlockingQueueSplit``` satisfies/implements the machine spec ```BlockingQueue``` by stating ```THEOREM Implements == Spec =\u003e BlockingQueue!Spec```.  \n\nWe are almost done. However, since ```BlockingQueueSplit``` differs slightly from ```BlockingQueue```, we have to provide a [refinement mapping](https://lamport.azurewebsites.net/tla/hiding-and-refinement.pdf) that relates the low-level spec to the high-level spec. In the case of ```BlockingQueueSplit```, it is fortunately straight forward: the union of ```waitC``` and ```waitP``` maps to ```waitSet```.\n\nWe can verify the correctness of the refinement mapping (for a finite model) with TLC again.  This time though, TLC does not check an invariant (state formula) but the (temporal) property ```BlockingQueue!Spec```.\n\nSidenote: \"TLA\" above is *not* a typo! Read Lamport's [The Temporal Logic of Actions](https://lamport.azurewebsites.net/pubs/lamport-actions.pdf) to understand why.\n\n### v22 (Refinement): Create BlockingQueueSplit with two sets waitP and waitC.\n\nThe bugfix below exploited the power of ([Zermelo-Fraenkel](https://en.wikipedia.org/wiki/Zermelo%E2%80%93Fraenkel_set_theory)) set theory to get away without changing ```waitSet``` into two (disjoint) sets; one for waiting ```Producers``` and one for waiting ```Consumers``` (we will call them ```waitP``` and ```waitC``` respectively).  In a real-world program, however, the elegance of math is likely too inefficient which is why a program would indeed maintain ```waitP``` and ```waitP``` to avoid intersecting ```Producers``` and ```Consumers``` from ```waitSet``` over and over again (which probably allocates temporary memory too).  In an actual project, we would probably spend no more than a few minutes on analyzing if separating ```waitSet``` into ```waitP``` and ```waitC``` can introduce a deadlock again.  Here, we have the luxury of time and thus write a new spec ```BlockingQueueSplit.tla```.  Fortunately, most of ```BlockingQueueSplit.tla``` is identical to ```BlockingQueue.tla``` which is why we copy\u0026paste from ```BlockingQueue``` before we modify ```NotifyOther```.\n\n### v21 (Traces): Validate long executions against the spec.\n\nThe previous step showed that trace validation is probabilistic and has no guarantees of finding violations of the high-level spec.  Thus, we want to increase the chance by checking a long or many traces.  However, copying long traces into the spec is not only a nuisance, but also slows down [SANY](https://github.com/tlaplus/tlaplus/issues/413#issuecomment-571024785).  This step introduces how to [serialize the app's output](./impl/src/org/kuppe/App2TLA.java) in a format that TLC can de-serialize efficiently with the help of the [IOUtils module](https://github.com/tlaplus/CommunityModules/blob/master/modules/IOUtils.tla).\n\n```bash\njava -XX:StartFlightRecording=disk=true,dumponexit=true,filename=app-$(date +%s).jfr -cp impl/src/ org.kuppe.App\n\n# Kill the process after a while.\n\n# app-XXXXXXX.jfr is the flight recording created by the previous command.\n# app-XXXXXXX.bin is the serialized app output.\njava -cp tla2tools.jar:impl/src/ org.kuppe.App2TLA app-XXXXXXX.jfr app-$(date +%s).bin\n```\n\nWith the longer trace (note the change in ```BlockingQueueTrace.tla```), we are lucky and TLC finds a violation:\n\n```tla\n$ java -cp /opt/TLA+Toolbox/tla2tools.jar:CommunityModules.jar tlc2.TLC BlockingQueueTrace\nTLC2 Version 2.15 of Day Month 20?? (rev: 52e91aa)\nWarning: Please run the Java VM which executes TLC with a throughput optimized garbage collector by passing the \"-XX:+UseParallelGC\" property.\n(Use the -nowarning option to disable this warning.)\nRunning breadth-first search Model-Checking with fp 10 and seed -2300318498630499187 with 1 worker on 4 cores with 5964MB heap and 64MB offheap memory [pid: 32602] (Linux 4.18.0-16-generic amd64, Azul Systems, Inc. 11.0.6 x86_64, MSBDiskFPSet, DiskStateQueue).\nParsing file /home/markus/src/TLA/_specs/models/tutorials/BlockingQueueTLA/BlockingQueueTrace.tla\nParsing file /tmp/TLC.tla\nParsing file /tmp/Sequences.tla\nParsing file /tmp/Naturals.tla\nParsing file /tmp/FiniteSets.tla\nParsing file /tmp/IOUtils.tla\nParsing file /home/markus/src/TLA/_specs/models/tutorials/BlockingQueueTLA/BlockingQueue.tla\nParsing file /home/markus/src/TLA/_specs/models/tutorials/BlockingQueueTLA/TLAPS.tla\nSemantic processing of module Naturals\nSemantic processing of module Sequences\nSemantic processing of module FiniteSets\nSemantic processing of module TLC\nSemantic processing of module IOUtils\nSemantic processing of module TLAPS\nSemantic processing of module BlockingQueue\nSemantic processing of module BlockingQueueTrace\nStarting... (2020-01-25 20:32:19)\nFailed to match TLCExt!AssertError operator override from jar:file:/home/markus/src/TLA/_specs/models/tutorials/BlockingQueueTLA/CommunityModules.jar!/tlc2/overrides/TLCExt.class with signature: \u003cJava Method: public static synchronized tlc2.value.impl.Value tlc2.overrides.TLCExt.assertError(tlc2.tool.impl.Tool,tla2sany.semantic.ExprOrOpArgNode[],tlc2.util.Context,tlc2.tool.TLCState,tlc2.tool.TLCState,int,tlc2.tool.coverage.CostModel)\u003e (no such module).\nFailed to match TLCExt!PickSuccessor operator override from jar:file:/home/markus/src/TLA/_specs/models/tutorials/BlockingQueueTLA/CommunityModules.jar!/tlc2/overrides/TLCExt.class with signature: \u003cJava Method: public static synchronized tlc2.value.impl.Value tlc2.overrides.TLCExt.pickSuccessor(tlc2.tool.impl.Tool,tla2sany.semantic.ExprOrOpArgNode[],tlc2.util.Context,tlc2.tool.TLCState,tlc2.tool.TLCState,int,tlc2.tool.coverage.CostModel)\u003e (no such module).\nLoading IODeserialize operator override from tlc2.overrides.IOUtils with signature: \u003cJava Method: public static final tlc2.value.IValue tlc2.overrides.IOUtils.deserialize(tlc2.value.impl.StringValue,tlc2.value.impl.BoolValue) throws java.io.IOException\u003e.\nLoading IOSerialize operator override from tlc2.overrides.IOUtils with signature: \u003cJava Method: public static final tlc2.value.IValue tlc2.overrides.IOUtils.serialize(tlc2.value.IValue,tlc2.value.impl.StringValue,tlc2.value.impl.BoolValue) throws java.io.IOException\u003e.\nImplied-temporal checking--satisfiability problem has 1 branches.\nComputing initial states...\nFinished computing initial states: 1 distinct state generated at 2020-01-25 20:32:19.\nError: Action property line 75, col 17 to line 75, col 29 of module BlockingQueue is violated.\nError: The behavior up to this point is:\nState 1: \u003cInitial predicate\u003e\n/\\ buffer = \u003c\u003c\u003e\u003e\n/\\ i = 1\n/\\ waitSet = {}\n\nState 2: \u003cwaitC line 149, col 10 to line 154, col 28 of module BlockingQueueTrace\u003e\n/\\ buffer = \u003c\u003c\u003e\u003e\n/\\ i = 2\n/\\ waitSet = {\"c1\"}\n\nState 3: \u003cput line 156, col 8 to line 166, col 73 of module BlockingQueueTrace\u003e\n/\\ buffer = \u003c\u003c\"p1\"\u003e\u003e\n/\\ i = 3\n/\\ waitSet = {}\n\nState 4: \u003cput line 156, col 8 to line 166, col 73 of module BlockingQueueTrace\u003e\n/\\ buffer = \u003c\u003c\"p1\", \"p1\"\u003e\u003e\n/\\ i = 4\n/\\ waitSet = {}\n\nState 5: \u003cget line 168, col 8 to line 173, col 32 of module BlockingQueueTrace\u003e\n/\\ buffer = \u003c\u003c\"p1\"\u003e\u003e\n/\\ i = 5\n/\\ waitSet = {}\n\nState 6: \u003cput line 156, col 8 to line 166, col 73 of module BlockingQueueTrace\u003e\n/\\ buffer = \u003c\u003c\"p1\", \"p1\"\u003e\u003e\n/\\ i = 6\n/\\ waitSet = {}\n\nState 7: \u003cget line 168, col 8 to line 173, col 32 of module BlockingQueueTrace\u003e\n/\\ buffer = \u003c\u003c\"p1\"\u003e\u003e\n/\\ i = 7\n/\\ waitSet = {}\n\nState 8: \u003cput line 156, col 8 to line 166, col 73 of module BlockingQueueTrace\u003e\n/\\ buffer = \u003c\u003c\"p1\", \"p1\"\u003e\u003e\n/\\ i = 8\n/\\ waitSet = {}\n\nState 9: \u003cput line 156, col 8 to line 166, col 73 of module BlockingQueueTrace\u003e\n/\\ buffer = \u003c\u003c\"p1\", \"p1\", \"p1\"\u003e\u003e\n/\\ i = 9\n/\\ waitSet = {}\n\nState 10: \u003cwaitP line 139, col 10 to line 147, col 28 of module BlockingQueueTrace\u003e\n/\\ buffer = \u003c\u003c\"p1\", \"p1\", \"p1\"\u003e\u003e\n/\\ i = 10\n/\\ waitSet = {\"p1\"}\n\nState 11: \u003cget line 168, col 8 to line 173, col 32 of module BlockingQueueTrace\u003e\n/\\ buffer = \u003c\u003c\"p1\", \"p1\"\u003e\u003e\n/\\ i = 11\n/\\ waitSet = {}\n\nState 12: \u003cput line 156, col 8 to line 166, col 73 of module BlockingQueueTrace\u003e\n/\\ buffer = \u003c\u003c\"p1\", \"p1\", \"p1\"\u003e\u003e\n/\\ i = 12\n/\\ waitSet = {}\n\nState 13: \u003cwaitP line 139, col 10 to line 147, col 28 of module BlockingQueueTrace\u003e\n/\\ buffer = \u003c\u003c\"p1\", \"p1\", \"p1\"\u003e\u003e\n/\\ i = 13\n/\\ waitSet = {\"p1\"}\n\nState 14: \u003cwaitP line 139, col 10 to line 147, col 28 of module BlockingQueueTrace\u003e\n/\\ buffer = \u003c\u003c\"p1\", \"p1\", \"p1\"\u003e\u003e\n/\\ i = 14\n/\\ waitSet = {\"p1\", \"p2\"}\n\nState 15: \u003cget line 168, col 8 to line 173, col 32 of module BlockingQueueTrace\u003e\n/\\ buffer = \u003c\u003c\"p1\", \"p1\"\u003e\u003e\n/\\ i = 15\n/\\ waitSet = {\"p1\"}\n\nState 16: \u003cput line 156, col 8 to line 166, col 73 of module BlockingQueueTrace\u003e\n/\\ buffer = \u003c\u003c\"p1\", \"p1\", \"p2\"\u003e\u003e\n/\\ i = 16\n/\\ waitSet = {}\n\n2529 states generated, 1169 distinct states found, 63 states left on queue.\nThe depth of the complete state graph search is 16.\nThe average outdegree of the complete state graph is 1 (minimum is 0, the maximum 4 and the 95th percentile is 4).\nFinished in 01s at (2020-01-25 20:32:20)\n```\n\nConvince yourself that TLC has indeed reported a violation of the high-level spec that is due to single-mutex bug.  Do so by re-running TLC with the two-mutex fix temporarily reverted (TLC reports no error):\n\n```diff\ndiff --git a/BlockingQueue.tla b/BlockingQueue.tla\nindex aba689d..d5d2e41 100644\n--- a/BlockingQueue.tla\n+++ b/BlockingQueue.tla\n@@ -19,7 +19,7 @@ vars == \u003c\u003cbuffer, waitSet\u003e\u003e\n RunningThreads == (Producers \\cup Consumers) \\ waitSet\n \n NotifyOther(t) == \n-          LET S == IF t \\in Producers THEN waitSet \\ Producers ELSE waitSet \\ Consumers\n+          LET S == waitSet\n           IN IF S # {}\n              THEN \\E x \\in S : waitSet' = waitSet \\ {x}\n              ELSE UNCHANGED waitSet\n```\n\nIn a real project, trace validation provides confidence that an implementation faithfully implements its high-level (TLA+) spec.  This is why we want to check as many traces as possible. However, we will never have proof that the implementation correctly implements its spec.  Note that [coverage](https://arxiv.org/abs/1912.10633) of the high-level spec (similar to code coverage measured for unit tests) indicates to what extent the trace explore the state space. \n\n### v20 (Traces): Validate implementation executions against the spec.\n\nThe top of the ```BlockingQueueTrace``` spec defines a ```Trace``` operator that is the the execution that we printed to stdout in the previous step.  The rest of the spec follows [Ron Pressler's](https://pron.github.io/tlaplus) ```Trace3``` method (refinement mapping) described in [\"Verifying Software Traces Against a Formal Specification with TLA+ and TLC\"](https://pron.github.io/files/Trace.pdf).  The comments in ```BlockingQueueTrace``` discuss how non-determinism compensates for the incomplete (actual value of buffer, ...) application log.\n\nChecking the spec shows that TLC found 4410 distinct states even though the trace is only ~107 states long.  This is because of the non-determinism we deliberately introduced.  Surprisingly, however, the implementation trace does not violate the high-level spec ```BlockingQueue```:\n\n```bash\n$ java -cp /opt/TLA+Toolbox/tla2tools.jar:CommunityModules.jar tlc2.TLC BlockingQueueTrace\nTLC2 Version 2.15 of Day Month 20?? (rev: 52e91aa)\nWarning: Please run the Java VM which executes TLC with a throughput optimized garbage collector by passing the \"-XX:+UseParallelGC\" property.\n(Use the -nowarning option to disable this warning.)\nRunning breadth-first search Model-Checking with fp 35 and seed -8345104333501302295 with 1 worker on 4 cores with 5964MB heap and 64MB offheap memory [pid: 27542] (Linux 4.18.0-16-generic amd64, Azul Systems, Inc. 11.0.6 x86_64, MSBDiskFPSet, DiskStateQueue).\nParsing file /home/markus/src/TLA/_specs/models/tutorials/BlockingQueueTLA/BlockingQueueTrace.tla\nParsing file /tmp/TLC.tla\nParsing file /tmp/Sequences.tla\nParsing file /tmp/Naturals.tla\nParsing file /tmp/FiniteSets.tla\nParsing file /home/markus/src/TLA/_specs/models/tutorials/BlockingQueueTLA/BlockingQueue.tla\nParsing file /home/markus/src/TLA/_specs/models/tutorials/BlockingQueueTLA/TLAPS.tla\nSemantic processing of module Naturals\nSemantic processing of module Sequences\nSemantic processing of module FiniteSets\nSemantic processing of module TLC\nSemantic processing of module TLAPS\nSemantic processing of module BlockingQueue\nSemantic processing of module BlockingQueueTrace\nStarting... (2020-01-25 17:48:08)\nFailed to match TLCExt!AssertError operator override from jar:file:/home/markus/src/TLA/_specs/models/tutorials/BlockingQueueTLA/CommunityModules.jar!/tlc2/overrides/TLCExt.class with signature: \u003cJava Method: public static synchronized tlc2.value.impl.Value tlc2.overrides.TLCExt.assertError(tlc2.tool.impl.Tool,tla2sany.semantic.ExprOrOpArgNode[],tlc2.util.Context,tlc2.tool.TLCState,tlc2.tool.TLCState,int,tlc2.tool.coverage.CostModel)\u003e (no such module).\nFailed to match TLCExt!PickSuccessor operator override from jar:file:/home/markus/src/TLA/_specs/models/tutorials/BlockingQueueTLA/CommunityModules.jar!/tlc2/overrides/TLCExt.class with signature: \u003cJava Method: public static synchronized tlc2.value.impl.Value tlc2.overrides.TLCExt.pickSuccessor(tlc2.tool.impl.Tool,tla2sany.semantic.ExprOrOpArgNode[],tlc2.util.Context,tlc2.tool.TLCState,tlc2.tool.TLCState,int,tlc2.tool.coverage.CostModel)\u003e (no such module).\nWarning: Failed to match IODeserialize operator override from IOUtils with signature: public static final tlc2.value.IValue tlc2.overrides.IOUtils.deserialize(tlc2.value.impl.StringValue,tlc2.value.impl.BoolValue) throws java.io.IOException (no such module).\nWarning: Failed to match IOSerialize operator override from IOUtils with signature: public static final tlc2.value.IValue tlc2.overrides.IOUtils.serialize(tlc2.value.IValue,tlc2.value.impl.StringValue,tlc2.value.impl.BoolValue) throws java.io.IOException (no such module).\nImplied-temporal checking--satisfiability problem has 1 branches.\nComputing initial states...\nFinished computing initial states: 1 distinct state generated at 2020-01-25 17:48:08.\nProgress(109) at 2020-01-25 17:48:09: 8,424 states generated, 4,410 distinct states found, 0 states left on queue.\nChecking temporal properties for the complete state space with 4410 total distinct states at (2020-01-25 17:48:09)\nFinished checking temporal properties in 00s at 2020-01-25 17:48:09\nModel checking completed. No error has been found.\n  Estimates of the probability that TLC did not check all reachable states\n  because two distinct states had the same fingerprint:\n  calculated (optimistic):  val = 9.6E-13\n8424 states generated, 4410 distinct states found, 0 states left on queue.\nThe depth of the complete state graph search is 109.\nThe average outdegree of the complete state graph is 1 (minimum is 0, the maximum 4 and the 95th percentile is 4).\nFinished in 01s at (2020-01-25 17:48:09)\n```\n\n### v19 (Traces): Print (partial) implementation executions.\n\nHaving finished the proof of the deadlock fix below, we shift our attention to the (Java) implementation that we assume can still deadlock.  However, before we apply the fix of two mutexes, we use TLC to check if the implementation allows executions that violate the ```BlockingQueue``` spec.  In other words, we check if the implementation correctly implements the TLA+ spec (which we know it does not).  To do so, we print an execution to stdout with the help of the low-overhead [Java Flight Recorder](https://en.wikipedia.org/wiki/JDK_Flight_Recorder) that we can consider a very powerful logging framework.  However, plain logging to stdout - with a high-precision timestamp - would have worked too.   To activate JFR, run the app with:\n\n```bash\njava -XX:StartFlightRecording=disk=true,dumponexit=true,filename=app-$(date +%s).jfr -cp impl/src/ org.kuppe.App\n\n# Kill the process after a few seconds.\n\n# app-1580000457.jfr is the flight recording created by the previous command.\njava -cp impl/src/ org.kuppe.App2TLA app-1580000457.jfr \n[ op |-\u003e \"w\", waiter |-\u003e \"c\" ],\n[ op |-\u003e \"e\" ],\n[ op |-\u003e \"e\" ],\n[ op |-\u003e \"d\" ],\n[ op |-\u003e \"e\" ],\n[ op |-\u003e \"d\" ],\n[ op |-\u003e \"d\" ],\n[ op |-\u003e \"e\" ],\n[ op |-\u003e \"e\" ],\n[ op |-\u003e \"d\" ],\n[ op |-\u003e \"d\" ],\n[ op |-\u003e \"e\" ],\n[ op |-\u003e \"e\" ],\n[ op |-\u003e \"e\" ],\n[ op |-\u003e \"d\" ],\n[ op |-\u003e \"d\" ],\n[ op |-\u003e \"e\" ],\n[ op |-\u003e \"e\" ],\n[ op |-\u003e \"w\", waiter |-\u003e \"p\" ],\n[ op |-\u003e \"d\" ],\n[ op |-\u003e \"e\" ],\n\n```\n\nIt is important to observe that the log statements only log the operation (deq|enq|wait) and the thread that calls wait. The log does not contain the size or content of the buffer, the thread that gets notified, or the thread that executes.\n\n(JFR requires Java 11 or newer).\n\n--------------------------------------------------------------------------\n\n### v18 (TLAPS): Co-domain of buffer not relevant for proof.\n\nNo need to say anything about the co-domain of buffer in the proof.  What is enqueued\nand dequeued has no relevance with regards to deadlock freedom.\n\nHowever, ```IInv``` by itself no longer completely specifies the (function) buffer, thus ```MCIInv``` has been added.\n\nPart I of Lamport's paper [Proving Safety Properties](https://lamport.azurewebsites.net/tla/proving-safety.pdf) has more exercises about finding and proving inductive invariants. \n\n### v17 (TLAPS): TLAPS proves DeadLockFreedom.\n\nTLAPS has no problem proving DeadlockFreedom.\n\n![TLAPS proves IInv](./screencasts/v14-IInvWin.gif)\n\n### v16 (TLAPS): Validating an inductive invariant candidate.\n\nFinding an inductive invariant is hard!  Fortunately, we can use TLC to - little by little - find and validate potential inductive invariants.  Some deeper thinking is at play too though:\n\n![IInv with TLC](./screencasts/v13-IInvTLC.gif)\n\nNote that we rewrite the last conjunct of ```TypeInv``` to ```\\in SUBSET```.  Check\nthe [discussion group](http://discuss.tlapl.us/msg00619.html) for the technical reason why.  We also re-define the Seq operator because its definition in the Sequences \nstandard module is not enumerable.  We only re-define Seq in BlockingQueue.tla because the vscode extensions doesn't have a model editor yet.  Generally though, we would do this in the model to not taint the actual spec.\n\n### v15 (TLAPS): Finding the inductive invariant.\n\nThe previous step was easy and straight forward!  Now comes the hard part: Finding\nthe inductive invariant that implies deadlock freedom.\n\nUnfortunately, naive approach does not work: ![Naive IInv](./screencasts/v12-IInv.gif)\n\nInvariant is not *inductive*, because ```Invariant /\\ Next_vars =\u003e Invariant'``` does not\nhold for e.g. the ```[buffer = \u003c\u003c\u003e\u003e /\\ Wait(t)]_vars``` step (assum we are in the \n(unreachable) state with ```RunnintThreads = {t}```, the successor state will have\n```RunningThreads' = {} /\\ waitSet' = (Producers \\cup Consumers)```.  A similar argument\napplies to a ```[Len(buffer) = BufCapacity /\\ Wait(t)]_vars``` step.\n\n### v14 (TLAPS): Proof that TypeInv is inductive.\n\nRecall that the deadlock originally only happened iff \n```2*BufCapacity \u003c Cardinality(Producers \\cup Consumers)```.  How do we know\nthat the solution with two mutexes hasn't a similar flaw, just for a different\ninequation?  All the model-checking in the world won't gives us absolute\nconfidence, because the domains for Producers, Consumers, and BufCapacity are\ninfinite.  We have to prove that the solution with two mutexes is correct for all configurations no matter what values we chose for producer, consumer, or BufCapacity.\n\nHere, we will use [TLAPS](https://tla.msr-inria.inria.fr/tlaps/content/Home.html) to take the first step towards an invariance proof of deadlock freedom by proving that ```TypeInv``` is inductive.  In the screencast below, TLAPS first checks the QED step,\nthen the 1\u003c1\u003e and 1\u003c2\u003e steps, and finally the top-level THEOREM.\n\n![Invoke TLAPS](./screencasts/v11-TypeInv.gif)\n\nNote that this step and the following ones require the TLA+ Toolbox and TLAPS! If\nyou don't have the Toolbox or TLAPS, you can skip this part (TLAPS.tla has been added to this repository to avoid parser errors).\n\n--------------------------------------------------------------------------\n\n### v13 (Bugfix): (Logically) two mutexes.\n\nRemove notifyAll and instead introduce two mutexes (one for Producers\nand one for Consumers).  A Consumer will notify the subset of\nProducers waiting on the Producer mutex (vice versa for a Producer).\n    \nThe spec does not have to introduce two mutexes.  Instead, we can\njust pick the right thread type from the set of waiting threads.\n    \nThis fix completely solves the bug, but are we fully satisfied yet?\n\n### v12 (Bugfix): Always notify all waiting threads.\n\nAlways notify all waiting threads instead of a non-deterministically\nselected one.  This fixes the deadlock bug but at a price: Load will\nspike when all (suspended) threads wake up at once.\n\nAs a bonus exercise, check if it necessary to notify all waiting threads in both ```Put``` and ```Get```.\n\nNote that this is the proposed solution to the bug in [Challenge 14](http://wiki.c2.com/?ExtremeProgrammingChallengeFourteen) of the c2 extreme programming wiki.  To the best of my knowledge, not a single comment mentions that just one ```notifyAll``` suffices. Neither does anybody mention a more elegant fix that has no performance implications (see next step).\n\n### v11 (Bugfix): Non-deterministically notify waiting threads.\n\nNon-deterministically notify waiting threads in an attempt to\nfix the deadlock situation.  This attempt fails because we\nmight end up waking the wrong thread up over and over again.\n\n--------------------------------------------------------------------------\n\n### v10 (Animation): Add visualization of error-traces.\n    \nThe trace with 2 procs, 1 cons, and a buffer of length 1 is\nminimal (length 8).  The other trace shows how quickly the\nissue becomes a) incomprehensible and b) the length of the\ntrace increases (46 states for 7 threads).  TLC takes 75 secs\non my machine to check this.\n\n![Animation for configuration p2c1b1](animation/BlockingQueue-Proc2_Cons1_Buff1_Len08.gif)\n\n![Animation for configuration p4c3b3](animation/BlockingQueue-Proc4_Cons3_Buff3_Len46.gif)\n\nThe animations are created in the Toolbox with:\n\n1. Check model [BlockingQueue.cfg](BlockingQueue.cfg)\n2. Set ```Animation``` as the trace expression in the Error-Trace console\n3. Hit Explore and export/copy the resulting trace to clipboard\n4. Paste into http://localhost:10996/files/animator.html\n\nWithout the Toolbox, something similar to this:\n\n1. Check model [BlockingQueue.cfg](BlockingQueue.cfg) with ```java -jar tla2tools.jar -deadlock -generateSpecTE BlockingQueue``` ('-generateSpecTE' causes TLC to generate [SpecTE.tla](SpecTE.tla)/.cfg)\n2. State trace expression ```Animation``` ([BlockingQueueAnim.tla](BlockingQueueAnim.tla))in SpecTE.tla\n3. Download https://github.com/tlaplus/CommunityModules/releases/download/20200107.1/CommunityModules-202001070430.jar\n4. Check SpecTE with ```java -jar tla2tools.jar:CommunityModules-202001070430.jar tlc2.TLC SpecTE```\n5. Copy trace into http://localhost:10996/files/animator.html (minor format changes needed)\n\n### v09 (Animation): Add prophecy variable to simplify animation.\n\nThe next-state relation has been restated to \"predict\"\nthe value of t (threads) in the successor state. We\nwill use the prophecy variable in the following\ncommit to come up with an animation.\n    \nThis and the following commit can be skipped unless\nyou are interested in the more advanced concept of\nprophecy (http://lamport.azurewebsites.net/pubs/auxiliary.pdf)\nor animations (https://youtu.be/mLF220fPrP4).\n\n--------------------------------------------------------------------------\n\n### v08b (trace): Check configurations up to 20 processes and buffer capacity 10.\n\nThe state-space reduction due to the introduction of the view in the previous commits, allows TLC to check the spec for larger configurations, with which we can infer the error-trace length.\n\n```bash\njava -jar /opt/TLA+Toolbox/tla2tools.jar -deadlock -note SimBlockingQueue\n```\n\n![SimBlockingQueue](./R/SimBlockingQueue.svg)\n\n### v08a (view): Define a view that abstracts buffer into a counter.\n\nWe exploit the insight that the order of elements in the (fifo) buffer is irrelevant for the correctness of the algorithm.  In other words, we can abstract the buffer into a simple counter of elements.  With this abstraction, the state-space for the current config shrinks from 2940 to 1797 distinct states. \n\n### v08 (continue): Infer inequation under which the system is deadlock free.\n\nBased on the scaffolding in the two previous steps, we run TLC with the [```-continue```](https://lamport.azurewebsites.net/tla/tlc-options.html?back-link=tools.html) option to not stop state space exploration after a violation of the invariant has been found.  In other words, we ask TLC to find all violations, not just one of the shortest ones (Breadth-First search guarantees that TLC finds the shortest counterexample first).\n\n```bash\njava -jar /opt/TLA+Toolbox/tla2tools.jar -deadlock -continue BlockingQueue | grep InvVio | sort | uniq\n\u003c\u003c\"InvVio\", 1, 3\u003e\u003e\n\u003c\u003c\"InvVio\", 1, 4\u003e\u003e\n\u003c\u003c\"InvVio\", 1, 5\u003e\u003e\n\u003c\u003c\"InvVio\", 1, 6\u003e\u003e\n\u003c\u003c\"InvVio\", 1, 7\u003e\u003e\n\u003c\u003c\"InvVio\", 1, 8\u003e\u003e\n\u003c\u003c\"InvVio\", 2, 5\u003e\u003e\n\u003c\u003c\"InvVio\", 2, 6\u003e\u003e\n\u003c\u003c\"InvVio\", 2, 7\u003e\u003e\n\u003c\u003c\"InvVio\", 2, 8\u003e\u003e\n\u003c\u003c\"InvVio\", 3, 7\u003e\u003e\n\u003c\u003c\"InvVio\", 3, 8\u003e\u003e\n```\n\nA little bit of bashery trims TLC's output so that we - with some squinting -  notice that ```BlockingQueue``` is deadlock free iff ```2*BufCapacity \u003e= Cardinality(Producers \\cup Consumers)```.  A simple [R plot](./R/ContinueInequation.R) makes this even more visible ('d' indicates a configuration that deadlocks):\n\n![ContinueInequation](./R/ContinueInequation.svg)\n\nCollecting even more [data](./R/TraceLength.csv), we can [correlate](./R/TraceLength.R) the length of the error trace with the constants (```Cardinality(Producers)```, ```Cardinality(Consumers)```, ```BufCapacity```, and ```Cardinality(Producers \\cup Consumers)```):\n\n![TraceLengthCorrelation](./R/TraceLengthCorrelation.svg) ([ggcorrplot](http://www.sthda.com/english/wiki/ggcorrplot-visualization-of-a-correlation-matrix-using-ggplot2))\n\n\n### v07 (continue): Declare Producers and Consumers to be symmetry sets.\n\nThe sets ```Producers``` and ```Consumers``` are symmetry sets for the ```BlockingQueue``` specification, meaning that permuting the elements in the sets does not change whether or not a behavior satisfies that behavior spec.  TLC can take advantage of this to reduce the number of (distinct) states it has to examine from 57254 to 1647!\n\nNote that TLC does not check if a set you declare to be a symmetry set really is one.  If you declare a set to be a symmetry set and it isn't, then TLC can fail to find an error that it otherwise would find.  An expression is symmetric for a set ```S``` if and only if interchanging any two values of ```S``` does not change the value of the expression.  The expression ```{{v1, v2}, {v1, v3}, {v2, v3}}``` is symmetric for the set ```{v1, v2, v3}``` -- for example, interchanging ```v1``` and ```v3``` in this expression produces ```{{v3, v2}, {v3, v1}, {v2, v1}}```, which is equal to the original expression.  You should declare a set S of model values to be a symmetry set only if the specification and all properties you are checking are symmetric for S after the substitutions for constants and defined operators specified by the model are made.  For example, you should not declare ```{v1, v2, v3}``` to be a symmetry set if the model substitutes v1 for some constant.  The only TLA+ operator that can produce a non-symmetric expression when applied to a symmetric expression is CHOOSE.  For example, the expression\n\n```tla\nCHOOSE x \\in {v1, v2, v3} : TRUE\n```\n\nis not symmetric for ```{v1, v2, v3}```. \n\nSymmetry sets should not be used when checking liveness properties.  Doing so can make TLC fail to find errors, or to report nonexistent errors.  The Toolbox adds a warning to the model when you try this.\n\n(The description of this step originates from https://tla.msr-inria.inria.fr/tlatoolbox/doc/model/model-values.html)\n\n### v06 (continue): Convert constants into variables.\n\nIn the section [\"Limitations of Model-Checking\"](http://www.cs.unh.edu/~charpov/programming-tlabuffer.html), Michel Charpentier points out that ```BlockingQueue``` is deadlock-free under some configurations, but that model checking is not helpful with finding the underlying mathematical function.  This observation is true in general because we cannot ask TLC to compute the set of all configurations for which ```BlockingQueue``` is deadlock-free, but at least we can ask it to find as many data points as possible. From those data points, we can try to infer/learn the function.\n\nIn this step, we rewrite ```BlockingQueue``` to check multiple configurations instead of a single one (p1c2b1) at once. Note that the rewrite increases the complete state space to 57254 distinct states, but TLC continues to find the behavior shown in the previous step. This is because TLC - by default - explores the state space with [breadth-first search](https://en.wikipedia.org/wiki/Breadth-first_search). This [search mode](https://tla.msr-inria.inria.fr/tlatoolbox/doc/model/tlc-options-page.html#checking) guarantees to always find the shortest counterexample (if TLC runs ```-workers N``` with N \u003e 1, it only returns the shortest counterexample with high probability).\n\nWe hope to [better support checking different constant values](https://github.com/tlaplus/tlaplus/issues/272) in the future.\n\n--------------------------------------------------------------------------\n\n### v05: Add Invariant to detect deadlocks.\n\nAdd Invariant to detect deadlocks (and TypeInv). TLC now finds the deadlock\nfor configuration p1c2b1 (see below) as well as the one matching the Java\napp p4c3b3.\n\n```tla\nError: Invariant Invariant is violated.\nError: The behavior up to this point is:\nState 1: \u003cInitial predicate\u003e\n/\\ buffer = \u003c\u003c\u003e\u003e\n/\\ waitSet = {}\n\nState 2: \u003cNext line 52, col 9 to line 55, col 45 of module BlockingQueue\u003e\n/\\ buffer = \u003c\u003c\u003e\u003e\n/\\ waitSet = {c1}\n\nState 3: \u003cNext line 52, col 9 to line 55, col 45 of module BlockingQueue\u003e\n/\\ buffer = \u003c\u003c\u003e\u003e\n/\\ waitSet = {c1, c2}\n\nState 4: \u003cNext line 52, col 9 to line 55, col 45 of module BlockingQueue\u003e\n/\\ buffer = \u003c\u003cp1\u003e\u003e\n/\\ waitSet = {c2}\n\nState 5: \u003cNext line 52, col 9 to line 55, col 45 of module BlockingQueue\u003e\n/\\ buffer = \u003c\u003cp1\u003e\u003e\n/\\ waitSet = {p1, c2}\n\nState 6: \u003cNext line 52, col 9 to line 55, col 45 of module BlockingQueue\u003e\n/\\ buffer = \u003c\u003c\u003e\u003e\n/\\ waitSet = {p1}\n\nState 7: \u003cNext line 52, col 9 to line 55, col 45 of module BlockingQueue\u003e\n/\\ buffer = \u003c\u003c\u003e\u003e\n/\\ waitSet = {p1, c1}\n\nState 8: \u003cNext line 52, col 9 to line 55, col 45 of module BlockingQueue\u003e\n/\\ buffer = \u003c\u003c\u003e\u003e\n/\\ waitSet = {p1, c1, c2}\n```\n\nNote that the Java app with p2c1b1 usually deadlocks only after it produced thousands of lines of log statements, which is considerably longer than the error trace above.  This makes it more difficult to understand the root cause of the deadlock.  For config p4c3b3, the C program has a high chance to deadlock after a few minutes and a couple million cycles of the consumer loop. \n\nSidenote: Compare the complexity of the behavior described in [Challenge 14](http://wiki.c2.com/?ExtremeProgrammingChallengeFourteenTheBug) of the c2 extreme programming wiki for configuration p2c2b1 with the TLA+ behavior below.  The explanation in the wiki requires 15 steps, whereas - for p2c2b1 - TLC already finds a deadlock after 9 states (and two more after 11 states).\n\n```tla\nInvariant Invariant is violated.\nThe behavior up to this point is:\n1: \u003cInitial predicate\u003e\n/\\ buffer = \u003c\u003c\u003e\u003e\n/\\ waitSet = {}\n2: \u003cNext line 53, col 9 to line 56, col 45 of module BlockingQueue\u003e\n/\\ buffer = \u003c\u003c\u003e\u003e\n/\\ waitSet = {c2}\n3: \u003cNext line 53, col 9 to line 56, col 45 of module BlockingQueue\u003e\n/\\ buffer = \u003c\u003c\u003e\u003e\n/\\ waitSet = {c1, c2}\n4: \u003cNext line 53, col 9 to line 56, col 45 of module BlockingQueue\u003e\n/\\ buffer = \u003c\u003cp1\u003e\u003e\n/\\ waitSet = {c2}\n5: \u003cNext line 53, col 9 to line 56, col 45 of module BlockingQueue\u003e\n/\\ buffer = \u003c\u003cp1\u003e\u003e\n/\\ waitSet = {p1, c2}\n6: \u003cNext line 53, col 9 to line 56, col 45 of module BlockingQueue\u003e\n/\\ buffer = \u003c\u003cp1\u003e\u003e\n/\\ waitSet = {p1, p2, c2}\n7: \u003cNext line 53, col 9 to line 56, col 45 of module BlockingQueue\u003e\n/\\ buffer = \u003c\u003c\u003e\u003e\n/\\ waitSet = {p1, p2}\n8: \u003cNext line 53, col 9 to line 56, col 45 of module BlockingQueue\u003e\n/\\ buffer = \u003c\u003c\u003e\u003e\n/\\ waitSet = {p1, p2, c1}\n9: \u003cNext line 53, col 9 to line 56, col 45 of module BlockingQueue\u003e\n/\\ buffer = \u003c\u003c\u003e\u003e\n/\\ waitSet = {p1, p2, c1, c2}\n```\n\n### v04: Debug state graph for configuration p2c1b1.\n    \nIn the previous step, we looked at the graphical representation of the state\ngraph.  With the help of TLCExt!PickSuccessor we build us a debugger\nwith which we study the state graph interactively.  We learn that with\nconfiguration p2c1b1 there are two deadlock states:\n\n![PickSuccessor](./screencasts/v04-PickSuccessor.gif)\n\nThe [CommunityModules](https://github.com/tlaplus/CommunityModules) release has to be added to TLC's command-line:\n\n```\njava -cp tla2tools.jar:CommunityModules.jar tlc2.TLC -deadlock BlockingQueue\n```\n\nNote that TLC's ```-continue``` flag would have also worked to find both\ndeadlock states.\n\n### v03: State graph for configurations p1c2b1 and p2c1b1.\n    \nSlightly larger configuration with which we can visually spot the\ndeadlock: ![p1c2b1](./p1c2b1.svg).\n\nBlockingQueueDebug.tla/.cfg shows how to interactively explore a\nstate graph for configuration p2c1b1 with TLC in combination with\nGraphViz (xdot):\n\n![Explore state graph](./screencasts/v03-StateGraph.gif)\n\n```\njava -jar tla2tools.jar -deadlock -dump dot,snapshot p2c1b1.dot BlockingQueueDebug\n```\n\n### v02: State graph for minimum configuration p1c1b1.\n    \nInitial TLA+ spec that models the existing (Java) code with all its\nbugs and shortcomings.\n    \nThe model uses the minimal parameters (1 producer, 1 consumer, and\na buffer of size one) possible.  When TLC generates the state graph with\n```java -jar tla2tools.jar -deadlock -dump dot p1c1b1.dot BlockingQueue```,\nwe can visually verify that no deadlock is possible with this\nconfiguration: ![p1c1b1](./p1c1b1.svg).\n\n### v01: Java and C implementations with configuration p4c3b3.\n    \nLegacy Java code with all its bugs and shortcomings.  At this point\nin the tutorial, we only know that the code can exhibit a deadlock,\nbut we don't know why.\n    \nWhat we will do is play a game with the universe (non-determinism).\nLaunch the Java app with ```java -cp impl/src/ org.kuppe.App``` in\nthe background and follow along with the tutorial.  If the Java app\ndeadlocks before you finish the tutorial, the universe wins.\n\n(For the c-affine among us, ```impl/producer_consumer.c``` is a C implementation of the blocking buffer sans most of the logging).\n\n### v00: IDE setup, nothing to see here.\n    \nAdd IDE setup for VSCode online and gitpod.io.\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flemmy%2Fblockingqueue","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flemmy%2Fblockingqueue","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flemmy%2Fblockingqueue/lists"}