{"id":33237168,"url":"https://github.com/melisgl/journal","last_synced_at":"2026-01-25T02:05:40.194Z","repository":{"id":40947664,"uuid":"292876864","full_name":"melisgl/journal","owner":"melisgl","description":"A Common Lisp library for logging, tracing, testing and persistence.","archived":false,"fork":false,"pushed_at":"2025-12-21T10:35:44.000Z","size":452,"stargazers_count":44,"open_issues_count":0,"forks_count":4,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-12-23T02:46:09.066Z","etag":null,"topics":["logging","persistence","testing","tracing"],"latest_commit_sha":null,"homepage":"https://melisgl.github.io/mgl-pax-world/journal-manual.html","language":"Common Lisp","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/melisgl.png","metadata":{"files":{"readme":"README","changelog":null,"contributing":null,"funding":null,"license":"COPYING","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2020-09-04T15:02:19.000Z","updated_at":"2025-12-21T10:35:48.000Z","dependencies_parsed_at":"2024-05-02T11:07:37.431Z","dependency_job_id":"0fbf1ae0-d960-41ff-9fbf-ad81f84b6c6d","html_url":"https://github.com/melisgl/journal","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/melisgl/journal","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/melisgl%2Fjournal","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/melisgl%2Fjournal/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/melisgl%2Fjournal/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/melisgl%2Fjournal/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/melisgl","download_url":"https://codeload.github.com/melisgl/journal/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/melisgl%2Fjournal/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28741649,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-25T01:40:51.112Z","status":"online","status_checked_at":"2026-01-25T02:00:06.841Z","response_time":113,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["logging","persistence","testing","tracing"],"created_at":"2025-11-16T19:00:28.322Z","updated_at":"2026-01-25T02:05:40.184Z","avatar_url":"https://github.com/melisgl.png","language":"Common Lisp","readme":"# Journal manual\n\n###### \\[in package JOURNAL with nicknames JRN\\]\n- [system] \"journal\"\n - _Version:_ 0.1.0\n - _Description:_ A library built around explicit execution traces for\n logging, tracing, testing and persistence.\n - _Licence:_ MIT, see COPYING.\n - _Author:_ Gábor Melis \u003cmega@retes.hu\u003e\n - _Homepage:_ [http://github.com/melisgl/journal](http://github.com/melisgl/journal)\n - _Bug tracker:_ [http://github.com/melisgl/journal/issues](http://github.com/melisgl/journal/issues)\n - _Source control:_ [GIT](https://github.com/melisgl/journal.git)\n - *Depends on:* alexandria, bordeaux-threads, local-time, mgl-pax, osicat(?), sb-posix(?), trivial-features, trivial-garbage\n\n## Links\n\nHere is the [official repository](https://github.com/melisgl/journal)\nand the [HTML\ndocumentation](http://melisgl.github.io/mgl-pax-world/journal-manual.html)\nfor the latest version.\n\n## Portability\n\nTested and supported on on ABCL, CCL, CMUCL, ECL, and SBCL.\nAllegroCL Express edition runs out of heap while running the tests.\nOn Lisps that seem to lack support for disabling and enabling of\ninterrupts, such as ABCL, durability is compromised, and any attempt\nto SYNC-JOURNAL (see @SYNCHRONIZATION-STRATEGIES and @SAFETY) will\nbe a runtime error.\n\nJournal depends on BORDEAUX-THREADS. Consequently, it does not load\non implementations without real thread such as CLISP.\n\n## Background\n\nLogging, tracing, testing, and persistence are about what happened\nduring code execution. Recording machine-readable logs and traces\ncan be repurposed for white-box testing. More, when the code is\nrerun, selected frames may return their recorded values without\nexecuting the code, which could serve as a @MOCK-OBJECT framework\nfor writing tests. This ability to isolate external interactions and\nto reexecute traces is sufficient to reconstruct the state of a\nprogram, achieving simple persistence not unlike a @JOURNALING-FS or\n@EVENT-SOURCING.\n\nJournal is the library to log, trace, test and persist. It has a\nsingle macro at its heart: JOURNALED, which does pretty much what\nwas described. It can be thought of as generating two events around\nits body: one that records the name and an argument list (as in a\nfunction call), and another that records the return values. In\nLisp-like pseudocode:\n\n```\n(defmacro journaled (name args \u0026body body)\n `(progn\n (record-event `(:in ,name :args ,args))\n (let ((,return-values (multiple-value-list (progn ,@body))))\n (record-event `(:out ,name :values ,return-values))\n (values-list ,return-values))))\n```\n\nThis is basically how recording works. When replaying events from a\nprevious run, the return values of BODY can be checked against the\nrecorded ones, or we may return the recorded values without even\nrunning BODY.\n\nIn summary, we can produce selective execution traces by wrapping\ncode in JOURNALED and use those traces for various purposes. The\nJournal library is this idea taken to its logical conclusion.\n\n## Distinguishing features\n\n##### As a logging facility\n\n- Nested contexts and single messages\n\n- Customizable content and format\n\n- Human- or machine-readable output\n\n```\n#68200.234: (\"some-context\")\n#68200.234: Informative log message\n#68200.250: =\u003e NIL\n```\n\nSee @LOGGING for a complete example.\n\n##### Compared to CL:TRACE\n\n- Ability to handle non-local exits\n\n- Customizable content and format\n\n- Optional timestamps, internal real- and run-time\n\n```\n(FOO 2.1)\n (1+ 2.1)\n =\u003e 3.1\n=E \"SIMPLE-ERROR\" \"The assertion (INTEGERP 3.1) failed.\"\n```\n\nSee @TRACING for a complete example.\n\n##### As a test framework\n\n- White-box testing based on execution traces\n\n- Isolation of external dependencies\n\n- Record-and-replay testing\n\n```\n(define-file-bundle-test (test-user-registration :directory \"registration\")\n (let ((username (replayed (\"ask-username\")\n (format t \"Please type your username: \")\n (read-line))))\n (add-user username)\n (assert (user-exists-p username))))\n```\n\nSee @TESTING for a complete example.\n\n##### As a solution for persistence\n\n- Event Sourcing: replay interactions with the external world\n\n- Unchanged control flow\n\n- Easy to implement history, undo\n\n```\n(defun my-resumable-autosaving-game-with-history ()\n (with-bundle (bundle)\n (play-guess-my-number)))\n```\n\nSee @PERSISTENCE for a complete example.\n\n## Basics\n\nThe JOURNALED macro does both recording and replaying of events,\npossibly at the same time. Recording is easy: events generated by\nJOURNALED are simply written to a journal, which is a sequence of\nevents much like a file. What events are generated is described in\nJOURNALED. @REPLAY is much more involved, thus it gets its own\nsection. The journals used for recording and replaying are specified\nby WITH-JOURNALING or by WITH-BUNDLE.\n\nThe @JOURNALS-REFERENCE is presented later, but for most purposes,\ncreating them (e.g. with MAKE-IN-MEMORY-JOURNAL, MAKE-FILE-JOURNAL)\nand maybe querying their contents with LIST-EVENTS will suffice.\nSome common cases of journal creation are handled by the convenience\nfunction TO-JOURNAL.\n\nBuilt on top of journals, @BUNDLES juggle repeated replay-and-record\ncycles focussing on persistence.\n\n- [generic-function] TO-JOURNAL DESIGNATOR\n\n Return the journal designated by DESIGNATOR or\n signal an error. The default implementation\n \n - returns DESIGNATOR itself if it is of type JOURNAL,\n \n - returns a new IN-MEMORY-JOURNAL if DESIGNATOR is T,\n \n - returns a new FILE-JOURNAL if DESIGNATOR is a PATHNAME.\n\n- [macro] WITH-JOURNALING (\u0026KEY RECORD REPLAY REPLAY-EOJ-ERROR-P) \u0026BODY BODY\n\n Turn recording and/or replaying of events on or off for the\n duration of BODY. Both RECORD and REPLAY should be a JOURNAL\n designator (in the sense of TO-JOURNAL) or NIL.\n \n If RECORD designates a JOURNAL, then events generated by enclosed\n JOURNALED @BLOCKs are written to that journal (with exceptions, see\n the LOG-RECORD argument of JOURNALED). If REPLAY designates a\n JOURNAL, then the generated events are matched against events from\n that journal according to the rules of @REPLAY.\n \n A JOURNAL-ERROR is signalled if RECORD is a JOURNAL that has been\n previously recorded to by another WITH-JOURNALING (that is, if its\n JOURNAL-STATE is not :NEW) or if REPLAY is a JOURNAL that is not a\n complete recording of successful replay (i.e. its JOURNAL-STATE is\n not :COMPLETED). These checks are intended to catch mistakes that\n would render the new or existing records unusable for replay. When\n WITH-JOURNALING finishes, the RECORD journal is marked :COMPLETED or\n :FAILED in its JOURNAL-STATE.\n \n REPLAY-EOJ-ERROR-P controls whether END-OF-JOURNAL is signalled when\n a new event is being matched to the replay journal from which there\n are no more events to read. If there was a JOURNALING-FAILURE or a\n REPLAY-FAILURE during execution, then END-OF-JOURNAL is not\n signalled.\n \n If BODY completes successfully, but REPLAY has unprocessed events,\n then REPLAY-INCOMPLETE is signalled.\n \n WITH-JOURNALING for different RECORD journals can be nested and run\n independently.\n\n- [glossary-term] block\n\n A journaled block, or simply block, is a number of forms wrapped in\n JOURNALED. When a block is executed, a @FRAME is created.\n\n- [glossary-term] frame\n\n A frame is an IN-EVENT, OUT-EVENT pair, which are created when a\n @BLOCK is entered and left, respectively.\n\n- [function] RECORD-JOURNAL\n\n Return the JOURNAL in which events are currently being\n recorded (see WITH-JOURNALING and WITH-BUNDLE) or NIL.\n\n- [function] REPLAY-JOURNAL\n\n Return the JOURNAL from which events are currently being\n replayed (see WITH-JOURNALING and WITH-BUNDLE) or NIL.\n\n- [macro] JOURNALED (NAME \u0026KEY (LOG-RECORD :RECORD) VERSION ARGS VALUES CONDITION INSERTABLE REPLAY-VALUES REPLAY-CONDITION) \u0026BODY BODY\n\n JOURNALED generates events upon entering and leaving the dynamic\n extent of BODY (also known as the journaled @BLOCK), which we call\n the @IN-EVENTS and @OUT-EVENTS. Between generating the two events,\n BODY is typically executed normally (except for\n @REPLAYING-THE-OUTCOME).\n \n Where the generated events are written is determined by the :RECORD\n argument of the enclosing WITH-JOURNALING. If there is no enclosing\n WITH-JOURNALING and LOG-RECORD is NIL, then event recording is\n turned off and JOURNALED imposes minimal overhead.\n \n - NAME can be of any type except NULL, not evaluated. For\n names, and for anything that gets written to a journal, a\n non-keyword symbol is a reasonable choice as it can be easily made\n unique. However, it also exposes the package structure, which\n might make reading stuff back more difficult. Keywords and strings\n do not have this problem.\n \n - ARGS can be of any type, but is typically a list.\n \n Also see @LOG-RECORD in the @LOGGING section. For a description of\n VERSION, INSERTABLE, REPLAY-VALUES and REPLAY-CONDITION, see\n @JOURNALED-FOR-REPLAY.\n\n### In-events\n\nUpon entering a @BLOCK, JOURNALED generates an IN-EVENT,\nwhich conceptually opens a new @FRAME. These in-events are created\nfrom the NAME, VERSION and ARGS arguments of JOURNALED. For example,\n\n```\n(journaled (name :version version :args args) ...)\n```\n\ncreates an event like this:\n\n```\n`(:in ,name :version ,version :args ,args)\n```\n\nwhere :VERSION and :ARGS may be omitted if they are NIL. Versions\nare used for @REPLAY.\n\n### Out-events\n\nUpon leaving a @BLOCK, JOURNALED generates an OUT-EVENT, closing\nthe @FRAME opened by the corresponding IN-EVENT. These out-events\nare property lists like this:\n\n```\n(:out foo :version 1 :values (42))\n```\n\nTheir NAME and VERSION (`FOO` and `1` in the example) are the same\nas in the in-event: they come from the corresponding arguments of\nJOURNALED. EXIT and OUTCOME are filled in differently depending on\nhow the block finished its execution.\n\n- [type] EVENT-EXIT\n\n One of :VALUES, :CONDITION, :ERROR and :NLX. Indicates whether a\n journaled @BLOCK\n \n - returned normally (:VALUES, see @VALUES-OUTCOME),\n \n - unwound on an expected condition (:CONDITION, see @CONDITION-OUTCOME),\n \n - unwound on an unexpected condition (:ERROR, see @ERROR-OUTCOME),\n \n - unwound by performing a non-local exit of some other kind\n such as a throw (:NLX, see @NLX-OUTCOME).\n \n The first two are @EXPECTED-OUTCOMEs, while the latter two are\n @UNEXPECTED-OUTCOMEs.\n\n- [glossary-term] values outcome\n\n If the JOURNALED @BLOCK returns normally, EVENT-EXIT is\n :VALUES, and the outcome is the list of values returned:\n \n ```\n (journaled (foo) (values 7 t))\n ;; generates the out-event\n (:out foo :values (7 t))\n ```\n \n The list of return values of the block is transformed by the VALUES\n argument of JOURNALED, whose default is `#'IDENTITY`. Also see\n @WORKING-WITH-UNREADABLE-VALUES).\n\n- [glossary-term] condition outcome\n\n If the @BLOCK unwound due to a condition, and JOURNALED's\n CONDITION argument (a function whose default is `(CONSTANTLY NIL)`)\n returns non-NIL when invoked on it, then EVENT-EXIT is\n :CONDITION, and the outcome is this return value:\n \n ```\n (journaled (foo :condition (lambda (c) (prin1-to-string c)))\n (error \"xxx\"))\n ;; generates the out-event\n (:out foo :condition \"xxx\")\n ```\n \n Conditions thus recognized are those that can be considered part of\n normal execution. Just like return values, these expected conditions\n may be required to match what's in the replay journal. Furthermore,\n given a suitable REPLAY-CONDITION in JOURNALED, they may be replayed\n without running the @BLOCK.\n\n- [glossary-term] error outcome\n\n If the JOURNALED @BLOCK unwound due to a condition, but\n JOURNALED's CONDITION argument returns NIL when invoked on it, then\n EVENT-EXIT is :ERROR and the outcome the string\n representations of the type of the condition and the condition\n itself.\n \n ```\n (journaled (foo)\n (error \"xxx\"))\n ;; generates this out-event:\n ;; (:out foo :error (\"simple-error\" \"xxx\"))\n ```\n \n The conversion to string is performed with PRINC in\n WITH-STANDARD-IO-SYNTAX. This scheme is intended to avoid leaking\n random implementation details into the journal, which would make\n `READ`ing it back difficult.\n \n In contrast with @CONDITION-OUTCOMEs, error outcomes are what the\n code is not prepared to handle or replay in a meaningful way.\n\n- [glossary-term] nlx outcome\n\n If the JOURNALED @BLOCK performed a non-local exit that\n was not due to a condition, then EVENT-EXIT is :NLX and the\n outcome is NIL.\n \n ```\n (catch 'xxx\n (journaled (foo)\n (throw 'xxx nil)))\n ;; generates the out-event\n (:out foo :nlx nil)\n ```\n \n Note that @CONDITION-OUTCOMEs and @ERROR-OUTCOMEs are also due to\n non-local exits but are distinct from nlx outcomes.\n \n Currently, nlx outcomes are detected rather heuristically as there\n is no portable way to detect what really caused the unwinding of the\n stack.\n\nThere is a further grouping of outcomes into expected and unexpected.\n\n- [glossary-term] expected outcome\n\n An OUT-EVENT is said to have an expected outcome if it had a\n @VALUES-OUTCOME or a @CONDITION-OUTCOME, or equivalently, when its\n EVENT-EXIT is :VALUES or :CONDITION.\n\n- [glossary-term] unexpected outcome\n\n An OUT-EVENT is said to have an unexpected outcome if it had an\n @ERROR-OUTCOME or an @NLX-OUTCOME, or equivalently, when its\n EVENT-EXIT is :ERROR or :NLX.\n\n### Working with unreadable values\n\nThe events recorded often need to be @READABLE. This is always\nrequired with FILE-JOURNALs, often with IN-MEMORY-JOURNALs, but\nnever with PPRINT-JOURNALs. By choosing an appropriate identifier or\nstring representation of the unreadable object to journal, this is\nnot a problem in practice. JOURNALED provides the VALUES\nhook for this purpose.\n\nWith EXTERNAL-EVENTs, whose outcome is replayed (see\n@REPLAYING-THE-OUTCOME), we also need to be able to reverse the\ntransformation of VALUES, and this is what the\nREPLAY-VALUES argument of JOURNALED is for.\n\nLet's see a complete example.\n\n```\n(defclass user ()\n ((id :initarg :id :reader user-id)))\n\n(defmethod print-object ((user user) stream)\n (print-unreadable-object (user stream :type t)\n (format stream \"~S\" (slot-value user 'id))))\n\n(defvar *users* (make-hash-table))\n\n(defun find-user (id)\n (gethash id *users*))\n\n(defun add-user (id)\n (setf (gethash id *users*) (make-instance 'user :id id)))\n\n(defvar *user7* (add-user 7))\n\n(defun get-message ()\n (replayed (listen :values (values-\u003e #'user-id)\n :replay-values (values\u003c- #'find-user))\n (values *user7* \"hello\")))\n\n(jtrace user-id find-user get-message)\n\n(let ((bundle (make-file-bundle \"/tmp/user-example/\")))\n (format t \"Recording\")\n (with-bundle (bundle)\n (get-message))\n (format t \"~%Replaying\")\n (with-bundle (bundle)\n (get-message)))\n.. Recording\n.. (GET-MESSAGE)\n.. (USER-ID #\u003cUSER 7\u003e)\n.. =\u003e 7\n.. =\u003e #\u003cUSER 7\u003e, \"hello\"\n.. Replaying\n.. (GET-MESSAGE)\n.. (FIND-USER 7)\n.. =\u003e #\u003cUSER 7\u003e, T\n.. =\u003e #\u003cUSER 7\u003e, \"hello\"\n==\u003e #\u003cUSER 7\u003e\n=\u003e \"hello\"\n```\n\nTo be able to journal the return values of `GET-MESSAGE`, the `USER`\nobject must be transformed to something @READABLE. On the\n`Recording` run, `(VALUES-\u003e #'USER-ID)` replaces the user object\nwith its id in the EVENT-OUTCOME recorded, but the original user\nobject is returned.\n\nWhen `Replaying`, the journaled OUT-EVENT is replayed (see\n@REPLAYING-THE-OUTCOME):\n\n```\n(:OUT GET-MESSAGE :VERSION :INFINITY :VALUES (7 \"hello\"))\n```\n\nThe user object is looked up according to :REPLAY-VALUES and is\nreturned along with `\"hello\"`.\n\n- [function] VALUES-\u003e \u0026REST FNS\n\n A utility to create a function suitable as the VALUES\n argument of JOURNALED. The VALUES function is called with the list\n of values returned by the @BLOCK and returns a transformed set of\n values that may be recorded in a journal. While arbitrary\n transformations are allowed, `VALUES-\u003e` handles the common case of\n transforming individual elements of the list independently by\n calling the functions in FN with the values of the list of the same\n position.\n \n ```\n (funcall (values-\u003e #'1+) '(7 :something))\n =\u003e (8 :SOMETHING)\n ```\n \n Note how `#'1+` is applied only to the first element of the values\n list. The list of functions is shorter than the values list, so\n `:SOMETHING` is not transformed. A value can be left explicitly\n untransformed by specifying #'IDENTITY or NIL as the function:\n \n ```\n (funcall (values-\u003e #'1+ nil #'symbol-name)\n '(7 :something :another))\n =\u003e (8 :SOMETHING \"ANOTHER\")\n ```\n\n- [function] VALUES\u003c- \u0026REST FNS\n\n The inverse of `VALUES-\u003e`, this returns a function suitable as\n the REPLAY-VALUES argument of JOURNALED. It does pretty much what\n `VALUES-\u003e` does, but the function returned returns the transformed\n list as multiple values instead of as a list.\n \n ```\n (funcall (values\u003c- #'1-) '(8 :something))\n =\u003e 7\n =\u003e :SOMETHING\n ```\n\n### Utilities\n\n- [function] LIST-EVENTS \u0026OPTIONAL (JOURNAL (RECORD-JOURNAL))\n\n Return a list of all the events in the journal designated by\n JOURNAL. Calls SYNC-JOURNAL first to make sure that all writes are\n taken into account.\n\n- [function] EVENTS-TO-FRAMES EVENTS\n\n Convert a flat list of events, such as those returned by LIST-EVENTS,\n to a nested list representing the @FRAMEs. Each frame is a list of\n the form `(\u003cin-event\u003e \u003cnested-frames\u003e* \u003cout-event\u003e?)`. Like in\n PRINT-EVENTS, EVENTS may be a JOURNAL.\n \n ```\n (events-to-frames '((:in foo :args (1 2))\n (:in bar :args (7))\n (:leaf \"leaf\")\n (:out bar :values (8))\n (:out foo :values (2))\n (:in foo :args (3 4))\n (:in bar :args (8))))\n =\u003e (((:IN FOO :ARGS (1 2))\n ((:IN BAR :ARGS (7))\n (:LEAF \"leaf\")\n (:OUT BAR :VALUES (8)))\n (:OUT FOO :VALUES (2)))\n ((:IN FOO :ARGS (3 4)) ((:IN BAR :ARGS (8)))))\n ```\n \n Note that, as in the above example, incomplete frames (those without\n an OUT-EVENT) are included in the output.\n\n- [function] EXPECTED-TYPE TYPE\n\n Return a function suitable as the CONDITION argument of JOURNALED,\n which returns the type of its single argument as a string if it is\n of TYPE, else NIL.\n\n### Pretty-printing\n\n- [function] PRINT-EVENTS EVENTS \u0026KEY STREAM\n\n Print EVENTS to STREAM as lists, starting a new line for each\n event and indenting them according to their nesting structure.\n EVENTS may be a sequence or a JOURNAL, in which case LIST-EVENTS is\n called on it first.\n \n ```\n (print-events '((:in log :args (\"first arg\" 2))\n (:in versioned :version 1 :args (3))\n (:out versioned :version 1 :values (42 t))\n (:out log :condition \"a :CONDITION outcome\")\n (:in log-2)\n (:out log-2 :nlx nil)\n (:in external :version :infinity)\n (:out external :version :infinity\n :error (\"ERROR\" \"an :ERROR outcome\"))))\n ..\n .. (:IN LOG :ARGS (\"first arg\" 2))\n .. (:IN VERSIONED :VERSION 1 :ARGS (3))\n .. (:OUT VERSIONED :VERSION 1 :VALUES (42 T))\n .. (:OUT LOG :CONDITION \"a :CONDITION outcome\")\n .. (:IN LOG-2)\n .. (:OUT LOG-2 :NLX NIL)\n .. (:IN EXTERNAL :VERSION :INFINITY)\n .. (:OUT EXTERNAL :VERSION :INFINITY :ERROR (\"ERROR\" \"an :ERROR outcome\"))\n =\u003e ; No value\n ```\n\n- [function] PPRINT-EVENTS EVENTS \u0026KEY STREAM (PRETTIFIER 'PRETTIFY-EVENT)\n\n Like PRINT-EVENTS, but produces terser, more human readable\n output.\n \n ```\n (pprint-events '((:in log :args (\"first arg\" 2))\n (:in versioned :version 1 :args (3))\n (:leaf \"This is a leaf, not a frame.\")\n (:out versioned :version 1 :values (42 t))\n (:out log :condition \"a :CONDITION outcome\")\n (:in log-2)\n (:out log-2 :nlx nil)\n (:in external :version :infinity)\n (:out external :version :infinity\n :error (\"ERROR\" \"an :ERROR outcome\"))))\n ..\n .. (LOG \"first arg\" 2)\n .. (VERSIONED 3) v1\n .. This is a leaf, not a frame.\n .. =\u003e 42, T\n .. =C \"a :CONDITION outcome\"\n .. (LOG-2)\n .. =X\n .. (EXTERNAL) ext\n .. =E \"ERROR\" \"an :ERROR outcome\"\n =\u003e ; No value\n ```\n \n The function given as the PRETTIFIER argument formats individual\n events. The above output was produced with PRETTIFY-EVENT. For a\n description of PRETTIFIER's arguments see PRETTIFY-EVENT.\n\n- [function] PRETTIFY-EVENT EVENT DEPTH STREAM\n\n Write EVENT to STREAM in a somewhat human-friendly format.\n This is the function PPRINT-JOURNAL, PPRINT-EVENTS, and @TRACING use\n by default. In addition to the basic example in PPRINT-EVENTS,\n @DECORATION on events is printed before normal, indented output like\n this:\n \n ```\n (pprint-events '((:leaf \"About to sleep\" :time \"19:57:00\" :function \"FOO\")))\n ..\n .. 19:57:00 FOO: About to sleep\n ```\n \n DEPTH is the nesting level of the EVENT. Top-level events have depth\n 0. PRETTIFY-EVENT prints indents the output after printing the\n decorations by 2 spaces per depth.\n\nInstead of collecting events and then printing them, events can\nbe pretty-printed to a stream as they generated. This is\naccomplished with @PPRINT-JOURNALS, discussed in detail later, in\nthe following way:\n\n```\n(let ((journal (make-pprint-journal)))\n (with-journaling (:record journal)\n (journaled (foo) \"Hello\")))\n..\n.. (FOO)\n.. =\u003e \"Hello\"\n```\n\nNote that @PPRINT-JOURNALS are not tied to WITH-JOURNALING and are\nmost often used for @LOGGING and @TRACING.\n\n### Error handling\n\n- [condition] JOURNALING-FAILURE SERIOUS-CONDITION\n\n Signalled during the dynamic extent of\n WITH-JOURNALING when an error threatens to leave the journaling\n mechanism in an inconsistent state. These include I/O errors\n encountered reading or writing journals by WITH-JOURNALING,\n JOURNALED, LOGGED, WITH-REPLAY-FILTER, SYNC-JOURNAL, and also\n STORAGE-CONDITIONs, assertion failures, errors calling JOURNALED's\n VALUES and CONDITION function arguments.\n Crucially, this does not apply to non-local exits from other\n code, such as JOURNALED @BLOCKs, whose error handling is largely\n unaltered (see @OUT-EVENTS and @REPLAY-FAILURES).\n \n In general, any non-local exit from critical parts of the\n code is turned into a JOURNALING-FAILURE to protect the integrity of\n the RECORD-JOURNAL. The condition that caused the unwinding is in\n JOURNALING-FAILURE-EMBEDDED-CONDITION, or NIL if it was a pure\n non-local exit like THROW. This is a SERIOUS-CONDITION, not\n to be handled within WITH-JOURNALING.\n \n After a JOURNALING-FAILURE, the journaling mechanism cannot be\n trusted anymore. The REPLAY-JOURNAL might have failed a read and be\n out-of-sync. The RECORD-JOURNAL may have missing events (or even\n half-written events with FILE-JOURNALs without SYNC, see\n @SYNCHRONIZATION-STRATEGIES), and further writes to it would risk\n replayability, which is equivalent to database corruption. Thus,\n upon signalling JOURNALING-FAILURE, JOURNAL-STATE is set to\n \n - :COMPLETED if the journal is in state :RECORDING or :LOGGING and\n the transition to :RECORDING was reflected in storage,\n \n - else it is set to :FAILED.\n \n After a JOURNALING-FAILURE, any further attempt within the affected\n WITH-JOURNALING to use the critical machinery mentioned\n above (JOURNALED, LOGGED, etc) resignals the same journal failure\n condition. As a consequence, the record journal cannot be changed,\n and the only way to recover is to leave WITH-JOURNALING. This does\n not affect processing in other threads, which by design cannot write\n to the record journal.\n \n Note that in contrast with JOURNALING-FAILURE and REPLAY-FAILURE,\n which necessitate leaving WITH-JOURNALING to recover from, the other\n conditions – JOURNAL-ERROR, and STREAMLET-ERROR – are subclasses of\n ERROR as the their handling need not be so\n heavy-handed.\n\n- [reader] JOURNALING-FAILURE-EMBEDDED-CONDITION [JOURNALING-FAILURE][3956] (:EMBEDDED-CONDITION)\n\n- [condition] RECORD-UNEXPECTED-OUTCOME\n\n Signalled (with SIGNAL: this is not an\n ERROR) by JOURNALED when a VERSIONED-EVENT or an\n EXTERNAL-EVENT had an UNEXPECTED-OUTCOME while in JOURNAL-STATE\n :RECORDING. Upon signalling this condition, JOURNAL-STATE is set to\n :LOGGING, thus no more events can be recorded that will affect\n replay of the journal being recorded. The event that triggered this\n condition is recorded in state :LOGGING, with its version\n downgraded. Since @REPLAY (except @INVOKED) is built on the\n assumption that control flow is deterministic, an unexpected outcome\n is significant because it makes this assumption to hold unlikely.\n \n Also see REPLAY-UNEXPECTED-OUTCOME.\n\n- [condition] DATA-EVENT-LOSSAGE JOURNALING-FAILURE\n\n Signalled when a @DATA-EVENT is about to be recorded\n in JOURNAL-STATE :MISMATCHED or :LOGGING. Since the data event will\n not be replayed that constitutes data loss.\n\n- [condition] JOURNAL-ERROR ERROR\n\n Signalled by WITH-JOURNALING, WITH-BUNDLE and by\n @LOG-RECORD. It is also signalled by the low-level streamlet\n interface (see @STREAMLETS-REFERENCE).\n\n- [condition] END-OF-JOURNAL JOURNAL-ERROR\n\n This might be signalled by the replay mechanism if\n WITH-JOURNALING's REPLAY-EOJ-ERROR-P is true. Unlike\n REPLAY-FAILUREs, this does not affect JOURNAL-STATE of\n RECORD-JOURNAL. At a lower level, it is signalled by READ-EVENT upon\n reading past the end of the JOURNAL if EOJ-ERROR-P.\n\n## Logging\n\nBefore we get into the details, here is a self-contained example\nthat demonstrates typical use.\n\n```\n(defvar *communication-log* nil)\n(defvar *logic-log* nil)\n(defvar *logic-log-level* 0)\n\n(defun call-with-connection (port fn)\n (framed (call-with-connection :log-record *communication-log*\n :args `(,port))\n (funcall fn)))\n\n(defun fetch-data (key)\n (let ((value 42))\n (logged ((and (\u003c= 1 *logic-log-level*) *logic-log*))\n \"The value of ~S is ~S.\" key value)\n value))\n\n(defun init-logging (\u0026key (logic-log-level 1))\n (let* ((stream (open \"/tmp/xxx.log\"\n :direction :output\n :if-does-not-exist :create\n :if-exists :append))\n (journal (make-pprint-journal\n :stream (make-broadcast-stream\n (make-synonym-stream '*standard-output*)\n stream))))\n (setq *communication-log* journal)\n (setq *logic-log* journal)\n (setq *logic-log-level* logic-log-level)))\n\n(init-logging)\n\n(call-with-connection 8080 (lambda () (fetch-data :foo)))\n..\n.. (CALL-WITH-CONNECTION 8080)\n.. The value of :FOO is 42.\n.. =\u003e 42\n=\u003e 42\n\n(setq *logic-log-level* 0)\n(call-with-connection 8080 (lambda () (fetch-data :foo)))\n..\n.. (CALL-WITH-CONNECTION 8080)\n.. =\u003e 42\n=\u003e 42\n\n(ignore-errors\n (call-with-connection 8080 (lambda () (error \"Something unexpected.\"))))\n..\n.. (CALL-WITH-CONNECTION 8080)\n.. =E \"SIMPLE-ERROR\" \"Something unexpected.\"\n```\n\n##### Default to muffling\n\nImagine a utility library called glib.\n\n```\n(defvar *glib-log* nil)\n(defvar *patience* 1)\n\n(defun sl33p (seconds)\n (logged (*glib-log*) \"Sleeping for ~As.\" seconds)\n (sleep (* *patience* seconds)))\n```\n\nGlib follows the recommendation to have a special variable globally\nbound to NIL by default. The value of `*GLIB-LOG*` is the journal to\nwhich glib log messages will be routed. Since it's NIL, the log\nmessages are muffled, and to record any log message, we need to\nchange its value.\n\n##### Routing logs to a journal\n\nLet's send the logs to a PPRINT-JOURNAL:\n\n```\n(setq *glib-log* (make-pprint-journal\n :log-decorator (make-log-decorator :time t)))\n(sl33p 0.01)\n..\n.. 2020-08-31T12:45:23.827172+02:00: Sleeping for 0.01s.\n```\n\nThat's a bit too wordy. For this tutorial, let's stick to less\nverbose output:\n\n```\n(setq *glib-log* (make-pprint-journal))\n(sl33p 0.01)\n..\n.. Sleeping for 0.01s.\n```\n\nTo log to a file:\n\n```\n(setq *glib-log* (make-pprint-journal\n :stream (open \"/tmp/glib.log\"\n :direction :output\n :if-does-not-exist :create\n :if-exists :append)))\n```\n\n##### Capturing logs in WITH-JOURNALING's RECORD-JOURNAL\n\nIf we were recording a journal for replay and wanted to include glib\nlogs in the journal, we would do something like this:\n\n```\n(with-journaling (:record t)\n (let ((*glib-log* :record))\n (sl33p 0.01)\n (journaled (non-glib-stuff :version 1)))\n (list-events))\n=\u003e ((:LEAF \"Sleeping for 0.01s.\")\n (:IN NON-GLIB-STUFF :VERSION 1)\n (:OUT NON-GLIB-STUFF :VERSION 1 :VALUES (NIL)))\n```\n\nWe could even `(SETQ *GLIB-LOG* :RECORD)` to make it so that glib\nmessages are included by default in the RECORD-JOURNAL. In this\nexample, the special `*GLIB-LOG*` acts like a log category for all\nthe log messages of the glib library (currently one).\n\n##### Rerouting a category\n\nNext, we route `*GLIB-LOG*` to wherever `*APP-LOG*` is pointing by\nbinding `*GLIB-LOG*` *to the symbol* `*APP-LOG*` (see @LOG-RECORD).\n\n```\n(defvar *app-log* nil)\n\n(let ((*glib-log* '*app-log*))\n (setq *app-log* nil)\n (logged (*glib-log*) \"This is not written anywhere.\")\n (setq *app-log* (make-pprint-journal :pretty nil))\n (sl33p 0.01))\n..\n.. (:LEAF \"Sleeping for 0.01s.\")\n```\n\nNote how pretty-printing was turned off, and we see the LEAF-EVENT\ngenerated by LOGGED in its raw plist form.\n\n##### Conditional routing\n\nFinally, to make routing decisions conditional we need to change\n`SL33P`:\n\n```\n(defvar *glib-log-level* 1)\n\n(defun sl33p (seconds)\n (logged ((and (\u003c= 2 *glib-log-level*) *glib-log*))\n \"Sleeping for ~As.\" (* *patience* seconds))\n (sleep seconds))\n\n;;; Check that it all works:\n(let ((*glib-log-level* 1)\n (*glib-log* (make-pprint-journal)))\n (format t \"~%With log-level ~A\" *glib-log-level*)\n (sl33p 0.01)\n (setq *glib-log-level* 2)\n (format t \"~%With log-level ~A\" *glib-log-level*)\n (sl33p 0.01))\n..\n.. With log-level 1\n.. With log-level 2\n.. Sleeping for 0.01s.\n```\n\n##### Nested log contexts\n\nLOGGED is for single messages. JOURNALED, or in this example FRAMED,\ncan provide nested context:\n\n```\n(defun callv (var value symbol \u0026rest args)\n \"Call SYMBOL-FUNCTION of SYMBOL with VAR dynamically bound to VALUE.\"\n (framed (\"glib:callv\" :log-record *glib-log*\n :args `(,var ,value ,symbol ,@args))\n (progv (list var) (list value)\n (apply (symbol-function symbol) args))))\n\n(callv '*print-base* 2 'print 10)\n..\n.. (\"glib:callv\" *PRINT-BASE* 2 PRINT 10)\n.. 1010 \n.. =\u003e 10\n=\u003e 10\n\n(let ((*glib-log-level* 2))\n (callv '*patience* 7 'sl33p 0.01))\n..\n.. (\"glib:callv\" *PATIENCE* 7 SL33P 0.01)\n.. Sleeping for 0.07s.\n.. =\u003e NIL\n```\n\n\n### Customizing logs\n\nCustomizing the output format is possible if we don't necessarily\nexpect to be able to read the logs back programmatically. There is\nan example in @TRACING, which is built on @PPRINT-JOURNALS.\n\nHere, we discuss how to make logs more informative.\n\n- [glossary-term] decoration\n\n JOURNAL-LOG-DECORATOR adds additional data to LOG-EVENTs as they\n are written to the journal. This data is called decoration, and it is\n to capture the context in which the event was triggered. See\n MAKE-LOG-DECORATOR for a typical example. Decorations, since they\n can be on LOG-EVENTs only, do not affect @REPLAY. Decorations are\n most often used with @PRETTY-PRINTING.\n\n- [accessor] JOURNAL-LOG-DECORATOR [JOURNAL][5082] (:LOG-DECORATOR = NIL)\n\n If non-NIL, this is a function to add @DECORATION\n to LOG-EVENTs before they are written to a journal. The only\n allowed transformation is to *append* a plist to the event, which\n is a plist itself. The keys can be anything.\n\n- [function] MAKE-LOG-DECORATOR \u0026KEY TIME REAL-TIME RUN-TIME THREAD DEPTH OUT-NAME\n\n Return a function suitable as JOURNAL-LOG-DECORATOR that may add\n a string timestamp, the internal real-time or run-time (both in\n seconds), the name of the thread, to events, which will be handled\n by PRETTIFY-EVENT. If DEPTH, then PRETTIFY-EVENT will the nesting\n level of the event being printed. If OUT-NAME, the PRETTIFY-EVENT\n will print the name of @OUT-EVENTS.\n \n All arguments are @BOOLEAN-VALUED-SYMBOLs.\n \n ```\n (funcall (make-log-decorator :depth t :out-name t :thread t\n :time t :real-time t :run-time t)\n (make-leaf-event :foo))\n =\u003e (:LEAF :FOO :DEPTH T :OUT-NAME T :THREAD \"worker\"\n :TIME \"2023-05-26T12:27:44.172614+01:00\"\n :REAL-TIME 2531.3254 :RUN-TIME 28.972797)\n ```\n\n### :LOG-RECORD\n\nWITH-JOURNALING and WITH-BUNDLE control replaying and recording\nwithin their dynamic extent, which is rather a necessity because\n@REPLAY needs to read the events in the same order as the JOURNALED\n@BLOCKs are being executed. However, LOG-EVENTs do not affect\nreplay, so we can allow more flexibility in routing them.\n\nThe LOG-RECORD argument of JOURNALED and LOGGED controls where\nLOG-EVENTs are written both within WITH-JOURNALING and without. The\nalgorithm to determine the target journal is this:\n\n1. If LOG-RECORD is :RECORD, then the RECORD-JOURNAL is returned.\n\n2. If LOG-RECORD is NIL, then it is returned.\n\n3. If LOG-RECORD is a JOURNAL, then it is returned.\n\n4. If LOG-RECORD is a symbol (other than NIL), then the SYMBOL-VALUE\n of that symbol is assigned to LOG-RECORD, and we go to step 1.\n\nIf the return value is NIL, then the event will not be written\nanywhere, else it is written to the journal returned.\n\nThis is reminiscent of SYNONYM-STREAMs, also in that it is possible\nend up in cycles in the resolution. For this reason, the algorithm\nstop with a JOURNAL-ERROR after 100 iterations.\n\n##### Interactions\n\nEvents may be written to LOG-RECORD even without an enclosing\nWITH-JOURNALING, and it does not affect the JOURNAL-STATE. However,\nit is a JOURNAL-ERROR to write to a :COMPLETED journal (see\nJOURNAL-STATE).\n\nWhen multiple threads log to the same journal, it is guaranteed that\nindividual events are written atomically, but frames from different\nthreads do not necessarily nest. To keep the log informative, the\nname of thread may be added to the events as @DECORATION.\n\nAlso, see notes on thread @SAFETY.\n\n### Logging with LEAF-EVENTs\n\n- [macro] LOGGED (\u0026OPTIONAL (LOG-RECORD :RECORD)) FORMAT-CONTROL \u0026REST FORMAT-ARGS\n\n LOGGED creates a single LEAF-EVENT, whose name is the string\n constructed by FORMAT. For example:\n \n ```\n (with-journaling (:record t)\n (logged () \"Hello, ~A.\" \"world\")\n (list-events))\n =\u003e ((:LEAF \"Hello, world.\"))\n ```\n \n LEAF-EVENTs are LOG-EVENTs with no separate in- and out-events. They\n have an EVENT-NAME and no other properties. Use LOGGED for\n point-in-time textual log messages, and JOURNALED with VERSION\n NIL (i.e. FRAMED) to provide context.\n \n Also, see @LOG-RECORD.\n\n## Tracing\n\nJTRACE behaves similarly to CL:TRACE but deals with\nnon-local exits gracefully.\n\n##### Basic tracing\n\n```common-lisp\n(defun foo (x)\n (sleep 0.12)\n (1+ x))\n\n(defun bar (x)\n (foo (+ x 2))\n (error \"xxx\"))\n\n(jtrace foo bar)\n\n(ignore-errors (bar 1))\n..\n.. 0: (BAR 1)\n.. 1: (FOO 3)\n.. 1: FOO =\u003e 4\n.. 0: BAR =E \"SIMPLE-ERROR\" \"xxx\"\n```\n\n##### Log-like output\n\nIt can also include the name of the originating thread and\ntimestamps in the output:\n\n```\n(let ((*trace-thread* t)\n (*trace-time* t)\n (*trace-depth* nil)\n (*trace-out-name* nil))\n (ignore-errors (bar 1)))\n..\n.. 2020-09-02T19:58:19.415204+02:00 worker: (BAR 1)\n.. 2020-09-02T19:58:19.415547+02:00 worker: (FOO 3)\n.. 2020-09-02T19:58:19.535766+02:00 worker: =\u003e 4\n.. 2020-09-02T19:58:19.535908+02:00 worker: =E \"SIMPLE-ERROR\" \"xxx\"\n```\n\n##### Profiler-like output\n\n```\n(let ((*trace-real-time* t)\n (*trace-run-time* t)\n (*trace-depth* nil)\n (*trace-out-name* nil))\n (ignore-errors (bar 1)))\n..\n.. #16735.736 !68.368: (BAR 1)\n.. #16735.736 !68.369: (FOO 3)\n.. #16735.857 !68.369: =\u003e 4\n.. #16735.857 !68.369: =E \"SIMPLE-ERROR\" \"xxx\"\n```\n\n##### Customizing the content and the format\n\nIf these options are insufficient, the content and the format of the\ntrace can be customized:\n\n```\n(let ((*trace-journal*\n (make-pprint-journal :pretty '*trace-pretty*\n :prettifier (lambda (event depth stream)\n (format stream \"~%Depth: ~A, event: ~S\"\n depth event))\n :stream (make-synonym-stream '*error-output*)\n :log-decorator (lambda (event)\n (append event '(:custom 7))))))\n (ignore-errors (bar 1)))\n..\n.. Depth: 0, event: (:IN BAR :ARGS (1) :CUSTOM 7)\n.. Depth: 1, event: (:IN FOO :ARGS (3) :CUSTOM 7)\n.. Depth: 1, event: (:OUT FOO :VALUES (4) :CUSTOM 7)\n.. Depth: 0, event: (:OUT BAR :ERROR (\"SIMPLE-ERROR\" \"xxx\") :CUSTOM 7)\n```\n\nIn the above, *TRACE-JOURNAL* was bound locally to keep the example\nfrom wrecking the global default, but the same effect could be\nachieved by `SETF`ing PPRINT-JOURNAL-PRETTIFIER,\nPPRINT-JOURNAL-STREAM and JOURNAL-LOG-DECORATOR.\n\n- [macro] JTRACE \u0026REST NAMES\n\n Like CL:TRACE, JTRACE takes a list of symbols. When functions\n denoted by those NAMES are invoked, their names, arguments and\n outcomes are printed in human readable form to *TRACE-OUTPUT*. These\n values may not be @READABLE, JTRACE does not care.\n \n The format of the output is the same as that of PPRINT-EVENTS.\n Behind the scenes, JTRACE encapsulates the global functions with\n NAMES in wrapper that behaves as if `FOO` in the example above was\n defined like this:\n \n ```\n (defun foo (x)\n (framed (foo :args `(,x) :log-record *trace-journal*)\n (1+ x)))\n ```\n \n If JTRACE is invoked with no arguments, it returns the list of\n symbols currently traced.\n \n On Lisps other than SBCL, where a function encapsulation facility is\n not available or it is not used by Journal, JTRACE simply sets\n SYMBOL-FUNCTION. This solution loses the tracing encapsulation when\n the function is recompiled. On these platforms, `(JTRACE)` also\n retraces all functions that should be traced but aren't.\n \n The main advantage of JTRACE over CL:TRACE is the ability to trace\n errors, not just normal return values. As it is built on JOURNALED,\n it can also detect – somewhat heuristically – THROWs and similar.\n\n- [macro] JUNTRACE \u0026REST NAMES\n\n Like CL:UNTRACE, JUNTRACE makes it so that the global functions\n denoted by the symbols NAMES are no longer traced by JTRACE. When\n invoked with no arguments, it untraces all traced functions.\n\n- [variable] *TRACE-PRETTY* T\n\n If *TRACE-PRETTY* is true, then JTRACE produces output like\n PPRINT-EVENTS, else it's like PRINT-EVENTS.\n\n- [variable] *TRACE-DEPTH* T\n\n Controls whether to decorate the trace with the depth of event.\n See MAKE-LOG-DECORATOR.\n\n- [variable] *TRACE-OUT-NAME* T\n\n Controls whether trace should print the EVENT-NAME of @OUT-EVENTS,\n which is redundant with the EVENT-NAME of the corresponding\n @IN-EVENTS. See MAKE-LOG-DECORATOR.\n\n- [variable] *TRACE-THREAD* NIL\n\n Controls whether to decorate the trace with the name of the\n originating thread. See MAKE-LOG-DECORATOR.\n\n- [variable] *TRACE-TIME* NIL\n\n Controls whether to decorate the trace with a timestamp. See\n MAKE-LOG-DECORATOR.\n\n- [variable] *TRACE-REAL-TIME* NIL\n\n Controls whether to decorate the trace with the internal real-time.\n See MAKE-LOG-DECORATOR.\n\n- [variable] *TRACE-RUN-TIME* NIL\n\n Controls whether to decorate the trace with the internal run-time.\n See MAKE-LOG-DECORATOR.\n\n- [variable] *TRACE-JOURNAL* \\#\\\u003cPPRINT-JOURNAL :NEW 1\u003e\n\n The JOURNAL where JTRACE writes LOG-EVENTs. By default, it is a\n PPRINT-JOURNAL that sets up a SYNONYM-STREAM to *TRACE-OUTPUT* and\n sends its output there. It pays attention to *TRACE-PRETTY*, and its\n log decorator is affected by *TRACE-TIME* and *TRACE-THREAD*.\n However, by changing JOURNAL-LOG-DECORATOR and\n PPRINT-JOURNAL-PRETTIFIER, content and output can be customized.\n\n### Slime integration\n\n[Slime](https://common-lisp.net/project/slime/), by default,\nbinds `C-c C-t` to toggling CL:TRACE. To integrate JTRACE into\nSlime, load `src/mgl-jrn.el` into Emacs.\n\n- If you installed Journal with Quicklisp, the location of\n `mgl-jrn.el` may change with updates, and you may want to copy the\n current version to a stable location:\n\n (journal:install-journal-elisp \"~/quicklisp/\")\n\nThen, assuming the Elisp file is in the quicklisp directory, add\nthis to your `.emacs`:\n\n```elisp\n(load \"~/quicklisp/mgl-jrn.el\")\n```\n\nSince JTRACE lacks some features of CL:TRACE, most notably that of\ntracing non-global functions, it is assigned a separate binding,\n`C-c C-j`.\n\n- [function] INSTALL-JOURNAL-ELISP TARGET-DIR\n\n Copy `mgl-jrn.el` distributed with this package to TARGET-DIR.\n\n## Replay\n\nDuring replay, code is executed normally with special rules for\n@BLOCKs. There are two modes for dealing with blocks: replaying the\ncode and replaying the outcome. When code is replayed, upon entering\nand leaving a block, the events generated are matched to events read\nfrom the journal being replayed. If the events don't match,\nREPLAY-FAILURE is signalled, which marks the record journal as having\nfailed the replay. This is intended to make sure that the state of\nthe program during the replay matches the state at the time of\nrecording. In the other mode, when the outcome is replayed, a block\nmay not be executed at all, but its recorded outcome is\nreproduced (i.e. the recorded return values are simply returned).\n\nReplay can be only be initiated with WITH-JOURNALING (or its close\nkin WITH-BUNDLE). After the per-event processing described below,\nwhen WITH-JOURNALING finishes, it might signal REPLAY-INCOMPLETE if\nthere are unprocessed non-log events left in the replay journal.\n\nReplay is deemed successful or failed depending on whether all\nevents are replayed from the replay journal without a\nREPLAY-FAILURE. A journal that records events from a successful\nreplay can be used in place of the journal that was replayed, and so\non. The logic of replacing journals with their successful replays is\nautomated by @BUNDLES. WITH-JOURNALING does not allow replay from\njournals that were failed replays themselves. The mechanism, in\nterms of which tracking success and failure of replays is\nimplemented, revolves around JOURNAL-STATE and\nEVENT-VERSIONs, which we discuss next.\n\n- [type] JOURNAL-STATE\n\n JOURNAL's state with respect to replay is updated during\n WITH-JOURNALING. The possible states are:\n \n - **:NEW**: This journal was just created but never recorded to.\n \n - **:REPLAYING**: Replaying events has started, some events may have\n been replayed successfully, but there are more non-log events to\n replay.\n \n - **:MISMATCHED**: There was a REPLAY-FAILURE. In this state,\n VERSIONED-EVENTs generated are downgraded to LOG-EVENTs,\n EXTERNAL-EVENTs and @INVOKED trigger DATA-EVENT-LOSSAGE.\n \n - **:RECORDING**: All events from the replay journal were\n successfully replayed, and now new events are being recorded\n without being matched to the replay journal.\n \n - **:LOGGING**: There was a RECORD-UNEXPECTED-OUTCOME. In this\n state, VERSIONED-EVENTs generated are downgraded to LOG-EVENTs,\n EXTERNAL-EVENTs and @INVOKED trigger DATA-EVENT-LOSSAGE.\n \n - **:FAILED**: The journal is to be discarded. It encountered a\n JOURNALING-FAILURE or a REPLAY-FAILURE without completing the\n replay and reaching :RECORDING.\n \n - **:COMPLETED**: All events were successfully replayed and\n WITH-JOURNALING finished or a JOURNALING-FAILURE occurred while\n :RECORDING or :LOGGING.\n \n The state transitions are:\n \n :NEW -\u003e :REPLAYING (on entering WITH-JOURNALING)\n :REPLAYING -\u003e :MISMATCHED (on REPLAY-FAILURE)\n :REPLAYING -\u003e :FAILED (on REPLAY-INCOMPLETE)\n :REPLAYING -\u003e :FAILED (on JOURNALING-FAILURE)\n :REPLAYING -\u003e :RECORDING (on successfully replaying all events)\n :MISMATCHED -\u003e :FAILED (on leaving WITH-JOURNALING)\n :RECORDING -\u003e :LOGGING (on RECORD-UNEXPECTED-OUTCOME)\n :RECORDING/:LOGGING -\u003e :COMPLETED (on leaving WITH-JOURNALING)\n :RECORDING/:LOGGING -\u003e :COMPLETED (on JOURNALING-FAILURE)\n \n :NEW is the starting state. It is a JOURNAL-ERROR to attempt to\n write to journals in :COMPLETED. Note that once in :RECORDING, the\n only possible terminal state is :COMPLETED.\n\n### Journaled for replay\n\nThe following arguments of JOURNALED control behaviour under replay.\n\n- VERSION: see EVENT-VERSION below.\n\n- INSERTABLE controls whether VERSIONED-EVENTs and EXTERNAL-EVENTs\n may be replayed with the *insert* replay strategy (see\n @THE-REPLAY-STRATEGY). Does not affect LOG-EVENTs, which are\n always \\_insert\\_ed. Note that inserting EXTERNAL-EVENTs while\n :REPLAYING is often not meaningful (e.g. asking the user for input\n may lead to a REPLAY-FAILURE). See PEEK-REPLAY-EVENT for an\n example on how to properly insert these kinds of EXTERNAL-EVENTs.\n\n- REPLAY-VALUES, a function or NIL, may be called with EVENT-OUTCOME\n when replaying and :VERSION :INFINITY. NIL is equivalent to\n VALUES-LIST. See `VALUES\u003c-` for an example.\n\n- REPLAY-CONDITION, a function or NIL, may be called with\n EVENT-OUTCOME (the return value of the function provided as\n :CONDITION) when replaying and :VERSION is :INFINITY. NIL is\n equivalent to the ERROR function. Replaying conditions is\n cumbersome and best avoided.\n\n\n- [variable] *FORCE-INSERTABLE* NIL\n\n The default value of the INSERTABLE argument of JOURNALED for\n VERSIONED-EVENTs. Binding this to T allows en-masse structural\n upgrades in combination with WITH-REPLAY-FILTER. Does not affect\n EXTERNAL-EVENTs. See @UPGRADES-AND-REPLAY.\n\n- [type] EVENT-VERSION\n\n An event's version is either NIL, a positive FIXNUM, or :INFINITY,\n which correspond to LOG-EVENTs, VERSIONED-EVENTs, and\n EXTERNAL-EVENTs, respectively, and have an increasingly strict\n behaviour with regards to @REPLAY. All EVENTs have versions. The\n versions of the in- and out-events belonging to the same @FRAME are\n the same.\n\n- [type] LOG-EVENT\n\n Events with EVENT-VERSION NIL called log events. During @REPLAY,\n they are never matched to events from the replay journal, and log\n events in the replay do not affect events being recorded either.\n These properties allow log events to be recorded in arbitrary\n journals with JOURNALED's LOG-RECORD argument. The convenience macro\n FRAMED is creating frames of log-events, while the LOGGED generates\n a log-event that's a LEAF-EVENT.\n\n- [type] VERSIONED-EVENT\n\n Events with a positive integer EVENT-VERSION are called\n versioned events. In @REPLAY, they undergo consistency checks unlike\n LOG-EVENTs, but the rules for them are less strict than for\n EXTERNAL-EVENTs. In particular, higher versions are always\n considered compatible with lower versions, they become an *upgrade*\n in terms of the @THE-REPLAY-STRATEGY, and versioned events can be\n inserted into the record without a corresponding @REPLAY-EVENT with\n JOURNALED's INSERTABLE.\n \n If a VERSIONED-EVENT has an @UNEXPECTED-OUTCOME,\n RECORD-UNEXPECTED-OUTCOME is signalled.\n\n- [type] EXTERNAL-EVENT\n\n Events with EVENT-VERSION :INFINITY are called external events.\n They are like VERSIONED-EVENTs whose version was bumped all the way\n to infinity, which rules out easy, non-matching upgrades. Also, they\n are never inserted to the record without a matching replay\n event (see @THE-REPLAY-STRATEGY).\n \n In return for these restrictions, external events can be replayed\n without running the corresponding @BLOCK (see\n @REPLAYING-THE-OUTCOME). This allows their out-event variety, called\n @DATA-EVENTs, to be non-deterministic. Data events play a crucial\n role in @PERSISTENCE.\n \n If an EXTERNAL-EVENT has an @UNEXPECTED-OUTCOME,\n RECORD-UNEXPECTED-OUTCOME is signalled.\n\nBuilt on top of JOURNALED, the macros below record a pair of\n@IN-EVENTS and @OUT-EVENTS but differ in how they are replayed and\nthe requirements on their @BLOCKs. The following table names the\ntype of EVENT produced (`Event`), how @IN-EVENTS are\nreplayed (`In-e.`), whether the block is always run (`Run`), how\n@OUT-EVENTS are replayed (`Out-e.`), whether the block must be\ndeterministic (`Det`) or side-effect free (`SEF`).\n\n | | Event | In-e. | Run | Out-e. | Det | SEF |\n |----------+-----------+--------+-----+--------+-----+-----|\n | FRAMED | log | skip | y | skip | n | n |\n | CHECKED | versioned | match | y | match | y | n |\n | REPLAYED | external | match | n | replay | n | y |\n | INVOKED | versioned | replay | y | match | y | n |\n\nNote that the replay-replay combination is not implemented because\nthere is nowhere to return values from replay-triggered functions.\n\n- [macro] FRAMED (NAME \u0026KEY LOG-RECORD ARGS VALUES CONDITION) \u0026BODY BODY\n\n A wrapper around JOURNALED to produce @FRAMEs of LOG-EVENTs. That\n is, VERSION is always NIL, and some irrelevant arguments are\n omitted. The related LOGGED creates a single LEAF-EVENT.\n \n With FRAMED, BODY is always run and no REPLAY-FAILUREs are\n triggered. BODY is not required to be deterministic, and it may have\n side-effects.\n\n- [macro] CHECKED (NAME \u0026KEY (VERSION 1) ARGS VALUES CONDITION INSERTABLE) \u0026BODY BODY\n\n A wrapper around JOURNALED to produce @FRAMEs of VERSIONED-EVENTs.\n VERSION defaults to 1. CHECKED is for ensuring that supposedly\n deterministic processing does not veer off the replay.\n \n With CHECKED, BODY – which must be deterministic – is always run and\n REPLAY-FAILUREs are triggered when the events generated do not match\n the events in the replay journal. BODY may have side-effects.\n \n For further discussion of determinism, see REPLAYED.\n\n- [macro] REPLAYED (NAME \u0026KEY ARGS VALUES CONDITION INSERTABLE REPLAY-VALUES REPLAY-CONDITION) \u0026BODY BODY\n\n A wrapper around JOURNALED to produce @FRAMEs of EXTERNAL-EVENTs.\n VERSION is :INFINITY. REPLAYED is for primarily for marking and\n isolating non-deterministic processing.\n \n With REPLAYED, the IN-EVENT is checked for consistency with the\n replay (as with CHECKED), but BODY is not run (assuming it has a\n recorded @EXPECTED-OUTCOME), and the outcome in the OUT-EVENT is\n reproduced (see @REPLAYING-THE-OUTCOME). For this scheme to work,\n REPLAYED requires its BODY to be side-effect free, but it may be\n non-deterministic.\n\n- [glossary-term] invoked\n\n Invoked refers to functions and blocks defined by DEFINE-INVOKED or\n FLET-INVOKED. Invoked frames may be recorded in response to\n asynchronous events, and at replay the presence of its in-event\n triggers the execution of the function associated with the name of\n the event.\n \n On the one hand, FRAMED, CHECKED, REPLAYED or plain JOURNALED have\n @IN-EVENTS that are always predictable from the code and the\n preceding events. The control flow – on the level of recorded frames\n – is deterministic in this sense. On the other hand, Invoked encodes\n in its IN-EVENT what function to call next, introducing\n non-deterministic control flow.\n \n By letting events choose the code to run, Invoked resembles typical\n @EVENT-SOURCING frameworks. When Invoked is used exclusively, the\n journal becomes a sequence of events. In contrast, JOURNALED and its\n wrappers put code first, and the journal will be a projection of the\n call tree.\n\n- [macro] DEFINE-INVOKED FUNCTION-NAME ARGS (NAME \u0026KEY (VERSION 1) INSERTABLE) \u0026BODY BODY\n\n DEFINE-INVOKED is intended for recording asynchronous function\n invocations like event or signal handlers. It defines a function\n that records VERSIONED-EVENTs with ARGS set to the actual arguments.\n At replay, it is invoked whenever the recorded IN-EVENT becomes the\n @REPLAY-EVENT.\n \n DEFUN and CHECKED rolled into one, DEFINE-INVOKED defines a\n top-level function with FUNCTION-NAME and ARGS (only simple\n positional arguments are allowed) and wraps CHECKED with NAME, the\n same ARGS and INSERTABLE around BODY. Whenever an IN-EVENT becomes\n the @REPLAY-EVENT, and it has a DEFINE-INVOKED defined with the name\n of the event, FUNCTION-NAME is invoked with EVENT-ARGS.\n \n While BODY's return values are recorded as usual, the defined\n function returns no values to make it less likely to affect control\n flow in a way that's not possible to reproduce when the function is\n called by the replay mechanism.\n \n ```\n (defvar *state*)\n \n (define-invoked foo (x) (\"foo\")\n (setq *state* (1+ x)))\n \n (define-invoked bar (x) (\"bar\")\n (setq *state* (+ 2 x)))\n \n (if (zerop (random 2))\n (foo 0)\n (bar 1))\n ```\n \n The above can be alternatively implemented with REPLAYED explicitly\n encapsulating the non-determinism:\n \n ```\n (let ((x (replayed (choose) (random 2))))\n (if (zerop x)\n (checked (foo :args `(,x))\n (setq *state* (1+ x)))\n (checked (bar :args `(,x))\n (setq *state* (+ 2 x)))))\n ```\n\n- [macro] FLET-INVOKED DEFINITIONS \u0026BODY BODY\n\n Like DEFINE-INVOKED, but with FLET instead of DEFUN. The event\n name and the function are associated in the dynamic extent of BODY.\n WITH-JOURNALING does not change the bindings. The example in\n DEFINE-INVOKED can be rewritten as:\n \n ```\n (let ((state nil))\n (flet-invoked ((foo (x) (\"foo\")\n (setq state (1+ x)))\n (bar (x) (\"bar\")\n (setq state (+ 2 x))))\n (if (zerop (random 2))\n (foo 0)\n (bar 1))))\n ```\n\n### Bundles\n\nConsider replaying the same code repeatedly, hoping to make\nprogress in the processing. Maybe based on the availability of\nexternal input, the code may error out. After each run, one has to\ndecide whether to keep the journal just recorded or stick with the\nreplay journal. A typical solution to this would look like this:\n\n```\n(let ((record nil))\n (loop\n (setq record (make-in-memory-journal))\n (with-journaling (:record record :replay replay)\n ...)\n (when (and\n ;; RECORD is a valid replay of REPLAY ...\n (eq (journal-state record) :completed)\n ;; ... and is also significantly different from it ...\n (journal-diverged-p record))\n ;; so use it for future replays.\n (setq replay record))))\n```\n\nThis is pretty much what bundles automate. The above becomes:\n\n```\n(let ((bundle (make-in-memory-bundle)))\n (loop\n (with-bundle (bundle)\n ...)))\n```\n\nWith FILE-JOURNALs, the motivating example above would be even more\ncomplicated, but FILE-BUNDLEs work the same way as\nIN-MEMORY-BUNDLEs.\n\n- [macro] WITH-BUNDLE (BUNDLE) \u0026BODY BODY\n\n This is like WITH-JOURNALING where the REPLAY-JOURNAL is the last\n successfully completed one in BUNDLE, and the RECORD-JOURNAL is a\n new one created in BUNDLE. When WITH-BUNDLE finishes, the record\n journal is in JOURNAL-STATE :FAILED or :COMPLETED.\n \n To avoid accumulating useless data, the new record is immediately\n deleted when WITH-BUNDLE finishes if it has not diverged from the\n replay journal (see JOURNAL-DIVERGENT-P). Because :FAILED journals\n are always divergent in this sense, they are deleted instead based\n on whether there is already a previous failed journal in the bundle\n and the new record is identical to that journal (see\n IDENTICAL-JOURNALS-P).\n \n It is a JOURNAL-ERROR to have concurrent or nested WITH-BUNDLEs on\n the same bundle.\n\n### The replay strategy\n\nThe replay process for both @IN-EVENTS and @OUT-EVENTS starts by\ndetermining how the generated event (the *new* event from now on)\nshall be replayed. Roughly, the decision is based on the NAME and\nVERSION of the new event and the @REPLAY-EVENT (the next event to be\nread from the replay). There are four possible strategies:\n\n- **match**: A new in-event must match the replay event in its ARGS.\n See @MATCHING-IN-EVENTS for details. A new out-event must match\n the replay event's EXIT and OUTCOME, see @MATCHING-OUT-EVENTS.\n\n- **upgrade**: The new event is not matched to any replay event, but\n an event is consumed from the replay journal. This happens if the\n next new event has the same name as the replay event, but its\n version is higher.\n\n- **insert**: The new event is not matched to any replay event, and\n no events are consumed from the replay journal, which may be\n empty. This is always the case for new LOG-EVENTs and when there\n are no more events to read from the replay journal (unless\n REPLAY-EOJ-ERROR-P). For VERSIONED-EVENTs, it is affected by\n setting JOURNALED's INSERTABLE to true (see\n @JOURNALED-FOR-REPLAY).\n\n The out-event's strategy is always *insert* if the strategy for\n the corresponding in-event was *insert*.\n\n- Also, END-OF-JOURNAL, REPLAY-NAME-MISMATCH and\n REPLAY-VERSION-DOWNGRADE may be signalled. See the algorithm below\n details.\n\nThe strategy is determined by the following algorithm, invoked\nwhenever an event is generated by a journaled @BLOCK:\n\n1. Log events are not matched to the replay. If the new event is a\n log event or a REPLAY-FAILURE has been signalled before (i.e. the\n record journal's JOURNAL-STATE is :MISMATCHED), then **insert**\n is returned.\n\n2. Else, log events to be read in the replay journal are skipped,\n and the next unread, non-log event is peeked at (without\n advancing the replay journal).\n\n - **end of replay**: If there are no replay events left, then:\n\n - If REPLAY-EOJ-ERROR-P is NIL in WITH-JOURNALING (the\n default), **insert** is returned.\n\n - If REPLAY-EOJ-ERROR-P is true, then **`END-OF-JOURNAL`**\n is signalled.\n\n - **mismatched name**: Else, if the next unread replay event's\n name is not EQUAL to the name of the new event, then:\n\n - For VERSIONED-EVENTs, **REPLAY-NAME-MISMATCH** is\n signalled if INSERTABLE is NIL, else **insert** is\n returned.\n\n - For EXTERNAL-EVENTs, **REPLAY-NAME-MISMATCH** is\n signalled.\n\n - **matching name**: Else, if the name of the next unread event\n in the replay journal is EQUAL to the name of new event, then\n it is chosen as the *replay* event.\n\n - If the replay event's version is higher than the new\n event's version, then **REPLAY-VERSION-DOWNGRADE** is\n signalled.\n\n - If the two versions are equal, then **match** is returned.\n\n - If the new event's version is higher, then **upgrade** is\n returned.\n\n Where :INFINITY is considered higher than any integer and\n equal to itself.\n\nIn summary:\n\n | new event | end-of-replay | mismatched name | matching name |\n |-----------+-------------------+-------------------+---------------|\n | Log | insert | insert | insert |\n | Versioned | insert/eoj-error | insert/name-error | match-version |\n | External | insert/eoj-error | insert/name-error | match-version |\n\nVersion matching (`match-version` above) is based on which event has\na higher version:\n\n | replay event | = | new event |\n |-----------------+-------+-----------|\n | downgrade-error | match | upgrade |\n\n\n- [glossary-term] replay event\n\n The replay event is the next event to be read from REPLAY-JOURNAL\n which is not to be skipped. There may be no replay event if there\n are no more unread events in the replay journal.\n \n An event in the replay journal is skipped if it is a LOG-EVENT or\n there is a WITH-REPLAY-FILTER with a matching :SKIP. If :SKIP is in\n effect, the replay event may be indeterminate.\n \n Events from the replay journal are read when they are `:MATCH`ed or\n `:UPGRADE`d (see @THE-REPLAY-STRATEGY), when nested events are\n echoed while @REPLAYING-THE-OUTCOME, or when there is an @INVOKED\n defined with the same name as the replay event.\n \n The replay event is available via PEEK-REPLAY-EVENT.\n\n### Matching in-events\n\nIf the replay strategy is *match*, then, for in-events, the\nmatching process continues like this:\n\n- If the EVENT-ARGS are not EQUAL, then **`REPLAY-ARGS-MISMATCH`**\n signalled.\n\n- At this point, two things might happen:\n\n - For VERSIONED-EVENTs, the @BLOCK will be executed as normal\n and its outcome will be matched to the @REPLAY-EVENT (see\n @MATCHING-OUT-EVENTS).\n\n - For EXTERNAL-EVENTs, the corresponding replay OUT-EVENT is\n looked at. If there is one, meaning that the frame finished\n with an @EXPECTED-OUTCOME, then its outcome will be\n replayed (see @REPLAYING-THE-OUTCOME). If the OUT-EVENT is\n missing, then EXTERNAL-EVENTs behave like VERSIONED-EVENTs,\n and the @BLOCK is executed.\n\n\n#### Replaying the outcome\n\nSo, if an in-event is triggered that matches the replay,\nEVENT-VERSION is :INFINITY, then normal execution is altered in the\nfollowing manner:\n\n- The journaled @BLOCK is not executed.\n\n- To keep execution and the replay journal in sync, events of frames\n nested in the current one are skipped over in the replay journal.\n\n- All events (including LOG-EVENTs) skipped over are echoed to the\n record journal. This serves to keep a trail of what happened\n during the original recording. Note that functions corresponding\n to @INVOKED frames are called when their IN-EVENT is skipped over.\n\n- The out-event corresponding to the in-event being processed is\n then read from the replay journal and is recorded again (to allow\n recording to function properly).\n\nTo be able to reproduce the outcome in the replay journal, some\nassistance may be required from REPLAY-VALUES and REPLAY-CONDITION:\n\n- If the @REPLAY-EVENT has a normal return (i.e. EVENT-EXIT :VALUES),\n then the recorded return values (in EVENT-OUTCOME) are returned\n immediately as in `(VALUES-LIST (EVENT-OUTCOME REPLAY-EVENT))`. If\n REPLAY-VALUES is specified, it is called instead of VALUES-LIST.\n See @WORKING-WITH-UNREADABLE-VALUES for an example.\n\n- Similarly, if the replay event has unwound with an expected\n condition (has EVENT-EXIT :CONDITION), then the recorded\n condition (in EVENT-OUTCOME) is signalled as\n IN `(ERROR (EVENT-OUTCOME REPLAY-EVENT))`. If REPLAY-CONDITION is\n specified, it is called instead of ERROR. REPLAY-CONDITION must\n not return normally, and it's a JOURNAL-ERROR if it does.\n\nWITH-REPLAY-FILTER's NO-REPLAY-OUTCOME can selectively turn off\nreplaying the outcome. See @TESTING-ON-MULTIPLE-LEVELS, for an\nexample.\n\n### Matching out-events\n\nIf there were no @REPLAY-FAILURES during the matching of the\nIN-EVENT, and the conditions for @REPLAYING-THE-OUTCOME were not\nmet, then the @BLOCK is executed. When the outcome of the block is\ndetermined, an OUT-EVENT is triggered and is matched to the replay\njournal. The matching of out-events starts out as in\n@THE-REPLAY-STRATEGY with checks for EVENT-NAME and\nEVENT-VERSION.\n\nIf the replay strategy is *insert* or *upgrade*, then the out-event\nis written to RECORD-JOURNAL, consuming an event with a matching\nname from the REPLAY-JOURNAL in the latter case. If the strategy is\n*match*, then:\n\n- If the new event has an @UNEXPECTED-OUTCOME, then\n **REPLAY-UNEXPECTED-OUTCOME** is signalled. Note that the replay\n event always has an @EXPECTED-OUTCOME due to the handling of\n RECORD-UNEXPECTED-OUTCOME.\n\n- If the new event has an @EXPECTED-OUTCOME, then unless the new and\n @REPLAY-EVENT's EVENT-EXITs are `EQ` and their EVENT-OUTCOMEs are\n EQUAL, **REPLAY-OUTCOME-MISMATCH** is signalled.\n\n- Else, the replay event is consumed and the new event is written\n the RECORD-JOURNAL.\n\nNote that @THE-REPLAY-STRATEGY for the in-event and the out-event of\nthe same @FRAME may differ if the corresponding out-event is not\npresent in REPLAY-JOURNAL, which may be the case when the recording\nprocess failed hard without unwinding properly, or when an\n@UNEXPECTED-OUTCOME triggered the transition to JOURNAL-STATE\n:LOGGING.\n\n### Replay failures\n\n- [condition] REPLAY-FAILURE SERIOUS-CONDITION\n\n A abstract superclass (never itself signalled) for\n all kinds of mismatches between the events produced and the replay\n journal. Signalled only in JOURNAL-STATE :REPLAYING and only once\n per WITH-JOURNALING. If a REPLAY-FAILURE is signalled for an EVENT,\n then the event will be recorded, but RECORD-JOURNAL will transition\n to JOURNAL-STATE :MISMATCHED. Like JOURNALING-FAILURE, this is a\n serious condition because it is to be handled outside the enclosing\n WITH-JOURNALING. If a REPLAY-FAILURE were to be handled inside the\n WITH-JOURNALING, keep in mind that in :MISMATCHED, replay always\n uses the *insert* replay strategy (see @THE-REPLAY-STRATEGY).\n\n- [reader] REPLAY-FAILURE-NEW-EVENT [REPLAY-FAILURE][2e9b] (:NEW-EVENT)\n\n- [reader] REPLAY-FAILURE-REPLAY-EVENT [REPLAY-FAILURE][2e9b] (:REPLAY-EVENT)\n\n- [reader] REPLAY-FAILURE-REPLAY-JOURNAL [REPLAY-FAILURE][2e9b] (= '(REPLAY-JOURNAL))\n\n- [condition] REPLAY-NAME-MISMATCH REPLAY-FAILURE\n\n Signalled when the new event's and @REPLAY-EVENT's\n EVENT-NAME are not EQUAL. The REPLAY-FORCE-INSERT,\n REPLAY-FORCE-UPGRADE restarts are provided.\n\n- [condition] REPLAY-VERSION-DOWNGRADE REPLAY-FAILURE\n\n Signalled when the new event and the @REPLAY-EVENT\n have the same EVENT-NAME, but the new event has a lower version. The\n REPLAY-FORCE-UPGRADE restart is provided.\n\n- [condition] REPLAY-ARGS-MISMATCH REPLAY-FAILURE\n\n Signalled when the new event's and @REPLAY-EVENT's\n EVENT-ARGS are not EQUAL. The REPLAY-FORCE-UPGRADE restart is\n provided.\n\n- [condition] REPLAY-OUTCOME-MISMATCH REPLAY-FAILURE\n\n Signalled when the new event's and @REPLAY-EVENT's\n EVENT-EXIT and/or EVENT-OUTCOME are not EQUAL. The\n REPLAY-FORCE-UPGRADE restart is provided.\n\n- [condition] REPLAY-UNEXPECTED-OUTCOME REPLAY-FAILURE\n\n Signalled when the new event has an\n @UNEXPECTED-OUTCOME. Note that the @REPLAY-EVENT always has an\n @EXPECTED-OUTCOME due to the logic of RECORD-UNEXPECTED-OUTCOME. No\n restarts are provided.\n\n- [condition] REPLAY-INCOMPLETE REPLAY-FAILURE\n\n Signalled if there are unprocessed non-log events in\n REPLAY-JOURNAL when WITH-JOURNALING finishes and the body of\n WITH-JOURNALING returned normally, which is to prevent this\n condition to cancel an ongoing unwinding. No restarts are provided.\n\n- [restart] REPLAY-FORCE-INSERT\n\n This restart forces @THE-REPLAY-STRATEGY to be :INSERT, overriding\n REPLAY-NAME-MISMATCH. This is intended for upgrades, and extreme\n care must be taken not to lose data.\n\n- [restart] REPLAY-FORCE-UPGRADE\n\n This restart forces @THE-REPLAY-STRATEGY to be :UPGRADE, overriding\n REPLAY-NAME-MISMATCH, REPLAY-VERSION-DOWNGRADE,\n REPLAY-ARGS-MISMATCH, REPLAY-OUTCOME-MISMATCH. This is intended for\n upgrades, and extreme care must be taken not to lose data.\n\n### Upgrades and replay\n\nThe replay mechanism is built on the assumption that the tree of\n@FRAMEs is the same when the code is replayed as it was when the\nreplay journal was originally recorded. Thus, non-deterministic\ncontrol flow poses a challenge, but non-determinism can be isolated\nwith EXTERNAL-EVENTs. However, when the code changes, we might find\nthe structure of frames in previous recordings hard to accommodate.\nIn this case, we might decide to alter the structure, giving up some\nof the safety provided by the replay mechanism. There are various\ntools at our disposal to control this tradeoff between safety and\nflexibility:\n\n- We can insert individual frames with JOURNALED's INSERTABLE,\n upgrade frames by bumping JOURNALED's VERSION, and filter frames\n with WITH-REPLAY-FILTER. This option allows for the most\n consistency checks.\n\n- The REPLAY-FORCE-UPGRADE and REPLAY-FORCE-INSERT restarts allow\n overriding @THE-REPLAY-STRATEGY, but their use requires great care\n to be taken.\n\n- Or we may decide to keep the bare minimum of the replay journal\n around and discard everything except for EXTERNAL-EVENTs. This\n option is equivalent to\n\n (let ((*force-insertable* t))\n (with-replay-filter (:skip '((:name nil)))\n 42))\n\n- Rerecording the journal without replay might be another option if\n there are no EXTERNAL-EVENTs to worry about.\n\n- Finally, we can rewrite the replay journal using the low-level\n interface (see @STREAMLETS-REFERENCE). In this case, extreme care\n must be taken not to corrupt the journal (and lose data) as there\n are no consistency checks to save us.\n\nWith that, let's see how WITH-REPLAY-FILTER works.\n\n- [macro] WITH-REPLAY-STREAMLET (VAR) \u0026BODY BODY\n\n Open REPLAY-JOURNAL for reading with WITH-OPEN-JOURNAL, set the\n READ-POSITION on it to the event next read by the @REPLAY\n mechanism (which is never a LOG-EVENT). The low-level\n @READING-FROM-STREAMLETS api is then available to inspect the\n contents of the replay. It is an error if REPLAY-JOURNAL is NIL.\n\n- [function] PEEK-REPLAY-EVENT\n\n Return the @REPLAY-EVENT to be read from REPLAY-JOURNAL. This is\n roughly equivalent to\n \n ```\n (when (replay-journal)\n (with-replay-streamlet (streamlet)\n (peek-event streamlet))\n ```\n \n except PEEK-REPLAY-EVENT takes into account WITH-REPLAY-FILTER\n :MAP, and it may return `(:INDETERMINATE)` if WITH-REPLAY-FILTER\n :SKIP is in effect and what events are to be skipped cannot be\n decided until the next in-event generated by the code.\n \n Imagine a business process for paying an invoice. In the first\n version of this process, we just pay the invoice:\n \n ```\n (replayed (pay))\n ```\n \n We have left the implementation of PAY blank. In the second version,\n we need to get an approval first:\n \n ```\n (when (replayed (get-approval)\n (= (random 2) 0))\n (replayed (pay)))\n ```\n \n Replaying a journal produced by the first version of the code with\n the second version would run into difficulties because inserting\n EXTERNAL-EVENTs is tricky.\n \n We have to first decide how to handle the lack of approval in the\n first version. Here, we just assume the processes started by the\n first version get approval automatically. The implementation is\n based on a dummy `PROCESS` block whose version is bumped when the\n payment process changes and is inspected at the start of journaling.\n \n When v1 is replayed with v2, we introduce an INSERTABLE, versioned\n `GET-APPROVAL` block that just returns T. When replaying the code\n again, still with v2, the `GET-APPROVAL` block will be upgraded to\n :INFINITY.\n \n ```\n (let ((bundle (make-in-memory-bundle)))\n ;; First version of the payment process. Just pay.\n (with-bundle (bundle)\n (checked (process :version 1))\n (replayed (pay)))\n ;; Second version of the payment process. Only pay if approved.\n (loop repeat 2 do\n (with-bundle (bundle)\n (let ((replay-process-event (peek-replay-event)))\n (checked (process :version 2))\n (when (if (and replay-process-event\n (\u003c (event-version replay-process-event) 2))\n ;; This will be upgraded to :INFINITY the second\n ;; time around the LOOP.\n (checked (get-approval :insertable t)\n t)\n (replayed (get-approval)\n (= (random 2) 0)))\n (replayed (pay)))))))\n ```\n\n- [macro] WITH-REPLAY-FILTER (\u0026KEY MAP SKIP NO-REPLAY-OUTCOME) \u0026BODY BODY\n\n WITH-REPLAY-FILTER performs journal upgrade during replay by\n allowing events to be transformed as they are read from the replay\n journal or skipped if they match some patterns. For how to add new\n blocks in a code upgrade, see JOURNALED's :INSERTABLE argument. In\n addition, it also allows some control over @REPLAYING-THE-OUTCOME.\n \n - MAP: A function called with an event read from the replay journal\n which returns a transformed event. See @EVENTS-REFERENCE. MAP\n takes effect before before SKIP.\n \n - SKIP: In addition to filtering out LOG-EVENTs (which always\n happens during replay), filter out all events that belong to\n frames that match any of its SKIP patterns. Filtered out events\n are never seen by JOURNALED as it replays events. SKIP patterns\n are of the format `(\u0026KEY NAME VERSION\u003c)`, where VERSION\\\u003c is a\n valid EVENT-VERSION, and NAME may be NIL, which acts as a\n wildcard.\n \n SKIP is for when JOURNALED @BLOCKs are removed from the code,\n which would render replaying previously recorded journals\n impossible. Note that, for reasons of safety, it is not possible\n to filter EXTERNAL-EVENTs.\n \n - NO-REPLAY-OUTCOME is a list of EVENT-NAMEs. @REPLAYING-THE-OUTCOME\n is prevented for frames with EQUAL names. See\n @TESTING-ON-MULTIPLE-LEVELS for an example.\n \n WITH-REPLAY-FILTER affects only the immediately enclosing\n WITH-JOURNALING. A WITH-REPLAY-FILTER nested within another in the\n same WITH-JOURNALING inherits the SKIP patterns of its parent, to\n which it adds its own. The MAP function is applied to before the\n parent's MAP.\n \n Examples of SKIP patterns:\n \n ```\n ;; Match events with name FOO and version 1, 2, 3 or 4\n (:name foo :version\u003c 5)\n ;; Match events with name BAR and any version\n (:name bar :version\u003c :infinity)\n ;; Same as the previous\n (:name bar)\n ;; Match all names\n (:name nil)\n ;; Same as the previous\n ()\n ```\n \n Skipping can be thought of as removing nodes of the tree of frames,\n connecting its children to its parent. The following example removes\n frames `J1` and `J2` from around `J3`, the `J1` frame from within\n `J3`, and the third `J1` frame.\n \n ```\n (let ((journal (make-in-memory-journal)))\n ;; Record trees J1 -\u003e J2 -\u003e J3 -\u003e J1, and J1.\n (with-journaling (:record journal)\n (checked (j1)\n (checked (j2)\n (checked (j3)\n (checked (j1)\n 42))))\n (checked (j1)\n 7))\n ;; Filter out all occurrences of VERSIONED-EVENTs named J1 and\n ;; J2 from the replay, leaving only J3 to match.\n (with-journaling (:replay journal :record t :replay-eoj-error-p t)\n (with-replay-filter (:skip '((:name j1) (:name j2)))\n (checked (j3)\n 42))))\n ```\n\n## Testing\n\nHaving discussed the @REPLAY mechanism, next are @TESTING and\n@PERSISTENCE, which rely heavily on replay. Suppose we want to unit\ntest user registration. Unfortunately, the code communicates with a\ndatabase service and also takes input from the user. A natural\nsolution is to create @MOCK-OBJECTs for these external systems to\nunshackle the test from the cumbersome database dependency and to\nallow it to run without user interaction.\n\nWe do this below by wrapping external interaction in JOURNALED with\n:VERSION :INFINITY (see @REPLAYING-THE-OUTCOME).\n\n```\n(defparameter *db* (make-hash-table))\n\n(defun set-key (key value)\n (replayed (\"set-key\" :args `(,key ,value))\n (format t \"Updating db~%\")\n (setf (gethash key *db*) value)\n nil))\n\n(defun get-key (key)\n (replayed (\"get-key\" :args `(,key))\n (format t \"Query db~%\")\n (gethash key *db*)))\n\n(defun ask-username ()\n (replayed (\"ask-username\")\n (format t \"Please type your username: \")\n (read-line)))\n\n(defun maybe-win-the-grand-prize ()\n (checked (\"maybe-win-the-grand-prize\")\n (when (= 1000000 (hash-table-count *db*))\n (format t \"You are the lucky one!\"))))\n\n(defun register-user (username)\n (unless (get-key username)\n (set-key username `(:user-object :username ,username))\n (maybe-win-the-grand-prize)))\n```\n\nNow, we write a test that records these interactions in a file when\nit's run for the first time.\n\n```\n(define-file-bundle-test (test-user-registration\n :directory (asdf:system-relative-pathname\n :journal \"test/registration/\"))\n (let ((username (ask-username)))\n (register-user username)\n (assert (get-key username))\n (register-user username)\n (assert (get-key username))))\n\n;; Original recording: everything is executed\nJRN\u003e (test-user-registration)\nPlease type your username: joe\nQuery db\nUpdating db\nQuery db\nQuery db\nQuery db\n=\u003e NIL\n```\n\nOn reruns, none of the external stuff is executed. The return values\nof the external JOURNALED blocks are replayed from the journal:\n\n```\n;; Replay: all external interactions are mocked.\nJRN\u003e (test-user-registration)\n=\u003e NIL\n```\n\nShould the code change, we might want to upgrade carefully (see\n@UPGRADES-AND-REPLAY) or just rerecord from scratch:\n\n```\nJRN\u003e (test-user-registration :rerecord t)\nPlease type your username: joe\nQuery db\nUpdating db\nQuery db\nQuery db\nQuery db\n=\u003e NIL\n```\n\nThus satisfied that our test runs, we can commit the journal file in\nthe bundle into version control. Its contents are:\n\n```\n\n(:IN \"ask-username\" :VERSION :INFINITY)\n(:OUT \"ask-username\" :VERSION :INFINITY :VALUES (\"joe\" NIL))\n(:IN \"get-key\" :VERSION :INFINITY :ARGS (\"joe\"))\n(:OUT \"get-key\" :VERSION :INFINITY :VALUES (NIL NIL))\n(:IN \"set-key\" :VERSION :INFINITY :ARGS (\"joe\" (:USER-OBJECT :USERNAME \"joe\")))\n(:OUT \"set-key\" :VERSION :INFINITY :VALUES (NIL))\n(:IN \"maybe-win-the-grand-prize\" :VERSION 1)\n(:OUT \"maybe-win-the-grand-prize\" :VERSION 1 :VALUES (NIL))\n(:IN \"get-key\" :VERSION :INFINITY :ARGS (\"joe\"))\n(:OUT \"get-key\" :VERSION :INFINITY :VALUES ((:USER-OBJECT :USERNAME \"joe\") T))\n(:IN \"get-key\" :VERSION :INFINITY :ARGS (\"joe\"))\n(:OUT \"get-key\" :VERSION :INFINITY :VALUES ((:USER-OBJECT :USERNAME \"joe\") T))\n(:IN \"get-key\" :VERSION :INFINITY :ARGS (\"joe\"))\n(:OUT \"get-key\" :VERSION :INFINITY :VALUES ((:USER-OBJECT :USERNAME \"joe\") T))\n```\n\nNote that when this journal is replayed, new VERSIONED-EVENTs are\nrequired to match the replay. So, after the original recording, we\ncan check by eyeballing that the record represents a correct\nexecution. Then on subsequent replays, even though\n`MAYBE-WIN-THE-GRAND-PRIZE` sits behind `REGISTER-USER` and is hard\nto test with ASSERTs, the replay mechanism verifies that it is\ncalled only for new users.\n\nThis record-and-replay style of testing is not the only possibility:\ndirect inspection of a journal with the low-level events api (see\n@EVENTS-REFERENCE) can facilitate checking non-local invariants.\n\n- [macro] DEFINE-FILE-BUNDLE-TEST (NAME \u0026KEY DIRECTORY (EQUIVALENTP T)) \u0026BODY BODY\n\n Define a function with NAME for record-and-replay testing. The\n function's BODY is executed in a WITH-BUNDLE to guarantee\n replayability. The bundle in question is a FILE-BUNDLE created in\n DIRECTORY. The function has a single keyword argument, RERECORD. If\n RERECORD is true, the bundle is deleted with DELETE-FILE-BUNDLE to\n start afresh.\n \n Furthermore, if BODY returns normally, and it is a replay of a\n previous run, and EQUIVALENTP, then it is ASSERTed that the record\n and replay journals are EQUIVALENT-REPLAY-JOURNALS-P. If this check\n fails, RECORD-JOURNAL is discarded when the function returns. In\n addition to the replay consistency, this checks that no inserts or\n upgrades were performed (see @THE-REPLAY-STRATEGY).\n\n### Testing on multiple levels\n\nNesting REPLAYEDs (that is, @FRAMEs of EXTERNAL-EVENTs) is not\nobviously useful since the outer REPLAYED will be replayed by\noutcome, and the inner one will be just echoed to the record\njournal. However, if we turn off @REPLAYING-THE-OUTCOME for the\nouter, the inner will be replayed.\n\nThis is useful for testing layered communication. For example, we\nmight have written code that takes input from an external\nsystem (READ-LINE) and does some complicated\nprocessing (READ-FROM-STRING) before returning the input in a form\nsuitable for further processing. Suppose we wrap REPLAYED around\nREAD-FROM-STRING for @PERSISTENCE because putting it around\nREAD-LINE would expose low-level protocol details in the journal,\nmaking protocol changes difficult.\n\nHowever, upon realizing that READ-FROM-STRING was not the best tool\nfor the job and switching to PARSE-INTEGER, we want to test by\nreplaying all previously recorded journals. For this, we prevent the\nouter REPLAYED from being replayed by outcome with\nWITH-REPLAY-FILTER:\n\n```\n(let ((bundle (make-in-memory-bundle)))\n ;; Original with READ-FROM-STRING\n (with-bundle (bundle)\n (replayed (\"accept-number\")\n (values (read-from-string (replayed (\"input-number\")\n (read-line))))))\n ;; Switch to PARSE-INTEGER and test by replay.\n (with-bundle (bundle)\n (with-replay-filter (:no-replay-outcome '(\"accept-number\"))\n (replayed (\"accept-number\")\n ;; 1+ is our bug.\n (values (1+ (parse-integer (replayed (\"input-number\")\n (read-line)))))))))\n```\n\nThe inner `input-number` block is replayed by outcome, and\nPARSE-INTEGER is called with the string READ-LINE returned in the\noriginal invocation. The outcome of the outer `accept-number` block\nchecked as if it was a VERSIONED-EVENT and we get a\nREPLAY-OUTCOME-MISMATCH due to the bug.\n\n## Persistence\n\n### Persistence tutorial\n\nLet's write a simple game.\n\n```\n(defun play-guess-my-number ()\n (let ((my-number (replayed (think-of-a-number)\n (random 10))))\n (format t \"~%I thought of a number.~%\")\n (loop for i upfrom 0 do\n (write-line \"Guess my number:\")\n (let ((guess (replayed (read-guess)\n (values (parse-integer (read-line))))))\n (format t \"You guessed ~D.~%\" guess)\n (when (= guess my-number)\n (checked (game-won :args `(,(1+ i))))\n (format t \"You guessed it in ~D tries!\" (1+ i))\n (return))))))\n\n(defparameter *the-evergreen-game* (make-in-memory-bundle))\n```\n\n##### Original recording\n\nUnfortunately, the implementation is lacking in the input validation\ndepartment. In the transcript below, PARSE-INTEGER fails with `junk\nin string` when the user enters `not a number`:\n\n```\nCL-USER\u003e (handler-case\n (with-bundle (*the-evergreen-game*)\n (play-guess-my-number))\n (error (e)\n (format t \"Oops. ~A~%\" e)))\nI thought of a number.\nGuess my number:\n7 ; real user input\nYou guessed 7.\nGuess my number:\nnot a number ; real user input\nOops. junk in string \"not a number\"\n```\n\n##### Replay and extension\n\nInstead of fixing this bug, we just restart the game from the\nbeginning, @REPLAYING-THE-OUTCOME of external interactions marked\nwith REPLAYED:\n\n```\nCL-USER\u003e (with-bundle (*the-evergreen-game*)\n (play-guess-my-number))\nI thought of a number.\nGuess my number:\nYou guessed 7.\nGuess my number: ; New recording starts here\n5 ; real user input\nYou guessed 5.\nGuess my number:\n4 ; real user input\nYou guessed 4.\nGuess my number:\n2 ; real user input\nYou guessed 2.\nYou guessed it in 4 tries!\n```\n\n##### It's evergreen\n\nWe can now replay this game many times without any user interaction:\n\n```\nCL-USER\u003e (with-bundle (*the-evergreen-game*)\n (play-guess-my-number))\nI thought of a number.\nGuess my number:\nYou guessed 7.\nGuess my number:\nYou guessed 5.\nGuess my number:\nYou guessed 4.\nGuess my number:\nYou guessed 2.\nYou guessed it in 4 tries!\n```\n\n##### The generated events\n\nThis simple mechanism allows us to isolate external interactions and\nwrite tests in record-and-replay style based on the events produced:\n\n```\nCL-USER\u003e (list-events *the-evergreen-game*)\n((:IN THINK-OF-A-NUMBER :VERSION :INFINITY)\n (:OUT THINK-OF-A-NUMBER :VERSION :INFINITY :VALUES (2))\n (:IN READ-GUESS :VERSION :INFINITY)\n (:OUT READ-GUESS :VERSION :INFINITY :VALUES (7))\n (:IN READ-GUESS :VERSION :INFINITY :ARGS NIL)\n (:OUT READ-GUESS :VERSION :INFINITY :VALUES (5))\n (:IN READ-GUESS :VERSION :INFINITY :ARGS NIL)\n (:OUT READ-GUESS :VERSION :INFINITY :VALUES (4))\n (:IN READ-GUESS :VERSION :INFINITY :ARGS NIL)\n (:OUT READ-GUESS :VERSION :INFINITY :VALUES (2))\n (:IN GAME-WON :VERSION 1 :ARGS (4))\n (:OUT GAME-WON :VERSION 1 :VALUES (NIL)))\n```\n\nIn fact, being able to replay this game at all already checks it\nthrough the `GAME-WON` event that the number of tries calculation is\ncorrect.\n\nIn addition, thus being able to reconstruct the internal state of\nthe program gives us persistence by replay. If instead of a\nIN-MEMORY-BUNDLE, we used a FILE-BUNDLE, the game would have been\nsaved on disk without having to write any code for saving and\nloading the game state.\n\n##### Discussion\n\nPersistence by replay, also known as @EVENT-SOURCING, is appropriate\nwhen the external interactions are well-defined and stable. Storing\nevents shines in comparison to persisting state when the control\nflow is too complicated to be interrupted and resumed easily.\nResuming execution in deeply nested function calls is fraught with\nsuch peril that it is often easier to flatten the program into a\nstate machine, which is as pleasant as manually managing\n@CONTINUATIONs.\n\nIn contrast, the Journal library does not favour certain styles of\ncontrol flow and only requires that non-determinism is packaged up\nin REPLAYED, which allows it to reconstruct the state of the program\nfrom the recorded events at any point during its execution and\nresume from there.\n\n### Synchronization to storage\n\nIn the following, we explore how journals can serve as a\npersistence mechanism and the guarantees they offer. The high-level\nsummary is that journals with SYNC can serve as a durable and\nconsistent storage medium. The other two\n[ACID](https://en.wikipedia.org/wiki/ACID) properties, atomicity and\nisolation, do not apply because Journal is single-client and does\nnot need transactions.\n\n- [glossary-term] aborted execution\n\n Aborted execution is when the operating system or the application\n crashes, calls `abort()`, is killed by a `SIGKILL` signal or there\n is a power outage. Synchronization guarantees are defined in the\n face of aborted execution and do not apply to hardware errors, Lisp\n or OS bugs.\n\n- [glossary-term] data event\n\n Data events are the only events that may be non-deterministic. They\n record information that could change if the same code were run\n multiple times. Data events typically correspond to interactions\n with the user, servers or even the random number generator. Due to\n their non-determinism, they are the only parts of the journal not\n reproducible by rerunning the code. In this sense, only the data\n events are not redundant with the code, and whether other events are\n persisted does not affect durability. There are two kinds of data\n events:\n \n - An EXTERNAL-EVENT that is also an OUT-EVENT.\n \n - The IN-EVENT of an @INVOKED function, which lies outside the\n normal, deterministic control flow.\n\n#### Synchronization strategies\n\nWhen a journal or bundle is created (see MAKE-IN-MEMORY-JOURNAL,\nMAKE-FILE-JOURNAL, MAKE-IN-MEMORY-BUNDLE, MAKE-FILE-BUNDLE), the\nSYNC option determines when – as a RECORD-JOURNAL – the recorded\nevents and JOURNAL-STATE changes are persisted durably. For\nFILE-JOURNALs, persisting means calling something like `fsync`,\nwhile for IN-MEMORY-JOURNALs, a user defined function is called to\npersist the data.\n\n- NIL: Never synchronize. A FILE-JOURNAL's file may be corrupted on\n @ABORTED-EXECUTION. In IN-MEMORY-JOURNALs, SYNC-FN is never\n called.\n\n- T: This is the *no data loss* setting with minimal\n synchronization. It guarantees *consistency* (i.e. no corruption)\n and *durability* up to the most recent @DATA-EVENT written in\n JOURNAL-STATE :RECORDING or for the entire record journal in\n states :FAILED and :COMPLETED. :FAILED or :COMPLETED is guaranteed\n when leaving WITH-JOURNALING at the latest.\n\n- Values other than NIL and T are reserved for future extensions.\n Using them triggers a JOURNAL-ERROR.\n\n\n#### Synchronization with in-memory journals\n\nUnlike FILE-JOURNALs, IN-MEMORY-JOURNALs do not have any built-in\npersistent storage backing them, but with SYNC-FN, persistence can\nbe tacked on. If non-NIL, SYNC-FN must be a function of a single\nargument, an IN-MEMORY-JOURNAL. SYNC-FN is called according to\n@SYNCHRONIZATION-STRATEGIES, and upon normal return the journal must\nbe stored durably.\n\nThe following example saves the entire journal history when a new\n@DATA-EVENT is recorded. Note how `SYNC-TO-DB` is careful to\noverwrite `*DB*` only if it is called with a journal that has not\nfailed the replay (as in @REPLAY-FAILURES) and is sufficiently\ndifferent from the replay journal as determined by\nJOURNAL-DIVERGENT-P.\n\n```\n(defparameter *db* ())\n\n(defun sync-to-db (journal)\n (when (and (member (journal-state journal)\n '(:recording :logging :completed))\n (journal-divergent-p journal))\n (setq *db* (journal-events journal))\n (format t \"Saved ~S~%New events from position ~S~%\" *db*\n (journal-previous-sync-position journal))))\n\n(defun make-db-backed-record-journal ()\n (make-in-memory-journal :sync-fn 'sync-to-db))\n\n(defun make-db-backed-replay-journal ()\n (make-in-memory-journal :events *db*))\n\n(with-journaling (:record (make-db-backed-record-journal)\n :replay (make-db-backed-replay-journal))\n (replayed (a)\n 2)\n (ignore-errors\n (replayed (b)\n (error \"Whoops\"))))\n.. Saved #((:IN A :VERSION :INFINITY)\n.. (:OUT A :VERSION :INFINITY :VALUES (2)))\n.. New events from position 0\n.. Saved #((:IN A :VERSION :INFINITY)\n.. (:OUT A :VERSION :INFINITY :VALUES (2))\n.. (:IN B :VERSION :INFINITY)\n.. (:OUT B :ERROR (\"SIMPLE-ERROR\" \"Whoops\")))\n.. New events from position 2\n..\n```\n\nIn a real application, external events often involve unreliable or\nhigh-latency communication. In the above example, block `B` signals\nan error, say, to simulate some kind of network condition. Now, a\nnew journal *for replay* is created and initialized with the saved\nevents, and the whole process is restarted.\n\n```\n(defun run-with-db ()\n (with-journaling (:record (make-db-backed-record-journal)\n :replay (make-db-backed-replay-journal))\n (replayed (a)\n (format t \"A~%\")\n 2)\n (replayed (b)\n (format t \"B~%\")\n 3)))\n\n(run-with-db)\n.. B\n.. Saved #((:IN A :VERSION :INFINITY)\n.. (:OUT A :VERSION :INFINITY :VALUES (2))\n.. (:IN B :VERSION :INFINITY)\n.. (:OUT B :VERSION :INFINITY :VALUES (3)))\n.. New events from position 0\n..\n=\u003e 3\n```\n\nNote that on the rerun, block `A` is not executed because external\nevents are replayed simply by reproducing their outcome, in this\ncase returning 2. See @REPLAYING-THE-OUTCOME. Block `B`, on the\nother hand, was rerun because it had an @UNEXPECTED-OUTCOME the\nfirst time around. This time it ran without error, a @DATA-EVENT was\ntriggered, and SYNC-FN was invoked.\n\nIf we were to invoke the now completed `RUN-WITH-DB` again, it would\nsimply return 3 without ever invoking SYNC-FN:\n\n```\n(run-with-db)\n=\u003e 3\n```\n\nWith JOURNAL-REPLAY-MISMATCH, SYNC-FN can be optimized to to reuse\nthe sequence of events in the replay journal up until the point of\ndivergence.\n\n#### Synchronization with file journals\n\nFor FILE-JOURNALs, SYNC determines when the events written to the\nRECORD-JOURNAL and its JOURNAL-STATE will be persisted durably in\nthe file. Syncing to the file involves two calls to `fsync` and is\nnot cheap.\n\nSyncing events to files is implemented as follows.\n\n- When the journal file is created, its parent directory is\n immediately fsynced to make sure that the file will not be lost on\n @ABORTED-EXECUTION.\n\n- When an event is about to be written the first time after file\n creation or after a sync, a transaction start marker is written to\n the file.\n\n- Any number of events may be subsequently written until syncing is\n deemed necessary (see @SYNCHRONIZATION-STRATEGIES).\n\n- At this point, `fsync` is called to flush all event data and state\n changes to the file, and the transaction start marker is\n *overwritten* with a transaction completed marker and another\n `fsync` is performed.\n\n- When reading back this file (e.g. for replay), an open transaction\n marker is treated as the end of file.\n\nNote that this implementation assumes that after writing the start\ntransaction marker, a crash cannot leave any kind of garbage bytes\naround: it must leave zeros. This is not true for all filesytems.\nFor example, ext3/ext4 with `data=writeback` can leave garbage\naround.\n\n## Safety\n\n##### Thread safety\n\nChanges to journals come in two varieties: adding an event and\nchanging the JOURNAL-STATE. Both are performed by JOURNALED only\nunless the low-level streamlet interface is used (see\n@STREAMLETS-REFERENCE). Using JOURNALED wrapped in a\nWITH-JOURNALING, WITH-BUNDLE, or @LOG-RECORD without WITH-JOURNALING\nis thread-safe.\n\n- Every journal is guaranteed to have at most a single writer active\n at any time. Writers are mainly WITH-JOURNALING and WITH-BUNDLE,\n but any journals directly logged to have a log writer stored in\n the journal object. See @LOGGING.\n\n- WITH-JOURNALING and WITH-BUNDLE have dynamic extent as writers,\n but log writers of journals have indefinite extent: once a journal\n is used as a LOG-RECORD, there remains a writer.\n\n- Attempting to create a second writer triggers a JOURNAL-ERROR.\n\n- Writing to the same journal via @LOG-RECORD from multiple threads\n concurrently is possible since this doesn't create multiple\n writers. It is ensured with locking that events are written\n atomically. Frames can be interleaved, but these are LOG-EVENTs,\n so this does not affect replay.\n\n- The juggling of replay and record journals performed by\n WITH-BUNDLE is also thread-safe.\n\n- It is ensured that there is at most one FILE-JOURNAL object in the\n same Lisp image is backed by the same file.\n\n- Similarly, there is at most FILE-BUNDLE object for a directory.\n\n##### Process safety\n\nCurrently, there is no protection against multiple OS processes\nwriting the same FILE-JOURNAL or FILE-BUNDLE.\n\n##### Signal safety\n\nJournal is *designed* to be @ASYNC-UNWIND safe but *not reentrant*.\nInterrupts are disabled only for the most critical cleanup forms. If\na thread is killed without unwinding, that constitutes\n@ABORTED-EXECUTION, so guarantees about @SYNCHRONIZATION apply, but\nJOURNAL objects written by the thread are not safe to access, and\nthe Lisp should probably be restarted.\n\n## Events reference\n\nEvents are normally triggered upon entering and leaving the\ndynamic extent of a JOURNALED @BLOCK (see @IN-EVENTS and\n@OUT-EVENTS) and also by LOGGED. Apart from being part of the\nlow-level substrate of the Journal library, working with events\ndirectly is sometimes useful when writing tests that inspect\nrecorded events. Otherwise, skip this entire section.\n\nAll EVENTs have EVENT-NAME and EVENT-VERSION, which feature\nprominently in @THE-REPLAY-STRATEGY. After the examples in\n@IN-EVENTS and @OUT-EVENTS, the following example is a reminder of\nhow events look in the simplest case.\n\n```\n(with-journaling (:record t)\n (journaled (foo :version 1 :args '(1 2))\n (+ 1 2))\n (logged () \"Oops\")\n (list-events))\n=\u003e ((:IN FOO :VERSION 1 :ARGS (1 2))\n (:OUT FOO :VERSION 1 :VALUES (3))\n (:LEAF \"Oops\"))\n```\n\nSo, a JOURNALED @BLOCK generates an IN-EVENT and an OUT-EVENT, which\nare simple property lists. The following reference lists these\nproperties, their semantics and the functions to read them.\n\n- [type] EVENT\n\n An event is either an IN-EVENT, an OUT-EVENT or a LEAF-EVENT.\n\n- [function] EVENT= EVENT-1 EVENT-2\n\n Return whether EVENT-1 and EVENT-2 represent the same event.\n In- and out-events belonging to the same @FRAME are *not* the same\n event. EVENT-OUTCOMEs are not compared when EVENT-EXIT is :ERROR to\n avoid undue dependence on implementation specific string\n representations. This function is useful in conjunction with\n MAKE-IN-EVENT and MAKE-OUT-EVENT to write tests.\n\n- [function] EVENT-NAME EVENT\n\n The name of an event can be of any type. It is often a symbol or a\n string. When replaying, names are compared with EQUAL. All EVENTs\n have names. The names of the in- and out-events belonging to the\n same @FRAME are the same.\n\n### Event versions\n\n- [function] EVENT-VERSION EVENT\n\n Return the version of EVENT of type EVENT-VERSION.\n\n- [function] LOG-EVENT-P EVENT\n\n See if EVENT is a LOG-EVENT.\n\n- [function] VERSIONED-EVENT-P EVENT\n\n See if EVENT is a VERSIONED-EVENT.\n\n- [function] EXTERNAL-EVENT-P EVENT\n\n See if EVENT is an EXTERNAL-EVENT.\n\n### In-events\n\n- [type] IN-EVENT\n\n IN-EVENTs are triggered upon entering the dynamic extent of a\n JOURNALED @BLOCK. IN-EVENTs have EVENT-NAME,\n EVENT-VERSION, and EVENT-ARGS. See @IN-EVENTS for a more\n introductory treatment.\n\n- [function] IN-EVENT-P EVENT\n\n See if EVENT is a IN-EVENT.\n\n- [function] MAKE-IN-EVENT \u0026KEY NAME VERSION ARGS\n\n Create an IN-EVENT with NAME, VERSION (of type EVENT-VERSION) and\n ARGS as its EVENT-NAME, EVENT-VERSION and EVENT-ARGS.\n\n- [function] EVENT-ARGS IN-EVENT\n\n Return the arguments of IN-EVENT, normally populated using the ARGS\n form in JOURNALED.\n\n### Out-events\n\n- [type] OUT-EVENT\n\n OUT-EVENTs are triggered upon leaving the dynamic extent of the\n JOURNALED @BLOCK. OUT-EVENTs have EVENT-NAME,\n EVENT-VERSION, EVENT-EXIT and EVENT-OUTCOME.\n See @OUT-EVENTS for a more introductory treatment.\n\n- [function] OUT-EVENT-P EVENT\n\n See if EVENT is an OUT-EVENT.\n\n- [function] MAKE-OUT-EVENT \u0026KEY NAME VERSION EXIT OUTCOME\n\n Create an OUT-EVENT with NAME, VERSION (of type EVENT-VERSION),\n EXIT (of type EVENT-EXIT), and OUTCOME as its EVENT-NAME,\n EVENT-VERSION, EVENT-EXIT and EVENT-OUTCOME.\n\n- [function] EVENT-EXIT OUT-EVENT\n\n Return how the journaled @BLOCK finished. See EVENT-EXIT\n for the possible types.\n\n- [function] EXPECTED-OUTCOME-P OUT-EVENT\n\n See if OUT-EVENT has an @EXPECTED-OUTCOME.\n\n- [function] UNEXPECTED-OUTCOME-P OUT-EVENT\n\n See if OUT-EVENT has an @UNEXPECTED-OUTCOME.\n\n- [function] EVENT-OUTCOME OUT-EVENT\n\n Return the outcome of the @FRAME (or loosely speaking of a @BLOCK)\n to which OUT-EVENT belongs.\n\n### Leaf-events\n\n- [type] LEAF-EVENT\n\n Leaf events are triggered by LOGGED. Unlike IN-EVENTs and\n OUT-EVENTs, which represent a @FRAME, leaf events represent a point\n in execution thus cannot have children. They are also the poorest of\n their kind: they only have an EVENT-NAME. Their VERSION is always\n NIL, which makes them LOG-EVENTs.\n\n- [function] LEAF-EVENT-P EVENT\n\n See if EVENT is a LEAF-EVENT.\n\n- [function] MAKE-LEAF-EVENT NAME\n\n Create a LEAF-EVENT with NAME.\n\n## Journals reference\n\nIn @JOURNAL-BASICS, we covered the bare minimum needed to work with\njournals. Here, we go into the details.\n\n- [class] JOURNAL\n\n JOURNAL is an abstract base class for a sequence of\n events. In case of FILE-JOURNALs, the events are stored in a file,\n while for IN-MEMORY-JOURNALs, they are in a Lisp array. When a\n journal is opened, it is possible to perform I/O on it (see\n @STREAMLETS-REFERENCE), which is normally taken care of by\n WITH-JOURNALING. For this reason, the user's involvement with\n journals normally only consists of creating and using them in\n WITH-JOURNALING.\n\n- [reader] JOURNAL-STATE [JOURNAL][5082] (:STATE)\n\n Return the state of JOURNAL, which is of type\n JOURNAL-STATE.\n\n- [reader] JOURNAL-SYNC [JOURNAL][5082] (:SYNC = NIL)\n\n The SYNC argument specified at instantiation. See\n @SYNCHRONIZATION-STRATEGIES.\n\n- [function] SYNC-JOURNAL \u0026OPTIONAL (JOURNAL (RECORD-JOURNAL))\n\n Durably persist changes made to JOURNAL if JOURNAL-SYNC is T.\n The changes that are persisted are \n \n - WRITE-EVENTs and JOURNAL-STATE changes made in an enclosing\n WITH-JOURNALING; and\n \n - LOG-RECORDs from any thread. \n \n In particular, writes made in a WITH-JOURNALING in another thread\n are not persisted. SYNC-JOURNAL is a noop if JOURNAL-SYNC is NIL. It\n is safe to call from any thread.\n\n- [reader] JOURNAL-REPLAY-MISMATCH [JOURNAL][5082] (= NIL)\n\n If JOURNAL-DIVERGENT-P, then this is a list of two\n elements: the READ-POSITIONs in the RECORD-JOURNAL and\n REPLAY-JOURNAL of the first events that were different (ignoring\n LOG-EVENTs). It is NIL, otherwise.\n\n- [function] JOURNAL-DIVERGENT-P JOURNAL\n\n See if WITH-JOURNALING recorded any event so far in this journal\n that was not EQUAL to its @REPLAY-EVENT or it had no corresponding\n replay event. This completely ignores LOG-EVENTs in both journals\n being compared and can be called any time during @REPLAY. It plays a\n role in WITH-BUNDLE deciding when a journal is important enough to\n keep and also in @SYNCHRONIZATION-WITH-IN-MEMORY-JOURNALS.\n \n The position of the first mismatch is available via\n JOURNAL-REPLAY-MISMATCH.\n\n### Comparing journals\n\nAfter replay finished (i.e. WITH-JOURNALING completed), we can ask\nwhether there were any changes produced. This is answered in the\nstrictest sense by IDENTICAL-JOURNALS-P and somewhat more\nfunctionally by EQUIVALENT-REPLAY-JOURNALS-P.\n\nAlso see JOURNAL-DIVERGENT-P.\n\n- [generic-function] IDENTICAL-JOURNALS-P JOURNAL-1 JOURNAL-2\n\n Compare two journals in a strict sense: whether\n they have the same JOURNAL-STATE and the lists of their events (as\n in LIST-EVENTS) are EQUAL.\n\n- [generic-function] EQUIVALENT-REPLAY-JOURNALS-P JOURNAL-1 JOURNAL-2\n\n See if two journals are equivalent when used the\n for REPLAY in WITH-JOURNALING. EQUIVALENT-REPLAY-JOURNALS-P is like\n IDENTICAL-JOURNALS-P, but it ignores LOG-EVENTs and allows events\n with EVENT-EXIT :ERROR to differ in their outcomes, which may very\n well be implementation specific, anyway. Also, it considers two\n groups of states as different :NEW, :REPLAYING, :MISMATCHED, :FAILED\n vs :RECORDING, :LOGGING, COMPLETED.\n\nThe rest of section is about concrete subclasses of JOURNAL.\n\n### In-memory journals\n\n- [class] IN-MEMORY-JOURNAL JOURNAL\n\n IN-MEMORY-JOURNALs are backed by a non-persistent\n Lisp array of events. Much quicker than FILE-JOURNALs, they are\n ideal for smallish journals persisted manually (see\n @SYNCHRONIZATION-WITH-IN-MEMORY-JOURNALS for an example).\n \n They are also useful for writing tests based on what events were\n generated. They differ from FILE-JOURNALs in that events written to\n IN-MEMORY-JOURNALs are not serialized (and deserialized on replay)\n with the following consequences for the objects recorded by\n JOURNALED (i.e. its NAME, ARGS arguments, and also the return VALUES\n of the block, or the value returned by CONDITION):\n \n - These objects need not be @READABLE.\n \n - Their identity (`EQ`ness) is not lost.\n \n - They must **must not be mutated** in any way.\n\n- [function] MAKE-IN-MEMORY-JOURNAL \u0026KEY (EVENTS NIL EVENTSP) STATE (SYNC NIL SYNCP) SYNC-FN\n\n Create an IN-MEMORY-JOURNAL.\n \n The returned journal's JOURNAL-STATE will be set to STATE. If STATE\n is NIL, then it is replaced by a default value, which is :COMPLETED\n if the EVENTS argument is provided, else it is :NEW.\n \n Thus, `(make-in-memory-journal)` creates a journal suitable for\n recording, and to make a replay journal, use :STATE :COMPLETED with\n some sequence of EVENTS:\n \n ```\n (make-in-memory-journal :events '((:in foo :version 1)) :state :completed)\n ```\n \n SYNC determines when SYNC-FN will be invoked on the RECORD-JOURNAL.\n SYNC defaults to T if SYNC-FN, else to NIL. For a description of\n possible values, see @SYNCHRONIZATION-STRATEGIES. For more\n discussion, see @SYNCHRONIZATION-WITH-IN-MEMORY-JOURNALS.\n\n- [reader] JOURNAL-EVENTS [IN-MEMORY-JOURNAL][b668] (:EVENTS)\n\n A sequence of events in the journal. Not to be\n mutated by client code.\n\n- [reader] JOURNAL-PREVIOUS-SYNC-POSITION [IN-MEMORY-JOURNAL][b668] (= 0)\n\n The length of JOURNAL-EVENTS at the time of the\n most recent invocation of SYNC-FN.\n\n### File journals\n\n- [class] FILE-JOURNAL JOURNAL\n\n A FILE-JOURNAL is a journal whose contents and\n JOURNAL-STATE are persisted in a file. This is the JOURNAL\n subclass with out-of-the-box persistence, but see @FILE-BUNDLES for\n a more full-featured solution for repeated @REPLAYs.\n \n Since serialization in FILE-JOURNALs is built on top of Lisp READ\n and WRITE, everything that JOURNALED records in events (i.e. its\n NAME, ARGS arguments, and also the return VALUES of the block, or\n the value returned by CONDITION) must be @READABLE.\n \n File journals are human-readable and editable by hand with some\n care. When editing, the following needs to be remembered:\n \n - The first character of the file represents its JOURNAL-STATE. It\n is a `#\\Space` (for state :NEW, :REPLAYING, :MISMATCHED and\n :FAILED), or a `#\\Newline` (for state :RECORDING, :LOGGING and\n :COMPLETED).\n \n - If the journal has SYNC (see @SYNCHRONIZATION-STRATEGIES), then\n between two events, there may be `#\\Del` (also called `#\\Rubout`)\n or `#\\Ack` characters (CHAR-CODE 127 and 6). `#\\Del` marks the end\n of the journal contents that may be read back: it's kind of an\n uncommitted-transaction marker for the events that follow it.\n `#\\Ack` characters, of which there may be many in the file, mark\n the sequence of events until the next marker of either kind as\n valid (or committed). `#\\Ack` characters are ignored when reading\n the journal.\n \n Thus, when editing a file, don't change the first character and\n leave the `#\\Del` character, if any, where it is. Also see\n @SYNCHRONIZATION-WITH-FILE-JOURNALS.\n\n- [function] MAKE-FILE-JOURNAL PATHNAME \u0026KEY SYNC\n\n Return a FILE-JOURNAL backed by the file with PATHNAME. The file is\n created when the journal is opened for writing. For a description of\n SYNC, see @SYNCHRONIZATION-STRATEGIES.\n \n If there is already an existing FILE-JOURNAL backed by the same\n file, then that object is returned. If the existing object has\n different options (e.g. it has SYNC T while the SYNC argument is NIL\n here), then a JOURNAL-ERROR is signalled.\n \n If there is already an existing FILE-JOURNAL backed by the same\n file, the JOURNAL-STATE is not :NEW, but the file doesn't exist,\n then the existing object is **invalidated**: attempts to write will\n fail with JOURNAL-ERROR. If the existing journal object is being\n written, then invalidation fails with a JOURNAL-ERROR. After\n invalidation, a new FILE-JOURNAL object is created.\n\n- [reader] PATHNAME-OF [FILE-JOURNAL][8428] (:PATHNAME)\n\n The pathname of the file backing the journal.\n\n### Pretty-printing journals\n\n- [class] PPRINT-JOURNAL JOURNAL\n\n Events written to a PPRINT-JOURNAL have a\n customizable output format. PPRINT-JOURNALs are intended for\n producing prettier output for @LOGGING and @TRACING, but they do not\n support reads, so they cannot be used as a REPLAY-JOURNAL or in\n LIST-EVENTS, for example. On the other hand, events written to\n PPRINT-JOURNALs need not be @READABLE.\n\n- [function] MAKE-PPRINT-JOURNAL \u0026KEY (STREAM (MAKE-SYNONYM-STREAM '\\*STANDARD-OUTPUT\\*)) (PRETTY T) (PRETTIFIER 'PRETTIFY-EVENT) LOG-DECORATOR\n\n Creates a PPRINT-JOURNAL.\n\n- [accessor] PPRINT-JOURNAL-STREAM [PPRINT-JOURNAL][9150] (:STREAM = \\*STANDARD-OUTPUT\\*)\n\n The stream where events are dumped. May be set any\n time to another STREAM.\n\n- [accessor] PPRINT-JOURNAL-PRETTY [PPRINT-JOURNAL][9150] (:PRETTY = T)\n\n Whether to use PPRINT-JOURNAL-PRETTIFIER or write\n events in as the property lists they are. A\n @BOOLEAN-VALUED-SYMBOL.\n\n- [accessor] PPRINT-JOURNAL-PRETTIFIER [PPRINT-JOURNAL][9150] (:PRETTIFIER = 'PRETTIFY-EVENT)\n\n A function like PRETTIFY-EVENT that writes an\n event to a stream. Only used when PPRINT-JOURNAL-PRETTY, this is\n the output format customization knob. Also see @DECORATIONs.\n\n## Bundles reference\n\nIn @BUNDLES, we covered the repeated replay problem that\nWITH-BUNDLE automates. Here, we provide a reference for the bundle\nclasses.\n\n- [class] BUNDLE\n\n A BUNDLE consists of a sequence of journals which\n are all reruns of the same code, hopefully making more and more\n progress towards completion. These journals are @REPLAYs of the\n previous successful one, extending it with new events. Upon\n replay (see WITH-BUNDLE), the latest journal in the bundle in\n JOURNAL-STATE :COMPLETED plays the role of the replay journal, and a\n new journal is added to the bundle for recording. If the replay\n succeeds, this new journal eventually becomes :COMPLETED and takes\n over the role of the replay journal for future replays until another\n replay succeeds. When the bundle is created and it has no journals\n yet, the replay journal is an empty, completed one.\n \n This is an abstract base class. Direct subclasses are\n IN-MEMORY-BUNDLE and FILE-BUNDLE.\n\n- [accessor] MAX-N-FAILED [BUNDLE][d9b6] (:MAX-N-FAILED = 1)\n\n If MAX-N-FAILED is non-NIL, and the number of\n journals of JOURNAL-STATE :FAILED in the bundle exceeds\n its value, then some journals (starting with the oldest) are\n deleted.\n\n- [accessor] MAX-N-COMPLETED [BUNDLE][d9b6] (:MAX-N-COMPLETED = 1)\n\n If MAX-N-COMPLETED is non-NIL, and the number of\n journals of JOURNAL-STATE :COMPLETED in the bundle exceeds\n its value, then some journals (starting with the oldest) are\n deleted.\n\n### In-memory bundles\n\n- [class] IN-MEMORY-BUNDLE BUNDLE\n\n An IN-MEMORY-BUNDLE is a BUNDLE that is built on\n IN-MEMORY-JOURNALs. IN-MEMORY-BUNDLEs have limited utility as a\n persistence mechanism and are provided mainly for reasons of\n symmetry and for testing. See\n @SYNCHRONIZATION-WITH-IN-MEMORY-JOURNALS for an example of how to\n achieve persistence without bundles.\n\n- [function] MAKE-IN-MEMORY-BUNDLE \u0026KEY (MAX-N-FAILED 1) (MAX-N-COMPLETED 1) SYNC SYNC-FN\n\n Create a new IN-MEMORY-BUNDLE with MAX-N-FAILED and MAX-N-COMPLETED. SYNC and SYNC-FN\n are passed on to MAKE-IN-MEMORY-JOURNAL.\n\n### File bundles\n\n- [class] FILE-BUNDLE BUNDLE\n\n A FILE-BUNDLE is a BUNDLE that is built on\n FILE-JOURNALs. It provides easy replay-based persistence.\n\n- [reader] DIRECTORY-OF [FILE-BUNDLE][1895] (:DIRECTORY)\n\n The directory where the files backing the\n FILE-JOURNALs in the FILE-BUNDLE are kept.\n\n- [function] MAKE-FILE-BUNDLE DIRECTORY \u0026KEY (MAX-N-FAILED 1) (MAX-N-COMPLETED 1) SYNC\n\n Return a FILE-BUNDLE object backed by FILE-JOURNALs in DIRECTORY.\n See MAX-N-FAILED and\n MAX-N-COMPLETED. For a description of SYNC, see\n @SYNCHRONIZATION-STRATEGIES.\n \n If there is already a FILE-BUNDLE with the same directory (according\n to TRUENAME), return that object is returned if it has the same\n MAX-N-FAILED, MAX-N-COMPLETED and SYNC options, else JOURNAL-ERROR\n is signalled.\n\n- [function] DELETE-FILE-BUNDLE DIRECTORY\n\n Delete all journal files (`*.jrn`) from DIRECTORY. Delete the\n directory if empty after the journal files were deleted, else signal\n an error. Existing FILE-BUNDLE objects are not updated, so\n MAKE-FILE-JOURNAL with FORCE-RELOAD may be required.\n\n## Streamlets reference\n\nThis section is relevant mostly for implementing new kinds of\nJOURNALs in addition to FILE-JOURNALs and IN-MEMORY-JOURNALs. In\nnormal operation, STREAMLETs are not worked with directly.\n\n### Opening and closing\n\n- [class] STREAMLET\n\n A STREAMLET is a handle to perform I/O on a\n JOURNAL. The high-level stuff (WITH-JOURNALING, JOURNALED, etc) is\n built on top of streamlets.\n\n- [reader] JOURNAL [STREAMLET][7a2f] (:JOURNAL)\n\n The JOURNAL that was passed to OPEN-STREAMLET.\n This is the journal STREAMLET operates on.\n\n- [generic-function] OPEN-STREAMLET JOURNAL \u0026KEY DIRECTION\n\n Return a STREAMLET suitable for performing I/O on\n JOURNAL. DIRECTION (defaults to :INPUT) is one of :INPUT, :OUTPUT,\n :IO, and it has the same purpose as the similarly named argument of\n CL:OPEN.\n\n- [generic-function] CLOSE-STREAMLET STREAMLET\n\n Close STREAMLET, which was returned by\n OPEN-STREAMLET. After closing, STREAMLET may not longer be used for\n IO.\n\n- [generic-function] MAKE-STREAMLET-FINALIZER STREAMLET\n\n Return NIL or a function of no arguments suitable\n as a finalizer for STREAMLET. That is, a function that closes\n STREAMLET but holds no reference to it. This is intended for\n streamlets that are not dynamic-extent, so using WITH-OPEN-JOURNAL\n is not appropriate.\n\n- [generic-function] OPEN-STREAMLET-P STREAMLET\n\n Return true if STREAMLET is open. STREAMLETs are\n open until they have been explicitly closed with CLOSE-STREAMLET.\n\n- [function] INPUT-STREAMLET-P STREAMLET\n\n See if STREAMLET was opened for input (the DIRECTION argument of\n OPEN-STREAMLET was :INPUT or :IO).\n\n- [function] OUTPUT-STREAMLET-P STREAMLET\n\n See if STREAMLET was opened for input (the DIRECTION argument of\n OPEN-STREAMLET was :OUTPUT or :IO).\n\n- [macro] WITH-OPEN-JOURNAL (VAR JOURNAL \u0026KEY (DIRECTION :INPUT)) \u0026BODY BODY\n\n This is like WITH-OPEN-FILE but for JOURNALs.\n Open the journal designated by JOURNAL (see TO-JOURNAL) with\n OPEN-STREAMLET, passing DIRECTION along, and bind VAR to the\n resulting STREAMLET. Call CLOSE-STREAMLET after BODY finishes. If\n JOURNAL is NIL, then VAR is bound to NIL and no streamlet is\n created.\n\n- [condition] STREAMLET-ERROR ERROR\n\n Like CL:STREAM-ERROR: failures pertaining to I/O on\n a closed STREAMLET or of the wrong DIRECTION. Actual I/O errors are\n *not* encapsulated in STREAMLET-ERROR.\n\n### Reading from streamlets\n\n- [generic-function] READ-EVENT STREAMLET \u0026OPTIONAL EOJ-ERROR-P\n\n Read the event at the current read position from\n STREAMLET, and move the read position to the event after. If there\n are no more events, signal END-OF-JOURNAL or return NIL depending on\n EOJ-ERROR-P. Signals STREAMLET-ERROR if STREAMLET is not\n INPUT-STREAMLET-P or not OPEN-STREAMLET-P.\n\n- [generic-function] READ-POSITION STREAMLET\n\n Return an integer that identifies the position of\n the next event to be read from STREAMLET. `SETF`able, see\n SET-READ-POSITION.\n\n- [generic-function] SET-READ-POSITION STREAMLET POSITION\n\n Set the read position of STREAMLET to POSITION,\n which must have been acquired from READ-POSITION.\n\n- [macro] SAVE-EXCURSION (STREAMLET) \u0026BODY BODY\n\n Save READ-POSITION of STREAMLET, execute BODY, and make sure to\n restore the saved read position.\n\n- [generic-function] PEEK-EVENT STREAMLET\n\n Read the next event from STREAMLET without changing\n the read position, or return NIL if there is no event to be read.\n\n- [method] PEEK-EVENT (STREAMLET STREAMLET)\n\n This is a slow default implementation, which relies on\n SAVE-EXCURSION and READ-EVENT.\n\n### Writing to streamlets\n\n- [generic-function] WRITE-EVENT EVENT STREAMLET\n\n Write EVENT to STREAMLET.\n Writing always happens at the end of STREAMLET's journal regardless\n of the READ-POSITION, and the read position is not changed. Signals\n STREAMLET-ERROR if STREAMLET is not OUTPUT-STREAMLET-P or not\n OPEN-STREAMLET-P.\n\n- [method] WRITE-EVENT EVENT (JOURNAL JOURNAL)\n\n For convenience, it is possible to write directly to a JOURNAL,\n in which case the journal's internal output streamlet is used.\n This internal streamlet is opened for :OUTPUT and may be used by\n @LOG-RECORD.\n\n- [generic-function] WRITE-POSITION STREAMLET\n\n Return an integer that identifies the position of\n the next event to be written to STREAMLET.\n\n- [generic-function] REQUEST-COMPLETED-ON-ABORT STREAMLET\n\n Make it so that upon @ABORTED-EXECUTION,\n STREAMLET's JOURNAL will be in JOURNAL-STATE :COMPLETED when loaded\n fresh (e.g. when creating a FILE-JOURNAL with an existing file). Any\n previously written events must be persisted before making this\n change. Before REQUEST-COMPLETED-ON-ABORT is called, a journal must\n be reloaded in state :FAILED.\n \n It is permissible to defer carrying out this request until the next\n SYNC-STREAMLET call. If the request was carried out, return true. If\n it was deferred, return NIL.\n\n- [generic-function] SYNC-STREAMLET STREAMLET\n\n Durably persist the effects of all preceding\n WRITE-EVENT calls made via STREAMLET to its journal and any deferred\n REQUEST-COMPLETED-ON-ABORT in this order.\n\n## Glossary\n\n- [glossary-term] async-unwind\n\n If an asynchronous event, say a `SIGINT` triggered by `C-c`, is\n delivered to a thread running Lisp or foreign code called from Lisp,\n a Lisp condition is typically signalled. If the handler for this\n condition unwinds the stack, then we have an asynchronous unwind.\n Another example is BT:INTERRUPT-THREAD, which, as it can execute\n arbitrary code, may unwind the stack in the target thread.\n\n- [glossary-term] boolean-valued symbol\n\n Imagine writing two STREAMs with a spaghetti of functions and\n wanting to have pretty-printed output on one of them. Unfortunately,\n binding *PRINT-PRETTY* to T will affect writes to both streams.\n \n One solution would be to have streams look up their own print-pretty\n flag with `(SYMBOL-VALUE (STREAM-PRETTY-PRINT STREAM))` and have the\n caller specify the dynamic variable they want:\n \n ```\n (defvar *print-pretty-1* nil)\n (setf (stream-print-pretty stream-1) '*print-pretty-1*)\n (let ((*print-pretty-1* t))\n (spaghetti stream-1 stream-2))\n ```\n \n Note that if the default `STREAM-PRINT-PRETTY` is `'*PRINT-PRETTY*`,\n then we have the normal Common Lisp behaviour. Setting\n `STREAM-PRINT-PRETTY` to NIL or T also works, because they are\n self-evaluating.\n \n The above hypothetical example demonstrates the concept of\n boolean-valued symbols on CL:STREAMs. In Journal, they are used by\n MAKE-LOG-DECORATOR and PPRINT-JOURNALs.\n\n- [glossary-term] readable\n\n In Common Lisp, readable objects are those that can be printed\n readably. Anything written to stream-based journals needs to\n be readable.\n\n [1895]: #JOURNAL:FILE-BUNDLE%20CLASS \"JOURNAL:FILE-BUNDLE CLASS\"\n [2e9b]: #JOURNAL:REPLAY-FAILURE%20CONDITION \"JOURNAL:REPLAY-FAILURE CONDITION\"\n [3956]: #JOURNAL:JOURNALING-FAILURE%20CONDITION \"JOURNAL:JOURNALING-FAILURE CONDITION\"\n [5082]: #JOURNAL:JOURNAL%20CLASS \"JOURNAL:JOURNAL CLASS\"\n [7a2f]: #JOURNAL:STREAMLET%20CLASS \"JOURNAL:STREAMLET CLASS\"\n [8428]: #JOURNAL:FILE-JOURNAL%20CLASS \"JOURNAL:FILE-JOURNAL CLASS\"\n [9150]: #JOURNAL:PPRINT-JOURNAL%20CLASS \"JOURNAL:PPRINT-JOURNAL CLASS\"\n [b668]: #JOURNAL:IN-MEMORY-JOURNAL%20CLASS \"JOURNAL:IN-MEMORY-JOURNAL CLASS\"\n [d9b6]: #JOURNAL:BUNDLE%20CLASS \"JOURNAL:BUNDLE CLASS\"\n\n* * *\n###### \\[generated by [MGL-PAX](https://github.com/melisgl/mgl-pax)\\]\n","funding_links":[],"categories":["Online editors ##"],"sub_categories":["Third-party APIs"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmelisgl%2Fjournal","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmelisgl%2Fjournal","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmelisgl%2Fjournal/lists"}