{"id":22360233,"url":"https://github.com/yitzchak/shasht","last_synced_at":"2025-07-29T18:18:07.904Z","repository":{"id":38397802,"uuid":"266177542","full_name":"yitzchak/shasht","owner":"yitzchak","description":"Common Lisp JSON reading and writing for the Kzinti.","archived":false,"fork":false,"pushed_at":"2024-07-20T16:14:17.000Z","size":172,"stargazers_count":44,"open_issues_count":3,"forks_count":3,"subscribers_count":9,"default_branch":"master","last_synced_at":"2024-07-20T17:33:03.444Z","etag":null,"topics":["common-lisp","json"],"latest_commit_sha":null,"homepage":"","language":"Common Lisp","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/yitzchak.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2020-05-22T18:12:05.000Z","updated_at":"2024-07-20T16:14:21.000Z","dependencies_parsed_at":"2024-01-08T22:12:18.303Z","dependency_job_id":null,"html_url":"https://github.com/yitzchak/shasht","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yitzchak%2Fshasht","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yitzchak%2Fshasht/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yitzchak%2Fshasht/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yitzchak%2Fshasht/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/yitzchak","download_url":"https://codeload.github.com/yitzchak/shasht/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":228137077,"owners_count":17875159,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["common-lisp","json"],"created_at":"2024-12-04T15:30:07.215Z","updated_at":"2025-07-29T18:18:07.858Z","avatar_url":"https://github.com/yitzchak.png","language":"Common Lisp","funding_links":[],"categories":["REPLs ##","Expert Systems"],"sub_categories":[],"readme":"# shasht\n\n[![Build Status][ci-badge]][ci]\n\nCommon Lisp JSON reading and writing for the Kzinti.\n\n## Reading\n\nThe primary interface to parsing and reading JSON is the `read-json` function.\n\n```lisp\n(read-json \u0026optional input-stream-or-string (eof-error-p t) eof-value single-value-p)\n```\n\nThe argument `input-stream-or-string` can be an stream, a string to read from,\nor `nil` to use `*standard-input*`. The arguments `eof-error-p` and `eof-value`\nhave the same affect as they do in the Common Lisp function `read`. If the\n`single-value-p` argument is true then the input to `read-json` is assumed to\nbe a single value, which means that extra tokens at the end will cause an\nerror to be generated.\n\nThere are a number of dynamic variables that will influence the parsing of JSON\ndata.\n\n- `common-lisp:*read-default-float-format*` — Controls the floating-point format \n   that is to be used when reading a floating-point number.\n- `*read-default-true-value*` — The default value to return when reading a true \n  token. Initially set to `t`.\n- `*read-default-false-value*` — The default value to return when reading a \n  false token. Initially set to `nil`.\n- `*read-default-null-value*` — The default value to return when reading a null\n  token. Initially set to `:null`.\n- `*read-default-array-format*` — The default format to use when reading an\n  array. Current supported formats are `:vector` or `:list`. Initially set to\n  `:vector`.\n- `*read-default-object-format*` — The default format to use when reading an\n  object. Current supported formats are `:hash-table`, `:alist` or `:plist`.\n  Initially set to `:hash-table`.\n- `*read-length*` — The maximum number of values in an array or an object.\n  Initially set to `nil` which disables length checking.\n- `*read-level*` — The maximum number of levels to allow during reading for\n  arrays and objects. Initially set to `nil` which disables level checking.\n  \nThere is also a keyword variant `read-json*` which will set the various dynamic\nvariables from supplied keywords.\n\n```lisp\n(read-json* :stream nil\n            :eof-error t\n            :eof-value nil \n            :single-value nil\n            :true-value t \n            :false-value nil \n            :null-value :null\n            :array-format :vector \n            :object-format :hash-table\n            :float-format 'single-float\n            :length nil\n            :level nil)\n```\n\n## Writing\n\nThe primary interface to serializing and writing JSON is the `write-json` \nfunction.\n\n```lisp\n(write-json value \u0026optional (output-stream t))\n```\n\nThe `output-stream` argument can be a stream, `t` for `*standard-output*`, or\n`nil` for output to a string. If the output is to a string then this string will\nbe returned, otherwise the original value will be returned.\n\nThere are a number of dynamic variables that will influence the serialization of\nJSON data.\n\n- `common-lisp:*print-pretty*` — If true then a simple indentation algorithm\n  will be used.\n- `*write-indent-string*` — The string to use when indenting objects and arrays.\n   Initially set to `#\\space`.\n- `*write-ascii-encoding*` — If true then any non ASCII values will be encoded \n  using Unicode escape sequences. Initially set to `nil`.\n- `*write-true-values*` — Values that will be written as a true token. Initially \n  set to `'(t :true)`.\n- `*write-false-values*` — Values that will be written as a false token. \n  Initially set to `'(nil :false)`.\n- `*write-null-values*` — Values that will be written as a null token. Initially \n  set to `(:null)`.\n- `*write-alist-as-object*` — If true then assocation lists will be written as \n  an object. Initially set to `nil`. \n- `*write-plist-as-object*` — If true then property lists will be written as an \n  object. Initially set to `nil`.\n- `*write-empty-array-values*` — A list of values that will be written as an \n  empty array.\n- `*write-empty-object-values*` — A list of values that will be written as an \n  empty object.\n- `*write-array-tags*` — A list of values whose appearance in the CAR of a list \n  indicates the CDR of the list should be written as an array. Initially set to \n  `'(:array)`.\n- `*write-object-alist-tags*` — A list of values whose appearance in the CAR of \n  a list indicates the CDR of the list is an alist and should be written as an \n  object. Initially set to `'(:object-alist)`.\n- `*write-object-plist-tags*` — A list of values whose appearance in the CAR of \n  a list indicates the CDR of the list is a plist and should be written as an \n  object. Initially set to `'(:object-plist)`.\n  \nThe actual serialization of JSON data is done by the generic function\n`print-json-value` which can be specialized for additional value types.\n\n```lisp\n(print-json-value value output-stream)\n```\n\nThere is also a keyword variant `write-json*` which will set the various dynamic\nvariables from supplied keywords and will default to the current dynamic value \nof each keyword.\n\n```lisp\n(write-json* value :stream t \n                   :ascii-encoding nil \n                   :true-values '(t :true)\n                   :false-values '(nil :false) \n                   :null-values '(:null)\n                   :empty-array-values '(:empty-array)\n                   :empty-object-values '(:empty-object)\n                   :array-tags '(:array)\n                   :object-alist-tags '(:object-alist) \n                   :object-plist-tags '(:object-plist) \n                   :alist-as-object nil \n                   :plist-as-object nil\n                   :pretty nil \n                   :indent-string \"  \")\n```\n### Serialization Helper Functions\n\nIn order to facilitate extending the serialization facilities of shasht there\nare a number of helper functions available. To aid in the printing of JSON\nstrings there is the following.\n\n```lisp\n(write-json-string value output-stream)\n```\n\nIn order to ease the serialization of objects and arrays there is \n`with-json-object` and `with-json-array`. Both of these macros take an\noutput stream as the first argument then enable indentation and automatic\nhandling of all delimiter tokens. Inside the body of `with-json-object`\nthe function `(print-json-key-value key value output-stream)` should be used\nto output a key value pair. Inside the body of `with-json-array` the function\n`(print-json-value value output-stream)` should be used to output a single\nvalue. Example usage can be seen in the source code.\n\n### JSON Array and Object Literals\n\nSometimes using incremental serialization via the serialization helper functions\nis not the best fit for the application. For example, maybe the JSON needs to be\nassembled from various fragments and then analyzed before serialization. This\nhappens in the Jupyter widget protocol. Binary data value in the JSON are\nextracted from the JSON and transmitted in binary form in order to decrease\nthe network load.\n\nSince constructing hash tables purely for the purpose of JSON serialization is \na bit difficult and potentially memory wasteful, there are various keyword\nliterals that can be used to construct JSON objects in order to avoid this\nsituation. There are also literals for JSON arrays, although they may not be as\nuseful. These literals exist in shasht mostly for consistency. The following\nshows some examples of the use of these literals.\n\n```\n* (shasht:write-json \n    '(:object-alist (\"a\" . :empty-array) \n                    (\"b\" . :empty-object) \n                    (\"c\" . (:object-plist \"d\" 1 \n                                          \"e\" (:array 1 2 3)))))\n{\n  \"a\": [],\n  \"b\": {},\n  \"c\": {\n    \"d\": 1,\n    \"e\": [\n      1,\n      2,\n      3\n    ]\n  }\n}\n(:OBJECT-ALIST (\"a\" . :EMPTY-ARRAY) (\"b\" . :EMPTY-OBJECT)\n (\"c\" :OBJECT-PLIST \"d\" 1 \"e\" (:ARRAY 1 2 3)))\n```\n\nThe values and tags that indicate these literals can be configured via the \ndynamic variables `*write-empty-array-values*`, `*write-empty-object-values*`,\n`*write-array-tags*`, `*write-object-alist-tags*`,\nand `*write-object-plist-tags*`.\n\nThese literals forms are only meant for serialization and not for round-trip\nmapping. Therefore there is no way to read JSON in the same format.\n\n## Mapping of Data Types\n\nThe mapping of types between Common Lisp and JSON is not one-to-one nor is it\nwithout ambiguity due to issues such as Common Lisp's treatment of `nil` as\nan empty list and as a false value. JSON also includes `null` which does not\nhave an obvious representation in Common Lisp. Because of this, shasht makes\ncertain choices in the default mapping of data types between Common Lisp and\nJSON that may not be ideal for all applications. shasht was primarily designed\nto be a good round-trip encoder/decoder for [common-lisp-jupyter][] in the\nnetwork protocol of Jupyter.\n\nA brief explanation of the default mapping is given in the table below. For \nmore detail regarding the mapping of individual types and how to configure\nthat mapping see the sections following this table.\n\n| Common Lisp                            |     | JSON                               |\n|----------------------------------------|----:|------------------------------------|\n| integer                                | \u003c-\u003e | number without decimal or exponent |\n| float                                  | \u003c-\u003e | number with decimal or exponent    |\n| ratio                                  |  -\u003e | number with decimal or exponent    |\n| rational                               |  -\u003e | number with decimal or exponent    |\n| string                                 | \u003c-\u003e | string                             |\n| character                              |  -\u003e | string                             |\n| pathname                               |  -\u003e | string                             |\n| symbol not matching other mapping      |  -\u003e | string                             |\n| vector                                 | \u003c-\u003e | array                              |\n| multi-dimensional array                |  -\u003e | nested array                       |\n| non-`nil` list                         |  -\u003e | array                              |\n| hash table                             | \u003c-\u003e | object                             |\n| standard object                        |  -\u003e | object                             |\n| structure object                       |  -\u003e | object                             |\n| `t`                                    | \u003c-\u003e | `true`                             |\n| `:true`                                |  -\u003e | `true`                             |\n| `nil`                                  | \u003c-\u003e | `false`                            |\n| `:false`                               |  -\u003e | `false`                            |\n| `:null`                                | \u003c-\u003e | `null`                             |\n| `:empty-array`                         |  -\u003e | `[]`                               |\n| `:empty-object`                        |  -\u003e | `{}`                               |\n| `'(:array 1 2 3)`                      |  -\u003e | `[1,2,3]`                          |\n| `'(:object-alist (\"a\" . 1) (\"b\" . 2))` |  -\u003e | `{\"a\":1,\"b\":2}`                    |\n| `'(:object-plist \"a\" 1 \"b\" 2)`         |  -\u003e | `{\"a\":1,\"b\":2}`                    |\n\n### Mapping of Number Types\n\nThe format of a number read from JSON when a decimal or an exponent is present\nin the number literal can be influenced with `cl:*read-default-float-format*`.\nThis is the same behavior of `cl:read`. In order to read JSON numbers with large\nexponents one would need do something like the following.\n\n```common-lisp\n(shasht:read-json \"[2.232e75]\" :float-format 'double-float)\n```\n### Mapping of Array Types\n\nThe dynamic variables `*read-default-array-format*`, \n`*write-empty-array-values*`, and `*write-array-tags*` all influence the mapping\nof JSON arrays to Common Lisp vectors and lists. Common Lisp vectors and \nmulti-dimensional arrays are always writen as JSON arrays. By default JSON\narrays are read as Common Lisp vectors. With the default settings only non-`nil`\nlists that don't satisfy some other mapping rule are written as JSON arrays.\n\nIf one wants to use lists as the default JSON array format then \n`*read-default-false-value*`, `*read-default-array-format*`, and \n`*write-false-value*` will need to need to be set to appropriate values since\nin the default mapping `nil` maps to `false`. For example, the following \ncould be done.\n\n```common-lisp\n(let ((shasht:*read-default-false-value* :false)\n      (shasht:*read-default-array-format* :list)\n      (shasht:*write-false-values* '(:false)))\n (shasht:read-json ...)\n (shasht:write-json ...))\n```\n\nLists with a CAR `eql` to a value in `*write-array-tags*`, \n`*write-object-alist-tags*`, `*write-object-plist-tags*` will still be written\nas an array or object as appropriate. To completely disable this behavior the\nvariables would need to be bound to `nil`. Or one could do the following.\n\n```common-lisp\n(shasht:write-json '(1 2 3) :false-value '(:false) :array-tags nil\n                            :object-alist-tags nil :object-plist-tags nil)\n```\n\nIn this case the mapping for array types would become:\n\n| Common Lisp                            |     | JSON                               |\n|----------------------------------------|----:|------------------------------------|\n| vector                                 |  -\u003e | array                              |\n| multi-dimensional array                |  -\u003e | nested array                       |\n| list                                   | \u003c-\u003e | array                              |\n\n### Mapping of Object Types\n\nThe dynamic variables `*read-default-object-format*`, `*write-alist-as-object*`,\n`*write-plist-as-object*`, `*write-empty-object-values*`, \n`*write-object-alist-tags*`, and `*write-object-plist-tags*` all influence the \nmapping of JSON objects to Common Lisp hash tables, alists, and plists. Common \nLisp hash tables are always written as JSON objects. By default JSON objects are \nread as Common Lisp hash tables.\n\nIn order to use alists as the default JSON object format the dynamic variables\n`*read-default-object-format*`, `*write-alist-as-object*`, \n`*read-default-false-value*`, and `*write-false-values*` will need to be set\nto appropriate values. For example, the following would use alists as the\ndefault JSON object format and `:false` as the JSON `false` value.\n\n```common-lisp\n(let ((shasht:*read-default-object-format* :alist)\n      (shasht:*write-alist-as-object* t)\n      (shasht:*read-default-false-value* :false)\n      (shasht:*write-false-values* '(:false)))\n (shasht:read-json ...)\n (shasht:write-json ...))\n```\n\nIn this case the mapping for object types would become:\n\n| Common Lisp                            |     | JSON                               |\n|----------------------------------------|----:|------------------------------------|\n| hash table                             |  -\u003e | object                             |\n| alist                                  | \u003c-\u003e | object                             |\n| standard object                        |  -\u003e | object                             |\n| structure object                       |  -\u003e | object                             |\n\nThe same could be accomplished for plists by doing the following.\n\n```common-lisp\n(let ((shasht:*read-default-object-format* :plist)\n      (shasht:*write-plist-as-object* t)\n      (shasht:*read-default-false-value* :false)\n      (shasht:*write-false-values* '(:false)))\n (shasht:read-json ...)\n (shasht:write-json ...))\n```\n\n## Compliance\n\nAlthough concise the JSON specification is very vague on a number of points and\nthus accurate compliance by implementations is often substandard. Without \ncomprehensive tests compliance is difficult to ascertain. The \n[JSONTestSuite](https://github.com/nst/JSONTestSuite) includes over 300 reading\ntests including those left ambiguous by the specification. The test suite of\nshasht includes all of these tests in addition to various write tests. For a\ncomparision of the compliance of the Common Lisp implementations of JSON see\n[Compliance Comparision](https://yitzchak.github.io/shasht/parsing.html).\n\n## Benchmarks\n\nA simple benchmark can be done with `tests/bench.lisp`. For SBCL the following \nresults are typical.\n\n```\n                                JSON Read Times                                 \n             0                     7.002169e-6                      1.4004338e-5\n             ˫--------------------------------+--------------------------------˧\n     cl-json ███████████████████████████████████▉\n    jonathan ████████▊\njson-streams ███████████████████████████████████████████████████████████████████\n       jsown █████████▋\n      shasht █████████████▏\n     st-json █████████████████████████████████▊\n       yason ████████████████████████████████████████▉\n\n\n                                JSON Write Times                                \n             0                    6.2018808e-6                     1.24037615e-5\n             ˫--------------------------------+--------------------------------˧\n     cl-json ███████████████████████████████████████████████████████████████████\n    jonathan ████████████████████████████████████▍\njson-streams ███████████████████████████████▎\n       jsown ████████████████████████████████████████▉\n      shasht ███████████▍\n     st-json ███████▋\n       yason ████████████████████████████████████▏\n\n\n                             JSON Read/Write Times                              \n             0                     1.1348141e-5                     2.2696282e-5\n             ˫--------------------------------+--------------------------------˧\n     cl-json ███████████████████████████████████████████████████████████████████\n    jonathan █████████████████████████████████████▍\njson-streams ██████████████████████████████████████████████████████████████▉\n       jsown ██████████████████████████████▎\n      shasht ██████████████████████▏\n     st-json ███████████████████████████▋\n       yason ███████████████████████████████████████████████████████████▉\n\n```\n\n\u003c!--refs--\u003e\n\n[ci]: https://github.com/yitzchak/shasht/actions/\n[ci-badge]: https://github.com/yitzchak/shasht/workflows/ci/badge.svg\n[common-lisp-jupyter]: https://github.com/yitzchak/common-lisp-jupyter/\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyitzchak%2Fshasht","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fyitzchak%2Fshasht","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyitzchak%2Fshasht/lists"}