https://github.com/atlas-engineer/njson

Common Lisp JSON handling library (not a parser!), with the aim for convenience and brevity.
https://github.com/atlas-engineer/njson
Last synced: about 1 year ago
JSON representation
Common Lisp JSON handling library (not a parser!), with the aim for convenience and brevity.
Host: GitHub
URL: https://github.com/atlas-engineer/njson
Owner: atlas-engineer
License: bsd-3-clause
Created: 2022-09-09T16:28:26.000Z (over 3 years ago)
Default Branch: master
Last Pushed: 2025-01-02T20:18:58.000Z (over 1 year ago)
Last Synced: 2025-01-16T06:58:16.683Z (about 1 year ago)
Language: Common Lisp
Homepage:
Size: 167 KB
Stars: 17
Watchers: 5
Forks: 3
Open Issues: 9
Metadata Files:
- Readme: README.org
- License: LICENSE
Awesome Lists containing this project

awesome-cl - NJSON - Parser-agnostic JSON indexing (with JSON Pointer support), destructuring, and validation framework. [BSD][15]. (Expert Systems)
README

          #+TITLE:njson

*A JSON handling framework aiming for convenience and brevity.*

NJSON aims to make it extremely convenient for you to decode,

validate, destructure, process, and encode JSON data, in the minimum

keystrokes/minutes possible.

* Getting started

Clone the Git repository:

#+begin_src sh

  git clone --recursive https://github.com/atlas-engineer/njson ~/common-lisp/njson

#+end_src

Load NJSON (with a jzon backend) in the REPL:

#+begin_src lisp

  ;; Show ASDF where NJSON is.

  (asdf:load-asd #p"/path/to/checkout/njson.asd")

  ;; Load it with ASDF.

  (asdf:load-system :njson/jzon)

  ;; Alternatively, load it with Quicklisp.

  (ql:quickload :njson/jzon)

#+end_src

And start parsing right away, be it from file:

#+begin_src lisp

  (njson:decode #p"/path/to/njson/checkout/tests/test.json")

  ;; => #(1 3.8 T NIL :NULL "foo" #(1 2 3) #("bar" T :NULL 1000000)

  ;;      #

  ;;      #)

#+end_src

or from string:

#+begin_src lisp

  (njson:decode "[\"hello\", 5]")

  ;; => #("hello", 5)

#+end_src

or other specializeable types. Default methods support:

- pathnames,

- strings,

- streams.

** Running tests

Given NJSON backend-agnostic nature, you can only test every particular backend against the uniform set of tests that NJSON provides. So, to test jzon backend, you can do:

#+begin_src lisp

  (asdf:test-system :njson/jzon)

#+end_src

And, for the CL-JSON backend,

#+begin_src lisp

  (asdf:test-system :njson/cl-json)

#+end_src

* What NJSON is not (and what it is, instead)

** NJSON is not a JSON parsing library.

It's one level higher: it's a convenience wrapper around your JSON

parser of choice. NJSON is made in such a way so as to be usable with

almost any JSON library out there. The bundled backends are jzon

(reliable, though new), and CL-JSON (fuzzy yet battle-proven).

- To make NJSON support your preferred JSON parser, you have to

  specialize as little as two methods: ~decode-from-stream~ and

  ~encode-to-stream~. If you care about correctness or proper type

  dispatching, you may also define ~(en|de)code-(to|from)-string~ and

  ~(en|de)code-(to|from)-file~.

** NJSON is not propagating unnecessary dependencies on you.

The core (~njson~ ASDF system) has no dependencies due to specifying

only the generics to implement.

Every other dependency is optional and depends on which backend you

want to use for parsing.

** NJSON is not the fastest JSON handling solution out there.

Plug-n-play usability and type variety are much higher of a priority

than the performance. The types NJSON returns from its methods (and

that your own methods extending NJSON should expect/return) are:

- Lisp ~real~-s for JSON numbers.

- Lisp strings for JSON strings.

- ~:null~ for JSON ~null~.

- ~t~ for ~true~ and ~nil~ for ~false~.

- Vectors for JSON arrays.

- Hash-tables for JSON objects.

With this basic (yet disjoint) set of types, you can easily ~typecase~

over NJSON output and make informed decisions about the JSON you

have. Even if it's some couple of CPU work milliseconds slower than

handling raw lists. It's faster in human work seconds, which are much

more valuable.

** NJSON is not minimalist.

NJSON has strict requirements on the returned data, but this

strictness enables a rich set of JSON-handling primitives/helpers. You

can

- ~(:use #:cl #:njson)~ in your packages if you want short and

  convenient JSON operations there. It's safe, because NJSON shadows

  no symbols from CL.

- Or you can define a package local nickname for ~:njson/aliases~ to

  be a mere ~j:~ (using ~trivial-package-local-nicknames~), so that

  even shorter helpers (just a couple of characters longer than the

  regular CL constructs) are available:

#+begin_src lisp

  (trivial-package-local-nicknames:add-package-local-nickname :j :njson/aliases :YOUR-PACKAGE)

  ;; And then use it like.

  (j:get ...)

  (j:decode ...)

  (j:if ...)

  (j:match ...)

#+end_src

See the next section for the functions/macros NJSON exports.

* API

** FUNCTION njson:jget (alias: njson/aliases:get)

Gets the value from the JSON object/array indexed by a certain

key. Note that the second value is a boolean denoting whether the

entry under key is found (like in ~gethash~).

#+begin_src lisp

  (defvar data (njson:decode "{\"key\": 5, \"second-key\": [1, 2, false]}"))

  (njson:jget "key" data)

  ;; => 5, T

  ;; Index using sequence:

  (njson:jget '("second-key" 1) data)

  ;; => 2, T

  ;; Index using JSON Pointer (as pathname):

  (njson:jget #p"/second-key/0" data)

  ;; => 1, T

  ;; Modify the element in place:

  (setf (njson:jget #p"/second-key/0" data) 3)

  ;; Another indexing syntax, for no particular reason:

  (njson:jget #("second-key" 0) data)

  ;; => 3, T

#+end_src

Note the pathname indexing—it uses the [[https://www.rfc-editor.org/rfc/rfc6901][JSON Pointer]] syntax for indexing convenience.

** FUNCTION njson:jget* (alias: njson/aliases:get*)

A stricter version of =jget= that throws =no-key= error when there's nothing under the given key in the provided object.

Will be merged into =jget= with the next major release.

** FUNCTION njson:jcopy (alias: njson/aliases:copy)

Copies the whole thing it's passed, no mater the nesting, into a fresh new equal object. Makes all the arrays adjustable and fillable for further possibly destructive use.

#+begin_src lisp

  (defvar data (njson:jget "key" (njson:decode "{\"key\": 5}")))

  ;; => 5, T

  (njson:jget "key" (njson:jcopy data))

  ;; => 5, T

#+end_src

** FUNCTION njson:jkeys (alias: njson/aliases:keys)

Gets all the keys present in the passed object. Integer keys for arrays, string keys for object, error for anything else.

#+begin_src lisp

  (njson:jkeys (njson:decode "{\"a\": 1, \"b\": 2}"))

  ;; ("a" "b")

  (njson:jkeys (njson:decode "[\"a\", \"b\"]"))

  ;; (0 1)

#+end_src

** FUNCTIONS njson:ensure-array, njson:ensure-object (aliases: njson/aliases:ensure-array, njson/aliases:ensure-object)

Ensure that the passed object is turned into array or object (respectively). If ~:convert-objects~ is provided in ~njson:ensure-array~, it creates an array with all the values of object, discarding keys.

#+begin_src lisp

  (njson:ensure-array #(1 2 3))

  ;; #(1 2 3)

  (njson:ensure-array 3)

  ;; #(3)

  (njson:ensure-array (njson:decode "{\"a\": 3}"))

  ;; #(#)

  (njson:ensure-array (njson:decode "{\"a\": 3}") :convert-objects t)

  ;; #(3)

  (njson:ensure-object "key" #)

  ;; #

  (njson:ensure-object "key" 3)

  ;; # with "key": 3

  (njson:ensure-object "key" #(1 2 3))

  ;; # with "key": #(1 2 3)

#+end_src

** FUNCTION njson:jtruep (aliases: njson:jtrue-p, njson:jtrue?, njson:truep, njson:true-p, njson:true?)

Checks whether the given value is true (in other words, neither ~false~, nor ~null~) per JSON.

All the macros below utilize it, so, if you want to change the behavior of those, specialize this function.

** MACRO njson:jwhen (alias: njson/aliases:when)

A regular CL ~when~ made aware of JSON's ~null~ and ~false~.

#+begin_src lisp

  (njson:jwhen (njson:decode "null")

    "This is never returned.")

  ;; nil

  (njson:jwhen (njson:decode "5")

    "This is always returned.")

  ;; "This is always returned"

#+end_src

** MACRO njson:if (alias: njson/aliases:if)

A regular Lisp ~if~ aware of JSON truths and lies.

#+begin_src lisp

  (njson:jif (njson:decode "5")

             "This is always returned."

             "This is never returned.")

  ;; "This is always returned"

#+end_src

** MACRO njson:jor, njson:jand, njson:jnot (and aliases: njson/aliases:or, njson/aliases:and, njson/aliases:not)

Regular Lisp logic operators, with awareness of JSON values.

** MACRO njson:jbind (alias njson/aliases:bind)

Destructures a JSON object against the provided destructuring pattern. This is most useful for deeply nested JSON structures often returned from old/corporate APIs. One example of such APIs is the Reddit one. To get to the title of the post, one has to go through half a dozen layers of nested objects and arrays:

#+begin_src js

  [{"kind": "Listing",

    "data": {"children": [{"kind": "t3",

                           "data": {"approved_at_utc": null,

                                    "subreddit": "programming",

                                    ...

                                    // Finally, a title!

                                    "title": "Henry Baker: Meta-circular semantics for Common Lisp special forms",

                                    "link_flair_richtext": [],

                                    "subreddit_name_prefixed": "r/programming",

                                    ...}}]

             ...}}

   ...]

#+end_src

One needs a strong destructuring facility with type checking to move through this mess of JSON data. ~jbind~ is exactly this facility. Here's how accessing the title of Reddit post would look like (array patterns access JSON arrays, list patterns access JSON objects) with ~jbind~:

#+begin_src lisp

  (njson:jbind #(("data" ("children" #(("data" ("title" title))))))

      ;; Dexador is not a dependency of NJSON, so load it separately

      (njson:decode

       (dex:get

        "https://www.reddit.com/r/programming/comments/6er9d/henry_baker_metacircular_semantics_for_common.json"))

    title)

  ;; "Henry Baker: Meta-circular semantics for Common Lisp special forms"

#+end_src

See documentation for more examples.

** MACRO njson:jmatch (alias njson/aliases:match)

Matches/destructures the provided form against patterns one by one, and executes the body of the successfully matching one with the bindings it established. Every pattern and body is essentially a ~jbind~ with checking for destructuring success. The use-case is dispatching over API responses that differ in structure.

Telegram Bot API, for example, has disjoint contents for error responses and success responses:

- Error responses have "ok" key set to false, and keys called "description" and "error_code".

- Successful responses have "ok" set to true and "result" as the payload they return.

Given these restrictions, we can ~jmatch~ the result of Bot API:

#+begin_src lisp

  (njson:jmatch

   parsed-api-data

   (("ok" :true "result" result)

    (values t result))

   (("ok" :false "error_code" _ "description" description)

    (values nil description))

   (t (error "Malformed data!")))

#+end_src

After parsing the data, we have clear value distinctions:

- On success, return (VALUES (EQL T) *) with the payload.

- On error, return (VALUES NULL &OPTIONAL STRING).

- And in the exceptional case of malformed data, error out.

~jmatch~ (and ~jbind~) also checks the value matching (see the ~"ok" :true~ and ~"ok" :false~ parts) with arbitrary JSON atomic type (number, string, ~:true~ (for T), ~:false~ (for NIL), and ~:null~). Arrays and lists are destructuring patterns already, so any value in them can be equality-checked.

** ERROR njson:jerror

An umbrella class for all the NJSON errors. If you want to play unsafe, simply ignore all of NJSON errors:

#+begin_src lisp

  (handler-case

      (njson:jget ...)

    ;; Or j:error if you nicknamed njson/aliases.

    (njson:jerror ()

      nil))

#+end_src

** ERROR njson:encode-to-stream-not-implemented, njson:decode-from-stream-not-implemented

These get thrown when the JSON parsing back-end does not define methods for =njson:encode-to-stream= and =njson:decode-from-stream=. These are the bare minimum a backend should have to work. Adding the string and file methods is nice, but not required.

** ERROR njson:invalid-key

This gets thrown when you try to index objects with integer indices and arrays with string keys. Because such an indexing wouldn't make sense.

To allow string indexing for arrays (to make ="1"= be recognized as a valid index), you can patch the =njson:jget= method for string indices:

#+begin_src lisp

  (defmethod njson:jget :around ((index string) (object array))

    (if (every #'digit-char-p index)

        (njson:jget (parse-integer index) object)

        (call-next-method)))

#+end_src

** ERROR njson:non-indexable

It doesn't make sense to index a number. This error reinforces the idea.

** ERROR njson:invalid-pointer

This error is JSON Pointer specific. It's thrown when there's something wrong with the pointer syntax.

** ERROR njson:no-key

This error is thrown in =njson:jget*= when the indexed object doesn't have the key it's indexed with.

** ERROR njson:value-mismatch

Some value validated in =njson:jbind= didn't match the expected value.

** ERROR njson:deprecated

Marks a certain function as deprecated.
ecosyste.ms

Data

Tools

Indexes

Applications

Experiments

Awesome

https://github.com/atlas-engineer/njson

Awesome Lists containing this project

README
