Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

https://github.com/spyzhov/ajson

Abstract JSON for Golang with JSONPath support
https://github.com/spyzhov/ajson

golang golang-package json json-parser jsonpath

Last synced: 24 days ago
JSON representation

Abstract JSON for Golang with JSONPath support

Lists

README 

        
# Abstract JSON 



[![Build](https://github.com/spyzhov/ajson/actions/workflows/main.yml/badge.svg)](https://github.com/spyzhov/ajson/actions/workflows/main.yml)

[![Go Report Card](https://goreportcard.com/badge/github.com/spyzhov/ajson)](https://goreportcard.com/report/github.com/spyzhov/ajson)

[![GoDoc](https://godoc.org/github.com/spyzhov/ajson?status.svg)](https://godoc.org/github.com/spyzhov/ajson)

[![codecov](https://codecov.io/gh/spyzhov/ajson/branch/master/graph/badge.svg)](https://codecov.io/gh/spyzhov/ajson)

[![Awesome](https://cdn.rawgit.com/sindresorhus/awesome/d7305f38d29fed78fa85652e3a63e154dd8e8829/media/badge.svg)](https://github.com/avelino/awesome-go#json)



Abstract [JSON](https://www.json.org/) is a small golang package provides a parser for JSON with support of JSONPath, in case when you are not sure in its structure.



Method `Unmarshal` will scan all the byte slice to create a root node of JSON structure, with all its behaviors.



Method `Marshal` will serialize current `Node` object to JSON structure.



Each `Node` has its own type and calculated value, which will be calculated on demand. 

Calculated value saves in `atomic.Value`, so it's thread safe.



Method `JSONPath` will returns slice of found elements in current JSON data, by [JSONPath](http://goessner.net/articles/JsonPath/) request.



## Compare with other solutions



Check the [cburgmer/json-path-comparison](https://cburgmer.github.io/json-path-comparison/) project.



# Usage



[Playground](https://play.golang.com/p/iIxkktxN0SK)



```go

package main



import (

	"fmt"

	"github.com/spyzhov/ajson"

)



func main() {

	json := []byte(`...`)



	root, _ := ajson.Unmarshal(json)

	nodes, _ := root.JSONPath("$..price")

	for _, node := range nodes {

		node.SetNumeric(node.MustNumeric() * 1.25)

		node.Parent().AppendObject("currency", ajson.StringNode("", "EUR"))

	}

	result, _ := ajson.Marshal(root)



	fmt.Printf("%s", result)

}

```



# Console application



You can download `ajson` cli from the [release page](https://github.com/spyzhov/ajson/releases), or install from the source:



```shell script

go get github.com/spyzhov/ajson/cmd/[email protected]

```



Usage:



```

Usage: ajson [-mq] "jsonpath" ["input"]

  Read JSON and evaluate it with JSONPath.

Parameters:

  -m, --multiline  Input file/stream will be read as a multiline JSON. Each line should have a full valid JSON.

  -q, --quiet      Do not print errors into the STDERR.

Argument:

  jsonpath         Valid JSONPath or evaluate string (Examples: "$..[?(@.price)]", "$..price", "avg($..price)")

  input            Path to the JSON file. Leave it blank to use STDIN.

```



Examples:



```shell script

  ajson "avg($..registered.age)" "https://randomuser.me/api/?results=5000"

  ajson "$.results.*.name" "https://randomuser.me/api/?results=10"

  curl -s "https://randomuser.me/api/?results=10" | ajson "$..coordinates"

  ajson "$" example.json

  echo "3" | ajson "2 * pi * $"

  docker logs image-name -f | ajson -qm 'root($[?(@=="ERROR" && key(@)=="severity")])'

```



# JSONPath



Current package supports JSONPath selection described at [http://goessner.net/articles/JsonPath/](http://goessner.net/articles/JsonPath/).



JSONPath expressions always refer to a JSON structure in the same way as XPath expression are used in combination with an XML document. 

Since a JSON structure is usually anonymous and doesn't necessarily have a "root member object" JSONPath assumes the abstract name $ assigned to the outer level object.



JSONPath expressions can use the dot–notation



`$.store.book[0].title`



or the bracket–notation



`$['store']['book'][0]['title']`



for input paths. Internal or output paths will always be converted to the more general bracket–notation.



JSONPath allows the wildcard symbol `*` for member names and array indices. 

It borrows the descendant operator `..` from E4X and the array slice syntax proposal `[start:end:step]` from ECMASCRIPT 4.



Expressions of the underlying scripting language `()` can be used as an alternative to explicit names or indices as in



`$.store.book[(@.length-1)].title`



using the symbol `@` for the current object. Filter expressions are supported via the syntax `?()` as in



`$.store.book[?(@.price < 10)].title`



Here is a complete overview and a side by side comparison of the JSONPath syntax elements with its XPath counterparts.



| JSONPath | Description |

|----------|---|

| `$`      | the root object/element |

| `@`      | the current object/element |

| `.` or `[]` | child operator |

| `..`     | recursive descent. JSONPath borrows this syntax from E4X. |

| `*`      | wildcard. All objects/elements regardless their names. |

| `[]`     | subscript operator. XPath uses it to iterate over element collections and for predicates. In Javascript and JSON it is the native array operator. |

| `[,]`    | Union operator in XPath results in a combination of node sets. JSONPath allows alternate names or array indices as a set. |

| `[start:end:step]` | array slice operator borrowed from ES4. |

| `?()`    | applies a filter (script) expression. |

| `()`     | script expression, using the underlying script engine. |



## Script engine



### Predefined constant



Package has several predefined constants. 



     e       math.E     float64

     pi      math.Pi    float64

     phi     math.Phi   float64

     

     sqrt2     math.Sqrt2   float64

     sqrte     math.SqrtE   float64

     sqrtpi    math.SqrtPi  float64

     sqrtphi   math.SqrtPhi float64

     

     ln2     math.Ln2    float64

     log2e   math.Log2E  float64

     ln10    math.Ln10   float64

     log10e  math.Log10E float64

          

     true    true       bool

     false   false      bool

     null    nil        interface{}

     

You are free to add new one with function `AddConstant`:



```go

    AddConstant("c", NumericNode("speed of light in vacuum", 299_792_458))

```



#### Examples



Using `true` in path



[Playground](https://play.golang.org/p/h0oFLaE11Tn)



```go

package main



import (

	"fmt"

	

	"github.com/spyzhov/ajson"

)



func main() {

	json := []byte(`{"foo": [true, null, false, 1, "bar", true, 1e3], "bar": [true, "baz", false]}`)

	result, _ := ajson.JSONPath(json, `$..[?(@ == true)]`)

	fmt.Printf("Count of `true` values: %d", len(result))

}

```

Output:

```

Count of `true` values: 3

```



Using `null` in eval



[Playground](https://play.golang.org/p/wpqh1Fw5vWE)



```go

package main



import (

	"fmt"

	

	"github.com/spyzhov/ajson"

)



func main() {

	json := []byte(`{"foo": [true, null, false, 1, "bar", true, 1e3], "bar": [true, "baz", false]}`)

	result, _ := ajson.JSONPath(json, `$..[?(@ == true)]`)

	fmt.Printf("Count of `true` values: %d", len(result))

}

```

Output:

```

Count of `true` values: 3

```



### Supported operations



Package has several predefined operators.



[Operator precedence](https://golang.org/ref/spec#Operator_precedence)



	Precedence    Operator

	    6	    	  **

	    5             *  /  %  <<  >>  &  &^

	    4             +  -  |  ^

	    3             ==  !=  <  <=  >  >= =~

	    2             &&

	    1             ||



[Arithmetic operators](https://golang.org/ref/spec#Arithmetic_operators)



	**   power                  integers, floats

	+    sum                    integers, floats, strings

	-    difference             integers, floats

	*    product                integers, floats

	/    quotient               integers, floats

	%    remainder              integers



	&    bitwise AND            integers

	|    bitwise OR             integers

	^    bitwise XOR            integers

	&^   bit clear (AND NOT)    integers



	<<   left shift             integer << unsigned integer

	>>   right shift            integer >> unsigned integer



	==  equals                  any

	!=  not equals              any

	<   less                    any

	<=  less or equals          any

	>   larger                  any

	>=  larger or equals        any

	=~  equals regex string     strings



You are free to add new one with function `AddOperation`:



```go

	AddOperation("<>", 3, false, func(left *ajson.Node, right *ajson.Node) (node *ajson.Node, err error) {

		result, err := left.Eq(right)

		if err != nil {

			return nil, err

		}

		return BoolNode("neq", !result), nil

	})

```



#### Examples



Using `regex` operator



[Playground](https://play.golang.org/p/Lm_F4OGTMWl)



```go

package main



import (

	"fmt"

	

	"github.com/spyzhov/ajson"

)



func main() {

	json := []byte(`[{"name":"Foo","mail":"[email protected]"},{"name":"bar","mail":"[email protected]"}]`)

	result, err := ajson.JSONPath(json, `$.[?(@.mail =~ '.+@example\\.com')]`)

	if err != nil {

		panic(err)

	}

	fmt.Printf("JSON: %s", result[0].Source())

	// Output:

	// JSON: {"name":"Foo","mail":"[email protected]"}

}



```

Output:

```

JSON: {"name":"Foo","mail":"[email protected]"}

```



### Supported functions



Package has several predefined functions.



    abs          math.Abs           integers, floats

    acos         math.Acos          integers, floats

    acosh        math.Acosh         integers, floats

    asin         math.Asin          integers, floats

    asinh        math.Asinh         integers, floats

    atan         math.Atan          integers, floats

    atanh        math.Atanh         integers, floats

    avg          Average            array of integers or floats

    b64decode    b64 Decoding       string

    b64encode    b64 Encoding       string 

    b64encoden   b64 Encoding (no padding)    string

    cbrt         math.Cbrt          integers, floats

    ceil         math.Ceil          integers, floats

    cos          math.Cos           integers, floats

    cosh         math.Cosh          integers, floats

    erf          math.Erf           integers, floats

    erfc         math.Erfc          integers, floats

    erfcinv      math.Erfcinv       integers, floats

    erfinv       math.Erfinv        integers, floats

    exp          math.Exp           integers, floats

    exp2         math.Exp2          integers, floats

    expm1        math.Expm1         integers, floats

    factorial    N!                 unsigned integer

    first        Get first element  any

    floor        math.Floor         integers, floats

    gamma        math.Gamma         integers, floats

    j0           math.J0            integers, floats

    j1           math.J1            integers, floats

    key          Key of element     string

    last         Get last element   any

    length       len                array

    log          math.Log           integers, floats

    log10        math.Log10         integers, floats

    log1p        math.Log1p         integers, floats

    log2         math.Log2          integers, floats

    logb         math.Logb          integers, floats

    not          not                any

    parent       Get parent element any

    pow10        math.Pow10         integer

    rand         N*rand.Float64     float

    randint      rand.Intn          integer

    root         Get root element   any

    round        math.Round         integers, floats

    roundtoeven  math.RoundToEven   integers, floats

    sin          math.Sin           integers, floats

    sinh         math.Sinh          integers, floats

    sum          Sum                array of integers or floats

    sqrt         math.Sqrt          integers, floats

    tan          math.Tan           integers, floats

    tanh         math.Tanh          integers, floats

    trunc        math.Trunc         integers, floats

    y0           math.Y0            integers, floats

    y1           math.Y1            integers, floats



You are free to add new one with function `AddFunction`:



```go

	AddFunction("trim", func(node *ajson.Node) (result *Node, err error) {

		if node.IsString() {

			return StringNode("trim", strings.TrimSpace(node.MustString())), nil

		}

		return

	})

```



#### Examples



Using `avg` for array



[Playground](https://play.golang.org/p/cM66hTE-CX1)



```go

package main



import (

	"fmt"

	

	"github.com/spyzhov/ajson"

)



func main() {

	json := []byte(`{"prices": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]}`)

	root, err := ajson.Unmarshal(json)

	if err != nil {

		panic(err)

	}

	result, err := ajson.Eval(root, `avg($.prices)`)

	if err != nil {

		panic(err)

	}

	fmt.Printf("Avg price: %0.1f", result.MustNumeric())

	// Output:

	// Avg price: 5.5

}

```

Output:

```

Avg price: 5.5

```



# Examples



Calculating `AVG(price)` when object is heterogeneous.



```json

{

  "store": {

    "book": [

      {

        "category": "reference",

        "author": "Nigel Rees",

        "title": "Sayings of the Century",

        "price": 8.95

      },

      {

        "category": "fiction",

        "author": "Evelyn Waugh",

        "title": "Sword of Honour",

        "price": 12.99

      },

      {

        "category": "fiction",

        "author": "Herman Melville",

        "title": "Moby Dick",

        "isbn": "0-553-21311-3",

        "price": 8.99

      },

      {

        "category": "fiction",

        "author": "J. R. R. Tolkien",

        "title": "The Lord of the Rings",

        "isbn": "0-395-19395-8",

        "price": 22.99

      }

    ],

    "bicycle": {

      "color": "red",

      "price": 19.95

    },

    "tools": null

  }

}

```



## Unmarshal



[Playground](https://play.golang.org/p/xny93dzjZCK)

```go

package main



import (

	"fmt"

	"github.com/spyzhov/ajson"

)



func main() {

	data := []byte(`{"store": {"book": [

{"category": "reference", "author": "Nigel Rees", "title": "Sayings of the Century", "price": 8.95}, 

{"category": "fiction", "author": "Evelyn Waugh", "title": "Sword of Honour", "price": 12.99}, 

{"category": "fiction", "author": "Herman Melville", "title": "Moby Dick", "isbn": "0-553-21311-3", "price": 8.99}, 

{"category": "fiction", "author": "J. R. R. Tolkien", "title": "The Lord of the Rings", "isbn": "0-395-19395-8", "price": 22.99}], 

"bicycle": {"color": "red", "price": 19.95}, "tools": null}}`)



	root, err := ajson.Unmarshal(data)

	if err != nil {

		panic(err)

	}



	store := root.MustKey("store").MustObject()



	var prices float64

	size := 0

	for _, objects := range store {

		if objects.IsArray() && objects.Size() > 0 {

			size += objects.Size()

			for _, object := range objects.MustArray() {

				prices += object.MustKey("price").MustNumeric()

			}

		} else if objects.IsObject() && objects.HasKey("price") {

			size++

			prices += objects.MustKey("price").MustNumeric()

		}

	}



	if size > 0 {

		fmt.Println("AVG price:", prices/float64(size))

	} else {

		fmt.Println("AVG price:", 0)

	}

}

```



## JSONPath:



[Playground](https://play.golang.org/p/7twZHOd6dbT)

```go

package main



import (

	"fmt"

	"github.com/spyzhov/ajson"

)



func main() {

	data := []byte(`{"store": {"book": [

{"category": "reference", "author": "Nigel Rees", "title": "Sayings of the Century", "price": 8.95}, 

{"category": "fiction", "author": "Evelyn Waugh", "title": "Sword of Honour", "price": 12.99}, 

{"category": "fiction", "author": "Herman Melville", "title": "Moby Dick", "isbn": "0-553-21311-3", "price": 8.99}, 

{"category": "fiction", "author": "J. R. R. Tolkien", "title": "The Lord of the Rings", "isbn": "0-395-19395-8", "price": 22.99}], 

"bicycle": {"color": "red", "price": 19.95}, "tools": null}}`)



	nodes, err := ajson.JSONPath(data, "$..price")

	if err != nil {

		panic(err)

	}



	var prices float64

	size := len(nodes)

	for _, node := range nodes {

		prices += node.MustNumeric()

	}



	if size > 0 {

		fmt.Println("AVG price:", prices/float64(size))

	} else {

		fmt.Println("AVG price:", 0)

	}

}

```



## Eval



[Playground](https://play.golang.org/p/lTXnlRU3sgR)

```go

package main



import (

	"fmt"

	"github.com/spyzhov/ajson"

)



func main() {

	json := []byte(`{"store": {"book": [

{"category": "reference", "author": "Nigel Rees", "title": "Sayings of the Century", "price": 8.95}, 

{"category": "fiction", "author": "Evelyn Waugh", "title": "Sword of Honour", "price": 12.99}, 

{"category": "fiction", "author": "Herman Melville", "title": "Moby Dick", "isbn": "0-553-21311-3", "price": 8.99}, 

{"category": "fiction", "author": "J. R. R. Tolkien", "title": "The Lord of the Rings", "isbn": "0-395-19395-8", "price": 22.99}], 

"bicycle": {"color": "red", "price": 19.95}, "tools": null}}`)

	root, err := ajson.Unmarshal(json)

	if err != nil {

		panic(err)

	}

	result, err := ajson.Eval(root, "avg($..price)")

	if err != nil {

		panic(err)

	}

	fmt.Println("AVG price:", result.MustNumeric())

}

```



## Marshal



[Playground](https://play.golang.org/p/i4gXXcA2VLU)

```go

package main



import (

	"fmt"

	"github.com/spyzhov/ajson"

)



func main() {

	json := []byte(`{"store": {"book": [

{"category": "reference", "author": "Nigel Rees", "title": "Sayings of the Century", "price": 8.95}, 

{"category": "fiction", "author": "Evelyn Waugh", "title": "Sword of Honour", "price": 12.99}, 

{"category": "fiction", "author": "Herman Melville", "title": "Moby Dick", "isbn": "0-553-21311-3", "price": 8.99}, 

{"category": "fiction", "author": "J. R. R. Tolkien", "title": "The Lord of the Rings", "isbn": "0-395-19395-8", "price": 22.99}], 

"bicycle": {"color": "red", "price": 19.95}, "tools": null}}`)

	root := ajson.Must(ajson.Unmarshal(json))

	result := ajson.Must(ajson.Eval(root, "avg($..price)"))

	err := root.AppendObject("price(avg)", result)

	if err != nil {

		panic(err)

	}

	marshalled, err := ajson.Marshal(root)

	if err != nil {

		panic(err)

	}

	fmt.Printf("%s", marshalled)

}

```



# Benchmarks



Current package is comparable with `encoding/json` package. 



Test data:

```json

{ "store": {

    "book": [ 

      { "category": "reference",

        "author": "Nigel Rees",

        "title": "Sayings of the Century",

        "price": 8.95

      },

      { "category": "fiction",

        "author": "Evelyn Waugh",

        "title": "Sword of Honour",

        "price": 12.99

      },

      { "category": "fiction",

        "author": "Herman Melville",

        "title": "Moby Dick",

        "isbn": "0-553-21311-3",

        "price": 8.99

      },

      { "category": "fiction",

        "author": "J. R. R. Tolkien",

        "title": "The Lord of the Rings",

        "isbn": "0-395-19395-8",

        "price": 22.99

      }

    ],

    "bicycle": {

      "color": "red",

      "price": 19.95

    }

  }

}

```

JSONPath: `$.store..price`



```

$ go test -bench=. -cpu=1 -benchmem

goos: linux

goarch: amd64

pkg: github.com/spyzhov/ajson

BenchmarkUnmarshal_AJSON          138032              8762 ns/op            5344 B/op         95 allocs/op

BenchmarkUnmarshal_JSON           117423             10502 ns/op             968 B/op         31 allocs/op

BenchmarkJSONPath_all_prices       80908             14394 ns/op            7128 B/op        153 allocs/op

```



# License



MIT licensed. See the [LICENSE](LICENSE) file for details.