Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/danielgtaylor/shorthand
Structured data & CLI shorthand syntax for Go
https://github.com/danielgtaylor/shorthand
cbor cli golang-library hacktoberfest json shorthand structured-data
Last synced: 3 months ago
JSON representation
Structured data & CLI shorthand syntax for Go
- Host: GitHub
- URL: https://github.com/danielgtaylor/shorthand
- Owner: danielgtaylor
- License: mit
- Created: 2021-10-18T03:37:10.000Z (over 3 years ago)
- Default Branch: main
- Last Pushed: 2024-01-30T17:28:47.000Z (about 1 year ago)
- Last Synced: 2024-10-30T00:27:46.355Z (3 months ago)
- Topics: cbor, cli, golang-library, hacktoberfest, json, shorthand, structured-data
- Language: Go
- Homepage:
- Size: 337 KB
- Stars: 29
- Watchers: 3
- Forks: 2
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Structured Data Shorthand Syntax
[](https://pkg.go.dev/github.com/danielgtaylor/shorthand?tab=doc) [](https://goreportcard.com/report/github.com/danielgtaylor/shorthand) [](https://github.com/danielgtaylor/shorthand/actions?query=workflow%3ACI+branch%3Amain++) [](https://codecov.io/gh/danielgtaylor/shorthand)
Shorthand is a superset and friendlier variant of JSON designed with several use-cases in mind:
| Use Case | Example |
| -------------------- | ------------------------------------------------ |
| CLI arguments/input | `my-cli post 'foo.bar[0]{baz: 1, hello: world}'` |
| Patch operations | `name: undefined, item.tags[]: appended` |
| Query language | `items[created before 2022-01-01].{id, tags}` |
| Configuration format | `{json.save.autoFormat: true}` |
The shorthand syntax supports the following features, described in more detail with examples below:
- Superset of JSON (valid JSON is valid shorthand)
- Optional commas, quotes, and sometimes colons
- Support for comments & trailing commas
- Automatic type coercion
- Support for bytes, datetimes, and maps with non-string keys
- Nested object & array creation
- Loading values from files
- Editing existing data
- Appending & inserting to arrays
- Unsetting properties
- Moving properties & items
- Querying, array filtering, and field selection
The following are all completely valid shorthand and result in the same output:
```
foo.bar[]{baz: 1, hello: world}
```
```
{
// This is a comment
foo.bar[]{
baz: 1
hello: world
}
}
```
```json
{
"foo": {
"bar": [
{
"baz": 1,
"hello": "world"
}
]
}
}
```
This library has excellent test coverage to ensure correctness and is additionally fuzz tested to prevent panics.
## Alternatives & Inspiration
The CLI shorthand syntax is not the only one you can use to generate data for CLI commands. Here are some alternatives:
- [jo](https://github.com/jpmens/jo)
- [jarg](https://github.com/jdp/jarg)
For example, the shorthand example given above could be rewritten as:
```sh
$ jo -p foo=$(jo -p bar=$(jo -a $(jo -p baz=1 hello=world)))
```
The shorthand syntax implementation described herein uses those and the following for inspiration:
- [YAML](http://yaml.org/)
- [W3C HTML JSON Forms](https://www.w3.org/TR/html-json-forms/)
- [jq](https://stedolan.github.io/jq/)
- [JMESPath](http://jmespath.org/)
It seems reasonable to ask, why create a new syntax?
1. Built-in. No extra executables required. Your tool ships ready-to-go.
2. No need to use sub-shells to build complex structured data.
3. Syntax is closer to YAML & JSON and mimics how you do queries using tools like `jq` and `jmespath`.
4. It's _optional_, so you can use your favorite tool/language instead, while at the same time it provides a minimum feature set everyone will have in common.
## Features in Depth
You can use the included `j` executable to try out the shorthand format examples below. Examples are shown in JSON, but the shorthand parses into structured data that can be marshalled as other formats, like YAML or TOML if you prefer.
```sh
go get -u github.com/danielgtaylor/shorthand/cmd/j
```
Also feel free to use this tool to generate structured data for input to other commands.
Here is a diagram overview of the language syntax, which is similar to [JSON's syntax](https://www.json.org/json-en.html) but adds a few things:
Note:
- `string` can be quoted (with `"`) or unquoted.
- The `query` syntax in the diagram above is described below in the [Querying](#querying) section.
### Keys & Values
At its most basic, a structure is built out of key & value pairs. They are separated by commas:
```sh
$ j hello: world, question: how are you?
{
"hello": "world",
"question": "how are you?"
}
```
### Types
Shorthand supports the standard JSON types, but adds some of its own as well to better support binary formats and its query features.
| Type | Description |
| --------- | ---------------------------------------------------------------- |
| `null` | JSON `null` |
| `boolean` | Either `true` or `false` |
| `number` | JSON number, e.g. `1`, `2.5`, or `1.4e5` |
| `string` | Quoted or unquoted strings, e.g. `hello` or `"hello"` |
| `bytes` | `%`-prefixed, unquoted, base64-encoded binary data, e.g. `%wg==` |
| `time` | Date/time in ISO8601, e.g. `2022-01-01T12:00:00Z` |
| `array` | JSON array, e.g. `[1, 2, 3]` |
| `object` | JSON object, e.g. `{"hello": "world"}` |
### Type Coercion
Well-known values like `null`, `true`, and `false` get converted to their respective types automatically. Numbers, bytes, and times also get converted. Similar to YAML, anything that doesn't fit one of those is treated as a string. This automatic coercion can be disabled by just wrapping your value in quotes.
```sh
# With coercion
$ j empty: null, bool: true, num: 1.5, string: hello
{
"bool": true,
"empty": null,
"num": 1.5,
"string": "hello"
}
# As strings
$ j empty: "null", bool: "true", num: "1.5", string: "hello"
{
"bool": "true",
"empty": "null",
"num": "1.5",
"string": "hello"
}
# Passing the empty string
$ j blank1: , blank2: ""
{
"blank1": "",
"blank2": ""
}
```
### Objects
Nested objects use a `.` separator when specifying the key.
```sh
$ j foo.bar.baz: 1
{
"foo": {
"bar": {
"baz": 1
}
}
}
```
Properties of nested objects can be grouped by placing them inside `{` and `}`. The `:` becomes optional for nested objects, so `foo.bar: {...}` is equivalent to `foo.bar{...}`.
```sh
$ j foo.bar{id: 1, count.clicks: 5}
{
"foo": {
"bar": {
"count": {
"clicks": 5
},
"id": 1
}
}
}
```
### Arrays
Arrays are surrounded by square brackets like in JSON:
```sh
# Simple array
$ j [1, 2, 3]
[
1,
2,
3
]
```
Array indexes use square brackets `[` and `]` to specify the zero-based index to set an item. If the index is out of bounds then `null` values are added as necessary to fill the array. Use an empty index `[]` to append to the an existing array. If the item is not an array, then a new one will be created.
```sh
# Nested arrays
$ j [0][2][0]: 1
[
[
null,
null,
[
1
]
]
]
# Appending arrays
$ j a[]: 1, a[]: 2, a[]: 3
{
"a": [
1,
2,
3
]
}
```
### Loading from Files
Sometimes a field makes more sense to load from a file than to be specified on the commandline. The `@` preprocessor lets you load structured data, text, and bytes depending on the file extension and whether all bytes are valid UTF-8:
```sh
# Load a file's value as a parameter
$ j foo: @hello.txt
{
"foo": "hello, world"
}
# Load structured data
$ j foo: @hello.json
{
"foo": {
"hello": "world"
}
}
```
Remember, it's possible to disable this behavior with quotes:
```sh
$ j 'twitter: "@user"'
{
"twitter": "@user"
}
```
### Patch (Partial Update)
Partial updates are supported on existing data, which can be used to implement HTTP `PATCH`, templating, and other similar features. The suggested content type for HTTP `PATCH` is `application/shorthand-patch`. This feature combines the best of both:
- [JSON Merge Patch](https://datatracker.ietf.org/doc/html/rfc7386)
- [JSON Patch](https://www.rfc-editor.org/rfc/rfc6902)
Partial updates support:
- Appending arrays via `[]`
- Inserting before via `[^index]`
- Removing fields or array items via `undefined`
- Moving/swapping fields or array items via `^`
- The right hand side is a path to the value to swap. See Querying below for the path syntax.
Note: When sending shorthand patches file loading via `@` should be disabled as the files will not exist on the server.
Some examples:
```sh
# First, let's create some data we'll modify later
$ j id: 1, tags: [a, b, c] >data.json
# Now let's append to the tags array
$ j
The `filter` syntax is described in the documentation for [mexpr](https://github.com/danielgtaylor/mexpr).
Examples:
```sh
# First, let's make a complex file to query
$ j 'users: [{id: 1, age: 5, friends: [a, b]}, {id: 2, age: 6, friends: [b, c]}, {id: 3, age: 5, friends: [c, d]}]' >data.json
# Query for each user's ID
$ j