Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/johnmurray/json-inspect
A command-line tool for inspecting and working with JSON files
https://github.com/johnmurray/json-inspect
Last synced: about 1 month ago
JSON representation
A command-line tool for inspecting and working with JSON files
- Host: GitHub
- URL: https://github.com/johnmurray/json-inspect
- Owner: JohnMurray
- License: apache-2.0
- Created: 2016-07-14T10:47:30.000Z (over 8 years ago)
- Default Branch: master
- Last Pushed: 2016-08-18T01:36:52.000Z (over 8 years ago)
- Last Synced: 2024-11-20T13:01:46.407Z (about 2 months ago)
- Language: Python
- Size: 28.3 KB
- Stars: 0
- Watchers: 2
- Forks: 0
- Open Issues: 1
-
Metadata Files:
- Readme: readme.md
- License: LICENSE
Awesome Lists containing this project
README
# json-inspect
[](https://badge.fury.io/py/json-inspect)
[](https://github.com/JohnMurray/json-inspect/blob/master/LICENSE)
A command-line tool for inspecting and working with JSON files. Current sub-commands supported include
__Table of Contents__
- [Installation](#installation)
- [Sub-Commands](#sub-commands)
- [histo](#histo)
- [ext](#ext)
- [keys](#keys)
- [Planned Improvements](#planned-improvements)
## Installation
You an simply install with the normal method for Python utils, such as
```sh
pip install json-inspect
```
## Sub-Commands
Each sub-command has it's own help file and options and serves different purposes. Please be sure to read the
docs for each command as it may not work like the others. To see the most up-to-date documentation on all available
sub-commands, run the utility with the `-h` options without providing a sub-command.
```text
$ json-inspect -h
usage: json-inspect [-h] [-v] [-f FILE] {histo,ext,keys,validate,format} ...
Utility for inspecting JSON files/input
positional arguments:
{histo,ext,keys,validate,format}
histo Create histograms from JSON values
ext Extract values from JSON
keys Lists keys in a JSON document
validate Validate text input as JSON (coming soon maybe)
format Nicely format JSON input (coming soon maybe)
optional arguments:
-h, --help show this help message and exit
-v, --version show program's version number and exit
-f FILE, --file FILE JSON file to read in. If not provided STDIN will be
used
```
Note that there are some global options. The main thing here is that some sort of JSON input is required
for this utility to work. This is defined with the global `-f` option, or by providing input via `STDIN`.
### histo
If you are processing a large number of JSON objects/arrays, then it may be useful to know what fields are present,
what values they contain, and the frequency of both. Starting off with the help file
```text
usage: json-inspect histo [-h] [-p PREFIX] [-c] paths [paths ...]
Generate a histogram based on values found using a JSON search expression
positional arguments:
paths search paths to create histograms for, prefixed with
optional value from the --prefix option
optional arguments:
-h, --help show this help message and exit
-p PREFIX, --prefix PREFIX
String to prefix all search-paths with
-c, --conflate Conflate non-empty responses to the same value
```
* The `-p` option will allow you to prefix all of your search paths. This is useful if you are performing
multiple searches that have a common prefix for deeply nested JSON searches.
* The `-c` option will conflate your histogram to two values, `__none__` and `__some__`. The first meaning
that no key/value was found for a given lookup in your search path and the latter meaning that _a_ value was
found. This is useful if you only care about the frequency of presence.
A search path is a dot-delimited expression used for traversing JSON objects. An example to get us started is
```text
[].*.user.demographic.regions.[].name
```
To start, note that search expressions contain 3 types of tokens
* `[]` - indicates an array. Each item in the array is collected and will be processed by the next token
* `*` - indicates an object in which all fields should be collected and will be processed by the next token
* `TOKEN` - a field-value of an object. It's value will be collected and processed by the next token
For our example above, it would be satisfied by the following JSON document
```json
[
{
"facebook": {
"user": {
"demographic": {
"regions": [ {"name": "US"}, {"name": "Kentucky"}, {"name": "Louisville"} ]
}
}
},
"google": {
"user": {
"demographic": {
"regions": [ {"name": "US"}, {"name": "Kentucky"}, {"name": "Highland Heights"} ]
}
}
}
},
{
"twitter": {
"user": {
"demographic": {
"regions": [ {"name": "UK"}, {"name": "Wales"} ]
}
}
}
}
]
```
Running the `histo` sub-command, we would get output such as
```sh
cat test.json | json-inspect histo '[].*.user.demographic.regions.[].name'
[].*.user.demographic.regions.[].name:
Highland Heig | ######################### | (1)
US | ################################################## | (2)
Louisville | ######################### | (1)
Kentucky | ################################################## | (2)
UK | ######################### | (1)
Wales | ######################### | (1)
```
The bar-chart represents the number found relative to the max found with a total count of finds per-element
in the rightmost column.
### ext
The `ext`, extraction command, is used for pulling data out of JSON files. It supports the
same prefix and search expressions as `histo` along with a few other options for value output.
```
$ json-inspect ext -h
usage: json-inspect ext [-h] [-p PREFIX] [-d DELIM] [-F] paths [paths ...]
Extract values from JSON using a JSON search expression
positional arguments:
paths search paths to return values for
optional arguments:
-h, --help show this help message and exit
-p PREFIX, --prefix PREFIX
String to prefix all search-paths with
-d DELIM, --delimiter DELIM
String to delimit all results by
-F, --flatten Flatten array and object values. For objects, only the
values (not the keys) are retained in the falttened
values
```
Using the same input test JSON file from the `histo` command, we can see an example of
output
```
$ cat test.json | json-inspect ext "[].*.user.demographic.regions.[].*"
Louisville,Kentucky,US,Highland Heights,Kentucky,US,Wales,UK
$ cat test.json | json-inspect ext -d '|' "[].*.user.demographic.regions.[].*"
Louisville|Kentucky|US|Highland Heights|Kentucky|US|Wales|UK
```
### keys
The `keys` command is used for listing alls keys found within a JSON document. Flags can be
provided to filter which keys are extracted.
```
$ json-inspect keys -h
usage: json-inspect keys [-h] [-n] [-o] [-p] [-e]
List all keys within the JSON document using a period-delimited notation
similar to search-paths
optional arguments:
-h, --help show this help message and exit
-n, --exclude-null Exclude keys that contain a null value
-o, --exclude-empty-objects
Exclude keys that contain an empty object
-p, --exclude-empty-primitives
Exclude keys that contain an empty primitive value
(zero and empty string)
-e, --exclude-empty-array
Exclude keys that contain an empty array value
```
Adding a new lines to each top-level object in our `test.json` with
```
"null": null,
"empty_object": {},
"empty_array": [],
"empty_string": "",
"empty_int": 0,
"empty_float": 0.0,
```
We can make some sample calls such as
```
$ cat test.json | json-inspect keys
facebook.null
facebook.empty_object
facebook.user.demographic.regions.name
facebook.empty_float
facebook.empty_array
# ...
# filter all keys with empty values
$ cat test.json | json-inspect keys -nope
facebook.user.demographic.regions.name
google.user.demographic.regions.name
twitter.user.demographic.regions.name
```
## Planned Improvements
[ ] Refactor code to be testable (maybe write some test)
[ ] Add support for `**`