https://github.com/aosingh/json-flattener
Python utility to flatten a JSON object
https://github.com/aosingh/json-flattener
Last synced: about 1 year ago
JSON representation
Python utility to flatten a JSON object
- Host: GitHub
- URL: https://github.com/aosingh/json-flattener
- Owner: aosingh
- Created: 2020-10-11T16:12:47.000Z (over 5 years ago)
- Default Branch: main
- Last Pushed: 2020-10-14T14:52:35.000Z (over 5 years ago)
- Last Synced: 2025-02-17T11:43:58.909Z (over 1 year ago)
- Language: Python
- Size: 24.4 KB
- Stars: 1
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# jflat [](https://travis-ci.com/github/aosingh/json-flattener)
[]((https://www.python.org/downloads/release/python-370/)) [](https://www.python.org/downloads/release/python-370/) [](https://www.python.org/downloads/release/python-380/)
[](https://www.pypy.org/index.html)
Python utility to flatten a JSON object
## Install
`jflat` can be installed using `pip` and `virtualenv`.
1. Create a local directory where you would like to work from
```bash
mkdir jflat_test
cd jflat_test
```
2. Create an isolated Python virtual environment.
`virtualenv` can be installed using pip as explained [here](https://virtualenv.pypa.io/en/latest/installation.html)
```bash
virtualenv --python=python3 .venv
source .venv/bin/activate
pip install --upgrade pip
```
3. Install `jflat` from source using `pip`
```bash
pip install git+https://github.com/aosingh/json-flattener.git#egg=jflat
```
### Dependencies
Dependencies are defined as install requirements in `setup.py`. You need not install them separately.
We use the following external libraries.
- click
- To create a Command Line Interface (CLI) for `jflat`
- pytest
- To run unit tests
## CLI
```bash
Usage: jflat [OPTIONS] [JSON_STRING]
Options:
--log-level [CRITICAL|FATAL|ERROR|WARN|WARNING|INFO|DEBUG|NOTSET]
--out-file TEXT Path to the output JSON file
--sort-keys / --no-sort-keys [default: False]
--help Show this message and exit.
```
## Examples
### Simple json string
At times it is convenient to test with an inline JSON string. Make sure you enclose the
JSON string in single quotes as shown below
```bash
jflat '{"username":"xyz","password":{"hash": "e8c400f94e807f0d374d9c971bec018"}}'
```
The following output will be written to stdout.
```json5
{
"password.hash": "e8c400f94e807f0d374d9c971bec018",
"username": "xyz"
}
```
### File input
Additionally, `jflat` can accept input from a file.
Let's say our input file `input.json` looks like below
```json5
{
"friends": {
"1": {
"name": "Mr. X",
"age": 13,
"hobbies": {
"football": true,
"vedic maths": false,
"origami": true
}
},
"2": {
"name": "Mr. Y",
"age": 56,
"hobbies": {
"football": false,
"vedic maths": true,
"origami": false
}
}
}
}
```
Invoke the JSON flat command as shown below
```bash
jflat < input.json
```
You will see the flattened JSON object printed to stdout
```json5
{
"friends.2.hobbies.origami": false,
"friends.2.hobbies.vedic maths": true,
"friends.2.hobbies.football": false,
"friends.2.age": 56,
"friends.2.name": "Mr. Y",
"friends.1.hobbies.origami": true,
"friends.1.hobbies.vedic maths": false,
"friends.1.hobbies.football": true,
"friends.1.age": 13,
"friends.1.name": "Mr. X"
}
```
### Sort keys
Pass the `--sort-keys` flag to sort the keys alphabetically in the flattened JSON.
```bash
jflat < input.json --sort-keys
```
```json5
{
"friends.1.age": 13,
"friends.1.hobbies.football": true,
"friends.1.hobbies.origami": true,
"friends.1.hobbies.vedic maths": false,
"friends.1.name": "Mr. X",
"friends.2.age": 56,
"friends.2.hobbies.football": false,
"friends.2.hobbies.origami": false,
"friends.2.hobbies.vedic maths": true,
"friends.2.name": "Mr. Y"
}
```
### Save the output to a file
The default behavior is to write the flattened JSON object to stdout. If you want to save the flattened JSON object to a file, you can use the `--out-file` option. An example
is shown below
```bash5
jflat < input.json --sort-keys --out-file=output.json
```
Verify the contents of the output file
```bash
cat output.json
```
```json5
{
"friends.1.age": 13,
"friends.1.hobbies.football": true,
"friends.1.hobbies.origami": true,
"friends.1.hobbies.vedic maths": false,
"friends.1.name": "Mr. X",
"friends.2.age": 56,
"friends.2.hobbies.football": false,
"friends.2.hobbies.origami": false,
"friends.2.hobbies.vedic maths": true,
"friends.2.name": "Mr. Y"
}
```
## Continuous Integration (CI)
The code repository is integrated with Travis CI. As shown below, in the build matrix, the build process tests
for Linux and different Python 3 versions.
```yaml
language: python
matrix:
include:
- os: linux
python: 3.6
- os: linux
python: 3.7
- os: linux
python: 3.8
- os: linux
python: pypy3
```
Additionally, github workflow is configured to run all tests with the latest version of `Ubuntu` and Python 3
## Unit Test & Coverage
Unit tests are integrated with the build process. We use `coverage` to run the test cases and generate a coverage report.
```bash
coverage run -m pytest --verbose
```
```bash
coverage report -i -m
Name Stmts Miss Cover Missing
---------------------------------------------------------
jflat/__init__.py 1 0 100%
jflat/cli.py 28 1 96% 38
jflat/flattener.py 25 0 100%
jflat/tests/__init__.py 0 0 100%
jflat/tests/test_jflat.py 98 0 100%
---------------------------------------------------------
TOTAL 152 1 99%
```
## API
The Python API can be used to access the flattened Python dictionary directly.
```python
from jflat.flattener import JSONFlattener
data = {
"friends": {
"1": {
"name": "Mr. X",
"age": 13,
"hobbies":{
"football": True,
"vedic maths": False,
"origami": True
}
},
"2": {
"name": "Mr. Y",
"age": 56,
"hobbies": {
"football": False,
"vedic maths": True,
"origami": False
}
}
}
}
j = JSONFlattener(data=data)
flattened_dict = j()
print(flattened_dict)
>>> {'friends.1.age': 13,
'friends.1.hobbies.football': True,
'friends.1.hobbies.origami': True,
'friends.1.hobbies.vedic maths': False,
'friends.1.name': 'Mr. X',
'friends.2.age': 56,
'friends.2.hobbies.football': False,
'friends.2.hobbies.origami': False,
'friends.2.hobbies.vedic maths': True,
'friends.2.name': 'Mr. Y'}
```
## Notes
- Standard python library `json` is used to parse the input JSON object and also
serialize the flattened dict.
- Iterative Depth First Search (DFS) is used to flatten the JSON object.
- The following JSON types are tested.
- boolean
- null
- number
- object
- string
- In the current implementation, if the JSON object has an array, the array will be skipped.
- The name `jflat` is inspired from command-line JSON processor `jq`.
## TODO
- Determine the maximum level of nesting supported.