https://github.com/multiprocessio/json-storage-benchmarking
https://github.com/multiprocessio/json-storage-benchmarking
Last synced: about 1 year ago
JSON representation
- Host: GitHub
- URL: https://github.com/multiprocessio/json-storage-benchmarking
- Owner: multiprocessio
- Created: 2021-10-18T13:53:25.000Z (over 4 years ago)
- Default Branch: main
- Last Pushed: 2021-10-18T18:08:59.000Z (over 4 years ago)
- Last Synced: 2025-03-16T13:28:09.749Z (over 1 year ago)
- Language: Python
- Size: 3.91 KB
- Stars: 1
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
## To generate fake data
```
pip3 install faker
python3 ./generate_schema_data.py 1_000_000
```
## To run the benchmarks
```
python3 ./benchmark.py
+------------------+--------------+--------------------+----------------------------------------+---------------------+-----------------------------+
| | Read JSON | Sum int field | Sort by first field and take first 100 | Sort by first field | Group by first field, count |
+------------------+--------------+--------------------+----------------------------------------+---------------------+-----------------------------+
| array of arrays | 40.54s (N/A) | 0.27s (4999013444) | 1.02s (1970-01-01T00:18:47) | 1.01s (N/A) | 0.69s (999668) |
+------------------+--------------+--------------------+----------------------------------------+---------------------+-----------------------------+
| array of objects | 47.35s (N/A) | 0.35s (4999013444) | 1.09s (1970-01-01T00:18:47) | 1.09s (N/A) | 0.72s (999668) |
+------------------+--------------+--------------------+----------------------------------------+---------------------+-----------------------------+
| columnar | 29.19s (N/A) | 0.01s (4999013444) | 0.51s (1970-01-01T00:18:47) | 49.30s (N/A) | 0.94s (999668) |
+------------------+--------------+--------------------+----------------------------------------+---------------------+-----------------------------+
```