https://github.com/davidbuchanan314/dag-cbor-benchmark
Test data for benchmarking dag-cbor decoders and encoders
https://github.com/davidbuchanan314/dag-cbor-benchmark
Last synced: 17 days ago
JSON representation
Test data for benchmarking dag-cbor decoders and encoders
- Host: GitHub
- URL: https://github.com/davidbuchanan314/dag-cbor-benchmark
- Owner: DavidBuchanan314
- Created: 2024-02-24T11:30:51.000Z (about 1 year ago)
- Default Branch: main
- Last Pushed: 2025-03-28T12:03:21.000Z (about 1 month ago)
- Last Synced: 2025-04-12T13:12:56.744Z (17 days ago)
- Language: Python
- Size: 3.69 MB
- Stars: 3
- Watchers: 3
- Forks: 2
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# dag-cbor-benchmark
Test data for benchmarking dag-cbor decoders and encoders
The following test cases came from https://github.com/serde-rs/json-benchmark, converted from JSON to DAG-CBOR. They're very "real world", but don't contain any CBOR-specific data types (bytes, CIDs).
```
data/canada.json.dagcbor 1_056_200 bytes - many lists of floats
data/citm_catalog.json.dagcbor 342_373 bytes - moderately nested objects, including unicode strings
data/twitter.json.dagcbor 402_814 btyes - ditto
```
Trivial test cases:
```
data/trivial_helloworld.dagcbor 14 bytes - A hello world string (sanity check and/or test constant overheads)
```
Synthetic torture tests:
```
data/torture_nested_lists.dagcbor 10_000_001 bytes - 10M nested lists
data/torture_nested_maps.dagcbor 20_000_001 bytes - 10M nested maps (each with empty-string keys)
data/torture_cids.dagcbor 4_100_005 btyes - 100K CIDs (in a flat list)
```
TODO: gather some more real-world DAG-CBOR from bluesky, ideally anonymised (maybe just the MST blocks?)
TODO: include CAR test cases
## Python shootout:
Resutlts of `bench.py`, on Python 3.12.9, Fedora 40, 2021 M1 Pro MBP
Contestants:
- [DavidBuchanan314/dag-cbrrr](https://github.com/DavidBuchanan314/dag-cbrrr) (release v1.0.1)
- [MarshalX/python-libipld](https://github.com/MarshalX/python-libipld) (release v3.0.1)
- [hashberg-io/dag-cbor](https://github.com/hashberg-io/dag-cbor) (release v0.3.3)
```
Hello World Decode:
===================
cbrrr : 193 ns
libipld : 122 ns
dag_cbor : 3936 ns
Hello World Encode:
===================
cbrrr : 124 ns
libipld : 109 ns
dag_cbor : 4564 ns
Realistic Decode Tests:
=======================
canada.json.dagcbor cbrrr : 3.87 ms (259.96 MB/s)
canada.json.dagcbor libipld : 4.26 ms (236.40 MB/s)
canada.json.dagcbor dag_cbor : 100.44 ms (10.03 MB/s)
citm_catalog.json.dagcbor cbrrr : 2.41 ms (135.31 MB/s)
citm_catalog.json.dagcbor libipld : 2.97 ms (109.81 MB/s)
citm_catalog.json.dagcbor dag_cbor : 43.82 ms (7.45 MB/s)
twitter.json.dagcbor cbrrr : 1.37 ms (279.79 MB/s)
twitter.json.dagcbor libipld : 1.64 ms (234.64 MB/s)
twitter.json.dagcbor dag_cbor : 18.42 ms (20.85 MB/s)
Realistic Encode Tests:
=======================
canada.json.dagcbor cbrrr : 0.80 ms (1259.39 MB/s)
canada.json.dagcbor libipld : 2.62 ms (383.93 MB/s)
canada.json.dagcbor dag_cbor : 208.59 ms (4.83 MB/s)
citm_catalog.json.dagcbor cbrrr : 1.31 ms (248.68 MB/s)
citm_catalog.json.dagcbor libipld : 1.61 ms (202.47 MB/s)
citm_catalog.json.dagcbor dag_cbor : 59.15 ms (5.52 MB/s)
twitter.json.dagcbor cbrrr : 0.61 ms (626.25 MB/s)
twitter.json.dagcbor libipld : 0.63 ms (612.57 MB/s)
twitter.json.dagcbor dag_cbor : 22.95 ms (16.74 MB/s)
Decode Torture Tests:
=====================
torture_cids.dagcbor cbrrr 30.4 ms (128.43 MB/s)
torture_cids.dagcbor libipld 20.4 ms (191.30 MB/s)
torture_cids.dagcbor dag_cbor 6988.5 ms (0.56 MB/s)
torture_nested_lists.dagcbor cbrrr 747.4 ms (12.76 MB/s)
torture_nested_lists.dagcbor libipld ERROR: RecursionError: maximum recursion depth exceeded in DAG-CBOR decoding
torture_nested_lists.dagcbor dag_cbor ERROR: maximum recursion depth exceeded
torture_nested_maps.dagcbor cbrrr 1207.0 ms (15.80 MB/s)
torture_nested_maps.dagcbor libipld ERROR: RecursionError: maximum recursion depth exceeded in DAG-CBOR decoding
torture_nested_maps.dagcbor dag_cbor ERROR: maximum recursion depth exceeded
```
Note: "maximum recursion depth exceeded" is an acceptable result for the recursion torture tests - a segfault isn't though :P