awesome-serialization-deserialization
A curated list of data serialization formats, libraries, tools, and resources across programming languages and use cases.
https://github.com/naranyala/awesome-serialization-deserialization
Last synced: 9 days ago
JSON representation
-
Learning Resources
-
Research Papers
- Performance Comparison of Messaging Protocols and Serialization Formats - Proos et al., 2020 (65 citations)
- Overview of Serialization Technologies (CERN Indico) - Pivarski, 2018
- Comparison of Data Serialization Formats (Wikipedia) - Comprehensive comparison table
- Heep: An FPGA-accelerated Reader for Apache Parquet - ETH Zürich, 2025
- Benchmarking Apache Arrow Flight - ACM Conference Paper
- Evaluating Serialization Formats for Space-Efficient Storage - 2024 study
- eishay/jvm-serializers - JVM serialization
- ProgrammerAL/SerializationBenchmarks - C#/.NET
-
Technical Documentation
- Protocol Buffers Language Guide - Official Protobuf documentation
- YAML 1.2.2 Specification - Official YAML specification
- MessagePack Specification - MessagePack format specification
- Cap'n Proto Style Guide - Cap'n Proto language reference
- Apache Arrow Format Specification - Arrow columnar format
- Apache Avro Specification - Avro format specification
-
-
Libraries by Language
-
C & C++
- RapidJSON
- nlohmann/json
- simdjson
- ArduinoJson
- Bitsery - only binary serialization | [GitHub](https://github.com/fraillt/bitsery) |
- Jansson
- jsmn
- protobuf-c - 2 | Protobuf implementation for C | [GitHub](https://github.com/protobuf-c/protobuf-c) |
- flatbuffers
- BitSerializer - format C++ serialization | [GitHub](https://github.com/PavelYevstifeev/bitserializer) |
-
C# & .NET
- System.Text.Json - in .NET JSON (.NET Core 3.0+) | [Docs](https://docs.microsoft.com/en-us/dotnet/api/system.text.json) |
- Newtonsoft.Json
- protobuf-net - net/protobuf-net) |
- MessagePack-CSharp - CSharp/MessagePack-CSharp) |
- ZeroFormatter
- System.Text.Json - in .NET JSON (.NET Core 3.0+) | [Docs](https://docs.microsoft.com/en-us/dotnet/api/system.text.json) |
- System.Text.Json - in .NET JSON (.NET Core 3.0+) | [Docs](https://docs.microsoft.com/en-us/dotnet/api/system.text.json) |
-
Go
- encoding/json - 3 | Built-in JSON package (stdlib) | [Docs](https://pkg.go.dev/encoding/json) |
- gob - 3 | Go's native binary format (stdlib) | [Docs](https://pkg.go.dev/encoding/gob) |
- protobuf-go - 3 | Official Go protobuf | [GitHub](https://github.com/protocolbuffers/protobuf-go) |
- vmihailenco/msgpack
- go-yaml/yaml - yaml/yaml) |
- segmentio/encoding - performance encoding | [GitHub](https://github.com/segmentio/encoding) |
- json-iterator/go - in JSON replacement | [GitHub](https://github.com/json-iterator/go) |
-
Java & JVM
- Jackson - used Java serialization | [GitHub](https://github.com/FasterXML/jackson) |
- Gson
- Moshi
- dsl-json - 3 | High-performance Java JSON library | [GitHub](https://github.com/ngs-doo/dsl-json) |
- Kryo - 3 | Fast, efficient Java serialization | [GitHub](https://github.com/EsotericSoftware/kryo) |
- protobuf-java - 3 | Official Java protobuf | [GitHub](https://github.com/protocolbuffers/protobuf) |
- Avro Java
-
JavaScript & TypeScript
- JSON (native) - in JSON support (ES5+) | [MDN](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/JSON) |
- BSON.js - bson) |
- msgpack-javascript - javascript) |
- avsc - 3 | Zero-dependency Avro implementation | [GitHub](https://github.com/mtth/avsc) |
- protobuf.js - 3 | Protocol Buffers for JavaScript | [GitHub](https://github.com/protobufjs/protobuf.js) |
- schemapack - based binary encoding | [GitHub](https://github.com/phretaddin/schemapack) |
- cbor-js - js) |
- msgpackr - performance MessagePack | [GitHub](https://github.com/kriszyp/msgpackr) |
-
PHP
- json_encode/decode - in PHP JSON (stdlib) | [Docs](https://www.php.net/manual/en/book.json.php) |
- webmozart/json
-
Python
- json - in JSON encoder/decoder (stdlib) | [Docs](https://docs.python.org/3/library/json.html) |
- pickle - specific | PSF | Python object serialization (stdlib) ⚠️ | [Docs](https://docs.python.org/3/library/pickle.html) |
- srsly - performance serialization wrapper | [GitHub](https://github.com/explosion/srsly) |
- avro-python3 - python3/) |
- pyarrow
- pyyaml
- orjson - based) | [GitHub](https://github.com/ijl/orjson) |
- ujson - 3 | Ultra-fast JSON encoder/decoder (C-based) | [GitHub](https://github.com/ultrajson/ultrajson) |
- safetensors
- msgpack-python
- avro-python3 - python3/) |
- protobuf - 3 | Official Google protobuf | [PyPI](https://pypi.org/project/protobuf/) |
-
Ruby
- oj
- MultiJSON
- msgpack-ruby - ruby) |
- json - in JSON (stdlib) | [Docs](https://ruby-doc.org/stdlib/libdoc/json/rdoc/JSON.html) |
-
Rust
-
Swift
- Codable - in Swift serialization (Swift 4+) | [Docs](https://developer.apple.com/documentation/swift/codable) |
- SwiftyJSON
- MessagePack.swift
-
-
Related Awesome Lists
-
Research Papers
- awesome-json - Curated list of JSON libraries and resources
- awesome-serialization - Curated list of serialization formats
- awesome-xml - Curated list of XML standards and libraries
- awesome-cpp - C++ libraries (serialization section)
- awesome-python - Python libraries (serialization section)
- awesome-go - Go libraries (serialization section)
- awesome-rust - Rust libraries (serialization section)
- awesome-hessian - Hessian serialization resources
-
-
Schema Evolution
-
Security Considerations
-
Documented CVEs
-
Mitigation Strategies
-
-
Serialization Formats
-
Binary Formats
- Apache Avro - in) | [Website](https://avro.apache.org/) |
- Apache Thrift
- MessagePack
- CBOR
- BSON
- Smile - format-specification) |
- Hessian
-
Domain-Specific Formats
- Parquet
- ORC
- HDF5
- ONNX
- NetCDF
- Delta Lake - based) | ✅ | ✅ | ACID transactions on data lakes | [Website](https://delta.io/) |
- Apache Iceberg
- SavedModel
- CoreML - device ML model format | [Docs](https://developer.apple.com/documentation/coreml) |
- ASN.1 - T standard) | [ITU](https://www.itu.int/en/ITU-T/asn1/) |
-
Text-Based Formats
-
Zero-Copy & High-Performance Formats
- Apache Arrow - memory analytics, DataFrame interchange | [GitHub](https://github.com/apache/arrow) |
- FlatBuffers - latency systems | [GitHub](https://github.com/google/flatbuffers) |
- Cap'n Proto - performance RPC, distributed systems | [GitHub](https://github.com/capnproto/capnproto) |
- Bebop - time applications | [GitHub](https://github.com/RainwayApp/bebop) |
-
-
Standards & Specifications
-
Tools & Utilities
-
ISO Standards
- jq - platform | Command-line JSON processor | [Website](https://stedolan.github.io/jq/) |
- yq - platform | Command-line YAML processor | [GitHub](https://github.com/mikefarah/yq) |
- Apache Parquet Tools - format/) |
- HDFView - platform | HDF5 file viewer and editor | [Download](https://www.hdfgroup.org/downloads/hdfview/) |
- JSON Schema Validator
- Buf - platform | Modern Protobuf toolchain | [Website](https://buf.build/) |
- Wire
- Prototool - platform | Uber's Protobuf toolkit (archived) | [GitHub](https://github.com/uber/prototool) |
- HDFView - platform | HDF5 file viewer and editor | [Download](https://www.hdfgroup.org/downloads/hdfview/) |
- Zebra - platform | Data file format viewer | [GitHub](https://github.com/ColinSu/zebra) |
-
-
Use Cases & Applications
-
APIs & Network Services
- gRPC
- RSocket
- CloudEvents
- AsyncAPI - driven API spec | ⚠️ | Event-driven architectures | [Website](https://www.asyncapi.com/) |
- Connect
-
Configuration Files
-
Graph Data
- JSON-LD - based) | JSON-LD Context | ✅ | JSON-LD Playground | [Website](https://json-ld.org/) |
- Turtle
- GraphML - based | GraphML Schema | ❌ | yEd, Gephi | [Website](http://graphml.graphdrawing.org/) |
- DOT
- GraphSON - based) | ❌ | ❌ | Apache TinkerPop | [Docs](https://tinkerpop.apache.org/docs/current/dev/io/#graphson) |
- GML
-
Machine Learning & AI
-
Scientific & Research
-
Security-Focused Serialization
- COSE - based | Cryptographic signatures | CBOR Object Signing and Encryption | [RFC 8152](https://datatracker.ietf.org/doc/html/rfc8152) |
- PASETO - proof tokens | Platform-Agnostic SEcure TOkens (JWT alternative) | [Website](https://paseto.io/) |
- SealedObject
- Capsule (Java)
- SealedObject
-
Workflow Management
- CWL
- Apache Airflow
- WDL
- Nextflow - based) | Nextflow runtime | Computational pipelines | [Website](https://www.nextflow.io/) |
-
Programming Languages
Categories
Sub Categories
Research Papers
17
ISO Standards
13
Python
12
C & C++
10
Domain-Specific Formats
10
Text-Based Formats
8
JavaScript & TypeScript
8
Go
7
Rust
7
Java & JVM
7
C# & .NET
7
Binary Formats
7
Graph Data
6
Documented CVEs
6
Technical Documentation
6
IETF RFC Standards
6
Security-Focused Serialization
5
APIs & Network Services
5
Machine Learning & AI
5
Ruby
4
Zero-Copy & High-Performance Formats
4
Mitigation Strategies
4
Workflow Management
4
Configuration Files
3
Swift
3
Evolution Rules by Format
3
W3C Standards
3
Scientific & Research
2
PHP
2
Keywords
json
15
serialization
13
msgpack
7
rust
7
json-parser
6
python
6
go
6
golang
6
protobuf
5
c
5
awesome-list
5
awesome
5
encoding
4
protocol-buffers
4
messagepack
4
c-plus-plus
3
serializer
3
typescript
3
yaml
3
c-sharp
3
javascript
3
java
3
unity
2
serde
2
marshalling
2
json-pointer
2
protoc
2
decoding
2
rpc
2
cpp11
2
deserialization
2
ujson
2
serialization-library
2
big-data
2
grpc
2
toml
2
xml
2
bson
2
json-patch
1
loongarch
1
json-merge-patch
1
neon
1
simd
1
sse42
1
vs2019
1
json-diff
1
x64
1
bash
1
header-only
1
cli
1