https://github.com/xiaodaigh/parquet-format-technical-notes
https://github.com/xiaodaigh/parquet-format-technical-notes
Last synced: 2 months ago
JSON representation
- Host: GitHub
- URL: https://github.com/xiaodaigh/parquet-format-technical-notes
- Owner: xiaodaigh
- Created: 2020-05-03T07:40:46.000Z (about 5 years ago)
- Default Branch: master
- Last Pushed: 2020-05-16T14:58:10.000Z (about 5 years ago)
- Last Synced: 2025-01-21T10:08:26.037Z (4 months ago)
- Size: 1.95 KB
- Stars: 0
- Watchers: 3
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# parquet-format-technical-notes
## Script
The first and last 4 bytes of a parquet file are "PAR1".
The 8th-5th last bytes are the metadata size = `meta_len`.
You can read the metadata by seeking to
```
filesize - 8 - metadatalen
```
where
```julia
sz = filesize(io)
data_size = (sz - meta_len - 2length("PAR1")-length(size of meta : Int32)) = sz - meta_len - 12
```