https://github.com/sameer/osm-pbf
Parse and write the OSM PBF Format in Rust
https://github.com/sameer/osm-pbf
Last synced: 8 months ago
JSON representation
Parse and write the OSM PBF Format in Rust
- Host: GitHub
- URL: https://github.com/sameer/osm-pbf
- Owner: sameer
- License: other
- Created: 2023-03-27T02:55:48.000Z (about 3 years ago)
- Default Branch: main
- Last Pushed: 2023-05-23T21:11:14.000Z (almost 3 years ago)
- Last Synced: 2025-04-02T11:54:18.920Z (12 months ago)
- Language: Rust
- Homepage:
- Size: 210 KB
- Stars: 0
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE-APACHE
Awesome Lists containing this project
README
# osm-pbf
Read and write the [PBF format](https://wiki.openstreetmap.org/wiki/PBF_Format) for [Open Street Map](https://www.openstreetmap.org/#map=13/47.4475/-122.3084) (OSM).
## Functionality
### Reading
There are two steps to reading the PBF format: parsing and decoding.
Parsing builds [fileblocks](https://wiki.openstreetmap.org/wiki/PBF_Format#Encoding_OSM_entities_into_fileblocks) out of the raw data.
Decoding converts fileblocks into the OSM [elements](https://wiki.openstreetmap.org/wiki/Elements) that they contain.
### Writing
Similarly, there are two steps to writing the PBF format: encoding and serialization.
Encoding converts OSM elements into fileblocks. This crate does not support encoding yet.
Serialization flattens fileblocks into raw data.
## Execution
This crate is written with async I/O for use with [tokio](https://tokio.rs/).
### Parallelism
The code is serial in nature but it's possible to parallelize encoding/decoding since fileblocks are independent in PBF.
Read parallelization example:
1. Call `get_osm_pbf_locations` to get a stream of fileblock locations
1. Call `parse_osm_pbf_at_location` for each location independently
1. Process blocks as desired
Write parallelization example:
1. Split your blocks into chunks
1. Call `write_osm_pbf` for each chunk independently with an in-memory vector as the writer
1. As each call completes, write them to their final destination (i.e. a file)
## Compression
There is a feature for each supported compression algorithm:
Name|Default Feature|Supported
---|---|---
Zlib|✅|✅
Zstd|❌|✅
Lzma|❌|✅
Lz4|❌|❌
Bzip2|❌|❌
Lz4 support is [not available yet](https://github.com/Nemo157/async-compression/issues/12).
Bzip2 has been deprecated for years so it is not supported.
There isn't any fine-grained control over encoding but feel free to file an issue if you are interested.