https://github.com/codesoap/pbf
A fast and simple library for reading OSM entities directly from PBF files
https://github.com/codesoap/pbf
high-performance open-street-map osm parser pbf
Last synced: 4 months ago
JSON representation
A fast and simple library for reading OSM entities directly from PBF files
- Host: GitHub
- URL: https://github.com/codesoap/pbf
- Owner: codesoap
- License: mit
- Created: 2024-09-30T08:46:39.000Z (over 1 year ago)
- Default Branch: master
- Last Pushed: 2024-10-04T11:08:08.000Z (over 1 year ago)
- Last Synced: 2026-01-13T18:22:11.680Z (5 months ago)
- Topics: high-performance, open-street-map, osm, parser, pbf
- Language: Go
- Homepage:
- Size: 41 KB
- Stars: 4
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
This is a library for extracting OSM entities from PBF files. It is
intended to be used for simple one-off tasks, searching an area of
no more than a few square kilometers, where setting up a PostgreSQL
database would be a disproportionate effort.
It performs reasonably well when working with
OSM extracts, such as the ones obtained from
[download.geofabrik.de](https://download.geofabrik.de/). With modest
hardware (e.g. an old ThinkPad T480) PBF files can be read at roughly
80MiB/s to 300MiB/s. When reading through an extract of the Czech
Republic (828MiB), roughly 320MiB of RAM are used (but this will depend
on the amount of cores on your CPU).
Performance can be improved, by changing the compression
inside PBF files to zstd. This can be done with the [zstd-pbf
tool](https://github.com/codesoap/zstd-pbf).
Find the full documentation of this library at
[godocs.io/github.com/codesoap/pbf](https://godocs.io/github.com/codesoap/pbf).
# Example
```go
filter := pbf.Filter{
Location: func(lat, lon int64) bool {
// A square filter matching the city center of Bremen, Germany.
return lat >= 53_071_495_496 &&
lat <= 53_080_504_504 &&
lon >= 8_799_510_372 &&
lon <= 8_814_489_628
},
Tags: map[string][]string{
// Find bicycle shops.
"shop": {"bicycle"},
},
}
// wget https://download.geofabrik.de/europe/germany/bremen-latest.osm.pbf
results, err := pbf.ExtractEntities("/tmp/bremen-latest.osm.pbf", filter)
resultCount := len(results.Nodes) + len(results.Ways) + len(results.Relations)
fmt.Printf("Found %d bicycle shop(s) in the center of Bremen.\n", resultCount)
```
# Development setup
To generate some protobuf related code, you need the `protoc` tool, the
`protoc-gen-go` tool and the `protoc-gen-go-vtproto` tool; the latter
two can be installed like this:
```
go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.34.2
go install github.com/planetscale/vtprotobuf/cmd/protoc-gen-go-vtproto@v0.6.0
```
# Ideas for the Future
For now, ways and relations will be incomplete, if they only partially
lie within the location filter or their members didn't match the tag
filter. This is undesirable, for example, when trying to render an
extracted area as an image. Thus it would be useful to have a flag that
makes the library return "ancillary entities". An idea for the interface
can be found at commit 9b673e35bd0510e4ad53a2875dcf4a7ea4ca085a.