Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/geopolars/geopolars
Geospatial extensions for Polars
https://github.com/geopolars/geopolars
apache-arrow geospatial polars pyo3 python rust
Last synced: 3 months ago
JSON representation
Geospatial extensions for Polars
- Host: GitHub
- URL: https://github.com/geopolars/geopolars
- Owner: geopolars
- License: mit
- Created: 2022-05-24T03:31:05.000Z (over 2 years ago)
- Default Branch: main
- Last Pushed: 2024-01-30T03:12:56.000Z (9 months ago)
- Last Synced: 2024-06-11T19:14:57.475Z (5 months ago)
- Topics: apache-arrow, geospatial, polars, pyo3, python, rust
- Language: Rust
- Homepage: https://geopolars.org
- Size: 5.82 MB
- Stars: 516
- Watchers: 28
- Forks: 23
- Open Issues: 48
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
- awesome-georust - GeoPolars - Geospatial extensions for Polars. (Watchlist)
README
# GeoPolars
Geospatial DataFrames for Rust and Python
> Update (July 2024): GeoPolars is [blocked](https://github.com/pola-rs/polars/issues/1830#issuecomment-2218102856) on Polars supporting [Arrow extension types](https://github.com/pola-rs/polars/issues/9112), which would allow GeoPolars to persist geometry type information and coordinate reference system (CRS) metadata. The primary workaround would be subclassing `polars.DataFrame`, similar to how the `geopandas.GeoDataFrame` is a subclass of `pandas.DataFrame`, but this requires a high maintenance burden and wouldn't work with polars' `LazyFrame`.
## Summary
GeoPolars extends the [Polars][polars] DataFrame library for use with geospatial data.
- Uses [GeoArrow][geo-arrow-spec] as the internal memory model.
- Written in Rust
- Bindings to Python
- Multithreading capable
At this point, GeoPolars is a **prototype** and should not be considered production-ready.
## Use from..
### Rust
GeoPolars is [published to crates.io](https://crates.io/crates/geopolars) under the name `geopolars`.
Documentation is available at [docs.rs/geopolars](https://docs.rs/geopolars).
### Python
An early alpha (`v0.1.0-alpha.4`) is published to PyPI:
```
pip install --pre geopolars
```
The publishing processs includes binary wheels for many platforms, so it should be easy to install, without needing to compile the underlying Rust code from source.
### WebAssembly
Polars itself does not yet exist in WebAssembly, though there has been discussion about adding bindings for it. The long-term goal of GeoPolars is to have a WebAssembly API as well.
## Comparison with GeoPandas
Imitation is the sincerest form of flattery! GeoPandas — and its underlying libraries of `shapely` and `GEOS` — is an incredible production-ready tool.
GeoPolars is nowhere near the functionality or stability of GeoPandas, but competition is good and, due to its pure-Rust core, GeoPolars will be much easier to use in WebAssembly.
## Future work
The biggest pieces of future work are:
- Store geometries in the efficient Arrow-native format, rather than as WKB buffers (as the prototype currently does). This is blocked on Polars, which doesn't currently support Arrow `FixedSizeList` data types, but they've recently [expressed openness](https://github.com/pola-rs/polars/issues/4014#issuecomment-1212376538) to adding minimal `FixedSizeList` support.
- Enable `georust/geo` algorithms to access Arrow data with zero copy. The prototype currently copies WKB geometries into `geo` structs on each geometry operation, which is expensive.
This is blocked on adding support to the `geo` library for geometry access traits, which is a large undertaking. See [georust/geo/discussions/838](https://github.com/georust/geo/discussions/838). I've started exploration on this
- Implement GeoArrow extension types for seamless handling of CRS metadata in Rust, rather than in the Python wrapper.
[polars]: https://github.com/pola-rs/polars
[geo-arrow-spec]: https://github.com/geoarrow/geoarrow