{"id":13782335,"url":"https://github.com/kylebarron/geo-index","last_synced_at":"2025-05-16T06:00:24.026Z","repository":{"id":200160046,"uuid":"704276948","full_name":"kylebarron/geo-index","owner":"kylebarron","description":"A Rust crate and Python library for packed, immutable, zero-copy spatial indexes.","archived":false,"fork":false,"pushed_at":"2025-02-28T15:42:54.000Z","size":1588,"stargazers_count":157,"open_issues_count":21,"forks_count":9,"subscribers_count":4,"default_branch":"main","last_synced_at":"2025-05-12T23:53:25.330Z","etag":null,"topics":["flatbush","georust","geospatial","kdbush","kdtree","pyo3","python","rtree","rust","spatial-indexes"],"latest_commit_sha":null,"homepage":"https://kylebarron.dev/geo-index","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/kylebarron.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE_APACHE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-10-12T23:35:55.000Z","updated_at":"2025-05-08T17:01:59.000Z","dependencies_parsed_at":"2024-06-04T23:34:44.449Z","dependency_job_id":"56aa3c11-5a6d-4679-89bb-debb035d439e","html_url":"https://github.com/kylebarron/geo-index","commit_stats":{"total_commits":62,"total_committers":2,"mean_commits":31.0,"dds":"0.016129032258064502","last_synced_commit":"87e1ec87e6f1b3a730622172e0c0c6b4a6683873"},"previous_names":["kylebarron/flatbush-rs","kylebarron/geo-index"],"tags_count":6,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kylebarron%2Fgeo-index","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kylebarron%2Fgeo-index/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kylebarron%2Fgeo-index/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kylebarron%2Fgeo-index/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/kylebarron","download_url":"https://codeload.github.com/kylebarron/geo-index/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253843185,"owners_count":21972870,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["flatbush","georust","geospatial","kdbush","kdtree","pyo3","python","rtree","rust","spatial-indexes"],"created_at":"2024-08-03T18:01:35.406Z","updated_at":"2025-05-16T06:00:24.005Z","avatar_url":"https://github.com/kylebarron.png","language":"Rust","funding_links":[],"categories":["Watchlist","Rust","Geospatial"],"sub_categories":[],"readme":"# geo-index\n\n[![crates.io version][crates.io_badge]][crates.io_link]\n[![docs.rs docs][docs.rs_badge]][docs.rs_link]\n[![PyPI][pypi_badge]][pypi_link]\n\n[crates.io_badge]: https://img.shields.io/crates/v/geo-index.svg\n[crates.io_link]: https://crates.io/crates/geo-index\n[docs.rs_badge]: https://docs.rs/geo-index/badge.svg\n[docs.rs_link]: https://docs.rs/geo-index\n[pypi_badge]: https://badge.fury.io/py/geoindex-rs.svg\n[pypi_link]: https://pypi.org/project/geoindex-rs/\n\nA Rust crate and [Python library](https://kylebarron.dev/geo-index/latest/) for packed, immutable, zero-copy spatial indexes.\n\n## Features\n\n- **An R-tree and k-d tree written in safe rust.**\n- **Fast.** Because of optimizations available by using immutable indexes, tends to be faster than dynamic implementations like [`rstar`](https://github.com/georust/rstar).\n- **Memory efficient.** The index is fully _packed_, meaning that all nodes are at full capacity (except for the last node at each tree level). This means the RTree and k-d tree use less memory. And because the index is backed by a single buffer, it exhibits excellent memory locality.\n- **Bounded memory**. For any given number of items and node size, you can infer the total memory used by the RTree or KDTree.\n- **Multiple R-tree sorting methods.** Currently, [hilbert](https://en.wikipedia.org/wiki/Hilbert_R-tree) and [sort-tile-recursive (STR)](https://ia600900.us.archive.org/27/items/nasa_techdoc_19970016975/19970016975.pdf) sorting methods are implemented, but it's extensible to other spatial sorting algorithms in the future, like [overlap-minimizing top-down (OMT)](https://ceur-ws.org/Vol-74/files/FORUM_18.pdf).\n- **ABI-stable:** the index is contained in a single `Vec\u003cu8\u003e`, compatible with the [`flatbush`](https://github.com/mourner/flatbush) and [`kdbush`](https://github.com/mourner/kdbush) JavaScript libraries. Being ABI-stable means that the spatial index can be persisted for later use or shared zero-copy between Rust and another program like Python.\n- **Generic over a set of coordinate types:** `i8`, `u8`, `i16`, `u16`, `i32`, `u32`, `f32`, `f64`. These are the only coordinate types currently supported to maintain binary compatibility with the upstream JS libraries.\n- **Efficient bulk loading.** As an immutable index, _only_ bulk loading is supported.\n- Optional `rayon` feature for parallelizing part of the sort in the sort-tile-recursive (`STRSort`) method.\n\n## Drawbacks\n\n- Trees are _immutable_. After creating the index, items can no longer be added or removed.\n- Only two-dimensional indexes is supported. This can still be used with higher-dimensional input data as long as it's fine to only index two of the dimensions.\n- Queries return insertion indexes into the input set, so you must manage your own collections.\n- Only the set of coordinate types that exist in JavaScript are allowed, to maintain FFI compatibility with the reference JavaScript implementations. Hence this does not support other types like `u64`.\n\n## Alternative crates\n\n- [`rstar`](https://github.com/georust/rstar): a dynamic RTree implementation.\n- [`kdtree`](https://github.com/mrhooray/kdtree-rs): a dynamic KDTree implementation.\n- [`kdbush`](https://github.com/pka/rust-kdbush): a port of `kdbush` but does not strive for FFI-compatibility.\n- [`static_aabb2d_index`](https://github.com/jbuckmccready/static_aabb2d_index): a port of `flatbush` but does not strive for FFI-compatibility.\n- [`static-bushes`](https://github.com/apendleton/static-bushes): a port of `flatbush` and `kdbush` but does not strive for FFI-compatibility.\n\n## Inspiration\n\n[@mourner](https://github.com/mourner)'s amazing [`flatbush`](https://github.com/mourner/flatbush) and [`kdbush`](https://github.com/mourner/kdbush) libraries are the fastest JavaScript libraries for immutable R-trees and k-d trees. Part of their appeal in the browser is that they're backed by a single, contiguous buffer, and thus can be moved from the main thread to another thread (a [web worker](https://developer.mozilla.org/en-US/docs/Web/API/Web_Workers_API)) [without any copies](https://developer.mozilla.org/en-US/docs/Web/API/Web_Workers_API/Transferable_objects).\n\nBy porting and expanding on those JavaScript libraries and ensuring that the internal memory layout is _exactly_ maintained, we can bootstrap zero-copy use cases both inside and outside the browser. In-browser use cases can interop between a rust-based WebAssembly module and the upstream JS libraries _without copies_. Outside-browser use cases can interop between multiple Rust libraries or between Rust and Python without copies.\n\n## Why zero-copy?\n\nI'm excited about Rust to speed up Python and JavaScript via compiled extension modules. It's true that you can create Python bindings to a Rust library, have Rust manage the memory, and never need to worry about zero-copy data structures. But when someone else writes a C library that would like to interface with your data, if you don't have an ABI-stable way to share the data, you need to serialize it and they need to deserialize it, which is costly.\n\nFor example, in Python, Shapely (and by extension the C library GEOS) is used for most geospatial data storage. But separate Python libraries with C extensions can't use the same GEOS memory because the underlying storage isn't ABI-stable. So there has to be a serde step in between.\n\n[GeoArrow](https://geoarrow.org/) solves this problem for geospatial vector data, because it defines a language-independent, ABI-stable memory layout. So you can safely move memory between Python/Rust/C just by exchanging pointer information.\n\nBut it's very useful to be able to share large spatial data, declare that the data is already spatially ordered, _and_ share a spatial index, all at no cost.\n\nCurrently, this library is used under the hood in [`geoarrow-rs`](https://github.com/geoarrow/geoarrow-rs) to speed up boolean operations and spatial joins. But over a medium-term time horizon, I believe that exposing the raw index data will enable exciting FFI use cases that are presently impossible.\n\n## Future work\n\n- Geographic queries. Currently all queries are planar.\n\n## Benchmarks\n\nThis is just _one_ benchmark; I recommend benchmarking with your own data, but this indicates construction is ~2x faster than `rstar` and search is ~33% faster.\n\n```ignore\ncargo bench --bench rtree\n```\n\n```ignore\nconstruction (geo-index, hilbert)\n                        time:   [80.503 ms 80.891 ms 81.350 ms]\nconstruction (geo-index, STRTree)\n                        time:   [115.60 ms 116.52 ms 117.64 ms]\nconstruction (geo-index, hilbert, f64 to f32, including casting)\n                        time:   [86.409 ms 86.681 ms 86.984 ms]\nconstruction (geo-index f32)\n                        time:   [78.292 ms 78.393 ms 78.514 ms]\nconstruction (rstar bulk)\n                        time:   [158.48 ms 159.34 ms 160.29 ms]\n\nsearch (flatbush)       time:   [115.97 µs 116.41 µs 116.86 µs]\nsearch (flatbush STRTree)\n                        time:   [115.85 µs 117.57 µs 118.95 µs]\nsearch (flatbush f32)   time:   [113.04 µs 114.56 µs 115.99 µs]\nsearch (rstar)          time:   [151.53 µs 153.62 µs 155.84 µs]\n```\n\nWith the `rayon` feature, the sorting phase of the `STRTree` is faster:\n\n```ignore\ncargo bench --bench rtree --features rayon\n```\n\n```ignore\nconstruction (geo-index, STRTree)\n                        time:   [71.825 ms 72.099 ms 72.382 ms]\n                        change: [-38.738% -38.125% -37.570%] (p = 0.00 \u003c 0.05)\n                        Performance has improved.\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkylebarron%2Fgeo-index","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkylebarron%2Fgeo-index","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkylebarron%2Fgeo-index/lists"}