Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/preligens-lab/buzzard
Advanced raster and geometry manipulations
https://github.com/preligens-lab/buzzard
data-science footprint gdal geometry geospatial gis image ogr osr python raster raster-pipelines vector
Last synced: 2 months ago
JSON representation
Advanced raster and geometry manipulations
- Host: GitHub
- URL: https://github.com/preligens-lab/buzzard
- Owner: preligens-lab
- License: apache-2.0
- Created: 2019-11-09T12:27:43.000Z (over 4 years ago)
- Default Branch: master
- Last Pushed: 2024-04-16T15:21:25.000Z (3 months ago)
- Last Synced: 2024-04-28T08:14:39.153Z (2 months ago)
- Topics: data-science, footprint, gdal, geometry, geospatial, gis, image, ogr, osr, python, raster, raster-pipelines, vector
- Language: Python
- Homepage:
- Size: 11.1 MB
- Stars: 37
- Watchers: 4
- Forks: 5
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- Changelog: changes.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
Lists
- awesome-stars - earthcube-lab/buzzard - Advanced raster and geometry manipulations (Python)
README
# `buzzard`
In a nutshell, the `buzzard` library provides powerful abstractions to manipulate together images and geometries that come from different kind of sources (`GeoTIFF`, `PNG`, `GeoJSON`, `Shapefile`, `numpy array`, `buzzard pipelines`, ...).
[](https://github.com/earthcube-lab/buzzard/blob/master/LICENSE)[](https://circleci.com/gh/earthcube-lab/buzzard/tree/master)[](https://codecov.io/gh/earthcube-lab/buzzard)[](https://buzzard.readthedocs.io/en/latest)
[
](https://join.slack.com/t/buzzard-python/shared_invite/enQtNjY0NDQ2MzU3MzgzLTJhNTZhNjAwOGIyM2RkOTdkZGE5MGUwZGEzZGQwODkyMzY2N2YwMTg5ZmI1NDc2MjY2MGM2ZTdhNDc3M2E1YTI)
## `buzzard` is
- A _python_ library.
- Primarily designed to hide all cumbersome operations when doing data-science with [GIS](https://en.wikipedia.org/wiki/Geographic_information_system) files.
- A multipurpose computer vision library, it can be used in all kind of situations where images or geometries are involved.
- A pythonic wrapper for _osgeo_'s _gdal_/_ogr_/_osr_.
- A solution to work with arbitrary large images by simplifying and automating the manipulation of image slices.
## `buzzard` contains
- A [`Dataset`](https://buzzard.readthedocs.io/en/latest/dataset.html) class that oversees a set of opened raster and vector files.
- An immutable toolbox class, the [`Footprint`](https://buzzard.readthedocs.io/en/latest/footprint.html), designed to locate a rectangle in both an image space and a geometry space.
## How to open and read files
This example demonstrates how to visualize a large raster polygon per polygon.
```py
import buzzard as buzz
import numpy as np
import matplotlib.pyplot as plt
# Open the files. Only metadata are kept in memory
r = buzz.open_raster('path/to/rgba-image.tif')
v = buzz.open_vector('path/to/polygons.geojson', driver='GeoJSON')
# Load the polygons from disk one by one as shapely objects
for poly in v.iter_data():
# Compute the Footprint bounding `poly`
fp = r.fp.intersection(poly)
print(fp)
# Load the image from disk at `fp` to a numpy array
rgb = r.get_data(fp=fp, channels=(0, 1, 2))
alpha = r.get_data(fp=fp, channels=3)
# Create a boolean mask as a numpy array from the shapely polygon
mask = np.invert(fp.burn_polygons(poly))
# Darken pixels outside of polygon and set transparent pixels to orange
rgb[mask] = (rgb[mask] * 0.5).astype(np.uint8)
rgb[alpha == 0] = [236, 120, 57]
# Show the result with matplotlib
plt.imshow(rgb)
plt.show()
```
Images from the [ISPRS's Potsdam dataset](http://www2.isprs.org/commissions/comm3/wg4/2d-sem-label-potsdam.html).
`Footprint(tl=(3183.600000, -914.550000), br=(3689.700000, -1170.450000), size=(506.100000, 255.900000), rsize=(3374, 1706))`
`Footprint(tl=(3171.600000, -1321.500000), br=(4553.400000, -2400.000000), size=(1381.800000, 1078.500000), rsize=(9212, 7190))`
## How to create files and manipulate _Footprints_
```py
import buzzard as buzz
import numpy as np
import matplotlib.pyplot as plt
import keras
r = buzz.open_raster('path/to/rgba-image.tif')
km = keras.models.load_model('path/to/deep-learning-model.hdf5')
# Chunk the raster's Footprint to Footprints of size
# 1920 x 1080 pixels stored in a 2d numpy array
tiles = r.fp.tile((1920, 1080))
all_roads = []
# Perform an inference for each tile
for i, fp in enumerate(tiles.flat):
rgb = r.get_data(fp=fp, channels=(0, 1, 2))
# Perform pixelwise semantic segmentation with a keras model
predictions_heatmap = km.predict(rgb[np.newaxis, ...])[0]
predictions_top1 = np.argmax(predictions_heatmap, axis=-1)
# Save the prediction to a `geotiff`
path = f'predictions_{i}.tif'
with buzz.create_raster(path=path, fp=fp, dtype='uint8', channel_count=1).close as out:
out.set_data(predictions_top1)
# Extract the road polygons by transforming a numpy boolean mask to shapely polygons
road_polygons = fp.find_polygons(predictions_top1 == 3)
all_roads += road_polygons
# Show the result with matplotlib for one tile
if i == 2:
plt.imshow(rgb)
plt.imshow(predictions_top1)
plt.show()
# Save all roads found to a single `shapefile`
with buzz.create_vector(path='roads.shp', type='polygon').close as out:
for poly in all_roads:
out.insert_data(poly)
```
## Advanced examples
Additional examples can be found here:
- [Files and _Footprints_ in depth](https://github.com/earthcube-lab/buzzard/blob/master/doc/examples.ipynb)
- [_async rasters_ in depth](https://github.com/earthcube-lab/buzzard/blob/master/doc/notebook2/async_rasters.ipynb)
## `buzzard` allows
- Opening and creating [raster](https://buzzard.readthedocs.io/en/latest/dataset_raster.html) and [vector](https://buzzard.readthedocs.io/en/latest/dataset_vector.html) files. Supports all [GDAL drivers (GTiff, PNG, ...)](https://www.gdal.org/formats_list.html) and all [OGR drivers (GeoJSON, DXF, Shapefile, ...)](https://www.gdal.org/ogr_formats.html).
- [Reading](https://buzzard.readthedocs.io/en/latest/source_gdal_file_raster.html#raster-file-get-data) raster files pixels from disk to _numpy.ndarray_.
- _Options:_ `sub-rectangle reading`, `rotated and scaled sub-rectangle reading (thanks to on-the-fly remapping with OpenCV)`, `automatic parallelization of read and remapping (soon)`, `async (soon)`, `be the source of an image processing pipeline (soon)`.
- _Properties:_ `thread-safe parallel reads`.
- [Writing](https://buzzard.readthedocs.io/en/latest/source_gdal_file_raster.html#raster-file-set-data) raster files pixels to disk from _numpy.ndarray_.
- _Options:_ `sub-rectangle writing`, `rotated and scaled sub-rectangle writing (thanks to on-the-fly remapping with OpenCV)`, `masked writing (slow)`.
- [Reading](https://buzzard.readthedocs.io/en/latest/source_gdal_file_vector.html#vector-file-iter-data) vector files geometries from disk to _shapely objects_, _geojson dict_ and _raw coordinates_.
- _Options:_ `masking`.
- _Properties:_ `thread-safe parallel reads`.
- [Writing](https://buzzard.readthedocs.io/en/latest/source_gdal_file_vector.html#vector-file-insert-data) vector files geometries to disk from _shapely objects_, _geojson dict_ and _raw coordinates_.
- Powerful manipulations of [raster windows](https://buzzard.readthedocs.io/en/latest/footprint.html)
- [Instantiation](https://buzzard.readthedocs.io/en/latest/dataset_recipe.html#buzzard.Dataset.create_raster_recipe) of image processing pipelines where each node is a raster, and each edge is a user defined python function transforming _numpy.ndarray_ objects (beta, partially implemented).
- _Options:_ `automatic parallelization using user defined thread or process pools`, `disk caching`.
- _Properties:_ `lazy evaluation`, `deterministic`, `automatic tasks chunking into tiles`, `fine grain task prioritization`, `backpressure prevention`.
- [Spatial reference homogenization](https://buzzard.readthedocs.io/en/latest/dataset.html#on-the-fly-re-projections-in-buzzard) between opened files like a GIS software does (beta)
## Documentation
https://buzzard.readthedocs.io/
## Dependencies
The following table lists dependencies along with the minimum version, their status for the project and the related license.
| Library | Version | Mandatory | License | Comment |
|------------------|----------|-----------|--------------------------------------------------------------------------------------|---------------------------------------------------------------|
| gdal | >=2.3.3 | Yes | [MIT/X](https://gdal.org/license.html) | Hard to install. Will be included in `buzzard` wheels |
| opencv-python | >=3.1.0 | Yes | [3-clause BSD](http://opencv.org/license.html) | Easy to install with `opencv-python` wheels. Will be optional |
| shapely | >=1.6.1 | Yes | [3-clause BSD](https://github.com/Toblerity/Shapely/blob/master/LICENSE.txt) | |
| affine | >=2.0.0 | Yes | [3-clause BSD](https://github.com/sgillies/affine/blob/master/LICENSE.txt) | |
| numpy | >=1.15.0 | Yes | [numpy](https://numpy.org/doc/stable/license.html) | |
| scipy | >=0.19.1 | Yes | [scipy](https://www.scipy.org/scipylib/license.html) | |
| pint | >=0.8.1 | Yes | [3-clause BSD](https://github.com/hgrecco/pint/blob/master/LICENSE) | || |
| sortedcontainers | >=1.5.9 | Yes | [apache](https://github.com/grantjenks/python-sortedcontainers/blob/master/LICENSE) | |
| Rtree | >=0.8.3 | Yes | [MIT](https://github.com/Toblerity/rtree/blob/master/LICENSE.txt) | |
| scikit-image | >=0.14.0 | Yes | [scikit-image](https://github.com/scikit-image/scikit-image/blob/master/LICENSE.txt) | |
| pytest | >=3.2.2 | No | [MIT](https://docs.pytest.org/en/latest/license.html) | Only for tests |
## How to install from terminal
### Anaconda and pip
```sh
# Step 1 - Install Anaconda
# https://www.anaconda.com/download/
# Step 2 - Create env
conda create -n buzz python gdal>=2.3.3 shapely rtree -c 'conda-forge'
# Step 3 - Activate env
conda activate buzz
# Step 4 - Install buzzard
pip install buzzard
```
### Docker
```sh
docker build -t buzz --build-arg PYTHON_VERSION=3.7 https://raw.githubusercontent.com/earthcube-lab/buzzard/master/.circleci/images/base-python/Dockerfile
docker run -it --rm buzz bash
pip install buzzard
```
### Package manager and pip
```sh
# Step 1 - Install GDAL and rtree ******************************************* **
# Windows
# https://www.lfd.uci.edu/~gohlke/pythonlibs/#gdal
# https://www.lfd.uci.edu/~gohlke/pythonlibs/#rtree
# MacOS
brew install gdal
brew tap osgeo/osgeo4mac
brew tap --repair
brew install gdal2
brew install spatialindex
export PATH="/usr/local/opt/gdal2/bin:$PATH"
python3 -m pip install 'gdal==2.3.3'
# Ubuntu
# Run the commands from the following Dockerfile:
# https://github.com/earthcube-lab/buzzard/blob/master/doc/ubuntu_install/Dockerfile
# Step 2 - Install buzzard ************************************************** **
python3 -m pip install buzzard
```
## Supported Python versions
To enjoy the latest buzzard features, update your python!
#### Full python support
- Latest tested version: `3.8` (June 2018)
- Oldest tested version: `3.6` (Sept 2015)
#### Partial python support
- For python `2.7`: use buzzard version `0.4.4`
- For python `3.4`: use buzzard version `0.6.3`
- For python `3.5`: use buzzard version `0.6.4`
## Slack
You want some help? You have a question? You want to contribute? Join us on Slack!
[
](https://join.slack.com/t/buzzard-python/shared_invite/enQtNjY0NDQ2MzU3MzgzLTJhNTZhNjAwOGIyM2RkOTdkZGE5MGUwZGEzZGQwODkyMzY2N2YwMTg5ZmI1NDc2MjY2MGM2ZTdhNDc3M2E1YTI)
## How to test
```sh
git clone https://github.com/earthcube-lab/buzzard
pip install -r buzzard/requirements-dev.txt
pytest buzzard/buzzard/test
```
## How to build documentation
```sh
cd docs
make html
open _build/html/index.html
```
## Contributions and feedback
Welcome to the `buzzard` project! We appreciate any contribution and feedback, your proposals and pull requests will be considered and responded to. For more information, see the [`CONTRIBUTING.md`](./CONTRIBUTING.md) file.
## Authors
See [AUTHORS](./AUTHORS.md)
## License and Notice
See [LICENSE](./LICENSE) and [NOTICE](./NOTICE).
## Other pages
- [TODO](https://www.notion.so/buzzard/2c94ef6ee8da4d6280834129cc00f4d2?v=334ead18796342feb32ba85ccdfcf69f) on `notion.so`
------------------------------------------------------------------------------------------------------------------------