Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/nismod/open-gira

Open-data Global Infrastructure Risk/Resilience Analysis
https://github.com/nismod/open-gira

Last synced: 3 months ago
JSON representation

Open-data Global Infrastructure Risk/Resilience Analysis

Awesome Lists containing this project

README 

        
# Open Global Infrastructure Risk/Resilience Analysis



[![mdBook Documentation](https://github.com/nismod/open-gira/actions/workflows/docs.yml/badge.svg?branch=main)](https://nismod.github.io/open-gira)

[![pyTest](https://github.com/nismod/open-gira/actions/workflows/test.yml/badge.svg?branch=main)](https://github.com/nismod/open-gira/actions/workflows/test.yml)

[![snakemake workflow](https://img.shields.io/badge/snakemake-open--gira-informational)](https://snakemake.github.io/snakemake-workflow-catalog/?usage=nismod/open-gira)



## Introduction



This open-source [snakemake](https://snakemake.readthedocs.io/en/stable/)

workflow can be used to analyse environmental risks to infrastructure

networks using global open data. It is a work in progress.



Goals:

- Automated pipeline for reproducible analysis anywhere in the world

- Maps per-country and of larger areas

- Charts/stats of exposure per admin region, per hazard type, scenario, epoch

- Consider transport, electricity, water, communications systems

- Consider river flooding, storm surge coastal flooding, tropical cyclones

- Estimate direct damages to physical networks

- Estimate indirect effects of disruption - people affected, economic activity disrupted



Non-goals:

- Using closed data, which may be appropriate for other projects or use-cases

- Detailed operational/engineering level simulation

- Long-term planning



## Installation



Install `open-gira` by cloning the repository:

```bash

git clone https://github.com/nismod/open-gira.git

```



The repository comes with a `environment.yml` file describing the `conda` and

`PyPI` packages required to run `open-gira`. The `open-gira` developers

recommend using either [micromamba](https://mamba.readthedocs.io/en/latest/user_guide/micromamba.html#micromamba)

or [mamba](https://mamba.readthedocs.io/en/latest/index.html) to install and

manage these `conda` packages.



Having installed one of the suggested package managers, to create the

`open-gira` conda environment:

```bash

micromamba create -f environment.yml -y

```



And to activate the environment:

```bash

micromamba activate open-gira

```



You are now ready to request result files, triggering analysis jobs in the

process.



Note that all subsequent commands given in the documentation assume that the

`open-gira` environment is already activated.



`exactextract` is used for zonal statistics in the tropical cyclones /

electricity grid analysis. It is not available via the `conda` package

management ecosystem and so must be installed separately. Please see

installation instructions [here](https://github.com/isciences/exactextract).



## Tests



Workflow steps are tested using small sample datasets.



To run the tests:

```bash

python -m pytest tests

```



## Usage



`open-gira` is comprised of a set of `snakemake` rules which call scripts and

library code to request data, process it and produce results.



The key idea of `snakemake` is similar to `make` in that the workflow is

determined from the end (the files users want) to the beginning (the files

users have, if any) by applying general rules with pattern matching on file and

folder names.



A example invocation looks like:

```bash

snakemake --cores 2 -- results/wales-latest_filter-road/edges.geoparquet

```



Here, we ask `snakemake` to use up to 2 CPUs to produce a target file, in this

case, the edges of the Welsh road network. `snakemake` pattern matches

`wales-latest` as the OSM dataset name and `filter-road` as the network type we

want to filter for.



To check what work we're going to request before commencing, use the `-n` flag:

```bash

snakemake -n --cores 2 -- results/wales-latest_filter-road/edges.geoparquet

```



This will explain which rules will be required to run to produce the target

file. It may be helpful to [visualise](https://snakemake.readthedocs.io/en/stable/executing/cli.html#visualization)

which rules are expected to run, too.



The workflow configuration details are in `config/config.yml`. You can edit

this to set the target OSM infrastructure datasets, number of spatial slices, and

hazard datasets.



See the [documentation](https://nismod.github.io/open-gira/)

and [config/README.md](https://github.com/nismod/open-gira/blob/main/config/README.md)

for more details on usage in general and on configuration.



## Documentation



Documentation is written using the [`mdbook`](https://rust-lang.github.io/mdBook/index.html)

format, using markdown files in the `./docs` directory.



Follow the [installation instructions](https://rust-lang.github.io/mdBook/guide/installation.html)

to get the `mdbook` command-line tool.



To build the docs locally:



```bash

cd docs

mdbook build

open book/index.html

```



Or run `mdbook serve` to run a server and rebuild the docs as you make changes.



## Related projects



Two libraries have been developed in tandem with `open-gira` and provide some

key functionality.



### snail



The open-source Python library [snail](https://github.com/nismod/snail)

is used for vector-raster intersection, e.g. identifying which road segments

might be affected by a set of flood map hazard rasters.



### snkit



The [snkit](https://github.com/tomalrussell/snkit) library is used for

network cleaning and assembly.



## Acknowledgments



This research received funding from the FCDO Climate Compatible Growth

Programme. The views expressed here do not necessarily reflect the UK

government's official policies.



This research has also been supported by funding from the World Bank

Group.