Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/mitrefireline/simharness

An open-source Reinforcement Learning (RL) harness written in Python to work with SimFire for training agents to fight wildfires on real data.
https://github.com/mitrefireline/simharness

python reinforcement-learning simulation wildfire

Last synced: about 1 month ago
JSON representation

An open-source Reinforcement Learning (RL) harness written in Python to work with SimFire for training agents to fight wildfires on real data.

Host: GitHub
URL: https://github.com/mitrefireline/simharness
Owner: mitrefireline
License: apache-2.0
Created: 2023-12-08T19:13:31.000Z (about 1 year ago)
Default Branch: dev
Last Pushed: 2024-09-18T21:56:45.000Z (5 months ago)
Last Synced: 2024-09-19T02:51:42.550Z (5 months ago)
Topics: python, reinforcement-learning, simulation, wildfire
Language: Python
Homepage: https://fireline.mitre.org
Size: 10.9 MB
Stars: 13
Watchers: 2
Forks: 1
Open Issues: 1
Metadata Files:
- Readme: README.md
- License: LICENSE

Awesome Lists containing this project

README

# SimHarness - Reinforcement Learning for Wildfire Mitigation in Simulated Disaster Environments

SimHarness is a Python-based harness that wraps a [SimFire](https://github.com/mitrefireline/simfire/tree/main)
environment to generate effective wildfire mitigation strategy responses via reinforcement
learning (RL). Through an easy-to-use API, SimHarness can train one, or multiple,
intelligent agents within the SimFire Simulation with a variety of different RL algorithms
defined by [RLlib](https://docs.ray.io/en/latest/rllib/index.html).

Documentation is available [here](https://mitrefireline.github.io/simharness/).

Conceptual workflow for training an RL model using SimHarness within the SimFire environment.

**Table of Contents**

- [About SimHarness](#about-simharness)
- [Installation](#installation)
- [Getting Started](#getting-started)
- [License](#license)
- [Citation](#citation)

# About SimHarness

SimHarness is a Python repository designed to support the training of
[RLlib](https://docs.ray.io/en/latest/rllib/index.html) RL algorithms within simulated
disaster environments defined by
[SimFire](https://github.com/mitrefireline/simfire/tree/main). SimHarness takes as input
an instance of the
[SimFire Simulation class](https://github.com/mitrefireline/simfire/blob/39abc5a34b103a306c776a3c2972c10a87d0e652/simfire/sim/simulation.py#L37),
such as SimFire's
[FireSimulation](https://github.com/mitrefireline/simfire/blob/39abc5a34b103a306c776a3c2972c10a87d0e652/simfire/sim/simulation.py#L173),
as the training environment.
The [Simulation](https://github.com/mitrefireline/simfire/blob/39abc5a34b103a306c776a3c2972c10a87d0e652/simfire/sim/simulation.py#L37)
object provides an API that allows SimHarness to move agents around the simulated
environment and interact with it by placing mitigations. The
[FireSimulation](https://github.com/mitrefireline/simfire/blob/39abc5a34b103a306c776a3c2972c10a87d0e652/simfire/sim/simulation.py#L173)
agents represent firefighters moving through an environment as a wildfire spreads,
placing mitigations such as firelines to limit the spread of the fire within the area.

The SimHarness training loop functions similarly to a traditional RL training loop, except
it expects the passed-in environment to be a child class of `Simulation` as opposed to a
[gymnasium](https://gymnasium.farama.org) environment. `Simulation` is currently a class
within the SimFire package, but is expected to be moved to a separate,
non-disaster-specific package in the future. The simulated environment outputs training
signals such as observations and rewards to the SimHarness agent(s) which use the
observations to predict optimal actions. The actions produced by the model provide both
`movement` and `interaction` information. `Movements` are how the agent is traversing
across the environment, such as `[nothing, up, down, left, right]`. `Interactions` are how
the agent is changing the environment itself. In the case of SimFire, this can be
`[nothing, fireline, wetline, scratchline]`. These actions are relayed back to the
simulated environment, which then affects the overall disaster scenario simulated by the
environment.

# Installation

*Note: SimHarness has only been tested on Ubuntu 18.04 and Python 3.9.18

1. **Clone the repository.**

```bash
git clone https://github.com/mitrefireline/simharness.git
cd simharness
```

2. **Setup Pyenv Virtual Environment**

```bash
curl https://pyenv.run | bash
```

```bash
export PYENV_ROOT="$HOME/.pyenv"
command -v pyenv >/dev/null || export PATH="$PYENV_ROOT/bin:$PATH"
eval "$(pyenv init -)"
```
```bash
sudo apt update
sudo apt install \
build-essential \
curl \
libbz2-dev \
libffi-dev \
liblzma-dev \
libncursesw5-dev \
libreadline-dev \
libsqlite3-dev \
libssl-dev \
libxml2-dev \
libxmlsec1-dev \
llvm \
make \
tk-dev \
wget \
xz-utils \
zlib1g-dev
```

```bash
pyenv install 3.9.18
pyenv local 3.9.18
```

3. **Install poetry**

```bash
curl -sSL https://install.python-poetry.org | python3 -
```

*Note: Don't forget to add `poetry` to your path.*
```bash
export PATH="$HOME/.local/bin:$PATH"
```

```bash
poetry config virtualenvs.in-project true
poetry env use 3.9.18
```

4. **Install required packages**

```bash
poetry shell
poetry install --only main
```

*Note: To re-enter the environment after this step, run `poetry shell`.*

# Getting Started

SimHarness provides a number of tutorials within the `tutorials` section of the
documentation. Documentation is available [here](https://mitrefireline.github.io/simharness/).

# License

SimHarness is released under the [Apache 2.0 license](LICENSE).

# Citation

If you use SimHarness for your work, please cite our white paper with the following BibTex entry.

```BibTeX
@misc{tapley2023simharness,
title={Reinforcement Learning for Wildfire Mitigation in Simulated Disaster Environments},
author={Alexander Tapley and Marissa Dotter and Michael Doyle and Aidan Fennelly and Dhanuj Gandikota and Savanna Smith and Michael Threet and Tim Welsh},
year={2023},
booktitle={"Proc. of the Tackling Climate Change with Machine Learning Workshop"},
series={NeurIPS},
howpublished={\url{https://arxiv.org/abs/2311.15925}}
}
```

This project contains content developed by The MITRE Corporation. If this code is used in
a deployment or embedded within another project, it is requested that you send an email
to [email protected] in order to let us know where this software is being used.