https://github.com/sisl/pomdpstresstesting.jl
Adaptive stress testing of black-box systems within POMDPs.jl
https://github.com/sisl/pomdpstresstesting.jl
black-box pomdps simulation solvers stress-test
Last synced: about 1 month ago
JSON representation
Adaptive stress testing of black-box systems within POMDPs.jl
- Host: GitHub
- URL: https://github.com/sisl/pomdpstresstesting.jl
- Owner: sisl
- License: other
- Created: 2020-02-11T20:48:05.000Z (over 5 years ago)
- Default Branch: master
- Last Pushed: 2024-02-06T17:20:06.000Z (over 1 year ago)
- Last Synced: 2025-09-11T03:42:37.821Z (about 1 month ago)
- Topics: black-box, pomdps, simulation, solvers, stress-test
- Language: Julia
- Homepage:
- Size: 2.91 MB
- Stars: 16
- Watchers: 4
- Forks: 6
- Open Issues: 7
-
Metadata Files:
- Readme: README.md
- License: LICENSE.md
Awesome Lists containing this project
README
# POMDPStressTesting.jl
[](https://sisl.github.io/POMDPStressTesting.jl/dev/)
[](https://github.com/sisl/POMDPStressTesting.jl/actions/workflows/CI.yml)
[](https://codecov.io/gh/sisl/POMDPStressTesting.jl)
Adaptive stress testing of black-box systems, implemented within the [POMDPs.jl](https://github.com/JuliaPOMDP/POMDPs.jl) ecosystem.
See the [documentation](https://sisl.github.io/POMDPStressTesting.jl/dev/) for more details.
# Citation
If you use this package for research purposes, please cite the following:
[](https://joss.theoj.org/papers/04dc39ea89e90938727d789a2e402b0b)
```
@article{moss2021pomdpstresstesting,
title = {{POMDPStressTesting.jl}: Adaptive Stress Testing for Black-Box Systems},
author = {Robert J. Moss},
journal = {Journal of Open Source Software},
year = {2021},
volume = {6},
number = {60},
pages = {2749},
doi = {10.21105/joss.02749}
}
```
# Interface
To stress test a new system, the user has to define the `GrayBox` and `BlackBox` interface outlined in [`src/GrayBox.jl`](https://github.com/mossr/POMDPStressTesting.jl/blob/master/src/GrayBox.jl) and [`src/BlackBox.jl`](https://github.com/mossr/POMDPStressTesting.jl/blob/master/src/BlackBox.jl).
### GrayBox Interface
The `GrayBox` simulator and environment interface includes:
* `GrayBox.Simulation` type to hold simulation variables
* `GrayBox.environment(sim::Simulation)` to return the collection of environment distributions
* `GrayBox.transition!(sim::Simulation)` to transition the simulator, returning the log-likelihood
### BlackBox Interface
The `BlackBox` system interface includes:
* `BlackBox.initialize!(sim::Simulation)` to initialize/reset the system under test
* `BlackBox.evaluate!(sim::Simulation)` to evaluate/execute the system under test
* `BlackBox.distance(sim::Simulation)` to return how close we are to an event
* `BlackBox.isevent(sim::Simulation)` to indicate if a failure event occurred
* `BlackBox.isterminal(sim::Simulation)` to indicate the simulation is in a terminal state
Functions ending with `!` may modify the `Simulation` object in place.
# Solvers
Several solvers are implemented.
#### Reinforcement learning solvers
* [`MCTSPWSolver`](https://github.com/mossr/POMDPStressTesting.jl/blob/master/src/solvers/mcts.jl)
#### Deep reinforcement learning solvers1
* [`TRPOSolver`](https://github.com/mossr/POMDPStressTesting.jl/blob/master/src/solvers/drl/trpo.jl)
* [`PPOSolver`](https://github.com/mossr/POMDPStressTesting.jl/blob/master/src/solvers/drl/ppo.jl)
#### Stochastic optimization solvers
* [`CEMSolver`](https://github.com/mossr/POMDPStressTesting.jl/blob/master/src/solvers/cem.jl)
#### Baseline solvers
* [`RandomSearchSolver`](https://github.com/mossr/POMDPStressTesting.jl/blob/master/src/solvers/random_search.jl)
# Example
[](https://nbviewer.jupyter.org/github/sisl/POMDPStressTesting.jl/blob/master/notebooks/Walk1D.ipynb)
An example implementation of the AST interface is provided for the Walk1D problem:
* **Julia source**: [`test/Walk1D.jl`](https://github.com/mossr/POMDPStressTesting.jl/blob/master/test/Walk1D.jl)
* **Jupyter notebook**: [`Walk1D.ipynb`](https://nbviewer.jupyter.org/github/sisl/POMDPStressTesting.jl/blob/master/notebooks/Walk1D.ipynb)
* **Descriptive tutorial-style write-up**: [`walk1d.pdf`](./test/pdf/walk1d.pdf) (created using [TeX.jl](https://github.com/mossr/TeX.jl))
# Installation
Install the `POMDPStressTesting.jl` package via:
```julia
] add POMDPStressTesting
```
### Testing
To run the test suite, you can use the Julia package manager.
```julia
] test POMDPStressTesting
```
# Contributing
We welcome contributions! Please fork the repository and submit a new Pull Request.
---
Package maintained by Robert Moss: mossr@cs.stanford.edu
1 TRPO and PPO thanks to [Shreyas Kowshik's](https://github.com/shreyas-kowshik/RL-baselines.jl) initial implementation.