https://github.com/epiforecasts/simplified-forecaster-evaluation
Evaluation of a simplified forecasting model designed to emulate the performance of the ECDC forecasting hub.
https://github.com/epiforecasts/simplified-forecaster-evaluation
covid-19 ensemble evaluation forecast-hub forecasting github-actions
Last synced: 3 months ago
JSON representation
Evaluation of a simplified forecasting model designed to emulate the performance of the ECDC forecasting hub.
- Host: GitHub
- URL: https://github.com/epiforecasts/simplified-forecaster-evaluation
- Owner: epiforecasts
- License: mit
- Created: 2022-05-17T15:03:49.000Z (about 4 years ago)
- Default Branch: main
- Last Pushed: 2023-01-09T10:18:24.000Z (over 3 years ago)
- Last Synced: 2025-11-09T09:12:16.523Z (7 months ago)
- Topics: covid-19, ensemble, evaluation, forecast-hub, forecasting, github-actions
- Language: TeX
- Homepage: https://epiforecasts.io/simplified-forecaster-evaluation/paper.pdf
- Size: 68.7 MB
- Stars: 5
- Watchers: 5
- Forks: 0
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- License: LICENSE.md
Awesome Lists containing this project
README
# Evaluating an epidemiologically motivated surrogate model of a multi-model ensemble
This repository contains the documentation, results, and code of a project evaluating a simplified forecasting model in comparison the European forecasting hub ensemble. See the documentation for further details.
## Citation
Please cite this using the following:
> Abbott, Sherratt, Bosse, Gruson, Bracher, and Funk. 2022 “Evaluating an Epidemiologically Motivated Surrogate Model of a Multi-Model Ensemble.” medRxiv. https://doi.org/10.1101/2022.10.12.22280917.
```
@UNPUBLISHED{Abbott_undated-tu,
title = "Evaluating an epidemiologically motivated surrogate model of a
multi-model ensemble",
author = "{Abbott} and {Sherratt} and {Bosse} and {Gruson} and {Bracher} and
{Funk}",
journal = "medRxiv",
year = 2022,
doi = "10.1101/2022.10.12.22280917"
}
```
## Project structure
Folder | Purpose
---|---
[`ecdc-weekly-growth-forecasts`](ecdc-weekly-growth-forecasts/) | The code for the simplified forecasting model evaluated in this work.
[`data-raw`](data-raw/) | Raw input data and scripts required to download and process it.
[`data`](data/) | Processed data from `data-raw` ready to be used in the paper analysis.
[`R`](R/) | R functions used in the analysis and for evaluation.
[`paper`](paper/) | Summary paper and additional supplementary information as `Rmarkdown` documents.
[`.github`](.github/) | GitHub actions used to build the docker image and render and publish the analysis paper.
[`.devcontainer`](.devcontainer/) | Resources for reproducibility using `vscode` and `docker`.
## Dependencies
Dependencies are managed using [`renv`](https://rstudio.github.io/renv/).
Alternatively a docker [container](https://github.com/epiforecasts/simplfied-forecaster-evaluation/blob/main/.devcontainer/Dockerfile) and [image](https://github.com/epiforecasts/simplfied-forecaster-evaluation/pkgs/container/simplfied-forecaster-evaluation) is provided. An easy way to make use of this is using the Remote development extension of `vscode`.
## Reproducibility
Once all dependencies are installed (see above) the paper analysis can be rerun using `paper/paper.Rmd` either interactively or rerendered as a document using `Rmarkdown`. To make this step easier we also provide a GitHub action to publish an updated version of the analysis to the `gh-pages` branch.
See `data-raw` for the code to re-extract forecasts and truth data, create metadata, normalise by population, and score forecasts against truth data. All steps of this process can be done automatically using `data-raw/update.sh`. Results from these steps will be stored in `data` as `.csv` files.