https://github.com/oliverhennhoefer/online-fdr
Online Multiple Hypothesis Testing.
https://github.com/oliverhennhoefer/online-fdr
addis alpha-investing alpha-spending anomaly-detection batching benjamini-hochberg bonferroni false-discovery-rate false-positive false-positive-control fdr fwer hypothesis-testing interim-analysis lond lord online-hypothesis-testing p-value saffron uncertainty-quantification
Last synced: about 2 months ago
JSON representation
Online Multiple Hypothesis Testing.
- Host: GitHub
- URL: https://github.com/oliverhennhoefer/online-fdr
- Owner: OliverHennhoefer
- License: bsd-3-clause
- Created: 2024-07-22T07:03:40.000Z (10 months ago)
- Default Branch: main
- Last Pushed: 2024-12-13T16:04:13.000Z (5 months ago)
- Last Synced: 2025-01-16T04:16:58.867Z (4 months ago)
- Topics: addis, alpha-investing, alpha-spending, anomaly-detection, batching, benjamini-hochberg, bonferroni, false-discovery-rate, false-positive, false-positive-control, fdr, fwer, hypothesis-testing, interim-analysis, lond, lord, online-hypothesis-testing, p-value, saffron, uncertainty-quantification
- Language: Python
- Homepage: https://projecteuclid.org/journals/statistical-science/volume-38/issue-4/Online-Multiple-Hypothesis-Testing/10.1214/23-STS901.full
- Size: 131 KB
- Stars: 1
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Online Multiple Hypothesis Testing
[](https://www.python.org)
[](https://github.com/OliverHennhoefer/online-fdr/issues)
[](https://opensource.org/licenses/BSD-3-Clause)
[](https://github.com/psf/black)
[](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7615519/)
The vast majority of implementations of online method for FDR control are either part of an experimental setup, that does
not straight-forwardly generalize towards applications outside this setup, or are geared towards tests for which all test
results are already available (i.e. they do not have an actual _online_ API).
For that reason, this repository implements a wide range of methods for FDR/FWER control for _actual_ online multiple
hypothesis testing with an intuitive `test_one()` method:
- **Alpha Spending** (Bonferroni, ...)
- [**Online Fallback**](https://journals.sagepub.com/doi/abs/10.1177/0962280220983381)
- **[[Generalized] Alpha Investing](https://www.jstor.org/stable/24774568)** ([Foster/Stine](http://deanfoster.net/research/edc.pdf), ...)
- [**LOND**](https://proceedings.neurips.cc/paper/2017/file/7f018eb7b301a66658931cb8a93fd6e8-Paper.pdf) (Original, Modified, Dependent)
- [**LORD**](https://projecteuclid.org/journals/annals-of-statistics/volume-46/issue-2/Online-rules-for-control-of-false-discovery-rate-and-false/10.1214/17-AOS1559.full) (LORD3, LOND++, D-LORD, Dependent, [DecayLORD](https://papers.nips.cc/paper_files/paper/2021/file/def130d0b67eb38b7a8f4e7121ed432c-Paper.pdf))
- [**SAFFRON**](https://proceedings.mlr.press/v80/ramdas18a/ramdas18a.pdf) (Standard, [DecaySAFFRON](https://papers.nips.cc/paper_files/paper/2021/file/def130d0b67eb38b7a8f4e7121ed432c-Paper.pdf))
- [**ADDIS**](https://proceedings.neurips.cc/paper_files/paper/2019/file/1d6408264d31d453d556c60fe7d0459e-Paper.pdf) (Standard, [DecayADDIS](https://papers.nips.cc/paper_files/paper/2021/file/def130d0b67eb38b7a8f4e7121ed432c-Paper.pdf))
- [**Batch-BH**](https://proceedings.mlr.press/v108/zrnic20a/zrnic20a.pdf) and [**Batch-StBH**](https://proceedings.mlr.press/v108/zrnic20a/zrnic20a.pdf)
Instantiate an online testing procedure (e.g. `Addis()`) and simply test _p_-values sequentially with `.test_one()`:
```python
from online_fdr.investing.addis.addis import Addis
from online_fdr.utils.generation import DataGenerator, StandardGaussianProcess
N = 100
generator = DataGenerator(n=N, contamination=0.1, dgp=StandardGaussianProcess())
addis = Addis(alpha=0.05, wealth=0.025, lambda_=0.25, tau=0.5) # procedure
for i in range(0, N):
p_value, label = generator.sample_one()
result = addis.test_one(p_value) # sequential testing
```
_This work is inspired by the R package '[onlineFDR](https://dsrobertson.github.io/onlineFDR/)'.
This package, and most of its methods, are largely validated by the [implementations](https://github.com/dsrobertson/onlineFDR) of said package.
**Key differentiator is the design choice in regard to method calls for sequential testing, as this implementation allows for truly temporal applications** ('onlineFDR' requires a [static] data.frame for testing)._
# Getting started
The code does not require external dependencies. It's recommended to use with Python 3.12, although previous versions (at least until 3.9) should generally be compatible.