https://github.com/fbartos/zcurve
zcurve R package for assessing the reliability and trustworthiness of published literature with the z-curve method
https://github.com/fbartos/zcurve
edr err replicability z-cruve
Last synced: 2 months ago
JSON representation
zcurve R package for assessing the reliability and trustworthiness of published literature with the z-curve method
- Host: GitHub
- URL: https://github.com/fbartos/zcurve
- Owner: FBartos
- Created: 2019-11-06T18:19:57.000Z (about 6 years ago)
- Default Branch: master
- Last Pushed: 2025-05-19T09:48:29.000Z (8 months ago)
- Last Synced: 2025-06-09T15:11:22.709Z (7 months ago)
- Topics: edr, err, replicability, z-cruve
- Language: R
- Homepage: https://fbartos.github.io/zcurve
- Size: 2.44 MB
- Stars: 13
- Watchers: 3
- Forks: 1
- Open Issues: 0
-
Metadata Files:
- Readme: README.Rmd
Awesome Lists containing this project
README
---
output: github_document
---
```{r, include = FALSE}
knitr::opts_chunk$set(
collapse = TRUE,
comment = "#>",
fig.path = "man/figures/README-",
out.width = "100%"
)
```
# zcurve
[](https://github.com/FBartos/zcurve/actions)
[](https://github.com/FBartos/zcurve/actions)
[](https://CRAN.R-project.org/package=zcurve)
This package implements z-curves - methods for estimating expected discovery
and replicability rates on bases of test-statistics of published studies. The package
provides functions for fitting the censored EM version (Schimmack & Bartoš, 2023),
the EM version (Bartoš & Schimmack, 2022) as well as the original density z-curve (Brunner & Schimmack, 2020).
Furthermore, the package provides summarizing and plotting functions for the
fitted z-curve objects. See the aforementioned articles for more information about
z-curve, expected discovery rate (EDR), expected replicability rate (ERR),
maximum false discovery rate (FDR), as well as validation studies, and limitations.
## Installation
You can install the current version of zcurve from CRAN with:
``` r
install.packages("zcurve")
```
or the development version from [GitHub](https://github.com/) with:
``` r
# install.packages("devtools")
devtools::install_github("FBartos/zcurve")
```
## Example
Z-curve can be used to estimate expected replicability rate (ERR), expected discovery rate (EDR), and
maximum false discovery rate (Soric FDR) using z-scores from a set of significant findings.
This is a reproduction of an example in Bartoš and Schimmack
(2022) where the z-curve is used to estimate ERR and EDR on a subset of studies used in
reproducibility project (OSC, 2015). Only studies with non-ambiguous original outcomes are used
- excluding studies with "marginally significant" original findings, leading to 90 studies. Out of these
90 studies, 35 were successfully replicated.
We included the recoded z-scores from the 90 OSC studies as a dataset in the package ('OSC.z'). The
expectation-maximization (EM) version of the z-curve is implemented as the default method and can be
fitted (with 1000 bootstraps) and summarized using 'zcurve and 'summary' functions.
The first argument to the function call is a vector of z-scores. Alternatively, a vector of two-sided
p-values can be also used, by specifying "zcurve(p = p.values)".
```{r}
set.seed(666)
library(zcurve)
fit <- zcurve(OSC.z)
summary(fit)
```
More details from the fitted object can be extracted from the fitted object. For more statistics, as
expected number of conducted studies, the file drawer ratio or Sorić's FDR specify 'all = TRUE' (see Schimmack & Bartoš, 2023) .
```{r}
summary(fit, all = TRUE)
```
For more information regarding the fitted model weights add 'type = "parameters"'.
```{r}
summary(fit, type = "parameters")
```
The package also provides a convenient plotting method for the z-curve fits.
```{r}
plot(fit)
```
The default plot can be further modified by using classic R plotting arguments as 'xlab', 'ylab',
'main', 'cex.axis', 'cex.lab'. Furthermore, an annotation with the main test statistics can be
added to the plot by specifying 'annotation = TRUE' and the pointwise confidence intervals of the
plot by specifying "CI = TRUE". For more options regarding the annotation see '?plot.zcurve".
```{r}
plot(fit, CI = TRUE, annotation = TRUE, main = "OSC 2015")
```
Other versions of the z-curves may be fitted by changing the method argument in the 'zcurve' function.
Set 'method = "density"' to fit the new version of z-curve using density method (KD2). The original
version of the density method as implemented in Brunner and Schimmack (2020) can be fitted by adding
'list(model = "KD1")' to the 'control' argument of 'zcurve'.
(We omit bootstrapping to speed the fitting process in this case)
```{r}
fit.KD2 <- zcurve(OSC.z, method = "density", bootstrap = FALSE)
fit.KD1 <- zcurve(OSC.z, method = "density", control = list(model = "KD1"), bootstrap = FALSE)
summary(fit.KD2)
summary(fit.KD1)
```
The 'control' argument can be used to change the number of iterations or reducing the convergence
criterion in cases of non-convergence. It can be also used for constructing custom z-curves by
changing the location of the mean components, their number or many other settings. However, it is
important to bear in mind that those custom models need to be validated first on simulation studies
prior to their usage. For more information about the control settings see '?control_EM', '?control_density',
and '?control_density_v1'.
If you encounter any problems or bugs, please, contact me at f.bartos96[at]gmail.com or submit an issue at
https://github.com/FBartos/zcurve/issues. If you like the package and use it in your work, please, cite it
as:
```{r}
citation(package = "zcurve")
```
## Sources
Schimmack, U., & Bartoš, F. (2023). Estimating the false discovery risk of (randomized) clinical trials in medical journals based on published p-values. _PLoS ONE, 18_(8), e0290084. https://doi.org/10.1371/journal.pone.0290084
Bartoš, F., & Schimmack, U. (2022). Z-curve 2.0: Estimating replication rates and discovery rates. _Meta-Psychology_, 6. https://doi.org/10.15626/MP.2021.2720
Brunner, J., & Schimmack, U. (2020). Estimating population mean power under conditions of heterogeneity and selection for significance. _Meta-Psychology_, 4. https://doi.org/10.15626/MP.2018.874
Open Science Collaboration. (2015). Estimating the reproducibility of psychological science. _Science, 349_(6251), aac4716. https://doi.org/10.1126/science.aac4716