{"id":32599783,"url":"https://github.com/fbartos/publicationbiasbenchmark","last_synced_at":"2025-10-30T06:52:18.033Z","repository":{"id":320351696,"uuid":"1024372290","full_name":"FBartos/PublicationBiasBenchmark","owner":"FBartos","description":null,"archived":false,"fork":false,"pushed_at":"2025-10-23T08:54:53.000Z","size":511506,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"master","last_synced_at":"2025-10-23T10:31:55.491Z","etag":null,"topics":["benchmark","meta-analysis","publication-bias","simulation"],"latest_commit_sha":null,"homepage":"https://fbartos.github.io/PublicationBiasBenchmark/","language":"R","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/FBartos.png","metadata":{"files":{"readme":"README.Rmd","changelog":"NEWS.md","contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-07-22T15:39:52.000Z","updated_at":"2025-10-23T08:54:57.000Z","dependencies_parsed_at":"2025-10-23T10:32:16.295Z","dependency_job_id":null,"html_url":"https://github.com/FBartos/PublicationBiasBenchmark","commit_stats":null,"previous_names":["fbartos/publicationbiasbenchmark"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/FBartos/PublicationBiasBenchmark","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/FBartos%2FPublicationBiasBenchmark","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/FBartos%2FPublicationBiasBenchmark/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/FBartos%2FPublicationBiasBenchmark/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/FBartos%2FPublicationBiasBenchmark/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/FBartos","download_url":"https://codeload.github.com/FBartos/PublicationBiasBenchmark/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/FBartos%2FPublicationBiasBenchmark/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":281761850,"owners_count":26557114,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-30T02:00:06.501Z","response_time":61,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["benchmark","meta-analysis","publication-bias","simulation"],"created_at":"2025-10-30T06:51:51.223Z","updated_at":"2025-10-30T06:52:18.020Z","avatar_url":"https://github.com/FBartos.png","language":"R","funding_links":[],"categories":[],"sub_categories":[],"readme":"---\ntitle:        \"README\"\nbibliography: inst/REFERENCES.bib\ncsl:          inst/apa.csl\noutput:       github_document\n---\n\n\u003c!-- README.md is generated from README.Rmd. Please edit that file --\u003e\n\n```{r, include = FALSE}\nknitr::opts_chunk$set(\n  collapse = TRUE,\n  comment = \"#\u003e\",\n  fig.path = \"man/figures/README-\",\n  out.width = \"100%\"\n)\n```\n\n# PublicationBiasBenchmark\n\n**PublicationBiasBenchmark** is an R package for benchmarking publication bias correction methods through simulation studies. It provides:  \n- Predefined data-generating mechanisms from the literature  \n- Functions for running meta-analytic methods on simulated data  \n- Pre-simulated datasets and pre-computed results for reproducible benchmarks  \n- Tools for visualizing and comparing method performance\n\nAll datasets and results are hosted on OSF: \u003chttps://doi.org/10.17605/OSF.IO/EXF3M\u003e\n\nFor the methodology of living synthetic benchmarks please cite:\n\n\u003e Bartoš, F., Pawel, S., \u0026 Siepe, B. S. (2025). Living synthetic benchmarks: A neutral and cumulative framework for simulation studies. _arXiv Preprint_.\n  \u003chttps://doi.org/10.48550/arXiv.2510.19489\u003e\n\nFor the publication bias benchmark R package please cite:\n\n\u003e Bartoš, F., Pawel, S., \u0026 Siepe, B. S. (2025). PublicationBiasBenchmark: Benchmark for publication bias correction methods (version 0.1.0). \u003chttps://github.com/FBartos/PublicationBiasBenchmark\u003e\n\nOverviews of the benchmark results are available as articles on the package website:\n\n - [Overall Results](https://fbartos.github.io/PublicationBiasBenchmark/articles/Results.html)\n - [Stanley (2017)](https://fbartos.github.io/PublicationBiasBenchmark/articles/Results_Stanley2017.html)\n - [Alinaghi (2018)](https://fbartos.github.io/PublicationBiasBenchmark/articles/Results_Alinaghi2018.html)\n - [Bom (2019)](https://fbartos.github.io/PublicationBiasBenchmark/articles/Results_Bom2019.html)\n - [Carter (2019)](https://fbartos.github.io/PublicationBiasBenchmark/articles/Results_Carter2019.html)\n \nContributor guidelines for extending the package with data-generating mechanisms, methods, and results are available at:\n\n - [How to add a new data-generating mechanism](https://fbartos.github.io/PublicationBiasBenchmark/articles/Adding_New_DGMs.html)\n - [How to add a new method](https://fbartos.github.io/PublicationBiasBenchmark/articles/Adding_New_Methods.html)\n - [How to compute method results](https://fbartos.github.io/PublicationBiasBenchmark/articles/Computing_Method_Results.html)\n - [How to compute method measures](https://fbartos.github.io/PublicationBiasBenchmark/articles/Computing_Method_Measures.html)\n \nIllustrations of how to use the precomputed datasets, results, and measures are available at:\n\n - [How to use presimulated datasets](https://fbartos.github.io/PublicationBiasBenchmark/articles/Using_Presimulated_Datasets.html)\n - [How to use precomputed results](https://fbartos.github.io/PublicationBiasBenchmark/articles/Using_Precomputed_Results.html)\n - [How to use precomputed measures](https://fbartos.github.io/PublicationBiasBenchmark/articles/Using_Precomputed_Measures.html)\n\nThe rest of this file overviews the main features of the package.\n\n## Installation\n\n```{r, eval = FALSE}\n# Install from GitHub\nremotes::install_github(\"FBartos/PublicationBiasBenchmark\")\n```\n\n## Usage\n\n```{r message=FALSE, warning=FALSE}\nlibrary(PublicationBiasBenchmark)\n```\n\n### Simulating From Existing Data-Generating Mechanisms\n\n```r\n# Obtain a data.frame with pre-defined conditions\ndgm_conditions(\"Stanley2017\")\n\n# simulate the data from the second condition\ndf \u003c- simulate_dgm(\"Stanley2017\", 2)\n\n# fit a method\nrun_method(\"RMA\", df)\n```\n\n\n### Using Pre-Simulated Datasets\n\n```r \n# download the pre-simulated datasets\n# the default settings downloads the datasets to the `resources` directory, use\n# PublicationBiasBenchmark.options(simulation_directory = \"/path/\")\n# to change the settings\ndownload_dgm_datasets(\"no_bias\")\n\n# retrieve first repetition of first condition from the downloaded datasets\nretrieve_dgm_dataset(\"no_bias\", condition_id = 1, repetition_id = 1)\n```\n\n### Using Pre-Computed Results\n\n```r \n# download the pre-computed results\ndownload_dgm_results(\"no_bias\")\n\n# retrieve results the first repetition of first condition of RMA from the downloaded results\nretrieve_dgm_results(\"no_bias\", method = \"RMA\", condition_id = 1, repetition_id = 1)\n\n# retrieve all results across all conditions and repetitions\nretrieve_dgm_results(\"no_bias\")\n\n```\n\n### Using Pre-Computed Measures\n\n```r \n# download the pre-computed measures\ndownload_dgm_measures(\"no_bias\")\n\n# retrieve measures of bias the first condition of RMA from the downloaded results\nretrieve_dgm_measures(\"no_bias\", measure = \"bias\", method = \"RMA\", condition_id = 1)\n\n# retrieve all measures across all conditions and measures\nretrieve_dgm_measures(\"no_bias\")\n```\n\n### Simulating From an Existing DGM With Custom Settings\n\n```r\n# define sim setting\nsim_settings \u003c- list(\n  n_studies     = 100,\n  mean_effect   = 0.3,\n  heterogeneity = 0.1\n)\n\n# check whether it is feasible\n# (defined outside of the function - not to decrease performance during simulation)\nvalidate_dgm_setting(\"no_bias\", sim_settings)\n\n# simulate the data\ndf \u003c- simulate_dgm(\"no_bias\", sim_settings)\n\n# fit a method\nrun_method(\"RMA\", df)\n\n```\n\n### Key Functions\n\n#### Data-Generating Mechanisms\n\n- `simulate_dgm()`: Generates simulated data according to specified data-generating mechanism and settings.\n- `dgm_conditions()`: Lists prespecified conditions of the data-generating mechanism.\n- `validate_dgm_setting()`: Validates (custom) setting of the data-generating mechanism.\n- `download_dgm_datasets()`: Downloads pre-simulated datasets from the OSF repository.\n- `retrieve_dgm_dataset()`: Retrieves the pre-simulated dataset of a given condition and repetition from downloaded from the pre-downloaded OSF repository.\n\n#### Method Estimation And Results\n\n- `run_method()`: Estimates method on a supplied data according to the specified settings.\n- `method_settings()`: Lists prespecified settings of the method.\n- `download_dgm_results()`: Downloads pre-computed results from the OSF repository.\n- `retrieve_dgm_results()`: Retrieves the pre-computed results of a given method, condition, and repetition from the pre-downloaded OSF repository.\n\n#### Performance measures And Results\n\n- `bias()`, `bias_mcse()`, etc.: Functions to compute performance measures and their Monte Carlo standard errors.\n- `download_dgm_measures()`: Downloads pre-computed performance measures from the OSF repository.\n- `retrieve_dgm_measures()`: Retrieves the pre-computed performance measures of a given method, condition, and repetition from the pre-downloaded OSF repository.\n\n### Available Data-Generating Mechanisms\n\nSee `methods(\"dgm\")` for the full list:\n\n- `\"no_bias\"`: Generates data without publication bias (a test simulation)\n- `\"Stanley2017\"`: @stanley2017finding\n- `\"Alinaghi2018\"`: @alinaghi2018meta\n- `\"Bom2019\"`: @bom2019kinked\n- `\"Carter2019\"`: @carter2019correcting\n\n### Available Methods\n\nSee `methods(\"method\")` for the full list:\n\n- `\"mean\"`: Mean effects size\n- `\"FMA\"`: Fixed effects meta-analysis\n- `\"RMA\"`: Random effects meta-analysis\n- `\"WLS\"`: Weighted Least Squares\n- `\"trimfill\"`: Trim-and-Fill [@duval2000trim]\n- `\"WAAPWLS\"`: Weighted Least Squares - Weighted Average of Adequately Power Studies [@stanley2017finding]\n- `\"WILS\"`: Weighted and Iterated Least Squares [@stanley2024harnessing]\n- `\"PET\"`: Precision-Effect Test (PET) publication bias adjustment [@stanley2014meta]\n- `\"PEESE\"`: Precision-Effect Estimate with Standard Errors (PEESE) publication bias adjustment [@stanley2014meta]\n- `\"PETPEESE\"`: Precision-Effect Test and Precision-Effect Estimate with Standard Errors (PET-PEESE) publication bias adjustment [@stanley2014meta]\n- `\"EK\"`: Endogenous Kink [@bom2019kinked]\n- `\"SM\"`: Selection Models (3PSM, 4PSM) [@vevea1995general] \n- `\"pcurve\"`: P-curve [@simonsohn2014pcurve]\n- `\"puniform\"`: P-uniform [@vanassen2015meta] and P-uniform* [@vanaert2025puniform]\n- `\"AK\"`: Andrews \u0026 Kasy selection models (AK1, AK2) [@andrews2019identification]\n- `\"RoBMA\"`: Robust Bayesian Meta-Analysis [@bartos2023robust]\n\n\n### Available Performance Measures\n\nSee `?measures` for the full list of performance measures and their Monte Carlo standard errors/\n\n### DGM OSF Repositories\n\nAll DGMs are linked to the OSF repository (\u003chttps://osf.io/exf3m/\u003e) and contain the following elements:\n\n- `data` : folder containing by-condition simulated datasets for all repetitions\n- `results` : folder containing by-method results for all conditions * repetitions\n- `measures` : folder containing by-measure performance for all methods * conditions \n- `metadata` : folder containing the following information:\n  - `dgm-conditions.csv` : file mapping of all conditions and the corresponding settings\n  - `dgm-generation.R` : file with code for exact reproduction of the pre-simulated datasets\n  - `dgm-sessionInfo.txt`: file with reproducibility details for the pre-simulated datasets\n  - `dgm-session.log`: file with reproducibility details for the pre-simulated datasets (based on sessioninfo package)\n  - `results.R` : file with code for exact reproduction of the by method results (might be method / method groups specific)\n  - `results-sessionInfo.txt`: file with reproducibility details for the precomputed results (might be method / method groups specific)\n  - `pm-computation.R` : file with code for computation of performance measures \n  \n### References\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffbartos%2Fpublicationbiasbenchmark","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffbartos%2Fpublicationbiasbenchmark","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffbartos%2Fpublicationbiasbenchmark/lists"}