{"id":27403025,"url":"https://github.com/s3alfisc/wildrwolf","last_synced_at":"2026-03-08T11:36:33.567Z","repository":{"id":88590916,"uuid":"376581327","full_name":"s3alfisc/wildrwolf","owner":"s3alfisc","description":"Romano-Wolf p-value adjustments for multiple hypotheses testing via the wild bootstrap for objects of type fixest and fixest_multi from the fixest package","archived":false,"fork":false,"pushed_at":"2024-01-14T11:28:25.000Z","size":4785,"stargazers_count":6,"open_issues_count":3,"forks_count":0,"subscribers_count":3,"default_branch":"main","last_synced_at":"2024-03-14T22:10:21.913Z","etag":null,"topics":["fixest","multiple-comparisons","r","romano-wolf","wild-bootstrap","wild-cluster-bootstrap"],"latest_commit_sha":null,"homepage":"https://s3alfisc.github.io/wildrwolf/","language":"R","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/s3alfisc.png","metadata":{"files":{"readme":"README.Rmd","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2021-06-13T15:46:54.000Z","updated_at":"2023-04-13T08:32:52.000Z","dependencies_parsed_at":"2023-03-07T00:45:25.430Z","dependency_job_id":null,"html_url":"https://github.com/s3alfisc/wildrwolf","commit_stats":null,"previous_names":[],"tags_count":3,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/s3alfisc%2Fwildrwolf","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/s3alfisc%2Fwildrwolf/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/s3alfisc%2Fwildrwolf/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/s3alfisc%2Fwildrwolf/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/s3alfisc","download_url":"https://codeload.github.com/s3alfisc/wildrwolf/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248824672,"owners_count":21167343,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["fixest","multiple-comparisons","r","romano-wolf","wild-bootstrap","wild-cluster-bootstrap"],"created_at":"2025-04-14T04:47:13.056Z","updated_at":"2026-03-08T11:36:28.514Z","avatar_url":"https://github.com/s3alfisc.png","language":"R","funding_links":[],"categories":[],"sub_categories":[],"readme":"---\noutput: github_document\n---\n\n\u003c!-- README.md is generated from README.Rmd. Please edit that file --\u003e\n\n```{r, include = FALSE}\nknitr::opts_chunk$set(\n  collapse = TRUE,\n  comment = \"#\u003e\",\n  fig.path = \"man/figures/README-\",\n  out.width = \"100%\"\n)\n```\n\n\n# wildrwolf `r emo::ji(\"wolf\")`\n\n\u003c!-- badges: start --\u003e\n[![R-CMD-check](https://github.com/s3alfisc/wildrwolf/workflows/R-CMD-check/badge.svg)](https://github.com/s3alfisc/wildrwolf/actions)\n[![](http://cranlogs.r-pkg.org/badges/last-month/wildrwolf)](https://cran.r-project.org/package=wildrwolf)\n[![Lifecycle: experimental](https://img.shields.io/badge/lifecycle-experimental-orange.svg)](https://lifecycle.r-lib.org/articles/stages.html)\n[![](https://www.r-pkg.org/badges/version/wildrwolf)](https://cran.r-project.org/package=wildrwolf)\n![runiverse-package](https://s3alfisc.r-universe.dev/badges/wildrwolf)\n[![Codecov test coverage](https://codecov.io/gh/s3alfisc/wildrwolf/branch/main/graph/badge.svg)](https://app.codecov.io/gh/s3alfisc/wildrwolf?branch=main)\n\u003c!-- badges: end --\u003e\n\nThe `wildrwolf` package implements Romano-Wolf multiple-hypothesis-adjusted p-values for objects of type `fixest` and `fixest_multi` from the `fixest` package via a wild (cluster) bootstrap. \n\nBecause the bootstrap-resampling is based on the [fwildclusterboot](https://github.com/s3alfisc/fwildclusterboot) package, `wildrwolf` is usually really fast. \n\nThe package is complementary to [wildwyoung](https://github.com/s3alfisc/wildwyoung) (still work in progress), which implements the multiple hypothesis adjustment method following Westfall and Young (1993).\n\nAdding support for multi-way clustering is work in progress.\n\n## Installation\n\nYou can install the package from CRAN and the development version from [GitHub](https://github.com/) with:\n\n``` r\ninstall.packages(\"wildrwolf\")\n\n# install.packages(\"devtools\")\ndevtools::install_github(\"s3alfisc/wildrwolf\")\n\n# from r-universe (windows \u0026 mac, compiled R \u003e 4.0 required)\ninstall.packages('wildrwolf', repos ='https://s3alfisc.r-universe.dev')\n```\n\n## Example I\n\n\u003c!-- As you can see in the example, there seems to be a bug in `rwolf()` for the pairs bootstrap. --\u003e\n\n```{r, warning=FALSE, message = FALSE}\nlibrary(wildrwolf)\nlibrary(fixest)\n\nset.seed(1412)\n\nN \u003c- 1000\nX1 \u003c- rnorm(N)\nX2 \u003c- rnorm(N)\nrho \u003c- 0.5\nsigma \u003c- matrix(rho, 4, 4); diag(sigma) \u003c- 1\nu \u003c- MASS::mvrnorm(n = N, mu = rep(0, 4), Sigma = sigma)\nY1 \u003c- 1 + 1 * X1 + X2 \nY2 \u003c- 1 + 0.01 * X1 + X2 \nY3 \u003c- 1 + 0.4 * X1 + X2\nY4 \u003c- 1 + -0.02 * X1 + X2 \nfor(x in 1:4){\n  var_char \u003c- paste0(\"Y\", x)\n  assign(var_char, get(var_char) + u[,x])\n}\n\ndata \u003c- data.frame(Y1 = Y1,\n                   Y2 = Y2,\n                   Y3 = Y3,\n                   Y4 = Y4,\n                   X1 = X1,\n                   X2 = X2,\n                   #group_id = group_id,\n                   splitvar = sample(1:2, N, TRUE))\n\nfit \u003c- feols(c(Y1, Y2, Y3, Y4) ~ csw(X1,X2),\n             data = data,\n             se = \"hetero\",\n             ssc = ssc(cluster.adj = TRUE))\n\n# clean workspace except for res \u0026 data\nrm(list= ls()[!(ls() %in% c('fit','data'))])\n\nres_rwolf1 \u003c- wildrwolf::rwolf(\n  models = fit,\n  param = \"X1\", \n  B = 9999\n)\n\npvals \u003c- lapply(fit, function(x) pvalue(x)[\"X1\"]) |\u003e unlist()\n\n# Romano-Wolf Corrected P-values\nres_rwolf1\n\n```\n\n## Example II\n\n```{r, warning = FALSE, message = FALSE}\nfit1 \u003c- feols(Y1 ~ X1 , data = data)\nfit2 \u003c- feols(Y1 ~ X1 + X2, data = data)\nfit3 \u003c- feols(Y2 ~ X1, data = data)\nfit4 \u003c- feols(Y2 ~ X1 + X2, data = data)\n\nres_rwolf2 \u003c- rwolf(\n  models = list(fit1, fit2, fit3, fit4), \n  param = \"X1\",  \n  B = 9999\n)\nres_rwolf2\n```\n\n## Performance\n\nThe above procedure with `S=8` hypotheses, `N=1000` observations and `k %in% (1,2)` parameters finishes in around 5 seconds.\n\n```{r, warning = FALSE, message = FALSE}\nif(requireNamespace(\"microbenchmark\")){\n  \n  microbenchmark::microbenchmark(\n    \"Romano-Wolf\" = wildrwolf::rwolf(\n      models = fit,\n      param = \"X1\", \n      B = 9999 \n    ), \n    times = 1\n  )\n \n}\n```\n\n\n## But does it work? Monte Carlo Experiments\n\nWe test $S=6$ hypotheses and generate data as \n\n$$Y_{i,s,g} = \\beta_{0} + \\beta_{1,s} D_{i} + u_{i,g} + \\epsilon_{i,s} $$\nwhere $D_i = 1(U_i \u003e 0.5)$ and $U_i$ is drawn from a uniform distribution, $u_{i,g}$ is a cluster level shock with intra-cluster correlation $0.5$, and the idiosyncratic error term is drawn from a multivariate random normal distribution with mean $0_S$ and covariance matrix \n\n```{r}\nS \u003c- 6\nrho \u003c- 0.5\nSigma \u003c- matrix(rho, 6, 6)\ndiag(Sigma) \u003c- 1\nSigma\n```\n\n\nwith $\\rho \\geq 0$. We assume that $\\beta_{1,s}= 0$ for all $s$. \n\nThis experiment imposes a data generating process as in equation (9) in [Clarke, Romano and Wolf](https://docs.iza.org/dp12845.pdf), with an additional error term $u_g$ for $G=20$ clusters and intra-cluster correlation 0.5 and $N=1000$ observations. \n\nYou can run the simulations via the `run_fwer_sim()` function attached in the package. \n\n\n```{r, message = FALSE, results = \"hide\"}\n# note that this will take some time\nres \u003c- run_fwer_sim(\n  seed = 76,\n  n_sims = 1000,\n  B = 499,\n  N = 1000,\n  s = 6, \n  rho = 0.5 #correlation between hypotheses, not intra-cluster!\n)\n```\n\nBoth Holm's method and `wildrwolf` control the family wise error rates, at both the 5 and 10% significance level. \n\n```{r}\nres\n```\n\n\n## Comparison with Stata's rwolf package\n\n```{r, eval = FALSE}\nlibrary(RStata)\n# initiate RStata\n    options(\"RStata.StataPath\" = \"\\\"C:\\\\Program Files\\\\Stata17\\\\StataBE-64\\\"\")\n    options(\"RStata.StataVersion\" = 17)\n# save the data set so it can be loaded into STATA\nwrite.csv(data, \"c:/Users/alexa/Dropbox/rwolf/inst/extdata/readme.csv\")\n\n# estimate with stata via Rstata\nstata_program \u003c- \"\nclear\nset more off\nimport delimited c:/Users/alexa/Dropbox/rwolf/inst/data/readme.csv\nset seed 1\nrwolf y1 y2 y3 y4, indepvar(x1) controls(x2) reps(9999)\n\"\nRStata::stata(stata_program, data.out = TRUE)\n\n\n# Romano-Wolf step-down adjusted p-values\n# \n# \n# Independent variable:  x1\n# Outcome variables:   y1 y2 y3 y4\n# Number of resamples: 9999\n# \n# \n# ------------------------------------------------------------------------------\n#    Outcome Variable | Model p-value    Resample p-value    Romano-Wolf p-value\n# --------------------+---------------------------------------------------------\n#                  y1 |    0.0000             0.0001              0.0001\n#                  y2 |    0.3904             0.3755              0.6070\n#                  y3 |    0.0000             0.0001              0.0001\n#                  y4 |    0.9586             0.9596              0.9596\n# ------------------------------------------------------------------------------\n\n\n```\n\nFor comparison,  `wildrwolf` produces the following output:\n\n```{r, warning = FALSE, message = FALSE, eval = FALSE}\nmodels \u003c- feols(c(Y1, Y2, Y3, Y4) ~ X1 + X2 \n                 , data = data, se = \"hetero\")\n```\n\n```{r, include = FALSE}\nmodels \u003c- feols(c(Y1, Y2, Y3, Y4) ~ X1 + X2 \n                 , data = data, se = \"hetero\")\n```\n\n```{r}\nrwolf(models, param = \"X1\", B = 9999)\n\n```\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fs3alfisc%2Fwildrwolf","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fs3alfisc%2Fwildrwolf","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fs3alfisc%2Fwildrwolf/lists"}