{"id":23480820,"url":"https://github.com/udialter/equivalence-testing-multiple-regression","last_synced_at":"2026-03-20T01:31:02.749Z","repository":{"id":49356955,"uuid":"277332279","full_name":"udialter/equivalence-testing-multiple-regression","owner":"udialter","description":" I constructed a simulation study to evaluate the statistical performance of two equivalence-based tests and compared it to the common, but inappropriate, method of concluding no effect by failing to reject the null hypothesis of the traditional test. I further propose two R functions to supply researchers with open-access and easy-to-use tools that they can flexibly adopt in their own research.","archived":false,"fork":false,"pushed_at":"2022-07-25T00:41:11.000Z","size":675,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-08-31T14:40:34.812Z","etag":null,"topics":["equivalence-testing","multiple-regression","null-hypothesis","psychology","quantitative-methods","quantitative-psychology"],"latest_commit_sha":null,"homepage":"","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/udialter.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-07-05T15:40:53.000Z","updated_at":"2022-12-16T21:14:09.000Z","dependencies_parsed_at":"2022-08-26T20:00:56.675Z","dependency_job_id":null,"html_url":"https://github.com/udialter/equivalence-testing-multiple-regression","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/udialter/equivalence-testing-multiple-regression","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/udialter%2Fequivalence-testing-multiple-regression","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/udialter%2Fequivalence-testing-multiple-regression/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/udialter%2Fequivalence-testing-multiple-regression/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/udialter%2Fequivalence-testing-multiple-regression/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/udialter","download_url":"https://codeload.github.com/udialter/equivalence-testing-multiple-regression/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/udialter%2Fequivalence-testing-multiple-regression/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29326180,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-11T02:08:56.257Z","status":"ssl_error","status_checked_at":"2026-02-11T02:08:51.338Z","response_time":97,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["equivalence-testing","multiple-regression","null-hypothesis","psychology","quantitative-methods","quantitative-psychology"],"created_at":"2024-12-24T20:42:46.903Z","updated_at":"2026-02-11T03:32:41.680Z","avatar_url":"https://github.com/udialter.png","language":null,"funding_links":[],"categories":[],"sub_categories":[],"readme":"# Equivalence Testing for Multiple Regression\n\nEquivalence testing can be applied to evaluate whether an observed\neffect from an individual predictor in a multiple regression model is\nsmall enough to be considered statistically and practically negligible\n([Alter \u0026 Counsell, 2021](https://psyarxiv.com/ugc9e/)). For more\ninformation, please refer to the [OSF\npage](https://osf.io/w96xe/ \"Equivalence Testing for Multiple Regression OSF Page\")\nand/or a freely available [preprint on\nPsyArXiv](https://psyarxiv.com/ugc9e/ \"PsyArXiv preprint\").\n\nThe following functions offer appropriate equivalence-based alternatives\nfor concluding negligible effect between a predictor and outcome in\nmultiple regression\n\nThese R functions are designed to accommodate multiple research contexts\neffortlessly, with or without access to the full dataset. The two\nfunctions, `reg.equiv.fd()` and `reg.equiv()`, provide similar output\nbut differ on the type of input information required by the user.\n\nSpecifically, the first function, `reg.equiv.fd()`, requires the full\ndataset and model in R (`lm` object), whereas the second does not. The\n`reg.equiv()` is intended for researchers who do not have access to the\ncomplete dataset but still wish to evaluate a certain predictor’s lack\nof association with the outcome variable in multiple regression, for\nexample, using information typically presented in a results section or\ntable reported in a published article.\n\n## `reg.equiv.fd()`: full dataset required\n\n### *Required* input:\n\n-   `datfra=` a data frame (e.g., mtcars)\n-   `model=` the model, an lm object (e.g., `mod1`, where\n    `mod1\u003c- mpg~hp+cyl`)\n-   `delta=` the smallest effect size of interest (SESOI), minimally\n    meaningful effect size (MMES), or upper bound of the equivalence\n    interval (𝛅) (e.g., .15)\n-   `predictor=` the name of the predictor to be tested (e.g., `\"cyl\"`)\n\n### Default settings:\n\n-   `test=` test type is set automatically to Two One-Sided Test (TOST;\n    Schuirmann, 1987), the other option is the Anderson-Hauck (AH;\n    Anderson \u0026 Hauck, 1983)\n-   `std=` the delta (or, SESOI) is the set as as standardized by\n    default. Indicate `std=FALSE` to assume unstandardized units\n-   `alpha=` the nominal Type I error rate is set to .05 by default. To\n    change, simply indicate the alpha level. E.g., `alpha=.10`\n\n#### `reg.equiv.fd()` example: \n![alttext](https://github.com/udialter/equivalence-testing-multiple-regression/blob/master/Figure%204%20reg.equiv.fd%20.png)\n\n## `reg.equiv()`: full dataset _not_ required\n\n### *Required* input:\n\n-   `b=` the estimated effect size associated with the predictor of\n    interest, this could be either standardized or unstandardized (e.g.,\n    .02)\n-   `se=` the standard error associated with the effect size of the\n    predictor of interest (if the effect size is standardized, make sure\n    the `se` value is tied to the standardized and not the raw effect)\n-   `p=` the number of total predictors in the regression model\n    (excluding intercept)\n-   `n=` sample size\n-   `delta=` the smallest effect size of interest (SESOI), minimally\n    meaningful effect size (MMES), or upper bound of the equivalence\n    interval (𝛅) (e.g., .15)\n-   `predictor=` the name of the predictor to be tested (e.g., `\"cyl\"`)\n\n### Default settings:\n\n-   `test=` test type is set automatically to Two One-Sided Test (TOST;\n    Schuirmann, 1987), the other option is the Anderson-Hauck (AH;\n    Anderson \u0026 Hauck, 1983)\n-   `std=` the delta (or, SESOI) and the indicated effect size are set\n    as as standardized by default. Indicate `std=FALSE` to assume\n    unstandardized units\n-   `alpha=` the nominal Type I error rate is set to .05 by default. To\n    change, simply indicate the alpha level. E.g., `alpha=.10`\n\n#### `reg.equiv()` example: \n![alttext](https://github.com/udialter/equivalence-testing-multiple-regression/blob/master/Figure%205%20reg.equiv.png)\n\n### Recommendations for Test Result Interpretation\n\nEquivalence testing is a method designed within the null-hypothesis\nsignificance testing (NHST) framework. NHST has been heavily criticized\nfor its overreliance on the dichotomous results of *p* values with\nlittle, or no consideration of the effect’s magnitude or its\nimplications in practice (e.g., Cumming, 2012; Fidler \u0026 Loftus, 2009;\nHarlow, 1997; Kirk, 2003; Lee, 2016 2014). Researchers must be mindful\nof the limitations of NHST, and disentangle the practical and\nstatistical aspects of the test results.\n\nTo minimize the limitations of *p* values, it is more informative to\n**interpret the observed effect’s magnitude and precision** beyond the\nconclusion of “negligible effects” or “insufficient evidence for\nnegligible effects.” **Observed effects should be construed in relation\nto the equivalence bounds, the extent of their uncertainty, and their\npractical implications (or lack thereof)**. For this reason, **the two R\nfunctions offered here also include a graphical representation of the\nobserved effect and its associated uncertainty in relation to the\nequivalence interval. The resulting plot aids in illustrating how close\nor far and wide or narrow the observed effect and its margin of error\nare from the equivalence bounds; inferring about the proportion and\nposition of the confidence band in relation to the equivalence interval\ncan help interpret the results over and above *p* values**.\n\n\n\n## For an in-depth literature review, disucssion, recommendations, and detailed tutorials, please refer to [Alter \u0026 Counsell (2021)](https://psyarxiv.com/ugc9e/).\n\n\n\n\n#### References\n\n1.  Alter, U., \u0026 Counsell, A. (2021, June 17). _Equivalence Testing for\n    Multiple Regression_. \u003chttps://doi.org/10.17605/OSF.IO/W96XE\u003e\n2.  Anderson, S., \u0026 Hauck, W. W. (1983). A new procedure for testing\n    equivalence in comparative bioavailability and other clinical\n    trials. _Statistics and Communications-Theory and Methods, 12_\n    ,2663-2692. \u003chttps://doi.org/10.1080/03610928308828634\u003e\n3.  Cumming, G. (2012). _Understanding The New Statistics: Effect Sizes,\n    Confidence Intervals, and Meta-Analysis_. New York, NY: Taylor \u0026\n    Francis Group, LLC\n4.  Cumming, G. (2014). The New Statistics: Why and How. _Psychological\n    Science, 25_(1), 7–29. \u003chttps://doi.org/10.1177/0956797613504966\u003e\n5.  Fidler, F., \u0026 Loftus, G. (2009). Why figures with error bars should\n    replace p values: Some conceptual arguments and empirical\n    demonstrations. _Zeitschrift Für Psychologie/Journal of Psychology,\n    217_(1), 27-37. \u003chttps://doi.org/10.1027/0044-3409.217.1.27\u003e\n6.  Harlow, L.L. (1997). _Significance Testing in Introduction and\n    Overview_. In L.L. Harlow, S.A. Muliak \u0026 J.H. Steiger (Eds.). What If\n    There Were No Significance Tests? (pp.1-17). Mahwah, NJ, USA:\n    Lawrence Erlbaum.\n7.  Hauck, W. W., \u0026 Anderson, S. (1984). A new statistical procedure for\n    testing equivalence in two-group comparative bioavailability trials.\n    _Journal of Pharmacokinetics and Biopharmaceutics, 12_(1), 83-91.\n    \u003chttps://doi.org/10.1007/BF01063612\u003e\n8.  Kirk, R. E. (2003). _The importance of effect magnitude_. In S. F.\n    Davis (Ed.), Handbook of research methods in experimental psychology\n    (pp. 83–105). Malden, MA: Blackwell.\n9.  Lee, D. K. (2016). Alternatives to P value: Confidence interval and\n    effect size. _Korean Journal of Anesthesiology, 69_(6), 555-562.\n    \u003chttps://doi.org/10.4097/kjae.2016.69.6.555\u003e\n10. Schuirmann, D. J. (1987). A comparison of the two one-sided tests\n    procedure and the power approach for assessing the equivalence of\n    average bioavailability. _Journal of Pharmacokinetics and\n    Biopharmaceutics, 15_, 657-680. \u003chttps://doi.org/10.1007/BF01068419\u003e\n11. Seli, P., Risko, E. F., Purdon, C., \u0026 Smilek, D. (2017). Intrusive thoughts: Linking spontaneous mind wandering and OCD symptomatology. _Psychological Research, 81_(2), 392-398. https://doi.org/10.1007/s00426-016-0756-3![image]\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fudialter%2Fequivalence-testing-multiple-regression","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fudialter%2Fequivalence-testing-multiple-regression","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fudialter%2Fequivalence-testing-multiple-regression/lists"}