https://github.com/lakens/follow_up_bias
Biased sample size estimates in a-priori power analysis due to the choice of the effect size index and follow-up bias
https://github.com/lakens/follow_up_bias
Last synced: about 1 year ago
JSON representation
Biased sample size estimates in a-priori power analysis due to the choice of the effect size index and follow-up bias
- Host: GitHub
- URL: https://github.com/lakens/follow_up_bias
- Owner: Lakens
- License: gpl-3.0
- Created: 2017-06-07T09:37:31.000Z (about 9 years ago)
- Default Branch: master
- Last Pushed: 2017-08-31T20:01:24.000Z (almost 9 years ago)
- Last Synced: 2025-03-23T19:05:14.677Z (about 1 year ago)
- Language: R
- Size: 52.7 KB
- Stars: 4
- Watchers: 3
- Forks: 2
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Albers, C. J. & Lakens, D. Biased sample size estimates in a-priori power analysis due to the choice of the effect size index and follow-up bias.
For more information, see the [OSF project page](https://osf.io/zq9mg/)
The figures generated by the R code is available from the [Figshare project page](https://figshare.com/projects/Biased_sample_size_estimates_in_a-priori_power_analysis_due_to_the_choice_of_the_effect_size_index_and_follow-up_bias/23326).
When designing a study, the planned sample size is often based on power analyses. One way to choose an effect size for power analyses is by relying on pilot data. A-priori power analyses are only accurate when the effect size estimate is accurate. In this paper we highlight two sources of bias when performing a-priori power analyses for between-subject designs based on pilot data. First, we examine how the choice of the effect size index (eta-squared, omage-squared and epsilon-squared) affects the sample size and power of the main study. Based on our observations, we recommend against the use of eta-squared in a-priori power analyses. Second, we examine how the maximum sample size researchers are willing to collect in a main study (e.g. due to time or financial constraints) leads to overestimated effect size estimates in the studies that are performed. Determining the required sample size exclusively based on the effect size estimates from pilot data, and following up on pilot studies only when the sample size estimate for the main study is considered feasible, creates what we term follow-up bias. We explain how follow-up bias leads to underpowered main studies.
Our simulations show that designing main studies based on effect sizes estimated from small pilot studies does not yield desired levels of power due to accuracy bias and follow-up bias, even when publication bias is not an issue. We urge researchers to consider alternative approaches to determining the sample size of their studies, and discuss several options.