Resources
A health psychologist is evaluating a mindfulness app designed to reduce resting heart rate. In a pilot trial with a known population standard deviation of heart rate differences , the app developers consider an average reduction of at least to represent practical clinical significance. (a) For a one-tailed test versus at significance level (), calculate the minimum sample size required to achieve a statistical power of () to detect this practically meaningful difference of . (b) If an investigator instead enrolls a massive sample of users and observes a statistically significant heart rate reduction with (), back-calculate the observed mean heart rate reduction . (c) Compare this observed reduction to the developer's benchmark of practical significance.
Answer and steps
Final answer
(a) , so a minimum sample size of participants is required. (b) . (c) Although is highly statistically significant (), it represents only a fraction of a single heartbeat per minute and falls far below the threshold required for practical clinical significance.
Steps
Part (a): Determining the Required Sample Size
To find the minimum sample size for a one-tailed test with significance level and power , we equate the critical value boundary under the null hypothesis to the boundary under the alternative hypothesis :
Substitute the given values:
- Critical value for :
- Critical value for power :
- Population standard deviation:
- Meaningful difference:
Because sample size must be an integer, always round up to ensure the desired power is achieved:
Part (b): Back-Calculating the Observed Mean Reduction
The standard error of the mean for a sample of size with is:
Using the one-sample -statistic formula:
Given :
Part (c): Comparing Statistical vs. Practical Significance
- Statistical Significance: With and , the result is overwhelmingly statistically significant, meaning the observed reduction of is extremely unlikely to be due to chance alone.
- Practical Significance: The observed drop of is only a small fraction of a single heartbeat per minute, which is clinically negligible and well below the developer's benchmark of .
Key Pitfall to Avoid: A massive sample size () makes the standard error extremely small. This allows even trivial, practically meaningless effect sizes to attain extreme statistical significance (very small -values). Thus, statistical significance does not imply practical importance.
Typical errors
01 The student forgets to square the ratio when calculating the sample size, yielding an unrealistically small sample size.
Sample Student Work
Reviewed by ACEpath TeacherStep-by-Step Review
Walk Through It with Your Teacher- Step 1Correct
You set up the ratio of the -scores and standard deviation to the target difference accurately, but you forgot to square the entire quantity. Because the standard error involves , solving for requires squaring the entire fraction.
- Step 2Correct
You rounded your calculated value up to the next integer correctly, but because the formula was not squared in the previous step, your final sample size was instead of .
- Step 3
- Step 4
WHAT YOU DID WELL
Compliments from Your TeacherYou correctly identified the critical values for and power.
You calculated the standard error and accurately back-calculated the sample mean difference for part (b).
You clearly articulated the distinction between statistical significance and practical clinical importance in part (c).
THE ERROR
Where Your ACEpath Teacher Spotted the MistakeYou forgot to square the ratio when solving for the required sample size .
WHY IT HAPPENED
ACEpath Teacher's DiagnosisConfusing the formula for with . In hypothesis testing power calculations, the margin of difference is equated to , so isolating requires squaring the entire right-hand side.
REASONING PATH
Your ACEpath teacher’s suggested approach- Deriving Sample Size from Standard Error
Because , isolating always requires squaring: .
Whenever you compute a required sample size, check that your formula squares the ratio before rounding up.
CORRECTED PATH
ACEpath Teacher's DemonstrationStart with the correct sample size formula: .
Substitute the given parameters: .
Always round up to ensure the minimum desired statistical power: .
(a) ; (b) ; (c) The effect is statistically significant () but clinically negligible because .
02 The student divides by n instead of the square root of n when calculating the standard error in part (b).
Sample Student Work
Reviewed by ACEpath TeacherStep-by-Step Review
Walk Through It with Your Teacher- Step 1
- Step 2Correct
You divided the population standard deviation by rather than when finding the standard error of the sampling distribution of the mean, making your standard error and resulting 200 times too small.
- Step 3
is much less than the clinical benchmark of , meaning it is not practically significant.
CorrectAlthough is highly statistically significant (), it is far below the benchmark and is clinically negligible.
Your conceptual reasoning is sound, but your sentence uses the incorrect numerical mean propagated from step 2.
WHAT YOU DID WELL
Compliments from Your TeacherYou correctly computed the required sample size in part (a) and rounded up to .
You recognized that statistical significance with a massive sample does not guarantee practical clinical significance.
THE ERROR
Where Your ACEpath Teacher Spotted the Mistakeno
WHY IT HAPPENED
ACEpath Teacher's DiagnosisYou confused the standard error of the mean with , omitting the square root on the sample size in the denominator of the -statistic formula.
REASONING PATH
Your ACEpath teacher’s suggested approach- Standard Error of the Mean
The standard deviation of the sampling distribution of is , not .
Whenever standardizing a sample mean into a -score, always verify that the denominator contains .
CORRECTED PATH
ACEpath Teacher's DemonstrationWrite the test statistic definition: .
Substitute the known values: .
Compute the standard error: .
Solve for the observed mean difference: .
(a) ; (b) ; (c) While statistically significant (), the observed drop of is clinically negligible compared to the practical benchmark.
03 The student rounds down the calculated sample size instead of rounding up to the next integer required to guarantee the specified power.
Sample Student Work
Reviewed by ACEpath TeacherStep-by-Step Review
Walk Through It with Your Teacher- Step 1
- Step 2Correct
You truncated or rounded down to , but sample size calculations for statistical power must always be rounded up (the ceiling function) to the next integer. Choosing would result in a statistical power slightly below the required .
- Step 3
- Step 4
is much smaller than , so the result lacks practical clinical significance despite .
WHAT YOU DID WELL
Compliments from Your TeacherYou correctly selected and computed the sample size formula for statistical power.
You correctly calculated the standard error for and back-calculated .
You correctly explained why a statistically significant finding with can still lack practical clinical significance.
THE ERROR
Where Your ACEpath Teacher Spotted the MistakeYou rounded down to instead of taking the ceiling integer .
WHY IT HAPPENED
ACEpath Teacher's DiagnosisYou treated sample size as an ordinary decimal number to round down or truncate, overlooking the rule that statistical power is a minimum threshold requirement that demands rounding up to the next integer.
REASONING PATH
Your ACEpath teacher’s suggested approach- Always Round Up Sample Size
When calculating a minimum sample size to guarantee a specified power or a maximum margin of error, any fractional part requires rounding up to the next whole integer: .
Whenever you compute a decimal value for sample size such as , automatically round up to rather than truncating or applying standard half-way rounding rules.
CORRECTED PATH
ACEpath Teacher's DemonstrationApply the ceiling rule for sample size: .
(a) participants; (b) ; (c) The observed reduction lacks practical clinical significance because .
Lock it in
What is the standard formula for the minimum sample size in a one-sample mean test given , power , standard deviation , and minimum detectable effect size ?
. Remember: because standard error uses , you must square the entire ratio.
Similar question
A researcher wants to detect a mean difference of in blood pressure with at (one-tailed, ) and power (). Calculate the minimum required sample size .
Answer
. Rounding up gives .
A study with finds an average increase in exam scores of points with (). Calculate the test statistic and explain why this result might be statistically significant but practically unimportant if a meaningful gain is defined as at least points.
Answer
. Then (). The tiny standard error from the massive sample makes points highly statistically significant, but points is only one-tenth of the practically meaningful -point benchmark.
To halve the margin of error or halve the minimum detectable effect size while keeping , power, and constant, by what factor must the required sample size change?
Answer
It must increase by a factor of . Because , cutting in half requires .