Resources

A health psychologist is evaluating a mindfulness app designed to reduce resting heart rate. In a pilot trial with a known population standard deviation of heart rate differences σ=6.0 bpm\sigma = 6.0\text{ bpm}, the app developers consider an average reduction of at least μd=2.5 bpm\mu_d = 2.5\text{ bpm} to represent practical clinical significance. (a) For a one-tailed test H0:μd=0H_0: \mu_d = 0 versus Ha:μd>0H_a: \mu_d > 0 at significance level α=0.05\alpha = 0.05 (z∗=1.645z^* = 1.645), calculate the minimum sample size nn required to achieve a statistical power of 1−β=0.801 - \beta = 0.80 (zβ=0.842z_\beta = 0.842) to detect this practically meaningful difference of 2.5 bpm2.5\text{ bpm}. (b) If an investigator instead enrolls a massive sample of n=40,000n = 40{,}000 users and observes a statistically significant heart rate reduction with z=4.00z = 4.00 (p<0.0001p < 0.0001), back-calculate the observed mean heart rate reduction xˉd\bar{x}_d. (c) Compare this observed reduction xˉd\bar{x}_d to the developer's benchmark of practical significance.

Answer and steps

Final answer

(a) n=((z∗+zβ)σμd)2=((1.645+0.842)×6.02.5)2=(2.487×6.02.5)2=(5.9688)2≈35.63n = \left(\frac{(z^* + z_\beta)\sigma}{\mu_d}\right)^2 = \left(\frac{(1.645 + 0.842) \times 6.0}{2.5}\right)^2 = \left(\frac{2.487 \times 6.0}{2.5}\right)^2 = (5.9688)^2 \approx 35.63, so a minimum sample size of n=36n = 36 participants is required. (b) z=xˉd−0σn  ⟹  4.00=xˉd6.040,000=xˉd0.030  ⟹  xˉd=4.00×0.030=0.120 bpmz = \frac{\bar{x}_d - 0}{\frac{\sigma}{\sqrt{n}}} \implies 4.00 = \frac{\bar{x}_d}{\frac{6.0}{\sqrt{40{,}000}}} = \frac{\bar{x}_d}{0.030} \implies \bar{x}_d = 4.00 \times 0.030 = 0.120\text{ bpm}. (c) Although xˉd=0.120 bpm\bar{x}_d = 0.120\text{ bpm} is highly statistically significant (p<0.0001p < 0.0001), it represents only a fraction of a single heartbeat per minute and falls far below the 2.5 bpm2.5\text{ bpm} threshold required for practical clinical significance.

Steps

Part (a): Determining the Required Sample Size

To find the minimum sample size nn for a one-tailed test with significance level α\alpha and power 1−β1 - \beta, we equate the critical value boundary under the null hypothesis H0:μd=0H_0: \mu_d = 0 to the boundary under the alternative hypothesis Ha:μd=2.5 bpmH_a: \mu_d = 2.5\text{ bpm}:

n=((z∗+zβ)σμd)2n = \left(\frac{(z^* + z_\beta)\sigma}{\mu_d}\right)^2

Substitute the given values:

  • Critical value for α=0.05\alpha = 0.05: z∗=1.645z^* = 1.645
  • Critical value for power 1−β=0.801 - \beta = 0.80: zβ=0.842z_\beta = 0.842
  • Population standard deviation: σ=6.0 bpm\sigma = 6.0\text{ bpm}
  • Meaningful difference: μd=2.5 bpm\mu_d = 2.5\text{ bpm}

n=((1.645+0.842)⋅6.02.5)2n = \left(\frac{(1.645 + 0.842) \cdot 6.0}{2.5}\right)^2

n=(2.487⋅6.02.5)2=(14.9222.5)2=(5.9688)2≈35.63n = \left(\frac{2.487 \cdot 6.0}{2.5}\right)^2 = \left(\frac{14.922}{2.5}\right)^2 = (5.9688)^2 \approx 35.63

Because sample size must be an integer, always round up to ensure the desired power is achieved:

n=36 participantsn = 36\text{ participants}


Part (b): Back-Calculating the Observed Mean Reduction

The standard error of the mean for a sample of size n=40,000n = 40{,}000 with σ=6.0 bpm\sigma = 6.0\text{ bpm} is:

SE=σn=6.040,000=6.0200=0.030 bpm\text{SE} = \frac{\sigma}{\sqrt{n}} = \frac{6.0}{\sqrt{40{,}000}} = \frac{6.0}{200} = 0.030\text{ bpm}

Using the one-sample zz-statistic formula:

z=xˉd−μ0SE=xˉd−00.030z = \frac{\bar{x}_d - \mu_0}{\text{SE}} = \frac{\bar{x}_d - 0}{0.030}

Given z=4.00z = 4.00:

4.00=xˉd0.030  ⟹  xˉd=4.00⋅0.030=0.120 bpm4.00 = \frac{\bar{x}_d}{0.030} \implies \bar{x}_d = 4.00 \cdot 0.030 = 0.120\text{ bpm}


Part (c): Comparing Statistical vs. Practical Significance

  • Statistical Significance: With z=4.00z = 4.00 and p<0.0001p < 0.0001, the result is overwhelmingly statistically significant, meaning the observed reduction of 0.120 bpm0.120\text{ bpm} is extremely unlikely to be due to chance alone.
  • Practical Significance: The observed drop of 0.120 bpm0.120\text{ bpm} is only a small fraction of a single heartbeat per minute, which is clinically negligible and well below the developer's benchmark of 2.5 bpm2.5\text{ bpm}.

Key Pitfall to Avoid: A massive sample size (nn) makes the standard error σ/n\sigma / \sqrt{n} extremely small. This allows even trivial, practically meaningless effect sizes to attain extreme statistical significance (very small pp-values). Thus, statistical significance does not imply practical importance.

Typical errors

OverviewACEpath Teacher's Diagnosis

01 The student forgets to square the ratio when calculating the sample size, yielding an unrealistically small sample size.

Sample Student Work

Reviewed by ACEpath Teacher
Step-by-Step Review
Walk Through It with Your Teacher
Partially correct
  1. Step 1

    n=(z∗+zβ)∗σμd  ⟹  n=(1.645+0.842)∗6.02.5  ⟹  n=14.9222.5=5.97n = \frac{(z^* + z_\beta) * \sigma}{\mu_d} \implies n = (1.645 + 0.842) * \frac{6.0}{2.5} \implies n = \frac{14.922}{2.5} = 5.97

    Correct

    n=((z∗+zβ)σμd)2=((1.645+0.842)×6.02.5)2=(5.9688)2≈35.63n = \left(\frac{(z^* + z_\beta)\sigma}{\mu_d}\right)^2 = \left(\frac{(1.645 + 0.842) \times 6.0}{2.5}\right)^2 = (5.9688)^2 \approx 35.63

    You set up the ratio of the zz-scores and standard deviation to the target difference accurately, but you forgot to square the entire quantity. Because the standard error involves n\sqrt{n}, solving for nn requires squaring the entire fraction.

  2. Step 2

    n=6n = 6

    Correct

    n=36n = 36

    You rounded your calculated value up to the next integer correctly, but because the formula was not squared in the previous step, your final sample size was 66 instead of 3636.

  3. Step 3

    z=xˉdσ/n  ⟹  4.00=xˉd6.0200  ⟹  xˉd=4.00∗0.03=0.12 bpmz = \frac{\bar{x}_d}{\sigma/\sqrt{n}} \implies 4.00 = \frac{\bar{x}_d}{\frac{6.0}{200}} \implies \bar{x}_d = 4.00 * 0.03 = 0.12\text{ bpm}

  4. Step 4

    0.12 bpm is much smaller than 2.5 bpm, so the result is statistically significant but clinically negligible.0.12\text{ bpm is much smaller than } 2.5\text{ bpm, so the result is statistically significant but clinically negligible.}

WHAT YOU DID WELL
Compliments from Your Teacher

You correctly identified the critical values for α=0.05\alpha = 0.05 and 80%80\% power.

You calculated the standard error and accurately back-calculated the sample mean difference for part (b).

You clearly articulated the distinction between statistical significance and practical clinical importance in part (c).

THE ERROR
Where Your ACEpath Teacher Spotted the Mistake
Omitted squaring in sample size formula

You forgot to square the ratio when solving for the required sample size nn.

WHY IT HAPPENED
ACEpath Teacher's Diagnosis

Confusing the formula for n\sqrt{n} with nn. In hypothesis testing power calculations, the margin of difference is equated to (z∗+zβ)σn(z^* + z_\beta)\frac{\sigma}{\sqrt{n}}, so isolating nn requires squaring the entire right-hand side.

REASONING PATH
Your ACEpath teacher’s suggested approach
Deriving Sample Size from Standard Error
  • Because SE=σn\text{SE} = \frac{\sigma}{\sqrt{n}}, isolating nn always requires squaring: n=((zα+zβ)σμa−μ0)2n = \left(\frac{(z_{\alpha} + z_{\beta})\sigma}{\mu_a - \mu_0}\right)^2.

  • Whenever you compute a required sample size, check that your formula squares the ratio before rounding up.

CORRECTED PATH
ACEpath Teacher's Demonstration
  1. Start with the correct sample size formula: n=((z∗+zβ)σμd)2n = \left(\frac{(z^* + z_\beta)\sigma}{\mu_d}\right)^2.

  2. Substitute the given parameters: n=((1.645+0.842)×6.02.5)2=(14.9222.5)2=(5.9688)2≈35.63n = \left(\frac{(1.645 + 0.842) \times 6.0}{2.5}\right)^2 = \left(\frac{14.922}{2.5}\right)^2 = (5.9688)^2 \approx 35.63.

  3. Always round up to ensure the minimum desired statistical power: n=36n = 36.

Final answer

(a) n=36n = 36; (b) xˉd=0.120 bpm\bar{x}_d = 0.120\text{ bpm}; (c) The effect is statistically significant (p<0.0001p < 0.0001) but clinically negligible because 0.120 bpm≪2.5 bpm0.120\text{ bpm} \ll 2.5\text{ bpm}.

NEXT STEPS
Final Takeaways from Your Teacher
  • Review how sample size relates to standard error: remember that nn appears under a square root in the denominator, so it must be squared.

  • Perform a sanity check on sample sizes: finding n=6n = 6 to detect a subtle difference of 2.5 bpm2.5\text{ bpm} with σ=6.0 bpm\sigma = 6.0\text{ bpm} is intuitively too small for 80%80\% power.

OverviewACEpath Teacher's Diagnosis

02 The student divides by n instead of the square root of n when calculating the standard error in part (b).

Sample Student Work

Reviewed by ACEpath Teacher
Step-by-Step Review
Walk Through It with Your Teacher
Partially correct
  1. Step 1

    n=((1.645+0.842)×6.0/2.5)2  ⟹  n=(14.922/2.5)2=35.63  ⟹  n=36n = ((1.645 + 0.842) \times 6.0 / 2.5)^2 \implies n = (14.922 / 2.5)^2 = 35.63 \implies n = 36

  2. Step 2

    z=xˉdσ/n  ⟹  4.00=xˉd6.0/40000=xˉd0.00015  ⟹  xˉd=4.00×0.00015=0.0006 bpmz = \frac{\bar{x}_d}{\sigma / n} \implies 4.00 = \frac{\bar{x}_d}{6.0 / 40000} = \frac{\bar{x}_d}{0.00015} \implies \bar{x}_d = 4.00 \times 0.00015 = 0.0006\text{ bpm}

    Correct

    z=xˉd−0σ/n  ⟹  4.00=xˉd6.0/40,000=xˉd0.030  ⟹  xˉd=4.00×0.030=0.120 bpmz = \frac{\bar{x}_d - 0}{\sigma / \sqrt{n}} \implies 4.00 = \frac{\bar{x}_d}{6.0 / \sqrt{40{,}000}} = \frac{\bar{x}_d}{0.030} \implies \bar{x}_d = 4.00 \times 0.030 = 0.120\text{ bpm}

    You divided the population standard deviation by nn rather than n\sqrt{n} when finding the standard error of the sampling distribution of the mean, making your standard error and resulting xˉd\bar{x}_d 200 times too small.

  3. Step 3

    0.0006 bpm0.0006\text{ bpm} is much less than the clinical benchmark of 2.5 bpm2.5\text{ bpm}, meaning it is not practically significant.

    Correct

    Although xˉd=0.120 bpm\bar{x}_d = 0.120\text{ bpm} is highly statistically significant (p<0.0001p < 0.0001), it is far below the 2.5 bpm2.5\text{ bpm} benchmark and is clinically negligible.

    Your conceptual reasoning is sound, but your sentence uses the incorrect numerical mean propagated from step 2.

WHAT YOU DID WELL
Compliments from Your Teacher

You correctly computed the required sample size in part (a) and rounded up to n=36n = 36.

You recognized that statistical significance with a massive sample does not guarantee practical clinical significance.

THE ERROR
Where Your ACEpath Teacher Spotted the Mistake
Standard Error Formula Error

no

WHY IT HAPPENED
ACEpath Teacher's Diagnosis

You confused the standard error of the mean SE=σn\text{SE} = \frac{\sigma}{\sqrt{n}} with σn\frac{\sigma}{n}, omitting the square root on the sample size in the denominator of the zz-statistic formula.

REASONING PATH
Your ACEpath teacher’s suggested approach
Standard Error of the Mean
  • The standard deviation of the sampling distribution of xˉ\bar{x} is σxˉ=σn\sigma_{\bar{x}} = \frac{\sigma}{\sqrt{n}}, not σn\frac{\sigma}{n}.

  • Whenever standardizing a sample mean xˉ\bar{x} into a zz-score, always verify that the denominator contains n\sqrt{n}.

CORRECTED PATH
ACEpath Teacher's Demonstration
  1. Write the test statistic definition: z=xˉd−μ0σ/nz = \frac{\bar{x}_d - \mu_0}{\sigma / \sqrt{n}}.

  2. Substitute the known values: 4.00=xˉd−06.0/40,0004.00 = \frac{\bar{x}_d - 0}{6.0 / \sqrt{40{,}000}}.

  3. Compute the standard error: SE=6.0200=0.030 bpm\text{SE} = \frac{6.0}{200} = 0.030\text{ bpm}.

  4. Solve for the observed mean difference: xˉd=4.00×0.030=0.120 bpm\bar{x}_d = 4.00 \times 0.030 = 0.120\text{ bpm}.

Final answer

(a) n=36n = 36; (b) xˉd=0.120 bpm\bar{x}_d = 0.120\text{ bpm}; (c) While statistically significant (z=4.00,p<0.0001z = 4.00, p < 0.0001), the observed drop of 0.120 bpm0.120\text{ bpm} is clinically negligible compared to the 2.5 bpm2.5\text{ bpm} practical benchmark.

NEXT STEPS
Final Takeaways from Your Teacher
  • Practice writing down SE=σn\text{SE} = \frac{\sigma}{\sqrt{n}} as a separate preliminary step before isolating unknown variables in test statistics.

  • Review how massive sample sizes shrink SE\text{SE} proportionally to 1n\frac{1}{\sqrt{n}}, leading to tiny pp-values for trivial effect sizes.

OverviewACEpath Teacher's Diagnosis

03 The student rounds down the calculated sample size instead of rounding up to the next integer required to guarantee the specified power.

Sample Student Work

Reviewed by ACEpath Teacher
Step-by-Step Review
Walk Through It with Your Teacher
Partially correct
  1. Step 1

    n=((1.645+0.842)×6.0/2.5)2  ⟹  n=(2.487×6.0/2.5)2  ⟹  n=5.96882=35.63n = ((1.645 + 0.842) \times 6.0 / 2.5)^2 \implies n = (2.487 \times 6.0 / 2.5)^2 \implies n = 5.9688^2 = 35.63

  2. Step 2

    n=35n = 35

    Correct

    n=36n = 36

    You truncated or rounded down to 3535, but sample size calculations for statistical power must always be rounded up (the ceiling function) to the next integer. Choosing n=35n = 35 would result in a statistical power slightly below the required 80%80\%.

  3. Step 3

    z=xˉdσ/n  ⟹  4.00=xˉd6.0/200  ⟹  xˉd=4.00×0.03=0.12 bpmz = \frac{\bar{x}_d}{\sigma / \sqrt{n}} \implies 4.00 = \frac{\bar{x}_d}{6.0 / 200} \implies \bar{x}_d = 4.00 \times 0.03 = 0.12\text{ bpm}

  4. Step 4

    0.12 bpm0.12\text{ bpm} is much smaller than 2.5 bpm2.5\text{ bpm}, so the result lacks practical clinical significance despite p<0.0001p < 0.0001.

WHAT YOU DID WELL
Compliments from Your Teacher

You correctly selected and computed the sample size formula for statistical power.

You correctly calculated the standard error for n=40,000n = 40{,}000 and back-calculated xˉd=0.12 bpm\bar{x}_d = 0.12\text{ bpm}.

You correctly explained why a statistically significant finding with p<0.0001p < 0.0001 can still lack practical clinical significance.

THE ERROR
Where Your ACEpath Teacher Spotted the Mistake
Sample size rounding error

You rounded 35.6335.63 down to 3535 instead of taking the ceiling integer 3636.

WHY IT HAPPENED
ACEpath Teacher's Diagnosis

You treated sample size as an ordinary decimal number to round down or truncate, overlooking the rule that statistical power is a minimum threshold requirement that demands rounding up to the next integer.

REASONING PATH
Your ACEpath teacher’s suggested approach
Always Round Up Sample Size
  • When calculating a minimum sample size nn to guarantee a specified power 1−β1 - \beta or a maximum margin of error, any fractional part requires rounding up to the next whole integer: n=⌈ncalc⌉n = \lceil n_{\text{calc}} \rceil.

  • Whenever you compute a decimal value for sample size such as 35.6335.63, automatically round up to 3636 rather than truncating or applying standard half-way rounding rules.

CORRECTED PATH
ACEpath Teacher's Demonstration
  1. Apply the ceiling rule for sample size: n=⌈35.63⌉=36n = \lceil 35.63 \rceil = 36.

Final answer

(a) n=36n = 36 participants; (b) xˉd=0.120 bpm\bar{x}_d = 0.120\text{ bpm}; (c) The observed reduction lacks practical clinical significance because 0.120 bpm≪2.5 bpm0.120\text{ bpm} \ll 2.5\text{ bpm}.

NEXT STEPS
Final Takeaways from Your Teacher
  • Review the ceiling integer rule for sample size calculations in power and confidence interval problems.

  • Practice identifying why rounding down reduces statistical power below the required target.

Lock it in

Similar question

01

A researcher wants to detect a mean difference of μd=4 mmHg\mu_d = 4\text{ mmHg} in blood pressure with σ=10 mmHg\sigma = 10\text{ mmHg} at α=0.05\alpha = 0.05 (one-tailed, z∗=1.645z^* = 1.645) and power 0.800.80 (zβ=0.842z_\beta = 0.842). Calculate the minimum required sample size nn.

Answer

n=((1.645+0.842)×104)2=(24.874)2=(6.2175)2≈38.66n = \left(\frac{(1.645 + 0.842) \times 10}{4}\right)^2 = \left(\frac{24.87}{4}\right)^2 = (6.2175)^2 \approx 38.66. Rounding up gives n=39n = 39.

02

A study with n=10,000n = 10{,}000 finds an average increase in exam scores of xˉ=0.5\bar{x} = 0.5 points with σ=15\sigma = 15 (H0:μ=0H_0: \mu = 0). Calculate the test statistic zz and explain why this result might be statistically significant but practically unimportant if a meaningful gain is defined as at least 55 points.

Answer

SE=1510,000=15100=0.15\text{SE} = \frac{15}{\sqrt{10{,}000}} = \frac{15}{100} = 0.15. Then z=0.5−00.15≈3.33z = \frac{0.5 - 0}{0.15} \approx 3.33 (p≈0.0004p \approx 0.0004). The tiny standard error from the massive sample makes 0.50.5 points highly statistically significant, but 0.50.5 points is only one-tenth of the practically meaningful 55-point benchmark.

03

To halve the margin of error or halve the minimum detectable effect size Δ\Delta while keeping α\alpha, power, and σ\sigma constant, by what factor must the required sample size nn change?

Answer

It must increase by a factor of 44. Because n∝(1Δ)2n \propto \left(\frac{1}{\Delta}\right)^2, cutting Δ\Delta in half requires nnew=(10.5)2n=4nn_{\text{new}} = \left(\frac{1}{0.5}\right)^2 n = 4n.

Loading workspace…