Resources

A clinical trial studies a new anti-inflammatory medication where doctors define a clinically meaningful reduction in serum C-reactive protein (CRP) as an effect size of at least d=0.40d = 0.40. The baseline standard deviation of CRP is σ=2.50 mg/L\sigma = 2.50\text{ mg/L}, so a clinically meaningful difference is defined as a reduction of Δcrit=1.00 mg/L\Delta_{\text{crit}} = 1.00\text{ mg/L}. A randomized experiment on n=36n = 36 patients yields a mean reduction of xˉd=0.82 mg/L\bar{x}_d = 0.82\text{ mg/L} with a standard error SE=0.35 mg/LSE = 0.35\text{ mg/L}. (a) Test H0:μd=0H_0: \mu_d = 0 against Ha:μd>0H_a: \mu_d > 0 at α=0.05\alpha = 0.05 (tcrit=1.690t_{\text{crit}} = 1.690 for df=35df = 35) and decide whether to reject or fail to reject H0H_0. (b) Construct an exact 90%90\% two-sided confidence interval for μd\mu_d (using t∗=1.690t^* = 1.690), which corresponds to a 5%5\% one-tailed lower and upper boundary. (c) At the threshold boundary of practical significance, is the data consistent with an effect that meets or exceeds Δcrit=1.00 mg/L\Delta_{\text{crit}} = 1.00\text{ mg/L}, and can practical significance be ruled out?

Answer and steps

Final answer

(a) t=0.82−00.35≈2.343t = \frac{0.82 - 0}{0.35} \approx 2.343. Because t=2.343>1.690t = 2.343 > 1.690 (corresponding to p≈0.0125<0.05p \approx 0.0125 < 0.05), we reject H0H_0; the reduction in CRP is statistically significant. (b) The 90%90\% confidence interval is 0.82±(1.690)(0.35)=0.82±0.59150.82 \pm (1.690)(0.35) = 0.82 \pm 0.5915, yielding [0.2285,1.4115] mg/L[0.2285, 1.4115]\text{ mg/L}, or [0.23,1.41] mg/L[0.23, 1.41]\text{ mg/L}. (c) Because the critical practical threshold Δcrit=1.00 mg/L\Delta_{\text{crit}} = 1.00\text{ mg/L} falls entirely inside the 90%90\% confidence interval [0.23,1.41] mg/L[0.23, 1.41]\text{ mg/L}, practical significance cannot be ruled out; the trial confirms statistical significance, but the true effect size is inconclusive regarding practical significance due to sampling variability.

Steps

Part (a): Hypothesis Test for Statistical Significance

  1. Formulate the Hypotheses and Test Statistic: We are testing whether the mean reduction in CRP, denoted by μd\mu_d, is significantly greater than zero: H0:μd=0versusHa:μd>0H_0: \mu_d = 0 \quad \text{versus} \quad H_a: \mu_d > 0 Given the sample mean reduction xˉd=0.82 mg/L\bar{x}_d = 0.82\text{ mg/L} and standard error SE=0.35 mg/LSE = 0.35\text{ mg/L}, compute the tt-statistic: t=xˉd−0SE=0.820.35≈2.343t = \frac{\bar{x}_d - 0}{SE} = \frac{0.82}{0.35} \approx 2.343
  2. Make the Statistical Decision:

    • At a significance level of α=0.05\alpha = 0.05 with degrees of freedom df=n−1=35df = n - 1 = 35, the critical value for a right-tailed test is tcrit=1.690t_{\text{crit}} = 1.690.
    • Since t=2.343>1.690t = 2.343 > 1.690 (which corresponds to a pp-value of approximately 0.0125<0.050.0125 < 0.05), we reject H0H_0.
    • Conclusion: The reduction in CRP is statistically significant, meaning the observed reduction is unlikely to have occurred by random sampling chance alone if the drug had zero effect.

Part (b): Construction of the Confidence Interval

  1. Formula for Confidence Interval: A two-sided 90%90\% confidence interval for the population mean μd\mu_d leaves 5%5\% in each tail, directly matching the one-tailed critical value t∗=1.690t^* = 1.690: CI=xˉd±t∗⋅SE\text{CI} = \bar{x}_d \pm t^* \cdot SE
  2. Calculate the Margin of Error and Bounds: Margin of Error=1.690⋅0.35=0.5915 mg/L\text{Margin of Error} = 1.690 \cdot 0.35 = 0.5915\text{ mg/L} Lower Limit=0.82−0.5915=0.2285 mg/L≈0.23 mg/L\text{Lower Limit} = 0.82 - 0.5915 = 0.2285\text{ mg/L} \approx 0.23\text{ mg/L} Upper Limit=0.82+0.5915=1.4115 mg/L≈1.41 mg/L\text{Upper Limit} = 0.82 + 0.5915 = 1.4115\text{ mg/L} \approx 1.41\text{ mg/L} Thus, the 90%90\% confidence interval is [0.2285,1.4115] mg/L[0.2285, 1.4115]\text{ mg/L}, or approximately [0.23,1.41] mg/L[0.23, 1.41]\text{ mg/L}.

Part (c): Evaluating Practical (Clinical) Significance

  1. Compare the Confidence Interval to the Practical Threshold:

    • Statistical significance tests whether μd>0\mu_d > 0. Because the entire interval [0.23,1.41] mg/L[0.23, 1.41]\text{ mg/L} lies strictly above 00, we are confident that the true effect is greater than zero.
    • Practical significance requires the effect to be meaningful in the real world—here, a clinically defined threshold of Δcrit=1.00 mg/L\Delta_{\text{crit}} = 1.00\text{ mg/L}.
  2. Interpret the Findings:

    • The threshold Δcrit=1.00 mg/L\Delta_{\text{crit}} = 1.00\text{ mg/L} lies inside the interval [0.23,1.41] mg/L[0.23, 1.41]\text{ mg/L}.
    • Because values ≥1.00 mg/L\ge 1.00\text{ mg/L} (such as 1.201.20 or 1.40 mg/L1.40\text{ mg/L}) are plausible given the data, practical significance cannot be ruled out. The data is consistent with an effect that meets or exceeds the critical threshold.
    • However, because values below 1.00 mg/L1.00\text{ mg/L} are also plausible, practical significance is not guaranteed. The trial conclusively demonstrates an effect (statistical significance), but remains inconclusive regarding whether the effect is large enough to be clinically meaningful (practical significance).

Typical errors

OverviewACEpath Teacher's Diagnosis

01 The student recomputes the standard error by dividing the already-given standard error by √n, inflating the t-statistic.

Sample Student Work

Reviewed by ACEpath Teacher
Step-by-Step Review
Walk Through It with Your Teacher
Incorrect
  1. Step 1

    SEcalc=0.35/36=0.35/6≈0.0583  ⟹  t=0.82/0.0583=14.07  ⟹  14.07>1.690, so reject H0SE_{\text{calc}} = 0.35 / \sqrt{36} = 0.35 / 6 \approx 0.0583 \implies t = 0.82 / 0.0583 = 14.07 \implies 14.07 > 1.690\text{, so reject } H_0

    Correct

    t=xˉd−0SE=0.820.35≈2.343  ⟹  2.343>1.690, so reject H0t = \frac{\bar{x}_d - 0}{SE} = \frac{0.82}{0.35} \approx 2.343 \implies 2.343 > 1.690\text{, so reject } H_0

    The problem already gave the standard error of the mean as SE=0.35 mg/LSE = 0.35\text{ mg/L}. You divided it again by n=36=6\sqrt{n} = \sqrt{36} = 6, confusing standard deviation ss with standard error SESE.

  2. Step 2

    CI=0.82±1.690×0.0583=0.82±0.0985=[0.7215,0.9185] mg/L\text{CI} = 0.82 \pm 1.690 \times 0.0583 = 0.82 \pm 0.0985 = [0.7215, 0.9185]\text{ mg/L}

    Correct

    CI=0.82±1.690×0.35=0.82±0.5915=[0.2285,1.4115] mg/L\text{CI} = 0.82 \pm 1.690 \times 0.35 = 0.82 \pm 0.5915 = [0.2285, 1.4115]\text{ mg/L}

    You applied the confidence interval formula correctly, but your interval bounds were affected by using the incorrectly deflated standard error 0.0583 mg/L0.0583\text{ mg/L} instead of 0.35 mg/L0.35\text{ mg/L}.

  3. Step 3

    The interval [0.7215,0.9185] does not contain 1.00 mg/L. The data are not consistent with an effect of 1.00 mg/L or greater, so practical significance can be ruled out.\text{The interval } [0.7215, 0.9185] \text{ does not contain } 1.00\text{ mg/L. The data are not consistent with an effect of } 1.00\text{ mg/L or greater, so practical significance can be ruled out.}

    Correct

    Because Δcrit=1.00 mg/L lies inside [0.23,1.41] mg/L, the data is consistent with an effect ≥1.00 mg/L, so practical significance cannot be ruled out.\text{Because } \Delta_{\text{crit}} = 1.00\text{ mg/L} \text{ lies inside } [0.23, 1.41]\text{ mg/L, the data is consistent with an effect } \ge 1.00\text{ mg/L, so practical significance cannot be ruled out.}

    Your comparison logic followed logically from your interval, but because the true interval is [0.23,1.41] mg/L[0.23, 1.41]\text{ mg/L}, it actually contains 1.00 mg/L1.00\text{ mg/L}, meaning practical significance cannot be ruled out.

WHAT YOU DID WELL
Compliments from Your Teacher

You correctly set up the decision rule by comparing the test statistic to tcrit=1.690t_{\text{crit}} = 1.690.

You used the correct confidence interval structure xˉd±t∗⋅SE\bar{x}_d \pm t^* \cdot SE.

You correctly recognized that whether practical significance can be ruled out depends on whether the critical threshold falls within the confidence interval.

THE ERROR
Where Your ACEpath Teacher Spotted the Mistake
Redividing standard error by square root of n

no

WHY IT HAPPENED
ACEpath Teacher's Diagnosis

You confused the sample standard deviation (ss or σ\sigma) with the standard error (SESE). Standard error is already equal to s/ns/\sqrt{n}, so dividing an already-stated standard error by n\sqrt{n} double-counts the sample size adjustment and severely underestimates uncertainty.

REASONING PATH
Your ACEpath teacher’s suggested approach
Check Given Quantities Before Computing
  • SE=snSE = \frac{s}{\sqrt{n}}. If a problem directly states 'standard error SESE', use it as given. Only divide by n\sqrt{n} if you are given the sample standard deviation ss.

  • Scan the problem statement specifically for the word 'error' versus 'deviation' before calculating the denominator of your test statistic.

CORRECTED PATH
ACEpath Teacher's Demonstration
  1. Use the given standard error directly: SE=0.35 mg/LSE = 0.35\text{ mg/L}.

  2. Calculate the test statistic: t=0.82−00.35≈2.343t = \frac{0.82 - 0}{0.35} \approx 2.343. Since 2.343>1.6902.343 > 1.690, reject H0H_0.

  3. Compute the 90%90\% confidence interval: 0.82±1.690(0.35)=0.82±0.5915=[0.2285,1.4115] mg/L0.82 \pm 1.690(0.35) = 0.82 \pm 0.5915 = [0.2285, 1.4115]\text{ mg/L}, or [0.23,1.41] mg/L[0.23, 1.41]\text{ mg/L}.

  4. Check the practical threshold: Δcrit=1.00 mg/L\Delta_{\text{crit}} = 1.00\text{ mg/L} falls inside [0.23,1.41] mg/L[0.23, 1.41]\text{ mg/L}. Values ≥1.00 mg/L\ge 1.00\text{ mg/L} are plausible, so practical significance cannot be ruled out.

Final answer

(a) t≈2.343>1.690t \approx 2.343 > 1.690, so reject H0H_0; (b) [0.23,1.41] mg/L[0.23, 1.41]\text{ mg/L}; (c) The data is consistent with an effect ≥1.00 mg/L\ge 1.00\text{ mg/L}, so practical significance cannot be ruled out.

NEXT STEPS
Final Takeaways from Your Teacher
  • Review the structural difference between standard deviation (ss) and standard error (SESE).

  • Re-solve parts (a) through (c) using SE=0.35 mg/LSE = 0.35\text{ mg/L} without dividing by 36\sqrt{36}.

  • Practice distinguishing whether a practical threshold lies inside or outside an estimated confidence interval.

OverviewACEpath Teacher's Diagnosis

02 The student confuses the sample mean being less than the threshold with ruling out practical significance, ignoring the uncertainty given by the confidence interval.

Sample Student Work

Reviewed by ACEpath Teacher
Step-by-Step Review
Walk Through It with Your Teacher
Partially correct
  1. Step 1

    t=0.82−00.35=2.343  ⟹  2.343>1.690t = \frac{0.82 - 0}{0.35} = 2.343 \implies 2.343 > 1.690, so reject H0H_0

  2. Step 2

    CI=0.82±1.690×0.35=0.82±0.5915=[0.2285,1.4115] mg/L\text{CI} = 0.82 \pm 1.690 \times 0.35 = 0.82 \pm 0.5915 = [0.2285, 1.4115]\text{ mg/L}

  3. Step 3

    xˉd=0.82 mg/L<1.00 mg/L\bar{x}_d = 0.82\text{ mg/L} < 1.00\text{ mg/L}. Since the observed reduction is less than 1.00 mg/L1.00\text{ mg/L}, the data are not consistent with reaching Δcrit\Delta_{\text{crit}}, so practical significance is ruled out.

    Correct

    Because Δcrit=1.00 mg/L\Delta_{\text{crit}} = 1.00\text{ mg/L} lies within [0.2285,1.4115] mg/L[0.2285, 1.4115]\text{ mg/L}, true values ≥1.00 mg/L\ge 1.00\text{ mg/L} remain plausible, so practical significance cannot be ruled out.

    You compared only the sample mean xˉd\bar{x}_d to the clinical threshold Δcrit\Delta_{\text{crit}}, which overlooks sampling variability. Plausible values for the true mean difference μd\mu_d range from 0.23 mg/L0.23\text{ mg/L} to 1.41 mg/L1.41\text{ mg/L}. Because values at or above 1.00 mg/L1.00\text{ mg/L} are contained in this interval, the data remain consistent with a clinically meaningful effect, and practical significance cannot be ruled out.

WHAT YOU DID WELL
Compliments from Your Teacher

You correctly computed the test statistic and rejected the null hypothesis in part (a).

You accurately calculated the margin of error and the 90%90\% confidence interval in part (b).

THE ERROR
Where Your ACEpath Teacher Spotted the Mistake
Evaluating practical significance using only a point estimate

You noted that xˉd=0.82<1.00\bar{x}_d = 0.82 < 1.00, but you treated the sample mean as the exact population effect rather than using the interval estimate.

WHY IT HAPPENED
ACEpath Teacher's Diagnosis

Conflating point estimate comparisons with inference under uncertainty: determining whether a threshold is plausible or ruled out requires examining the entire confidence interval, not just whether the sample mean falls below the threshold.

REASONING PATH
Your ACEpath teacher’s suggested approach
Point Estimate vs. Interval Estimate
  • A sample mean xˉ\bar{x} only estimates the true mean μ\mu. Sampling error means the true effect could be higher or lower than xˉ\bar{x}.

  • Always refer to the confidence interval when determining what range of true parameter values is consistent with the data.

Ruling Out Practical Significance
  • An effect threshold Δcrit\Delta_{\text{crit}} is only ruled out if the entire confidence interval falls strictly below Δcrit\Delta_{\text{crit}}.

  • Check whether the upper bound of the confidence interval is less than the practical threshold before concluding practical significance is ruled out.

CORRECTED PATH
ACEpath Teacher's Demonstration
  1. Identify the practical significance threshold Δcrit=1.00 mg/L\Delta_{\text{crit}} = 1.00\text{ mg/L}.

  2. Compare Δcrit\Delta_{\text{crit}} to the 90%90\% confidence interval: 0.23≤1.00≤1.410.23 \le 1.00 \le 1.41.

  3. Conclude that because values at or above 1.00 mg/L1.00\text{ mg/L} are inside the confidence interval, an effect meeting or exceeding Δcrit\Delta_{\text{crit}} is plausible.

Final answer

Because Δcrit=1.00 mg/L\Delta_{\text{crit}} = 1.00\text{ mg/L} lies inside the 90%90\% confidence interval [0.23,1.41] mg/L[0.23, 1.41]\text{ mg/L}, the data are consistent with reaching or exceeding the threshold, and practical significance cannot be ruled out.

NEXT STEPS
Final Takeaways from Your Teacher
  • Practice sketching confidence intervals alongside null and practical thresholds on a number line.

  • Interpret study results by categorizing them as conclusively practical, conclusively not practical, or inconclusive based on where the interval lies relative to the threshold.

OverviewACEpath Teacher's Diagnosis

03 The student uses the critical threshold Δ_crit = 1.00 mg/L instead of 0 in the null hypothesis test for part (a).

Sample Student Work

Reviewed by ACEpath Teacher
Step-by-Step Review
Walk Through It with Your Teacher
Partially correct
  1. Step 1

    t=0.82−1.000.35=−0.180.35≈−0.514  ⟹  −0.514<1.690t = \frac{0.82 - 1.00}{0.35} = \frac{-0.18}{0.35} \approx -0.514 \implies -0.514 < 1.690, so fail to reject H0H_0

    Correct

    t=0.82−00.35≈2.343  ⟹  2.343>1.690t = \frac{0.82 - 0}{0.35} \approx 2.343 \implies 2.343 > 1.690, so reject H0H_0

    The standard null hypothesis for statistical significance tests for any non-zero effect (H0:μd=0H_0: \mu_d = 0). You subtracted the practical threshold Δcrit=1.00\Delta_{\text{crit}} = 1.00 instead of the null value μ0=0\mu_0 = 0 in the numerator.

  2. Step 2

    CI=0.82±1.690×0.35  ⟹  CI=0.82±0.5915  ⟹  CI=[0.2285,1.4115] mg/L\text{CI} = 0.82 \pm 1.690 \times 0.35 \implies \text{CI} = 0.82 \pm 0.5915 \implies \text{CI} = [0.2285, 1.4115]\text{ mg/L}

  3. Step 3

    The confidence interval [0.2285,1.4115][0.2285, 1.4115] contains 1.00 mg/L1.00\text{ mg/L}. The data are consistent with reaching Δcrit=1.00 mg/L\Delta_{\text{crit}} = 1.00\text{ mg/L}, so practical significance cannot be ruled out.

WHAT YOU DID WELL
Compliments from Your Teacher

You correctly calculated the margin of error and the exact boundaries of the 90% confidence interval.

You demonstrated a solid understanding of how confidence intervals evaluate practical significance relative to a threshold value.

THE ERROR
Where Your ACEpath Teacher Spotted the Mistake
Conflating Statistical and Practical Null Baselines

No, this was a conceptual confusion between the null hypothesis value for statistical significance and the clinical benchmark for practical significance.

WHY IT HAPPENED
ACEpath Teacher's Diagnosis

You substituted the real-world threshold of practical importance (Δcrit=1.00\Delta_{\text{crit}} = 1.00) into the test statistic formula where the null value μ0=0\mu_0 = 0 belongs. Statistical significance tests whether an effect differs from zero chance variation, whereas practical significance evaluates whether the interval covers a meaningful target value.

REASONING PATH
Your ACEpath teacher’s suggested approach
Statistical Significance vs. Practical Significance
  • Statistical significance tests H0:μ=0H_0: \mu = 0 (evaluating if any effect exists beyond random noise), whereas practical significance compares the estimated interval to an external threshold Δcrit\Delta_{\text{crit}}.

  • Always set the null value in t=xˉ−μ0SEt = \frac{\bar{x} - \mu_0}{SE} to the value specified in H0H_0 (typically μ0=0\mu_0 = 0). Save threshold comparisons for interval estimation.

CORRECTED PATH
ACEpath Teacher's Demonstration
  1. Set μ0=0\mu_0 = 0 according to H0:μd=0H_0: \mu_d = 0.

  2. Compute the test statistic: t=0.82−00.35=0.820.35≈2.343t = \frac{0.82 - 0}{0.35} = \frac{0.82}{0.35} \approx 2.343.

  3. Compare tt to the critical value: because 2.343>1.6902.343 > 1.690, reject H0H_0 at α=0.05\alpha = 0.05.

Final answer

(a) t≈2.343>1.690t \approx 2.343 > 1.690, reject H0H_0; (b) [0.2285,1.4115] mg/L[0.2285, 1.4115]\text{ mg/L}; (c) Practical significance cannot be ruled out because 1.00 mg/L1.00\text{ mg/L} lies within the interval.

NEXT STEPS
Final Takeaways from Your Teacher
  • Practice identifying the baseline parameter μ0\mu_0 explicitly stated in the null hypothesis before writing the test statistic formula.

  • Use confidence intervals rather than the baseline tt-statistic to compare sample findings against non-zero practical or clinical benchmarks.

Lock it in

Similar question

01

A researcher tests a new cognitive training method on n=49n = 49 participants. The sample mean improvement score is xˉ=4.2 points\bar{x} = 4.2\text{ points} with a reported standard error of SE=1.4 pointsSE = 1.4\text{ points}. Construct an approximate 95%95\% confidence interval for the true mean improvement using z∗=1.96z^* = 1.96, and state whether an improvement of zero is ruled out.

Answer

Margin of error=1.96×1.4=2.744 points\text{Margin of error} = 1.96 \times 1.4 = 2.744\text{ points}. The 95%95\% confidence interval is 4.2±2.744=[1.456,6.944] points4.2 \pm 2.744 = [1.456, 6.944]\text{ points}. Because zero is strictly outside this interval, an improvement of zero is ruled out (statistically significant).

02

An educational intervention achieves a mean reading score increase of xˉd=3.8 points\bar{x}_d = 3.8\text{ points} with sample standard deviation s=12.0 pointss = 12.0\text{ points} in a sample of n=64n = 64 students. A school district considers an improvement practically significant only if μd≥5.0 points\mu_d \ge 5.0\text{ points}. Using z∗=1.645z^* = 1.645 for a 90%90\% confidence interval, calculate the interval and decide whether practical significance can be ruled out.

Answer

First, compute standard error: SE=sn=12.064=1.5 pointsSE = \frac{s}{\sqrt{n}} = \frac{12.0}{\sqrt{64}} = 1.5\text{ points}. Margin of error is 1.645×1.5=2.4675 points1.645 \times 1.5 = 2.4675\text{ points}. The 90%90\% confidence interval is 3.8±2.4675=[1.3325,6.2675] points3.8 \pm 2.4675 = [1.3325, 6.2675]\text{ points}. Because the threshold 5.0 points5.0\text{ points} lies inside the interval, practical significance cannot be ruled out.

03

In a weight-loss trial with n=100n = 100, the mean loss is xˉ=2.1 kg\bar{x} = 2.1\text{ kg} with standard error SE=0.40 kgSE = 0.40\text{ kg}. Doctors consider a reduction practically significant only if μ≥3.5 kg\mu \ge 3.5\text{ kg}. (1) Compute the 95%95\% confidence interval using t∗=1.984t^* = 1.984. (2) Is the result statistically significant at α=0.05\alpha = 0.05 (testing H0:μ=0H_0: \mu = 0)? (3) Can practical significance be ruled out?

Answer

(1) CI=2.1±(1.984)(0.40)=2.1±0.7936=[1.3064,2.8936] kg\text{CI} = 2.1 \pm (1.984)(0.40) = 2.1 \pm 0.7936 = [1.3064, 2.8936]\text{ kg}, or [1.31,2.89] kg[1.31, 2.89]\text{ kg}. (2) Yes, it is statistically significant because the entire interval is strictly above 00. (3) Yes, practical significance can be ruled out because the upper limit (2.89 kg2.89\text{ kg}) is strictly below the clinical threshold of 3.5 kg3.5\text{ kg}.

Loading workspace…