Resources
A clinical trial studies a new anti-inflammatory medication where doctors define a clinically meaningful reduction in serum C-reactive protein (CRP) as an effect size of at least . The baseline standard deviation of CRP is , so a clinically meaningful difference is defined as a reduction of . A randomized experiment on patients yields a mean reduction of with a standard error . (a) Test against at ( for ) and decide whether to reject or fail to reject . (b) Construct an exact two-sided confidence interval for (using ), which corresponds to a one-tailed lower and upper boundary. (c) At the threshold boundary of practical significance, is the data consistent with an effect that meets or exceeds , and can practical significance be ruled out?
Answer and steps
Final answer
(a) . Because (corresponding to ), we reject ; the reduction in CRP is statistically significant. (b) The confidence interval is , yielding , or . (c) Because the critical practical threshold falls entirely inside the confidence interval , practical significance cannot be ruled out; the trial confirms statistical significance, but the true effect size is inconclusive regarding practical significance due to sampling variability.
Steps
Part (a): Hypothesis Test for Statistical Significance
- Formulate the Hypotheses and Test Statistic: We are testing whether the mean reduction in CRP, denoted by , is significantly greater than zero: Given the sample mean reduction and standard error , compute the -statistic:
-
Make the Statistical Decision:
- At a significance level of with degrees of freedom , the critical value for a right-tailed test is .
- Since (which corresponds to a -value of approximately ), we reject .
- Conclusion: The reduction in CRP is statistically significant, meaning the observed reduction is unlikely to have occurred by random sampling chance alone if the drug had zero effect.
Part (b): Construction of the Confidence Interval
- Formula for Confidence Interval: A two-sided confidence interval for the population mean leaves in each tail, directly matching the one-tailed critical value :
- Calculate the Margin of Error and Bounds: Thus, the confidence interval is , or approximately .
Part (c): Evaluating Practical (Clinical) Significance
-
Compare the Confidence Interval to the Practical Threshold:
- Statistical significance tests whether . Because the entire interval lies strictly above , we are confident that the true effect is greater than zero.
- Practical significance requires the effect to be meaningful in the real world—here, a clinically defined threshold of .
-
Interpret the Findings:
- The threshold lies inside the interval .
- Because values (such as or ) are plausible given the data, practical significance cannot be ruled out. The data is consistent with an effect that meets or exceeds the critical threshold.
- However, because values below are also plausible, practical significance is not guaranteed. The trial conclusively demonstrates an effect (statistical significance), but remains inconclusive regarding whether the effect is large enough to be clinically meaningful (practical significance).
Typical errors
01 The student recomputes the standard error by dividing the already-given standard error by √n, inflating the t-statistic.
Sample Student Work
Reviewed by ACEpath TeacherStep-by-Step Review
Walk Through It with Your Teacher- Step 1Correct
The problem already gave the standard error of the mean as . You divided it again by , confusing standard deviation with standard error .
- Step 2Correct
You applied the confidence interval formula correctly, but your interval bounds were affected by using the incorrectly deflated standard error instead of .
- Step 3Correct
Your comparison logic followed logically from your interval, but because the true interval is , it actually contains , meaning practical significance cannot be ruled out.
WHAT YOU DID WELL
Compliments from Your TeacherYou correctly set up the decision rule by comparing the test statistic to .
You used the correct confidence interval structure .
You correctly recognized that whether practical significance can be ruled out depends on whether the critical threshold falls within the confidence interval.
THE ERROR
Where Your ACEpath Teacher Spotted the Mistakeno
WHY IT HAPPENED
ACEpath Teacher's DiagnosisYou confused the sample standard deviation ( or ) with the standard error (). Standard error is already equal to , so dividing an already-stated standard error by double-counts the sample size adjustment and severely underestimates uncertainty.
REASONING PATH
Your ACEpath teacher’s suggested approach- Check Given Quantities Before Computing
. If a problem directly states 'standard error ', use it as given. Only divide by if you are given the sample standard deviation .
Scan the problem statement specifically for the word 'error' versus 'deviation' before calculating the denominator of your test statistic.
CORRECTED PATH
ACEpath Teacher's DemonstrationUse the given standard error directly: .
Calculate the test statistic: . Since , reject .
Compute the confidence interval: , or .
Check the practical threshold: falls inside . Values are plausible, so practical significance cannot be ruled out.
(a) , so reject ; (b) ; (c) The data is consistent with an effect , so practical significance cannot be ruled out.
02 The student confuses the sample mean being less than the threshold with ruling out practical significance, ignoring the uncertainty given by the confidence interval.
Sample Student Work
Reviewed by ACEpath TeacherStep-by-Step Review
Walk Through It with Your Teacher- Step 1
, so reject
- Step 2
- Step 3
. Since the observed reduction is less than , the data are not consistent with reaching , so practical significance is ruled out.
CorrectBecause lies within , true values remain plausible, so practical significance cannot be ruled out.
You compared only the sample mean to the clinical threshold , which overlooks sampling variability. Plausible values for the true mean difference range from to . Because values at or above are contained in this interval, the data remain consistent with a clinically meaningful effect, and practical significance cannot be ruled out.
WHAT YOU DID WELL
Compliments from Your TeacherYou correctly computed the test statistic and rejected the null hypothesis in part (a).
You accurately calculated the margin of error and the confidence interval in part (b).
THE ERROR
Where Your ACEpath Teacher Spotted the MistakeYou noted that , but you treated the sample mean as the exact population effect rather than using the interval estimate.
WHY IT HAPPENED
ACEpath Teacher's DiagnosisConflating point estimate comparisons with inference under uncertainty: determining whether a threshold is plausible or ruled out requires examining the entire confidence interval, not just whether the sample mean falls below the threshold.
REASONING PATH
Your ACEpath teacher’s suggested approach- Point Estimate vs. Interval Estimate
A sample mean only estimates the true mean . Sampling error means the true effect could be higher or lower than .
Always refer to the confidence interval when determining what range of true parameter values is consistent with the data.
- Ruling Out Practical Significance
An effect threshold is only ruled out if the entire confidence interval falls strictly below .
Check whether the upper bound of the confidence interval is less than the practical threshold before concluding practical significance is ruled out.
CORRECTED PATH
ACEpath Teacher's DemonstrationIdentify the practical significance threshold .
Compare to the confidence interval: .
Conclude that because values at or above are inside the confidence interval, an effect meeting or exceeding is plausible.
Because lies inside the confidence interval , the data are consistent with reaching or exceeding the threshold, and practical significance cannot be ruled out.
03 The student uses the critical threshold Δ_crit = 1.00 mg/L instead of 0 in the null hypothesis test for part (a).
Sample Student Work
Reviewed by ACEpath TeacherStep-by-Step Review
Walk Through It with Your Teacher- Step 1
, so fail to reject
Correct, so reject
The standard null hypothesis for statistical significance tests for any non-zero effect (). You subtracted the practical threshold instead of the null value in the numerator.
- Step 2
- Step 3
The confidence interval contains . The data are consistent with reaching , so practical significance cannot be ruled out.
WHAT YOU DID WELL
Compliments from Your TeacherYou correctly calculated the margin of error and the exact boundaries of the 90% confidence interval.
You demonstrated a solid understanding of how confidence intervals evaluate practical significance relative to a threshold value.
THE ERROR
Where Your ACEpath Teacher Spotted the MistakeNo, this was a conceptual confusion between the null hypothesis value for statistical significance and the clinical benchmark for practical significance.
WHY IT HAPPENED
ACEpath Teacher's DiagnosisYou substituted the real-world threshold of practical importance () into the test statistic formula where the null value belongs. Statistical significance tests whether an effect differs from zero chance variation, whereas practical significance evaluates whether the interval covers a meaningful target value.
REASONING PATH
Your ACEpath teacher’s suggested approach- Statistical Significance vs. Practical Significance
Statistical significance tests (evaluating if any effect exists beyond random noise), whereas practical significance compares the estimated interval to an external threshold .
Always set the null value in to the value specified in (typically ). Save threshold comparisons for interval estimation.
CORRECTED PATH
ACEpath Teacher's DemonstrationSet according to .
Compute the test statistic: .
Compare to the critical value: because , reject at .
(a) , reject ; (b) ; (c) Practical significance cannot be ruled out because lies within the interval.
Lock it in
What is the mathematical distinction between standard deviation () and standard error ()?
measures the variability among individual observations, while measures the variability of the sample mean across repeated samples.
Similar question
A researcher tests a new cognitive training method on participants. The sample mean improvement score is with a reported standard error of . Construct an approximate confidence interval for the true mean improvement using , and state whether an improvement of zero is ruled out.
Answer
. The confidence interval is . Because zero is strictly outside this interval, an improvement of zero is ruled out (statistically significant).
An educational intervention achieves a mean reading score increase of with sample standard deviation in a sample of students. A school district considers an improvement practically significant only if . Using for a confidence interval, calculate the interval and decide whether practical significance can be ruled out.
Answer
First, compute standard error: . Margin of error is . The confidence interval is . Because the threshold lies inside the interval, practical significance cannot be ruled out.
In a weight-loss trial with , the mean loss is with standard error . Doctors consider a reduction practically significant only if . (1) Compute the confidence interval using . (2) Is the result statistically significant at (testing )? (3) Can practical significance be ruled out?
Answer
(1) , or . (2) Yes, it is statistically significant because the entire interval is strictly above . (3) Yes, practical significance can be ruled out because the upper limit () is strictly below the clinical threshold of .