core practice · coding problem
Type I vs Type II errors
A security patch test: H₀ = 'no change in error rate'. You fail to reject H₀, but errors actually rose 0.5%. (a) Which error type? (b) If α=0.05 and power=0.8, what does each number mean in one sentence?
The mechanism
Hypothesis testing asks: 'how surprising is this data if nothing real changed?' A p-value is not the probability the null is true — it's the probability of seeing data this extreme under the null. Low p → reject the null, but 'not significant' is not proof of no effect.
Expected evidence
(a) Type II error (false negative) — missed a real regression. α=5% false-alarm rate if H₀ true; power=80% chance to detect a real effect of the assumed size.
Try this drill in the practice workspace →
Hints when you need them
- Type I: reject true H₀. Type II: fail to reject false H₀.
- Power = 1 − P(Type II) for a specific alternative.
Watch for these mistakes
- Treating p < 0.05 as proof the effect is real
- Running many tests without correction (p-hacking)
- Confusing statistical significance with practical significance