Mathematics · core
Sampling & the Central Limit Theorem
Sampling distributions, standard error of the mean, and why averages become Normal as n grows.
Mental model
Your data is one draw from a bigger game. The CLT says sample means from many independent pieces look Normal even when the underlying data is not — which is why confidence intervals and z-tests work on proportions and averages.
How to study Sampling & the Central Limit Theorem
Begin by restating the mental model in your own words, then connect it to a concrete system you have built or operated. Name the mechanism, the constraint it addresses, and the trade-off it introduces. Use Seeing Theory — Probability Distributions, StatQuest — Central Limit Theorem to check details, but close the source before writing your explanation. Retrieval is the learning step; rereading is only preparation.
Next, compare Sampling & the Central Limit Theorem with Estimation & Confidence Intervals, Classical Distributions, A/B Testing for Engineers. Ask what changes in correctness, latency, resource use, operability, and failure recovery. Complete CLT and required sample size, Standard error vs standard deviation and preserve the command, input, output, and one failed attempt as evidence. Finish by explaining the idea without jargon to someone who has not studied the track.
Proof of understanding
- Explain the mechanism from first principles and identify the state it reads or changes.
- Give one situation where the concept is the right choice and one where it is not.
- Predict a realistic failure mode before running the drill, then compare the prediction with evidence.
- Connect the result to a roadmap or build artifact instead of treating the concept as isolated trivia.
Where it matters
Poll margins of error, experiment power calculations, simulation validation.
Common mistakes
- Applying CLT with n=5 and calling it rigorous
- Treating the sample as the population
- Ignoring dependence across samples (e.g. time-series)
Learn from primary sources
Practice and explain it back
CLT and required sample size
A/B test on conversion rate: baseline 5%, MDE 1pp (to 6%). Rough rule: need ~16p(1−p)/δ² samples per arm for Normal approx (p=0.05, δ=0.01). Estimate n per arm. Why is n quadratic in 1/δ?
Expected evidence: n ≈ 16×0.05×0.95/0.0001 ≈ 7,600 per arm. Halving MDE quadruples required n — detecting small lifts is expensive.
Open the interactive drill →Standard error vs standard deviation
n=100 samples, sample mean x̄=50, sample SD s=10. Compute SE = s/√n and a 95% CI for the population mean (use z=1.96). Why is SE smaller than s?
Expected evidence: SE=10/10=1; CI≈[48.04, 51.96]. SE describes uncertainty of the mean; s describes spread of individual points.
Open the interactive drill →Review prompts
- What does the Central Limit Theorem say about sample means?
- Standard error vs standard deviation?
Build evidence
Analyze a fake trading signal
Given (or generate) a spurious 'alpha' signal on synthetic or historical noise, show why it fails: multiple testing, in-sample overfit, or non-stationarity.
- Reproduce the 'great' in-sample metric
- Show out-of-sample or holdout collapse
- Name the statistical mistake (p-hacking, data snooping, regime shift, etc.)