core practice · system design prompt
Design exercise: Model Pre-training
Data mixtures, next-token objectives, scaling laws, distributed training, checkpoints, and training stability. Implement designOutline() returning non-empty values for: dataMixture, trainingLoop, scalingPlan. Each value must name a concrete mechanism or decision.
The mechanism
Pre-training converts data and compute into general capability. The core loop is data sampling, forward/backward passes, optimizer updates, checkpointing, and continuous loss/quality diagnosis.
Expected evidence
A design outline with dataMixture, trainingLoop, scalingPlan plus an explicit failure mode or trade-off.
Try this drill in the practice workspace →
Hints when you need them
- Name the failure mode before selecting the mechanism.