core practice · system design prompt
Design exercise: Model Quantization
Post-training and quantization-aware methods, integer and low-bit formats, calibration, kernels, and quality trade-offs. Implement designOutline() returning non-empty values for: numericFormat, calibration, qualityGate. Each value must name a concrete mechanism or decision.
The mechanism
Quantization stores and computes approximate weights or activations with fewer bits. The serving win is real only when hardware kernels support the format and evals bound quality loss.
Expected evidence
A design outline with numericFormat, calibration, qualityGate plus an explicit failure mode or trade-off.
Try this drill in the practice workspace →
Hints when you need them
- Name the failure mode before selecting the mechanism.
Check the sources
- AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration
- LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale
- SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models
- Democratizing Foundation Models via k-bit Quantization — Tim Dettmers (Stanford MLSys #82)
- GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers
- Hugging Face Transformers — Quantization