core practice · system design prompt
Design exercise: GPU Utilization
Compute occupancy, memory bandwidth, kernel launch overhead, tensor parallelism, profiling, and saturation. Implement designOutline() returning non-empty values for: profile, bottleneckClass, saturationPlan. Each value must name a concrete mechanism or decision.
The mechanism
GPU utilization is not one percentage. Profile kernel occupancy, memory bandwidth, queue gaps, communication, and batch shape to find whether the workload is compute-, memory-, or launch-bound.
Expected evidence
A design outline with profile, bottleneckClass, saturationPlan plus an explicit failure mode or trade-off.
Try this drill in the practice workspace →
Hints when you need them
- Name the failure mode before selecting the mechanism.
Check the sources
- Stanford CS149 Lecture 7 — GPU Architecture and CUDA Programming
- Efficiently Scaling Transformer Inference
- FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness
- Stanford CS336 — Lecture 5: GPUs (Language Modeling from Scratch, Spring 2025)
- Roofline: An Insightful Visual Performance Model for Multicore Architectures (UCB/EECS-2008-134)
- NVIDIA Nsight Systems Documentation