core practice · system design prompt
Design exercise: Inference Hardware
GPUs, TPUs, NPUs, CPUs, memory bandwidth, interconnects, topology, precision support, and deployment fit. Implement designOutline() returning non-empty values for: modelFit, memoryBandwidth, topology. Each value must name a concrete mechanism or decision.
The mechanism
Inference hardware is a memory-and-interconnect system around matrix engines. Choose by model fit, precision, bandwidth, topology, power, software support, and workload shape.
Expected evidence
A design outline with modelFit, memoryBandwidth, topology plus an explicit failure mode or trade-off.
Try this drill in the practice workspace →
Hints when you need them
- Name the failure mode before selecting the mechanism.
Check the sources
- TPU v4: An Optically Reconfigurable Supercomputer for Machine Learning
- Stanford Lecture 15 — Efficient Methods and Hardware for Deep Learning (Song Han)
- In-Datacenter Performance Analysis of a Tensor Processing Unit (ISCA '17)
- Stanford CS 217 — Hardware Accelerators for Machine Learning
- NVIDIA Hopper Architecture