core practice · system design prompt
Design exercise: FlashAttention & Attention Kernels
IO-aware tiling, fused kernels, SRAM/HBM movement, numerical stability, and hardware-aware attention. Implement designOutline() returning non-empty values for: memoryTraffic, tiling, numericalStability. Each value must name a concrete mechanism or decision.
The mechanism
FlashAttention is exact attention reorganized around the memory hierarchy. Tiling and fusion reduce expensive HBM traffic without materializing the full attention matrix.
Expected evidence
A design outline with memoryTraffic, tiling, numericalStability plus an explicit failure mode or trade-off.
Try this drill in the practice workspace →
Hints when you need them
- Name the failure mode before selecting the mechanism.