core practice · coding problem
One SGD step on a quadratic
Loss L(w)=(w−3)². At w=0 with lr=0.1, compute gradient and one gradient-descent update.
The mechanism
Gradient descent is repeated tiny adjustments. At each step you move the weights a small amount in the direction that lowers the loss. The "learning rate" is how big each step is — too big and you overshoot the answer, too small and training crawls.
Expected evidence
∇L=−6, w₁=0.6.
Try this drill in the practice workspace →
Hints when you need them
- d/dw (w−3)² = 2(w−3).
- w ← w − lr·∇L.