advanced practice · coding problem
NDCG@k
Implement ndcgAtK(relevances, k) where relevances are graded gains in RANKED order. DCG = sum over i of rel_i / log2(i + 2) for i from 0. IDCG uses the same relevances sorted descending. Return DCG/IDCG rounded to 4 decimals, or 0 when IDCG is 0.
The mechanism
Ranking quality is measured by NDCG, which depends on sort order and therefore has zero gradient almost everywhere — so learning-to-rank is largely the story of optimising something else that correlates. Pointwise regression ignores that only relative order matters; pairwise methods learn 'a should outrank b' and get much closer; LambdaMART's trick is to weight each pair by how much swapping it would change NDCG, smuggling the real metric into the gradient without ever differentiating it.
Expected evidence
ndcgAtK([3,2,3,0,1,2],6) -> 0.9608
Try this drill in the practice workspace →
Hints when you need them
- The discount is log2(rank+1) with ranks starting at 1, i.e. log2(i+2) for zero-based i.
- IDCG is the best achievable ordering of the SAME relevance values, truncated to k as well.
Watch for these mistakes
- Training pointwise on graded labels and being surprised the ordering is poor
- Evaluating on a metric the loss never optimised, then tuning the loss to the metric by accident
- Ignoring query-level grouping, so pairs are formed across unrelated queries
- Training on clicks as if they were relevance labels, without correcting for position bias