Search & IR · advanced
Learning to Rank
Pointwise, pairwise, and listwise objectives — and why the metric you care about is not differentiable.
Mental model
Ranking quality is measured by NDCG, which depends on sort order and therefore has zero gradient almost everywhere — so learning-to-rank is largely the story of optimising something else that correlates. Pointwise regression ignores that only relative order matters; pairwise methods learn 'a should outrank b' and get much closer; LambdaMART's trick is to weight each pair by how much swapping it would change NDCG, smuggling the real metric into the gradient without ever differentiating it.
How to study Learning to Rank
Begin by restating the mental model in your own words, then connect it to a concrete system you have built or operated. Name the mechanism, the constraint it addresses, and the trade-off it introduces. Use the linked roadmap and primary implementation references to check details, but close the source before writing your explanation. Retrieval is the learning step; rereading is only preparation.
Next, compare Learning to Rank with Reranking, Click Models & Position Bias. Ask what changes in correctness, latency, resource use, operability, and failure recovery. Complete NDCG@k and preserve the command, input, output, and one failed attempt as evidence. Finish by explaining the idea without jargon to someone who has not studied the track.
Proof of understanding
- Explain the mechanism from first principles and identify the state it reads or changes.
- Give one situation where the concept is the right choice and one where it is not.
- Predict a realistic failure mode before running the drill, then compare the prediction with evidence.
- Connect the result to a roadmap or build artifact instead of treating the concept as isolated trivia.
Where it matters
The reranking stage in every production search system, and the reason retrieval and ranking are trained separately.
Common mistakes
- Training pointwise on graded labels and being surprised the ordering is poor
- Evaluating on a metric the loss never optimised, then tuning the loss to the metric by accident
- Ignoring query-level grouping, so pairs are formed across unrelated queries
- Training on clicks as if they were relevance labels, without correcting for position bias
Learn from primary sources
Use the linked roadmap context and practice prompt.
Practice and explain it back
NDCG@k
Implement ndcgAtK(relevances, k) where relevances are graded gains in RANKED order. DCG = sum over i of rel_i / log2(i + 2) for i from 0. IDCG uses the same relevances sorted descending. Return DCG/IDCG rounded to 4 decimals, or 0 when IDCG is 0.
Expected evidence: ndcgAtK([3,2,3,0,1,2],6) -> 0.9608
Open the interactive drill →Review prompts
- NDCG has zero gradient almost everywhere. How does LambdaMART optimise it anyway?
Build evidence
Use a roadmap capstone to turn this concept into working evidence.