Mathematics · core
Queueing Theory
Little's law and the utilisation curve — why latency explodes before a system runs out of capacity.
Mental model
Little's law (L = λW) is an accounting identity, not a model: it holds for any stable system regardless of arrival or service distribution, which is why it works on a queue, a thread pool, or a whole company's hiring pipeline. The result that actually changes designs is the utilisation curve — waiting time scales as 1/(1−ρ), so a server at 50% has a small queue and one at 95% has twenty times that. Systems do not fall over when utilisation hits 100%; they fall over well before, because latency goes vertical as ρ approaches 1.
How to study Queueing Theory
Begin by restating the mental model in your own words, then connect it to a concrete system you have built or operated. Name the mechanism, the constraint it addresses, and the trade-off it introduces. Use the linked roadmap and primary implementation references to check details, but close the source before writing your explanation. Retrieval is the learning step; rereading is only preparation.
Next, compare Queueing Theory with Capacity Estimation, Rate Limiting. Ask what changes in correctness, latency, resource use, operability, and failure recovery. Complete Latency against utilisation and preserve the command, input, output, and one failed attempt as evidence. Finish by explaining the idea without jargon to someone who has not studied the track.
Proof of understanding
- Explain the mechanism from first principles and identify the state it reads or changes.
- Give one situation where the concept is the right choice and one where it is not.
- Predict a realistic failure mode before running the drill, then compare the prediction with evidence.
- Connect the result to a roadmap or build artifact instead of treating the concept as isolated trivia.
Where it matters
Thread-pool and connection-pool sizing, autoscaling thresholds, and the reason SRE targets utilisation well below saturation.
Common mistakes
- Sizing for average utilisation and being surprised by tail latency — the curve is nonlinear, so the mean hides it
- Applying Little's law to an unstable system where the queue grows without bound; it requires λ < µ
- Adding servers to fix latency caused by variance rather than by load
- Reading 70% CPU as '30% headroom' when the queueing delay at 70% is already several times the service time
Learn from primary sources
Use the linked roadmap context and practice prompt.
Practice and explain it back
Latency against utilisation
For an M/M/1 queue implement mm1(lambdaRate, muRate) returning { rho, avgInSystem, avgWaitSeconds } where rho = λ/µ, L = ρ/(1−ρ), W = 1/(µ−λ). Return null when the queue is unstable (λ >= µ). Round rho to 2 decimals, avgInSystem to 2, avgWaitSeconds to 4.
Expected evidence: mm1(50,100) -> { rho:0.5, avgInSystem:1, avgWaitSeconds:0.02 }; mm1(95,100) -> avgInSystem 19
Open the interactive drill →Review prompts
- A service sits at 70% CPU and someone reports '30% headroom'. Using the utilisation curve, why is that reading wrong?
Build evidence
Use a roadmap capstone to turn this concept into working evidence.