Databases & Storage · core
Buffer Pool
The database's own page cache — why it does not simply trust the OS.
Mental model
A database manages its own page cache because it knows things the OS cannot: which pages a query will touch next, which are dirty and pinned by an open transaction, and that a sequential scan should not evict the working set. That last point is why plain LRU is wrong for a database — one big scan touches every page once and flushes everything useful, so real systems use LRU-K or clock-sweep variants that require a second reference before promoting a page.
How to study Buffer Pool
Begin by restating the mental model in your own words, then connect it to a concrete system you have built or operated. Name the mechanism, the constraint it addresses, and the trade-off it introduces. Use the linked roadmap and primary implementation references to check details, but close the source before writing your explanation. Retrieval is the learning step; rereading is only preparation.
Next, compare Buffer Pool with Compaction, Isolation Levels & MVCC. Ask what changes in correctness, latency, resource use, operability, and failure recovery. Complete Why LRU-K survives a scan and preserve the command, input, output, and one failed attempt as evidence. Finish by explaining the idea without jargon to someone who has not studied the track.
Proof of understanding
- Explain the mechanism from first principles and identify the state it reads or changes.
- Give one situation where the concept is the right choice and one where it is not.
- Predict a realistic failure mode before running the drill, then compare the prediction with evidence.
- Connect the result to a roadmap or build artifact instead of treating the concept as isolated trivia.
Where it matters
The first thing to size on any database server, and the explanation for a cache hit rate that collapses after a nightly batch job.
Common mistakes
- Assuming the OS page cache is equivalent — it cannot honour pin counts or write ordering for recovery
- Plain LRU, which a single sequential scan pollutes end to end
- Sizing the pool to fill RAM and leaving nothing for sorts, hash joins, and the OS
- Ignoring that a dirty page cannot be evicted until its log record is durable
Learn from primary sources
Use the linked roadmap context and practice prompt.
Practice and explain it back
Why LRU-K survives a scan
Implement simulate(capacity, accesses, policy) returning the number of hits. policy is 'lru' or 'lru2'. Under 'lru2' a page is only promoted into the resident set on its SECOND access — a page seen once goes to a probationary list and is evicted before any twice-seen page. Evict the least-recently-used within each tier, probationary first.
Expected evidence: A hot working set plus a long one-shot scan: lru2 keeps the working set, lru loses it.
Open the interactive drill →Review prompts
- A nightly report scans a large table and the morning cache hit rate collapses. What did plain LRU do, and how does LRU-K prevent it?
Build evidence
Use a roadmap capstone to turn this concept into working evidence.