Databases & Storage · advanced
Compaction
Merging sorted runs to reclaim space and bound read amplification.
Mental model
An LSM tree keeps producing sorted files; without compaction reads get slower and deleted data lingers. Compaction merges files, drops tombstones, and bounds read amplification — at the cost of write amplification and IO spikes.
How to study Compaction
Begin by restating the mental model in your own words, then connect it to a concrete system you have built or operated. Name the mechanism, the constraint it addresses, and the trade-off it introduces. Use RocksDB Compaction to check details, but close the source before writing your explanation. Retrieval is the learning step; rereading is only preparation.
Next, compare Compaction with Write-Ahead Log. Ask what changes in correctness, latency, resource use, operability, and failure recovery. Complete LSM compaction levels and preserve the command, input, output, and one failed attempt as evidence. Finish by explaining the idea without jargon to someone who has not studied the track.
Proof of understanding
- Explain the mechanism from first principles and identify the state it reads or changes.
- Give one situation where the concept is the right choice and one where it is not.
- Predict a realistic failure mode before running the drill, then compare the prediction with evidence.
- Connect the result to a roadmap or build artifact instead of treating the concept as isolated trivia.
Where it matters
RocksDB/Cassandra compaction strategies; segment merges in search engines.
Common mistakes
- Ignoring the write-amplification cost of compaction
- Not understanding leveled vs size-tiered strategies
- Compaction storms that starve foreground traffic
Learn from primary sources
- RocksDB Compaction (doc)
Practice and explain it back
LSM compaction levels
L0 has 4 SSTables, threshold 4 triggers compaction to L1. Why not compact every write?
Expected evidence: Compaction is expensive; batching amortizes I/O; reads use bloom + levels.
Open the interactive drill →Review prompts
- What does compaction cost, and what does it buy?
Build evidence
Toy LSM tree
A minimal LSM storage engine: memtable, SSTables, compaction.
- Writes buffered in a memtable and flushed to immutable sorted files
- Reads merge across files; deletes are tombstones
- A simple compaction pass