← All work
Systems/Storage engine/2024 — 2025/Built

MiniDB

Paged storage engine with slotted pages, a buffer pool, B+ tree indexes, write-ahead logging and crash recovery — then a parse → plan → execute query path on top.

C++B+ treesWALBuffer pool
Personal systems work
01

The problem

Durability and isolation are easy to use and hard to explain. I wrote a storage engine from the page layout up so that the reasons behind WAL ordering, buffer pool eviction and B+ tree splits came from my own bugs rather than from a diagram.

02

Who uses it

2 groups

Me, as a backend engineer

Understand what an fsync, an index choice and an isolation level actually cost underneath.

Anyone reading the code

A small engine where the whole path from SQL text to page write is readable in one sitting.

03

Workflow

6 stages
01Parse SQL
02Plan
03Execute
04Buffer pool fetch
05WAL append then page write
06Recover on restart
01 · ParseTriggerSQL statement submittedSystemTokenizer and recursive-descent parserOutputAn abstract syntax tree, or a syntax error with a position
02 · PlanTriggerStatement parsed and validated against the catalogSystemPlanner choosing sequential scan or index scanOutputAn operator tree
03 · ExecuteTriggerPlan readySystemOperator tree pulling tuples through the access layerOutputResult rows, or a mutation staged against pages
04 · PageTriggerAccess layer needs a pageSystemBuffer pool: hit, or evict a victim and read from diskOutputA pinned frame; dirty victims flushed under WAL ordering
05 · LogTriggerA page is about to be modifiedSystemWrite-ahead log appended and flushed before the page writeOutputA durable record of the change ahead of the change itself
06 · RecoverTriggerProcess starts after a crashSystemLog scan, redo committed work, undo uncommitted workOutputA consistent database with no partial transactions
04

How it was built

6 parts

Slotted pages

Fixed-size pages with a header, a slot directory growing from the front and tuple data growing from the back. Variable-length rows fit without rewriting the page, and deletes leave a reusable slot — the layout that makes everything above it possible.

Buffer pool

A fixed set of frames with pin counts, dirty flags and an eviction policy. Nothing pinned can be evicted; a dirty victim is flushed first, and always after its log records. This is where the WAL rule becomes concrete rather than theoretical.

B+ tree indexes

Keys in internal nodes for routing, values in leaves, leaves chained for range scans. Splits propagate upward and can grow a new root. Getting split and redistribution correct under a page-sized node was the part that took the longest.

Write-ahead logging

Every page modification appends a log record, and the log is flushed before the page write. Commit is a log record reaching disk — not the data page. That ordering is the entire durability guarantee.

Crash recovery, tested by crashing

Recovery scans the log, redoes committed transactions and undoes uncommitted ones. I tested it by killing the process mid-write, repeatedly, at different points — which is how I found the ordering bugs that a clean shutdown never exposes.

Query path

A tokenizer and recursive-descent parser produce an AST, a planner picks between a sequential scan and an index scan, and an operator tree executes it. Small, but complete from SQL text to page write.

05

What it is built with

Language

  • C++
  • RAII
  • Manual memory layout

Storage

  • Slotted pages
  • Buffer pool
  • B+ trees
  • Catalog

Durability

  • Write-ahead log
  • Redo / undo recovery
  • fsync ordering

Query

  • Tokenizer
  • Recursive-descent parser
  • Planner
  • Operators
06

Outcome