storage-format_skill

Overview

This skill documents the on-disk storage format used in TursoDB, compatible with SQLite. It explains database file layout, page types, B-tree structures, record and cell formats, overflow handling, and the freelist. It also points to the Rust modules that implement these structures in Turso.

How this skill works

It inspects the first 100-byte database header and parses big-endian fields such as page size, page count, freelist pointers, and encoding. It walks B-tree pages (interior and leaf) to interpret cells and row payloads, follows overflow chains for large payloads, and reads freelist trunk/leaf lists to identify unused pages. The skill ties these concepts to the Turso Rust modules that implement pager, buffer pool, and B-tree logic.

When to use it

• When you need to debug or inspect a Turso/SQLite file at the page level.
• When implementing or modifying on-disk storage behavior in Turso.
• When diagnosing corruption, freelist issues, or unexpected page counts.
• When tracing how a row is stored across leaf/interior and overflow pages.
• When optimizing page size or cache behavior for embedded deployments.

Best practices

• Always treat multi-byte integers in the header and pages as big-endian.
• Start analysis from page 1 (schema root) — it contains the database header and sqlite_schema root.
• Follow overflow chains by reading the 4-byte next_page pointer at the start of each overflow page.
• Use PRAGMA integrity_check and page_count to verify file consistency before manual edits.
• Prefer reading Turso’s core/storage modules (ont-disk, btree, pager, buffer_pool) to see canonical parsing and invariants.

Example use cases

• Locate and extract a row whose payload spans overflow pages by following overflow pointers and reassembling data segments.
• Identify freed pages via freelist trunk entries to estimate reclaimable space or detect freelist corruption.
• Confirm page size and encoding from the header to ensure client compatibility with database file.
• Trace why a specific rowid is stored on a particular leaf by walking interior page pointers to the leaf page.
• Validate an on-disk change by running PRAGMA integrity_check and comparing page counts before/after.