Skip to content

Reimplement: core: Store system tables in commitlog #4800

@clockwork-labs-bot

Description

@clockwork-labs-bot

This issue tracks reimplementation of the work from stale PR #1535, which is being closed because it is too out of date to merge directly.

Original PR: https://github.com/clockworklabs/SpacetimeDB/pull/1535
Original author: @kim
Original branch: `kim/st-in-commitlog`
Base branch: `master`

## Original PR summary

Changes the bootstrap sequence, such that the initial system table

schemas and contents are stored in the commitlog.

Opening a database proceeds as follows:

  • If a snapshot exists, restore it and replay the history suffix
  • Otherwise:
    • Create the "seed tables" st_table and st_columns in-memory
    • Replay the history
      • Whenever a table in the reserved system table range is found,
        reset the system table schema and layout information in the
        page manager, so as to repair incomplete schema information.
    • If the history was empty:
      • Create the rest of the system tables (i.e. sans seed tables) in a
        transaction.
      • Commit and persist that transaction.

Alternatives Considered

The already brittle bootrapping process is now even more brittle. In particular
the various incantations of repair / rebuild routines is hard to follow and easy
to break.

It is also unfortunate that the commitlog will be non-empty in case the
initialization from the user-supplied module fails.

As an alternative, a snapshot could be taken right after the database was fully
initialized, incl. the module. Instead of hashes, this snapshot could contain
their preimages and be inlined into the commitlog.

Restoring a database from such a snapshot requires only the statically known
schemas of the seed tables and a single repair pass.

API and ABI breaking changes

It will not be possible to open a database from a commitlog created by an older
version, because bootstrapping the system tables will only occur if the history
is empty.

Expected complexity level and risk

5 - database bootstrap is highly fragile

## Follow-up

- Reimplement this change in a fresh PR against current `master`.
- Carry forward any still-relevant context from the original PR discussion and review.
- Link the new implementation PR back to the original stale PR for historical context.

Metadata

Metadata

Assignees

Labels

abi-breakA PR that makes an ABI breaking changereassessingTaking a step back to reconsider the problem/scope

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions