Crash recovery by truncating partial tail records

[mairinkdev]

Summary

On open, validate the last segment and find the last fully written, valid record. Truncate any partial or corrupt tail (in file and in memory). When an index exists (from later days), ensure it is aligned with the truncated log (e.g. truncate index entries past the new end).

Scope

• Recovery procedure: open last segment (by base_offset); scan from start or from a safe point, validate each record (checksum, length); record the position/offset of the last valid record; truncate the file to that position.
• Truncate in place: call file.set_len() (or equivalent) to drop bytes after the last valid record.
• If index (.idx) is present: remove or truncate index entries that refer to offsets beyond the new log end so index and log stay consistent.
• Tests: write a few valid records; simulate crash by writing a partial record (e.g. header only, or header + half payload); reopen log; assert recovery truncates tail, next append continues after last valid offset, and scan returns only valid records.

Acceptance criteria

• Reopening after a “crash” (partial write) never exposes partial/corrupt data.
• Log and index (when present) remain consistent after recovery.
• Tests demonstrate recovery with partial header and partial payload scenarios.

Suggested commit

feat: crash recovery by truncating partial tail records