Skip to content

Add upgrade tests for persisted serde format (Event, WorkItem, KvEntry) #14

Description

@tjgreen42

Problem

The provider persists several duroxide Rust types to PostgreSQL via serde_json, but there is no test that catches an incompatible change to that on-disk format. A change to these types (or their serde derives), or in principle a change in serialization behavior pulled in via a dependency bump, could silently break deserialization of already-persisted rows — i.e. existing instances fail to replay after an upgrade — and nothing in CI would flag it.

What is actually persisted (all in src/provider.rs)

Type Storage Notes
duroxide::Event (Vec<Event> / single) history.event_data orchestration history; long-lived, must deserialize across many versions
duroxide::providers::WorkItem worker_queue / orchestrator_queue payloads (incl. completion) wide enum (ActivityExecute/Completed/Failed, StartOrchestration, TimerFired, ExternalRaised, CancelInstance, ContinueAsNew, QueueMessage, …); the largest format surface
duroxide::providers::KvEntry kv_store KV snapshot

Serialize sites: serde_json::to_string/to_value around L1134, L1157/1164, L1419, L1467, L1676, L1906.
Deserialize sites: serde_json::from_value::<Vec<Event>> L1001, Vec<WorkItem> L1016, KvEntry L1024, Event L1384/2029/2256, WorkItem L1576.

Why the contract lives here, not in duroxide

duroxide core knows nothing about its providers or whether/how its types are persisted — its serde representation is, from its own perspective, an internal detail it may change. The durability contract ("rows written by an older version must still deserialize") is created by this provider's choice to serde_json these types into durable columns. So the compatibility guard belongs at the provider boundary. A nice property of putting it here: a breaking duroxide change can only surface via this repo's duroxide dependency bump, so a provider-side compat test fires at exactly the right moment.

Existing coverage (insufficient)

  • corrupt_instance_history writes {"garbage": true} to verify graceful failure, not format compatibility.
  • version-related tests cover orchestration/code versioning and capability/pinned-version filtering — unrelated to serde wire format.
  • No golden/fixture of an older serialized blob is checked in; nothing asserts current code can read prior-version Event / WorkItem / KvEntry.

Proposed work

  1. Golden-fixture round-trip tests: check in serialized fixtures (representative Event, every WorkItem variant, KvEntry) captured under prior versions, and assert the current provider deserializes them. Regenerate-on-purpose, fail-on-accidental-change.
  2. Fixture replay / in-flight straddle test: persist history + queue rows under an older build (or fixtures), then drive them to completion with the current binary — covers both long-lived history and short-lived in-flight queue payloads across a binary swap.
  3. Wire these into CI so a duroxide dependency bump that changes any persisted type's representation fails loudly.

Context: surfaced while reviewing pg_durable's upgrade testing, whose Scenario A explicitly excludes the duroxide schema and delegates this format-compat concern to the provider.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions