Skip to content

Sheaf re-import completeness + S3 SigV4 fix (v0.2.2)#74

Merged
SiteRelEnby merged 3 commits into
mainfrom
import-completeness
May 24, 2026
Merged

Sheaf re-import completeness + S3 SigV4 fix (v0.2.2)#74
SiteRelEnby merged 3 commits into
mainfrom
import-completeness

Conversation

@SiteRelEnby
Copy link
Copy Markdown
Contributor

Summary

The Sheaf-to-Sheaf re-import was only consuming system / members / fronts / groups / tags / custom fields, so journals (and their edit history), board messages, polls, reminders, and the notification config silently vanished on re-import even though the export had always carried them. This makes the importer round-trip the full export, adds per-section selectors, and fixes the KMS export-download bug found while testing. Tagged v0.2.2.

Import completeness

  • New importer sections for journals, content revisions (member-bio + journal-entry history), messages, polls (options/votes/audit log), notification config (watch tokens + channels + group/member rules), and reminders. All cross-references are remapped onto the freshly minted IDs (revision targets, poll option/vote refs, channel rules, reminder channel + scope members).
  • Per-section import selectors (journals, messages, polls, reminders, notifications) added to the options schema, runner, preview counts, and the import UI. Reminders attach to a channel, so they ride the notifications toggle.
  • Restored-reference decisions:
    • Journal/revision authorship re-points at the importing user; the poll audit-log actor is nulled (old-instance account UUIDs are meaningless here).
    • Notification channels land in pending_registration so nothing dispatches to external recipients until the owner re-activates. Webhook secrets and recipient subscriptions are not carried by the export and are not reconstructed.
    • delete_confirmation is intentionally not restored (it would otherwise lock destructive actions on an account without the matching TOTP enrolment). Safety toggles, grace period, and retention caps are restored.

S3 SigV4 fix

  • Export download was failing with "requests specifying Server Side Encryption with AWS KMS managed keys require AWS Signature Version 4". S3 only serves a presigned GET for an SSE-KMS object (incl. a bucket-default KMS policy) when the URL is SigV4-signed; the boto3 clients were not pinning a signature version and could fall back to SigV2. Both the export-artefact and image storage clients now pin s3v4 (harmless for non-KMS buckets and MinIO).
  • Note: this can only be verified against the KMS-encrypted export bucket; MinIO will not reproduce it.

Release

  • Bumped to v0.2.2 (pyproject + web package.json), changelog entry added, and corrected the web lockfile version which had drifted from package.json since v0.2.0.

Tests

  • Full hand-crafted-export round-trip exercising every section and its cross-references, plus a section-toggle test (proving deselection works, and that member-bio revisions survive journals=off while journal revisions do not).
  • 46 passed across the import suite; ruff / tsc / eslint clean.

Migrations

None. The importer writes into existing tables.

…re-import

The Sheaf importer only consumed system/members/fronts/groups/tags/
custom_fields, so journals, content revisions, messages, polls,
reminders and the notification config (watch tokens + channels + rules)
silently vanished on re-import even though the exporter emitted them.

Add importer sections for each, in dependency order, remapping old
export IDs to the freshly minted ones (bio/journal revision targets,
poll option/vote refs, channel group/member rules, reminder channel and
scope members). Add per-section import selectors (journals, messages,
polls, reminders, notifications) to the options schema, runner, preview
counts and the import UI.

Notes on restored references:
- journal/revision authorship re-points at the importing user; poll
  audit-log actor is nulled (old-instance account UUIDs are meaningless
  here).
- notification channels land pending_registration: recipient
  subscriptions, activation hashes and webhook secrets are omitted by
  the exporter and not reconstructed, so nothing dispatches until the
  owner re-activates.
- delete_confirmation is deliberately not restored to avoid locking
  destructive actions on an account without the matching TOTP enrolment.
- reminders attach to a channel (NOT NULL FK), so they ride the
  notifications toggle and are skipped with a warning if their channel
  didn't import.
S3 only serves a presigned GET for an SSE-KMS-encrypted object (incl.
objects covered by a bucket-default KMS encryption policy) when the URL
is signed with SigV4. The boto3 clients weren't pinning a signature
version, so presigning could fall back to SigV2 and S3 rejected the
download with "requests specifying Server Side Encryption with AWS KMS
managed keys require AWS Signature Version 4".

Pin signature_version=s3v4 on both the export-artefact client and the
image storage client. Harmless for non-KMS buckets and MinIO, which
already speak SigV4.
Import completeness (journals, messages, polls, reminders, notification
config round-trip on re-import) and the S3 SigV4 pin for KMS-encrypted
export downloads. Also corrects the web lockfile version, which had
drifted from package.json since v0.2.0.
@SiteRelEnby SiteRelEnby merged commit d4d18bf into main May 24, 2026
4 checks passed
@SiteRelEnby SiteRelEnby deleted the import-completeness branch May 24, 2026 15:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant