Skip to content

feat: bundled features — archived-emails all-sources view, ingestion-source details, advanced search, date-format preference#383

Open
kisst wants to merge 63 commits into
LogicLabs-OU:mainfrom
kisst:feat/kisst-fork-bundle
Open

feat: bundled features — archived-emails all-sources view, ingestion-source details, advanced search, date-format preference#383
kisst wants to merge 63 commits into
LogicLabs-OU:mainfrom
kisst:feat/kisst-fork-bundle

Conversation

@kisst
Copy link
Copy Markdown

@kisst kisst commented May 25, 2026

Why a bundle

Started out wanting to test just feat/archived-emails-view-all-sources against my own production archive (214k emails). Once that worked I tried to merge the other in-flight branches one by one and hit conflicts and regressions that only surface when these features run together. Sending them as a bundle so the integration fixes don't get split across separate PRs and reviewers can see the failure modes I had to resolve.

What's in here

Issues found while integrating

  1. ArchivedEmailService.ts uses desc() from drizzle-orm but the import was never added — only the cross-source list endpoint uses it, so it never tripped before the merge. Fix: 5f1b723.
  2. Five frontend merge conflicts (archived-emails list/detail, ingestions, search, EmailThread). Date-format reformats date displays in exactly the files the other branches restructure. Resolved by keeping the new feature code AND wiring formatDateTimeStore through (including patching SearchResults.svelte, which the date-format branch didn't know about). Fix: be80056.
  3. Ingestions table's Oldest, Newest, Last sync columns were hardcoded to toLocaleDateString while Created at already went through formatDateStore. Inconsistent once date-format was in. Fix: 9af3c79.
  4. Reindex and date-backfill workers were added to the codebase but not wired into start:workers in package.json. Reindex jobs sat in waiting forever in my test. Fix: 648170c.
  5. Meilisearch containsFilter experimental feature wasn't enabled. Subject/from CONTAINS filters returned 4xx until I added a client.updateExperimentalFeatures({ containsFilter: true }) call in configureEmailIndex(). Fix: fee3251.

What I tested

  • Built the OSS Docker image, ran the stack against 214k archived emails + 5 ingestion sources
  • Verified each of the four features works end-to-end with real data
  • Triggered a full reindex via POST /v1/admin/reindex (scope=full): orchestrator enqueued 213,723 rows in 428 batches; indexing worker drains them with containsFilter enabled and the new filterable attributes live

Notes for maintainers

  • Happy to split into per-feature PRs if you'd rather review that way — the five integration commits would still need to land somewhere.
  • Migration 0035_lonely_roulette.sql and the Meilisearch settings update are required for advanced-search filtering. The reindex orchestrator in this PR is what populates the new fields on existing data.
  • On an existing populated meilidata volume, settings-update tasks can sit behind queued document tasks. In my testing I had to recreate the volume once to apply the new filterable attributes; the reindex worker handles it cleanly on a fresh index.

kisst added 30 commits May 25, 2026 11:03
Adds vitest.config.ts and one smoke test per package, plus
test/test:watch scripts. Dependency declared in package.json;
install deferred to verification phase.
…bs-OU#372)

Allow archived emails to record that the original Date header was missing
or unparseable, rather than silently substituting the ingestion timestamp.
Adds OriginalDateSource union and ArchivedEmail.originalDateSource.
Adds popover, calendar, range-calendar, date-picker, command,
combobox, toggle-group, tabs, sheet, and tooltip components.
No business-logic consumers yet; primitives only.
The Bulgarian locale ships in packages/frontend/src/lib/translations/bg/
and the settings page already lists it as a selectable language, but the
type union omitted it, causing a svelte-check error on the settings page.

Pre-existing on main; surfacing it during this PR's svelte-check runs.
kisst added 22 commits May 25, 2026 13:44
@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 25, 2026

All contributors have signed the CLA ✍️ ✅
Posted by the CLA Assistant Lite bot.

@kisst
Copy link
Copy Markdown
Author

kisst commented May 25, 2026

I have read the CLA Document and I hereby sign the CLA

@kisst kisst force-pushed the feat/kisst-fork-bundle branch from fee3251 to ea371ca Compare May 25, 2026 15:58
@kisst kisst marked this pull request as ready for review May 25, 2026 16:08
@kisst
Copy link
Copy Markdown
Author

kisst commented May 25, 2026

@wayneshn would appreciate your eyes on this when you have time — it's a chunky bundle but the integration fixes (5 commits) are small and isolated. Happy to split into per-feature PRs if you'd prefer.

…efore insert

Some IMAP/MIME-parsed emails contain U+0000 or unpaired UTF-16 surrogates
in headers or decoded bodies. Postgres rejects these with error 22021
(invalid byte sequence for encoding UTF8), and the worker's retry loop
then blocks the entire mailbox sync. Add a helper that strips NULs and
replaces lone surrogates with U+FFFD, and apply it to every text field
that goes into the archived_emails INSERT.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Set default search option or remember.

1 participant