1698 cms implement hybridquery#1714
Open
johan-bell wants to merge 72 commits into
Open
Conversation
johan-bell
force-pushed
the
1698-cms-implement-hybridquery
branch
from
8c205db to
91f26c3
Compare
ivanslabbert
force-pushed
the
1698-cms-implement-hybridquery
branch
from
91f26c3 to
8c205db
Compare
Rename the editable-adapter utility to align with the Vue/VueUse `to*` conversion convention (toRef/toRefs/toReactive) and the existing `...AsEditable` wrappers in the same module. The function clones a source ref into new editable reactive state, so `to*` is more accurate than the `create*` factory prefix. - Move shared/src/util/createEditable/ -> toEditable/ (function, type CreateEditableOptions -> ToEditableOptions, spec, barrel) - Keep deprecated `createEditable` / `CreateEditableOptions` aliases so external luminary-shared consumers keep working - Existing callers (useDexieLiveQueryAsEditable, ApiLiveQueryAsEditable) only get their import path updated; they keep using the deprecated alias - Mark useDexieLiveQueryAsEditable as @deprecated Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…s` file to fix tests.
… fetching, replacing ApiLiveQuery and useDexieLiveQuery.
…lector in CreateOrEditRedirectModal.vue
- README.md: document the exported surface grouped by area (querying, database, sync, transport, permissions, FTS, S3, types) with links to the per-module deep-dive docs - useDexieLiveQuery README: rewrite the raw upstream copy to match the other module docs (correct luminary-shared imports, intro, origin link) - HybridQuery README: genericize consumer-specific references - sync README: use luminary-shared imports for consistency - MangoQuery: add example-led guide.md and link it - CLAUDE.md: drop LFormData from the public surface (not re-exported) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Cover the remaining operators ($allMatch, $regex, $beginsWith, $mod, $keyMapMatch) with examples, add an operator quick-reference table, and document the syntax the API validator rejects (no $regex/$where, $elemMatch field allowlist, limit cap, null-in-$in, use_index allowlist, depth/clause caps) — distinguishing local-engine vs remote-API behaviour. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Replace EditContent's imperative loading (db.get / db.whereParent) and hand-rolled dirty tracking (existingParent/existingContent clones + _.isEqual) with the shared luminary-shared primitives: - Load the parent (Post/Tag) and content children via useHybridQuery (live), keyed on a reactive currentId. Eager one-shot hydration + latched sources avoid a placeholder-before-load save race and a blink-to-empty wipe. - Track dirty state with toEditable over the live source; derive existing* baselines and isDirty from it. Extract the data layer and orchestration into colocated, testable units under components/content/: - composables/useEditContentSource — loading, editable copies, dirty tracking, save / revert / duplicate primitives. - composables/useContentLanguage — selected language/content + lists (also drops a dead writable-computed setter). - composables/useContentPermissions — canTranslate/Publish/Edit/Delete via a shared access() helper; fixes a latent canTranslate null-check ordering bug that could throw during load. - util/buildContentDuplicate, util/buildRedirects — pure helpers. Add [type+parentId] and [type+_id] to CMS_DOCS_INDEX so the content- children and parent HybridQuery reads use an index (no full table scan). Adjust EditContent specs for the now-async load. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Load parent/content via useHybridQuery and track dirty state with toEditable, replacing db.get/db.whereParent and manual _.isEqual diffing. Extract the data layer, language, and permissions into colocated composables (+ pure buildContentDuplicate/buildRedirects helpers), and add [type+parentId]/[type+_id] to the CMS Dexie index. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…sePageScroll key for scroll management.
Replace ContentOverview's pull-all-docs + in-memory filter + offset paginator with HybridQuery browse + FTS search behind a unified list, plus infinite scroll. - Browse via useHybridQuery (local-first); search via useFtsSearch, dispatched on whether the search box has >=3 chars. Fold ContentTable into ContentOverview. - Reuse the app's language-priority "fill untranslated" selector, derived from the CMS working language -> default -> the rest (cmsLanguageSelector). - Search-result cards show match context: highlighted title/author + a content snippet (searchHighlight, duplicated from the app SearchModal engine). - Add CouchDB design docs for title/expiryDate sorts. - shared FTS: forward type/tag/status/publishDate filters to /fts and apply the same filters in local ftsSearch (offline parity); thread reactive filters through useFtsSearch. - Route CMS FTS to local: shouldUseApiFts keys purely off the sync cutoff, so the CMS (no cutoff) searches its full local index -- symmetric with its HybridQuery browse, which never calls the API with no cutoff (ADR 0011 amended). - mangoToDexie: build the index-pushdown collection lazily so the sort+limit path no longer emits a spurious full-table-scan warning. Remove the now-unused query.ts and its specs. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
… more content. - Update tests for useInfiniteScrollLoadMore to ensure correct behavior when sentinel intersects. - Enhance useInfiniteScrollList with new loadMore functionality for better pagination control.
Add a save(id) method to toEditable that routes each save per document based on whether it is persisted offline: - Persisted offline (synced, or persistOffline set) -> local write via db.upsert (docs table + queued localChange). Below-cutoff content persisted this way is retention-stamped so it isn't evicted. - Otherwise -> direct API change request (LFormData when the doc carries binary upload data). Persistence is decided per doc: content uses the publishDate sync-window cutoff, other types use isSyncableDoc, and a new persistOffline option forces the local path. filterFn is applied before saving; the baseline is promoted only on an accepted result. The deprecated useDexieLiveQueryAsEditable / ApiLiveQueryAsEditable wrappers are left untouched. Includes save() tests and toEditable README docs (plus related shared README cross-links). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ionObserver. This allows for better control over intersection behavior during testing, ensuring accurate simulation of observer states. Updated mock implementation to return a consistent intersection observer return value.
johan-bell
force-pushed
the
1698-cms-implement-hybridquery
branch
from
d6e1671 to
e3dc2db
Compare
ChrisTouo
force-pushed
the
1698-cms-implement-hybridquery
branch
from
e3dc2db to
d8deda1
Compare
MrDirkelz
force-pushed
the
1698-cms-implement-hybridquery
branch
from
d8deda1 to
e3dc2db
Compare
… improve state management - Removed lodash dependency and replaced deep cloning with a more efficient approach. - Introduced `currentId` and `createTemplate` functions for better ID management and template creation. - Enhanced the `watch` functionality to hydrate redirect data more effectively. - Updated `editable` to use a computed property for cleaner state management. - Improved slug validation and dirty checking logic for better user experience.
…te handling and live updates - Introduced `currentId` and `createTemplate` functions for better user ID management and template creation. - Replaced original user cloning with a more efficient hydration method using `liveUsers`. - Updated `editable` to utilize a computed property for cleaner state management. - Removed unnecessary lodash dependency and improved dirty checking logic. - Enhanced delete confirmation dialog to handle undefined user names gracefully. refactor(UserFilterOptions): simplify query options by removing pagination properties - Removed `pageSize` and `pageIndex` from query options and reset logic for a cleaner implementation. refactor(UserOverview): implement infinite scroll for user listing - Replaced pagination with infinite scroll functionality using `useInfiniteScrollList`. - Updated user listing to reflect changes in the data structure and removed paginator component.
db.upgrade.ts gained a v18 step but the spec never mocked it, so the real v18 ran against the empty mockDb and threw "db.getSchemaVersion is not a function". Mock v18 and assert it is called, mirroring v9..v17. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
inheritAttrs:false (added to silence the production <Teleport> fallthrough warning) also stopped attrs from landing on the test-only <div> branch, breaking specs that locate modals via a passed-through `name` (e.g. LanguageModal.spec, EditLanguage.spec). Re-bind $attrs on the test <div> so fallthrough still reaches the rendered node; the <Teleport> branch (production) cannot receive attrs and stays warning-free. Also document GroupOverview.spec's pre-existing failure (its useHybridQuery migration left the API-mock-based spec stale) alongside the other known groups-domain issues. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Root-caused why deleteCmd:group and deleteCmd:storage syncList entries are
pinned at blockStart:0/blockEnd:0: an empty base-type initial sync pushes no
syncList entry, so merge() returns {0,0} and the deleteCmd column is seeded
from it (firstSync stays true → REST catch-up skipped). Captures reproduction
(deterministic spec + runtime) and the Minimal/Full fix options. Fix is
blocked on the senior (lives in shared/src/api/sync).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two fixes so a freshly-seeded database is correct out of the box: 1. Group seeding docs (group-*.json) now carry memberOf=[self._id], mirroring processGroupDto. Seeding inserts raw JSON via db.upsertDoc and bypasses the change-request pipeline, so seeded groups previously had no memberOf — making the group sync query (memberOf $elemMatch) return zero and groups never sync to clients. 2. New initSchemaVersion default upgrade runs first in the chain. On a fresh DB (no dbSchema doc → getSchemaVersion() === 0) every versioned upgrade no-ops (each guards on an exact prior version), so the dbSchema doc was never created and the version stayed at 0 forever. initSchemaVersion stamps a fresh DB at FRESH_DB_SCHEMA_VERSION (17 — one below the v18 FTS backfill) so the version is tracked AND v18 still runs, computing the server-side fts index over the freshly-seeded User/Redirect docs. No-op on any DB that already has a dbSchema doc. v18 is DB-only (no S3), so it is safe on a fresh DB. Adds initSchemaVersion.spec.ts and asserts the init step in db.upgrade.spec.ts. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Move Language/Settings/Sandbox into the scrollable nav, directly below the main nav items behind a border-t divider (mirrors the app's Theme/Language/Settings block) instead of pinning them near the bottom. Combine Sign out + the avatar/email row into one account block pinned at the bottom, styled like the app (logout uses ArrowRightEndOnRectangleIcon; icons and avatar share one left edge). Move connectivity to the logo row: OnlineIndicator now renders only when offline (v-if !isConnected) next to the logo, and its tooltip opens downward (bottom-center) since it sits at the top. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
FilterOptionsMobile carried z-20 but no positioning context, so the z-index was a no-op; FilterOptionsDesktop had no z-index at all. The filter bar and the scroll container are sibling flex items, so the later-painted cards covered the header's shadow (and would overlay its filter dropdowns). Add `relative z-20` to both so the header forms a positioned stacking context above the auto-level content; z-20 stays below modals/sidebar (z-40/z-50). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…idQuery
GroupSelector read all groups via db.whereTypeAsRef — the last caller of
any db.toRef-family method across cms/, app/, and shared/. That family is
the only code backed by @vueuse/rxjs + rxjs (db.toRef wraps Dexie
liveQuery as an rxjs Observable via useObservable). Switch it to
useHybridQuery({ type: Group }), which is Dexie-first for the synced Group
type and subscribes to liveQuery directly (no rxjs).
The CMS now has zero deprecated, RxJS-backed reactive reads. This unblocks
deleting the toRef/*AsRef family from shared/src/db/database.ts and dropping
@vueuse/rxjs + rxjs from shared (a shared-owned follow-up). todos.md updated.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…p rxjs The db.toRef family on the Database class (toRef, getAsRef, getBySlugAsRef, someByTypeAsRef, whereTypeAsRef, whereParentAsRef, tagsWhereTagTypeAsRef, contentWhereTagAsRef, isLocalChangeAsRef) wrapped Dexie liveQuery as an rxjs Observable via @vueuse/rxjs' useObservable. All were @deprecated in favour of useHybridQuery / useDexieLiveQuery, and after the GroupSelector migration no consumer (cms/app) or shared-internal code called any of them. Delete the whole family and the now-dead imports (useObservable, rxjs Observable, dexie liveQuery, vue Ref, PostType), and drop @vueuse/rxjs + rxjs from package.json — db.toRef was their only user. The non-reactive raw helpers (get, whereParent, tagsWhereTagType, contentWhereTag) are kept. database.spec: remove the deprecated-method tests; rewrite the two that also covered upsert (localChangesOnly + queue-on-upsert) to assert against the docs/localChanges tables directly. Docs (shared CLAUDE.md, useDexieLiveQuery README, cms todos) updated to reflect the removal. Verified: shared build + tests green; cms dist refreshed, full cms suite shows only the pre-existing groups-domain failures (no regression). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Revalidate the HybridQuery-migration todo list against source and drop the finished work (toEditable.save/backPatchFields, HybridQuery isFetching/error/hasLocalChanges, RxJS db.*AsRef removal, deleteRevoked over-purge fix, GroupOverview migration). Remove the deleteCmd:group/ storage syncList entry — confirmed an API issue, not a shared bug. Keep only genuinely remaining/blocked items. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ADR 0012) Captures why CMS reactive document reads were unified on useHybridQuery (offline-first Dexie-first + API supplement vs the old useDexieLiveQuery / ApiLiveQuery split), the construction-time routing rule and the two useDexieLiveQuery carve-outs (globalConfig language, DashboardPage localChanges), and why the deprecated RxJS-backed db.toRef/*AsRef family was deleted and @vueuse/rxjs + rxjs dropped from shared. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ed + backfilled The `seed` command exited via process.exit(0) before upgradeDbSchema ran, so a freshly-seeded DB never reached the upgrade chain: initSchemaVersion never stamped the dbSchema doc and the v18 server-side fts backfill never ran over the seeded User/Redirect docs (seeding writes raw JSON, bypassing process*Dto). This defeated the whole point of FRESH_DB_SCHEMA_VERSION=17. Run upgradeDbSchema in the seed path before exiting. Safe in seed mode: on a fresh DB only v18 runs, which is DB-only (no S3, no PermissionSystem). Assert it in main.spec.ts. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
CMS edit screen — behind the `lightPolish` dev flag (content/lightPolish.ts) so the
whole pass can be toggled off:
- LCard gains a `bare` prop (drops border/shadow/rounding) so a card can nest as a
plain section. The sidebar collapses to two cards: "{type} settings" (group /
categories / topics / toggles + image + media + video) and "Basic" (the Translations
switcher + the per-translation fields).
- EditContentBasic: stack each label above its input, full-width Title, expiry behind a
disclosure, separators between field groups; parent toggles decluttered.
- Rich-text editor bleeds to the edges (no card border); its toolbar pins as a sticky
overlay on mobile; a mobile quick language switcher (app SingleContent-style) changes
the edited translation via the same route navigation the badges use.
Sidebar / misc:
- Desktop collapsible sidebar rail (useDesktopSidebar) + OnlineIndicator icon-only badge.
- Remove the internal ComponentSandbox page; minor LButton / LDropdown / router tweaks.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ccsa/luminary into 1698-cms-implement-hybridquery
…itor layout styling
…alculation and enforcing static min-height
The generic GET /search endpoint (X-Query header) was served only by the ApiLiveQuery / ApiLiveQueryAsEditable module in luminary-shared, which is dead code: not instantiated anywhere in app/ or cms/ (consumers migrated to HybridQuery / toEditable). Remove the endpoint and everything that existed only to serve it. API: - delete search.controller/service, SearchReqDto, db.searchFunctions (SearchOptions/calcGroups), and the orphaned x-query validator (+ specs) - drop SearchController/SearchService from app.module - remove DbService.search()/searchBySlug() (keep getContentByParentId, DbQueryResult) and the search test block; drop the X-Query CORS header - /fts (POST) stack is untouched shared: - delete the util/ApiLiveQuery module and the now-orphaned http.get() X-Query method; remove rest.search() and the ApiSearchQuery type - trim the matching tests; refresh comments/docs (CLAUDE.md, READMEs) - prune the stale @vueuse/rxjs/rxjs entries from the lockfile app/cms: - EditGroup.spec: retype the mock off the deleted ApiLiveQueryAsEditable onto the component's real toEditable contract (type-only) - tidy stale ApiLiveQuery comments; prune dead lockfile entries Pre-existing GroupOverview/EditGroup test failures (specs stale vs the HybridQuery/toEditable migrations) are left untouched. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
app/ and cms/ imported the built dist/index.js via `file:../shared` + `--install-links`, so every shared change needed a rebuild and re-install and there was no HMR. The `--install-links` copy existed to dodge a symlink bug where shared bundled its own vue/dexie and a plain symlink produced duplicate instances (broken Dexie/Vue reactivity). Restructure so the monorepo consumes shared source directly while the package stays standalone-publishable (dist build unchanged): shared: - move vue + dexie to peerDependencies (also devDependencies for own build/test) - add an `exports` map (types -> dist/index.d.ts, default -> dist/index.js); fix `files`. auto-external still keeps vue/dexie external in dist. app + cms: - vite.config: alias `luminary-shared` -> ../shared/src/index.ts, dedupe vue/dexie/@vueuse/core, server.fs.allow `..` (Vitest inherits via mergeConfig) - add `dexie` dependency (now a peer of shared) - tsconfig.app.json paths mirror the dedupe so vue-tsc resolves a single vue (the symlink otherwise exposes shared/node_modules/vue -> duplicate Ref types) - install plain (symlink), drop --install-links Result: editing shared/src hot-reloads with no rebuild/reinstall; a shared TYPE change still needs `npm run build` in shared/ (consumers resolve types from dist). Verified: app 535/535 and cms (only the 4 pre-existing content/group-overview spec files still fail, unchanged by this work) green; app+cms type-check and production builds pass; dev server serves shared/src. Docs (root/app/cms/shared) updated. The rebuild-shared skill and post-checkout hook still reference --install-links and remain to be updated. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The post-checkout git hook rebuilt shared and re-installed app/cms with --install-links on every branch switch. Now that app/cms consume shared/src directly via a Vite alias (HMR), it's unnecessary. - delete scripts/post-checkout and the premade docs/.../automation/post-checkout - setup-dev.sh: drop setup_git_hooks() + its call; app/cms install with plain `npm ci` (no --install-links) - rebuild-shared skill: rewrite for the new model (rebuild only for shared type/signature changes; behaviour hot-reloads; no reinstall) - diff-vs-main skill: update the shared/src contract row accordingly - docs (root CLAUDE.md, scripts/README.md, shared/development.md, docs/setup-vue-app.md, project-automation.md): drop --install-links and the git-hook setup instructions Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Capture the rationale for keeping the DeleteCmd approach over pruning the original doc into a tombstone: sync pulls filtered Mango query windows (not CouchDB _changes), so a doc leaving scope vanishes silently while its stale local copy lingers. DeleteCmd is the always-syncable, independently-routed signal that closes that gap. Pruning fails the two-audience cases (permission change: keepers vs droppers; status change: CMS full draft vs app drop signal) and isn't worth dual mechanisms + ADR 0005 cost for the single-audience delete. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Show posts/tags that have no translation in the user's synced languages by fetching the fallback translation on demand and caching it offline, so the CMS default language no longer needs to be force-synced to every device. Shared (HybridQuery / sync): - decideContentApiQuery folds the fallback (language $nin synced, no cutoff) into ONE combined supplement ($or[publishDate<=cutoff, $nin]) — a single scan per content feed instead of two. - persistOffline gates on type === Content (not isSyncableDoc): below-cutoff / fallback content persists even when sync registered no syncList entry (it only does so after fetching a doc), while still excluding non-content PII. - isSyncable keeps the best-available fallback translation on the live socket feed (language-priority gate), used only by the socket persister now. - db.purge also clears the HybridQuery response cache + retention; add clearResponseCache and pruneUnsyncedLanguageContent (un-tick cleanup). App: - Split preferred display order (appLanguageIdsAsRef + appDisplayLanguageIdsAsRef) from the synced/downloaded subset (appSyncedLanguageIdsAsRef, normalized to always include the primary); sync + config key off the synced subset. - LanguageModal: staged edits with Save/Cancel, per-row "Available offline" checkboxes, and offline guards (block removing/un-downloading a synced language with a toast; defer adds). Toast z-index raised above modals. - FallbackLanguageBadge on the article header, flagged off the synced set. - KeepAlive of Home/Explore/Watch invalidated on local-cache clear. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.