Skip to content

feat: Bring-your-own-database (Postgres/MariaDB) behind a query-layer abstraction #119

Description

@jherforth

Split out of #105, which now focuses on plugin extensibility. This issue tracks the database-flexibility half of that discussion.

Goal

Let power users run HomeGlow against their own database server while keeping the default install exactly as simple as it is today.

  • SQLite remains the zero-config default, forever. A basic install never asks a database question and never will.
  • Postgres and MariaDB become opt-in targets for users who want external/managed storage.

User experience (refined)

  • Connecting an external DB is a power-user feature with its own setup guide — it must never complicate the basic install.
  • A user who starts on the basic SQLite install and later wants an external DB configures the connection in the Admin Panel (credentials stored via the existing encryption helpers), HomeGlow migrates the data, and the change takes effect on a container restart — no docker-compose.yml edits, no image rebuild, no re-deploy.
  • Restarting the container is acceptable; a literal zero-restart hot-switch of a live database is explicitly not a requirement (connection lifecycle + in-flight requests + sync timers make it disproportionately hard for the value).

Technical reality (why this is a large lift)

  • The server talks to better-sqlite3 synchronously throughout server/index.js (~4,400 lines — every route does db.prepare(...).get/all/run inline). Every multi-database option (Prisma, Drizzle, Knex, node-postgres…) is async, so this work forces an async refactor of essentially every route. That's the real cost hiding in this issue, independent of which layer we pick.
  • The custom schemaMigrations system (numbered, frozen migration scripts) would need porting to — or coexistence with — the chosen layer's migration story.
  • Answering the ORM questions from the original feat: Plugin extensibility #105 discussion:
    Language consistency? Not a preference question — the server is a Node process, so the ORM/query layer must be JavaScript. Django is a Python web framework and cannot be adopted here without rewriting the backend. Candidates: Prisma, Drizzle, Knex (evaluation below).

Evaluation criteria for the query layer

  • SQLite + Postgres + MariaDB parity (same schema & queries on all three)
  • Migration tooling vs. our existing schemaMigrations (can old migrations be preserved/replayed?)
  • Runtime weight & cold-start on constrained hardware (we just shrank the backend image ~1.4GB → ~300MB in Docker optimization - low priority #104 — the query layer shouldn't claw that back)
  • Query ergonomics for a 4,400-line single-file route server (incremental adoption matters; a layer that allows raw SQL escape hatches eases the transition)

Suggested phasing

  1. Spike: pick the query layer against the criteria above (small proof: users + prizes routes on SQLite and Postgres).
  2. Async refactor + layer adoption route-group by route-group, SQLite-only, behavior-identical (test suite green throughout).
  3. Postgres/MariaDB enablement + data migration tool (SQLite → external DB) + Admin Panel connection UI.
  4. Docs: the dedicated setup guide.

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Projects

Status
Backlog

Relationships

None yet

Development

No branches or pull requests

Issue actions