fix(api,web): migration orchestration, quota precision, AST i18n gate, extended browser CI#41
Open
mikkelchokolate wants to merge 74 commits into
Open
fix(api,web): migration orchestration, quota precision, AST i18n gate, extended browser CI#41mikkelchokolate wants to merge 74 commits into
mikkelchokolate wants to merge 74 commits into
Conversation
Add internal/storage backed by modernc.org/sqlite (pure-Go, no CGO) with WAL, foreign_keys, busy_timeout, and a single connection for the embedded single-node workload. Migrate() applies ordered, transactional, idempotent schema migrations recorded in schema_migrations. Migration 1 creates the normalized domain tables used by the architecture rework: revisions, apply_jobs, clients, client_bindings, client_credentials, subscription_tokens, traffic_counters, traffic_runtime_state, traffic_samples.
Introduce internal/apply on top of the SQLite store: - RevisionStore: desired_revision bumped by every committed mutation, applied_revision advanced only after a job succeeds; marking applied beyond desired is rejected. - JobStore: persistent apply_jobs (never rewritten after terminal state) so history survives a panel restart; failure code/message recorded. - Runner: creates a durable job pinned to an exact immutable revision, serializes concurrent applies (ErrApplyBusy), runs the executor seam over the staged-apply pipeline, and only then advances applied_revision. A job applies its pinned revision, never a newer mutable latest state.
… API
Wire the durable apply subsystem into the management API:
- initApplySubsystem opens the SQLite store next to state.json and wires
RevisionStore/JobStore/Runner (skipped when no StatePath).
- Every committed mutation bumps desired_revision; applied_revision advances
only after a successful apply job.
- autoApplyResultLocked runs a durable apply job via the serialized Runner
(executeApplyRevision seam over the staged-apply pipeline) and returns
revision + job. The executor must not re-acquire s.mu (callers hold it) to
avoid self-deadlock.
- CRUD endpoints (settings, inbounds, routing rules, warp) now return the
mutated object top-level (backward compatible) plus additive revision,
applyJob, and an honest success flag — apply failures are surfaced, never
silently dropped. DELETE returns the envelope (200) instead of bare 204.
- New endpoints: GET /api/apply/state, GET /api/apply/jobs,
GET /api/apply/jobs/{id}, POST /api/apply/jobs/{id}/retry (new job, never
rewrites history), POST /api/apply/reconcile (idempotent converge).
- System state derived from revisions + latest job: synced/pending/applying/
failed/rolling_back/rolled_back.
Tests cover revision tracking, mutation envelope, job list/get/retry,
reconcile, auth, and history persistence.
…ntials, migration, and /api/v1/clients Stage 1 of the client/subscription/traffic rework: Domain model (internal/client): - Client: immutable UUID id (rename keeps id), optional email (never identity), quota bytes + reset policy + resetAt, expiresAt, deviceLimit, notes, depleted flag, optimistic-locking version. - ClientBinding: (client_id, inbound_id) unique, per-binding protocol settings, enabled; one client -> many inbounds. - ClientCredential: AES-256-GCM encrypted at rest via secrets.Cipher, keyed by binding + kind, versioned with rotation (old revoked), never logged or returned by list endpoints. - EffectiveStatus computed with deterministic priority: apply_failed > expired > depleted > disabled > pending_apply > orphaned > active. Repository: parameterized SQL, optimistic locking (409 on version conflict), unique binding (409 on duplicate), cascade delete (client -> bindings + credentials; inbound delete -> orphan clients), SQL-level pagination/filter/ search/sort, orphan detection. Migration: legacy inbound profiles -> Client+Binding+Credential, idempotent via stable (inbound, username)-derived IDs, disabled profiles skippable or preserved-disabled. API /api/v1/clients: CRUD with 400/404/409 mapping, pagination+filter+search, bindings + credentials subresources, rotate, and /api/v1/clients/bulk (enable/disable/delete/extend/reset_quota) with per-client result reporting. Wiring: initClientSubsystem runs after cipher load (fixed ordering bug where it ran before, yielding a nil cipher) onto the same SQLite store; mutations trigger the durable apply notifier.
…nt + formats + meta headers + HTML landing
- TokenStore: SHA-256 hashed storage, plaintext shown once, prefix, last_used_at, revoke, rotate (revoke+reissue), expiry, enable/disable.
- Public /s/{token} outside /api/ (no session; token IS the capability). Unknown/disabled/revoked/expired -> indistinguishable 404.
- Formats via existing BuildClientSubscription (base64 default, raw). SubscriptionMetaHeaders: subscription-userinfo (upload/download/total/expire), profile-update-interval, support-url, profile-title, sanitized content-disposition.
- HTML landing page for browsers (Accept: text/html, no format param).
- SubscriptionRenderer: per-client links from bindings+credentials via clientaccess registry; skips missing/disabled inbounds and unrevealable credentials.
- Disabled/expired/depleted client -> 200 with empty subscription (access revocation propagates cleanly).
- Token mgmt: GET/POST /api/v1/clients/{id}/tokens, DELETE .../tokens/{tid}, POST .../tokens/{tid}/rotate (returns new plaintext once).
- tokenStore + subRenderer wired in apply_subsystem (after cipher load).
…SSE stream
- TrafficStore: absolute counters + per-minute bucketed samples + monotonic
runtime_state. Correct baseline on first provider reading and reset-safe
diffing (no negative deltas after a runtime restart).
- Collector: polls TrafficProvider implementations on an interval, diffs
successive monotonic readings, tolerates broken providers.
- Reconciler: marks clients depleted at quota, clears on quota raise, resets
counters and the depleted flag when a reset window rolls over; onChange
triggers an apply so bindings/enforcement react.
- /api/v1/traffic/{id} totals (+remaining/depleted), /history bucketed,
/top talkers ranked by usage, /stream SSE live snapshots (5s).
- Fix SSE through middleware: HTTPStatusRecorder gains Unwrap()+Flush() and
the stream handler uses http.ResponseController, so metrics-wrapped writers
no longer 500 with 'streaming unsupported'.
…-client render
- ProtocolCapability: the clientaccess registry now reports per protocol
whether it supports per-client credentials, cross-inbound aggregation
(Mieru), and credential-less fallback links. Lets the panel and the
subscription renderer reason about protocols without hardcoding names.
- SubscriptionRenderer.WithSettings: render per-client links against global
settings (domain). Mieru's per-profile config requires a settings domain;
hysteria2/naiveproxy resolve per-inbound. /s/{token} now passes the panel
domain so all four protocols render for subscribed clients.
- Contract test asserting per-client render semantics across all 4 protocols
(hysteria2/naiveproxy embed the per-client password; olcRTC carries the
per-client username against a shared inbound key; mieru a per-profile
config), locking the legacy-BuildClientLinks -> clientaccess facade
delegation to the same rendering path.
…generated SDK
- Document /api/v1/clients (list/create/get/patch/delete), subscription-token
rotate/revoke, /api/v1/traffic (totals/history/top/SSE stream), and the
unauthenticated /s/{token} subscription endpoint (token-as-capability, 404
for unknown/revoked/expired). Add ClientUpsertRequest/ClientView/TrafficTotals
and related schemas plus a shared ClientId path parameter.
- Sync the OpenAPI routes contract test with the newly registered routes.
- Regenerate sdk/go via oapi-codegen: new typed client methods for all v1
client/traffic routes.
… tests+OpenAPI+SDK Pre-existing mismatch since 5250779: the apply workflow changed DELETE to return the apply outcome (revision+job) with status 200, but the management tests still expected 204 and the OpenAPI/SDK documented 204. Update tests to assert 200, document the 200 response (deletion applied; apply outcome returned), regenerate the SDK, and fix a context-leak vet finding in the traffic SSE test (cancel the timeout context when done).
…ken/traffic routes
Live verification on the live host surfaced two real defects:
1. stripBasePathMiddleware rejected every path outside the mounted web base
path, so the public subscription endpoint /s/{token} returned 404 whenever
the panel ran behind a rotated base path (the production default). The
unguessable token is the capability, not the panel path — exempt /s/ from
the base-path gate so subscription URLs keep working. Verified end-to-end:
created inbound+client+token over the API, then /s/{token} served the
per-client hysteria2 link with Profile-Title/Subscription-Userinfo headers
in both base64 and raw formats; unknown tokens still return an
indistinguishable 404.
2. OpenAPI/SDK drifted from the implemented routes: token management lives at
/api/v1/clients/{id}/tokens (list/issue) and /tokens/{tokenId}[/rotate]
(revoke/rotate), and top talkers at /api/v1/traffic/top — not the
/subscription-token and bare /api/v1/traffic paths documented earlier.
Rewrote the spec to match the handlers, synced the routes contract test,
and regenerated the SDK.
Fix divergences between spec and registered handlers:
- client update PATCH->PUT; delete 204->200 with body+404
- GET endpoints no longer marked csrfToken (CSRF only for cookie mutations)
- traffic history documents limit param (handler reads limit, not bucket)
- /s/{token}: add HEAD variant + text/html content
- document previously-missing routes: /api/apply/state, /api/apply/jobs,
/api/apply/jobs/{id}, /api/apply/jobs/{id}/retry, /api/apply/reconcile,
/api/v1/clients/bulk, client bindings and credential routes
- update hand-map contract test to match; regenerate Go SDK
Add tests that exercise the live router rather than a hand-maintained map, so OpenAPI and implementation cannot drift in the same wrong direction: - TestRealRouterCoversOpenAPIStageEndpoints probes documented stage routes against the real router (405 => method drift, unexpected 404 => missing) - TestOpenAPIDocumentsEveryRegisteredStageRoute asserts the spec documents every stage 0-3 route
Each desired revision now records an immutable, encrypted snapshot of the full runtime-affecting configuration (settings, inbounds, routing, warp, users). Apply jobs render from the snapshot pinned to their revision, not from newer mutable state, so retry/reconcile of revision N never applies revision N+1. - migration v2: revision_snapshots table - apply.SnapshotStore: idempotent first-write-wins save, load, has - bumpDesiredRevisionLocked captures + encrypts snapshot on each mutation - executeApplyRevision pins renderer to the revision snapshot and restores - applyRenderSnapshot does full field replacement (not the load-time merge) - tests: store round-trip/immutability/missing, pin swap+restore, no plaintext credential in stored snapshot (encrypted, decrypted on render)
Client create/update/delete and binding/credential mutations previously
triggered apply only via a void notifier whose outcome was discarded; the
HTTP response carried the bare object with no revision or apply result.
Now each client mutation explicitly bumps the desired revision, runs the
apply job, and returns the standard mutation envelope: the object plus
revision{desired,applied,state}, applyJob, and success (success=false means
desired saved but runtime not updated). applyAfterClientMutation records the
revision explicitly because the client store is separate from the management
state file.
Client create previously ignored per-binding errors and returned 201 even when a requested binding or credential failed, persisting a half-configured client. Now any binding/credential failure rolls back the whole create (the created bindings and the client row) and returns the error instead of 201. Adds TestV1ClientCreateTransactionalRollback (duplicate inboundId triggers a unique violation -> non-201, no orphaned client) and a happy-path test.
The runtime renderer only built client access from legacy inbound-embedded profiles, so any client created via the normalized Client+Binding+Credential store was silently dropped from generated live configs. - model.RuntimeCredential: runtime-only (json:"-") carrier on Inbound - clientaccess.BuildClientAccess auto-merges Inbound.RuntimeCredentials and accepts WithCredentials extra; normalized credentials override legacy profiles with the same username (single source of truth) - client.Service.CredentialsForInbound resolves active credential plaintext for enabled clients/bindings on an inbound - management renderer attaches runtime credentials per inbound before render - tests: clientaccess merge/override; api render test proving a normalized client's credential reaches generated/hysteria2/hy2.yaml
Expose the existing idempotent client.Migrator over HTTP: POST /api/v1/clients/migrate-legacy converts every inbound's embedded legacy profiles into normalized Client+Binding+Credential rows (stable derived IDs, safe to re-run), returns per-inbound results plus the mutation envelope, and triggers an apply so migrated clients reach live configs via A6a. Test: migrate -> client visible via v1 API -> second run is a no-op.
…ies (A7)
The client view only carried bare inboundIds, so API/UI consumers could not
tell what a binding supports. Add client.BindingView + BindingCapability:
each binding now reports its id, enabled flag, and the bound inbound's
protocol capabilities (protocol, transports, perClientCredentials,
requiresCaddy) resolved via Service.WithInboundLookup from the protocol
registry metadata. inboundIds kept for backward compatibility.
Test: GET /api/v1/clients/{id} returns bindings[].capability for a hysteria2
binding.
Complete the A7 client read model and binding endpoints:
- BindingView now carries version and metadata-only credential
(configured/kind/version/rotatedAt) via CredentialStore.ActiveForBinding;
never any encrypted/plaintext material.
- PATCH /api/v1/clients/{id}/bindings/{bindingId} toggles enabled with
optimistic locking (Service.SetBindingEnabled).
- POST /credentials/{bindingId}/rotate with empty value performs a
server-generated rotate (Service.RotateCredentialGenerated): high-entropy
plaintext returned exactly once, capability-driven. Explicit value keeps
the caller-supplied path.
Tests: binding patch disable, server rotate returns one-time plaintext,
credential metadata configured in read model.
Bulk operations now return the honest mutation envelope (revision/applyJob/ success) and a per-action skipped count alongside succeeded/failed. Replace the misleading reset_quota (which only cleared the depleted flag) with reset_traffic that actually clears recorded usage via TrafficStore. ResetForClient plus the depleted flag; reset_quota kept as a deprecated alias. Add set_quota, extend_expiry, attach_inbound, detach_inbound; enable/disable no longer no-op-skip (always applied) to keep the existing per-client failure accounting intact. Test: bulk reset_traffic carries revision+applyJob+skipped, bad id counted as failed.
…iler Add GET /api/v1/traffic/summary reporting aggregate totals plus an honest telemetry state (unsupported/collecting) and providerCount. Collector gains ProviderCount. Start collector and reconciler at subsystem init; with zero providers collection is a no-op and summary reports state=unsupported (no fake zeros). stopTrafficSubsystem added for clean shutdown. Test: summary reports state, never 'collecting' with zero providers.
Subscription-Userinfo was hardcoded upload=0; download=0. Now resolves the client's real recorded totals via TrafficStore.TotalsForClient so proxy clients show honest usage. Formats already restricted to base64+raw. Test: recorded traffic surfaces in Subscription-Userinfo, not zeros.
Add to spec: PATCH binding, server-generated rotate (optional value), migrate-legacy, traffic/summary; update bulk action enum + response envelope (skipped/revision/applyJob). Regenerate sdk/go/veilclient.gen.go (+349 lines). Update route contract test to match. Spec==routes now green.
Latest deps (React 19, Vite 8/rolldown, TS 7, TanStack Query/Router/Table,
RHF, Zod 4, Orval 8, Biome 2, MSW, Vitest). Relative base './' so the SPA
serves under both / and /<secret> WebBasePath; /s/{token} stays outside.
Central fetch mutator (src/api/fetcher.ts): WebBasePath prefix, same-origin
credentials, X-CSRF-Token only for cookie mutations, error normalisation,
no tokens/creds in storage. Orval tags-split clean-gen from docs/openapi.yaml
(never hand-edit generated). pnpm build green (react+tanstack+vendor chunks).
Auth: AuthContext (login/status/logout), CSRF from login+status into central
fetcher, admin/viewer RBAC hook, Login + first-run Setup views, session-expiry
refresh. WebBasePath: relative Vite base + <base href> + router basepath read
from it, so client-side nav works under /<secret>; /s/{token} untouched.
Shell: dark design tokens matching legacy panel (canvas/surface/accent,
260px sidebar, nav ordering, top bar, cards, tables, buttons, badges) with a
global ApplyStatusIndicator on every authenticated page. TanStack Router with
all top-level sections. Build green (165 modules).
Clients list: server pagination via /api/v1/clients?page&pageSize&search& status&sort (hand-typed wrapper, spec gap recorded), URL-synced filters, sorting, admin-only bulk select with enable/disable/reset_traffic/delete returning honest succeeded/skipped/failed. Effective-status badges (active/expired/depleted/pending_apply/apply_failed/orphaned). Create: General/Limits/Access/Review wizard; binds inbounds at creation; server-generated credentials rotated once and shown as one-time plaintext. Detail: tabs Overview/Access/Subscription/Traffic; bindings read model with protocol capability + credential metadata (configured/kind/version, never the secret); PATCH enable/disable with optimistic locking; rotate credential returns one-time plaintext; no persistent Reveal, nothing cached in storage. Build green (169 modules).
Apply page (B5): honest synchronous semantics — desired vs applied revision, drift indicator, derived system state badge, last error code/message, admin reconcile + per-job retry. Global ApplyStatusIndicator polls /api/apply/state. Traffic dashboard (B9): honest telemetry states — shows 'no source feeding counters' explicitly instead of a fake graph; when collecting, real aggregate totals + top-clients-by-usage table. Inbounds (B10): read-only list (name/protocol/transport/port/status) with a note that client credentials live under Clients → Access, not legacy profile edits. Build green (169 modules).
Subscription tokens (B8): list/create/rotate/revoke per client; one-time plaintext + subscription URL + QR (canvas, no CDN) shown once at issue/rotate, never persisted; prefix-only display thereafter; status badges (active/revoked/disabled), created/last-used/expires. Per-client traffic (B9): usage totals, quota progress bar with depleted state, remaining bytes, collected-at timestamp. Detail page tabs now fully wired: Overview / Access / Subscription / Traffic. Build green (200 modules).
web/web.go embeds web/dist (go:embed all:dist). internal/api/spa.go serves the
React SPA as the primary UI: index shell at / and all client-side routes
(history-API fallback) with <base href> rewritten to the secret WebBasePath so
relative assets resolve under /<secret>; hashed /assets/* served immutable;
API/metrics/health//s/favicon paths excluded so /s/{token} stays public and
outside the base path. handlePanel routes SPA-matching paths to the shell;
legacy server-rendered panel still reachable for non-SPA paths.
make web builds the SPA before dist. SPA serving tests (base-path rewrite +
client-route fallback + API exclusion) green. go vet/build clean.
SPA now serves at '/' and all client-side routes (history-API fallback) with
<base href> rewritten to the secret WebBasePath. Preserved fail-closed guard:
public listener with no users still returns 503 (no HTML) — verified by test.
HEAD returns headers only. /assets/* served immutable; API/metrics/health/
/s/favicon excluded from SPA fallback so /s/{token} stays public.
CSP tightened for the SPA (B12): no unsafe-inline scripts/styles, no CDN,
self-hosted bundle, form-action 'self'. Legacy server-rendered panel superseded
as primary UI.
Tests updated to assert SPA shell + client-route fallback + first-run SPA
setup; go test (non-e2e) fully green; go vet clean.
…e plaintext Backend: CreateWithBindingsIssued generates a protocol-compatible high-entropy credential inside the SAME SQL transaction for every binding without an explicit credential, persists only the encrypted form, and returns the plaintext exactly once via issuedCredentials in the create response. Test asserts the plaintext round-trips through the encrypted store and the created client lands in the immutable snapshot. Frontend: ClientNewPage no longer calls the rotate endpoint (which wrongly used inboundId as the credential id); it reads issuedCredentials from the create response and shows them in a one-time dialog held only in component state — never Query cache/URL/storage — cleared on Done, navigation, unmount, and after a 5-minute timeout.
S3 (Clients UI):
- Zod-validated typed search params (page/pageSize/search/status/sort)
- debounced server-side search, column visibility menu, aggregate summary
- per-client bulk results (not just aggregate), delete confirmation
- ClientDetail: RHF+Zod edit, enable/disable, delete confirm, attach/detach
inbound, binding toggle, credential rotate, new Audit tab
- revision/apply feedback card surfaced after every mutation
- byte counters kept as integer strings in forms (never Number() lossy parse)
- new per-client audit endpoint GET /api/v1/clients/{id}/audit
S4 (admin parity):
- InboundsPage: full CRUD + enable/disable + normalized attached clients + apply result
- BackupsPage: download/verify/restore(confirm)/prune + restore-job tracking
- UsersPage: edit role/password, delete with confirm, active sessions + revoke
- SettingsPage: full editable field set + state key rotation
Tests: fix create-response decoding to tolerate the S2 nested client envelope.
S5 (Apply UI): - job detail surfaces the operation breakdown (type/target/result/detail) - rendered plan on demand (valid, configs/actions, errors, issues, operations with interruption risk + rollback availability) - legacy apply history with full validation / service-action / health-check / rollback-action breakdown per entry - synchronous retry preserved; plan endpoint is POST S6 (traffic providers): - Collector.ResetProviders replaces the provider set atomically - buildTrafficProvidersLocked builds providers from current clients/bindings/ inbounds; registerTrafficProvidersLocked now resets rather than appends - autoApplyResultLocked re-registers providers after every apply so client/binding/credential/inbound changes update attribution without restart - test: provider count increases after an enabled hysteria2 inbound apply
…ight SPA suite
- AppShell rendered OUTSIDE the RouterProvider, so its navigation hooks
(useRouterState/Link) resolved a null router store and crashed with
'Cannot read properties of null (reading stores)' on any authenticated
render. Moved AppShell into the router root layout so the shell is a child
of RouterProvider.
- AuthContext used the Orval-generated auth client but read res.status/res.data;
the fetch mutator returns the parsed body directly (and throws on non-2xx),
so a successful 200 login was misread as a failure ('Invalid username or
password'). AuthContext now uses apiFetch directly (consistent with the rest
of the app) and Session carries csrfToken.
- Regenerated Orval clients (no hand edits).
- test/browser/panel.spec.js: replaced the legacy hash-route SPA suite with a
React SPA suite (login, invalid creds, clients, inbounds, wizard create with
one-time credentials modal, logout). 6/6 pass against a live binary.
- new 'frontend' CI job: pnpm install --frozen-lockfile -> pnpm gen -> git diff --exit-code on src/api/generated -> typecheck -> biome check -> unit tests -> build. Fails on any Orval drift, type error, lint error, test failure, or build failure. - web/biome.json: scope checks to source (exclude dist/node_modules/generated) so `pnpm check` is CI-green instead of failing on the built bundle. - App.tsx spinner uses role=status (a11y/useAriaPropsSupportedByRole). - ApplyJobDetailPage: biome-ignore for index-keyed API rows that carry a stable discriminating prefix (noArrayIndexKey).
- orval client switched fetch -> react-query so every operation now also has a generated useQuery/useMutation hook (e.g. useGetApiV1Clients) typed to the OpenAPI contract, not hand-rolled DTOs. - new veilZod output project generates .zod.ts schemas per tag for runtime payload validation. - new veilMsw output project generates .msw.ts handlers per tag so browser and component tests can mock the API at the network edge. - all generated output typechecks and builds clean; unit tests still pass.
- migrated the hand-rolled code-based router to @tanstack/router-plugin file-based routing under src/routes/** (typed path params + search via the generated route tree). __root.tsx renders AppShell inside the router. - routeTree.gen.ts is a build artifact: gitignored, regenerated by the Vite plugin on every build and by `tsr generate` ahead of typecheck/build so a fresh clone compiles. build/typecheck scripts prepend `tsr generate`. - vite.config adds tanstackRouter() before react() with autoCodeSplitting. - ApplyPage job links updated /apply/jobs/$jobId -> /apply/$jobId to match the file route. - web/tsconfig.tsbuildinfo untracked + gitignored (build cache). - biome excludes routeTree.gen.ts. 6/6 Playwright SPA tests still pass.
- replaced inline SVG path strings with lucide-react components (tree-shaken, no icon-font/CDN so the strict CSP still holds). NAV_ENTRIES now carries a LucideIcon per entry; AppShell renders <entry.icon/>. Added the previously missing Apply nav entry.
- Add @radix-ui primitives, cva, clsx, tailwind-merge, lucide-react - New ui/ primitives: Button, Input, Select, Dialog, AlertDialog, Tabs, Table, Form, Badge, Toast, DropdownMenu, Label, Card (shadcn-style, CSS-var themed) - Migrate all 15 pages off custom form/table/modal markup onto the primitives; confirm flows now use AlertDialog, one-time credentials use Dialog, client detail uses Tabs - Fix vite dev proxy: '/s' prefix matched /src/* and proxied module requests to the backend - Verified: typecheck + biome + vitest + build green; Playwright (28 checks incl. dialogs, tabs, dropdown, wizard) all pass
- Extract every user-facing string from all 15 pages, auth views, shell, apply/subscription components, router 404 and ui/dialog sr-only label into the en/ru catalogs (~460 keys each, kept in exact parity) - Namespaces: one per page (clients.*, clientDetail.*, applyJob.*, ...) plus shared common.*/nav.*; zod validation messages localized via a per-render schema factory; i18next-style _one/_other keys for counts - Fix provider topology: I18nProvider now wraps every auth branch (login/setup/spinner also use t()), keyed by session locale so it remounts when the authenticated locale becomes known - Add scripts/i18n_check.py (pnpm i18n:check): en/ru key parity, missing t() keys, and hardcoded-string leak scan — CLEAN - Verified: typecheck, biome check, vitest (4/4), vite build, Playwright smoke (28/28 en) + RU smoke (nav/overview/clients in Russian with locale=ru)
Every client-domain mutation (create/update/delete, enable, quota, expiry, credential rotate/revoke, import, legacy migration, traffic quota reconciliation) now runs through commitClientMutationLocked: state change + desired-revision bump + immutable encrypted snapshot commit in ONE SQLite transaction, and exactly one apply job per revision. Before this, mutations committed state first and bumped the revision/snapshot separately, so a crash mid-sequence left revision tracking lying about the actual state, and the reconciler flipped quota-depleted flags with no revision/snapshot at all. - client.Repository: BeginTx/WithTx + tx-scoped query set; atomic credential rotate/revoke; migration moves under the same tx surface - api: commitClientMutationLocked + recordRevisionInTxLocked (snapshot reads through the same tx); startup legacy migration runs inside the orchestration on boot and reports per-client outcomes - apply: tx-scoped BumpDesired/SaveSnapshot helpers - reconciler onChange routes flag flips through the orchestration - atomic-tx tests: crash-injection at every commit stage proves no partial state (client_v1_atomic_tx_test.go)
W4: the client create request/response is now fully declared in docs/openapi.yaml (ClientCreateRequest with credentials + atomic flag, ClientCreateResponse = client + issuedCredentials + applyJob + revision + success), and the SPA consumes the generated Orval types instead of the hand-written apiFetch envelope. Wizard submit path sends the generated ClientCreateRequest shape. W5: Orval outputs are split — models/ holds pure DTOs, msw/models/ holds the generated MSW handlers, and 'pnpm gen' regenerates both deterministically. The SPA's own MSW test doubles now import the generated handlers instead of drifting hand copies. Also fixes the wizard footer: the footer was a <form> whose buttons swapped under the cursor, so the click on Review advanced the step and the trailing mouse-up hit the submit button that took its place — the create fired without the user ever seeing the review screen. The footer is now a plain div with explicit onClick handlers.
W6: the file route now owns search-param validation via
validateSearch + a Zod schema (page/pageSize/search/status/
inboundId/sort). Every field is optional-with-catch: an absent param
stays undefined (links don't need to enumerate defaults), an invalid
hand-edited value falls back to its default — the page never sees
unvalidated URL input. ClientsPage consumes typed Route.useSearch()
instead of useSearch({ strict: false }) + manual decoding, and the
test suite composes the real route tree.
Resolves the src tree from the script's own location so the i18n parity/leak gate runs identically from any CWD (CI, local, release scripts); it previously hardcoded a developer-machine absolute path.
… in CI W8: the web suite now runs in two projects — jsdom (default, pnpm test) and a real-Chromium browser project (pnpm test:browser) that executes the app through an actual browser engine with the MSW Service Worker (shared isomorphic handlers in src/test/handlers.ts; node server and browser worker both consume them). Catches jsdom blind spots: real fetch, real DOM, real event loop. CI: the frontend job runs pnpm i18n:check and the browser project (playwright chromium). The browser-e2e job now builds web/dist BEFORE compiling the Go binary — the binary go:embeds the dist tree, so previously the e2e panel served stale embedded assets. Playwright SPA suite gains the required critical flows beyond login/navigation smoke: - wizard create bound to a seeded inbound: one-time credentials dialog surfaces the server-generated plaintext exactly once - rename through the detail page persists across a hard reload (committed state, not local cache) - the pre-existing wizard test was stale (it never reached the review-submit step) and is fixed to the real 4-step flow - inbound seeds use a per-run unique port so reused state never collides
Deep-link reloads (e.g. F5 on /clients/{id}) were broken: the SPA
ships a static <base href> that the server rewrites to the active
WebBasePath, but the CSP said base-uri 'none', so the browser blocked
the base element, the relative asset URLs resolved against the deep
route, every module 404'd, and the app never booted. SPA-internal
navigation masked this — only full reloads on deep links hit it.
base-uri 'self' keeps the actual protection (an injected base cannot
point at a foreign origin) while allowing the app's own base tag.
Found by the new Playwright persistence flow, which hard-reloads the
client detail page.
…mode/ownership Issue 1: startup legacy migration committed normalized clients via a raw clientRepo.WithTx — no desired revision, no immutable snapshot, no apply job, so the panel could report synced while the runtime differed from the migrated state. The migration now runs through commitClientMutationLocked (migration + verification + marker + revision + snapshot in ONE tx) and triggers exactly one apply job for the new revision. Issue 2: the version marker was a blind skip gate. A restored older state file may carry legacy profiles that were not represented when the marker was written. Every boot now fingerprints the CURRENT legacy set (MissingInboundProfiles: stable client ID + binding representation) and idempotently migrates only unrepresented profiles; the marker degrades to an audit record (with the set fingerprint in its details). Also: the privileged restore wrote state/key replacements root-owned 0600, leaving the unprivileged panel unable to re-read its own state after a helper restore (reload failed -> restore job 'failed' despite correct on-disk state). stageRestoreFile now preserves the original file's mode, and as root its owner/group. Added unit tests; .gitignore the runtime generated/ artifacts written by api tests.
Issue 3: quotaBytes crossed the API as an int64 but TypeScript/JS consumes
it as a number; Number() and Number.parseInt() silently lose precision
above 2^53-1. The contract now caps quotas at Number.MAX_SAFE_INTEGER
end-to-end: OpenAPI maximum + regenerated Orval outputs, backend
validation (client.validate + bulk set_quota), and the SPA's quota form
schemas. Form validation compares decimal strings lexicographically so the
check itself never round-trips through a float (9007199254740993 must be
rejected; Number() cannot tell it apart from the cap). New clientNew
format/precision error keys in both locales.
Issue 4: the regex i18n leak scan excluded newlines, so every multiline
JSX text node escaped it. Replaced with an AST scanner
(scripts/i18n_leaks.mjs, @babel/parser — declared as a real devDependency):
JSX text nodes, user-facing attribute strings, and string literals in
conditional/logical branches rendered as element children (attribute-bound
expressions like variant={...} are prop values, not text). It found 14
real leaks; all extracted — most keys already existed in the catalogs
(delete dialog, feedback revision line, quota label, rotate/detach/attach,
credential state, issued-note), plus genuinely new shell.panelTitle and
subTokens.urlLabel. Removed the dead || "Logout" fallback (t() already
falls back to the en catalog). i18n:check now runs catalog parity (python)
+ AST leak scan (node) and both are CLEAN.
…, apply, tokens, backups, WebBasePath Seven new real-panel flows (no mocks): - two-binding create: one dialog, one-time plaintext per inbound, then proof the plaintext never reaches localStorage, sessionStorage, the URL, network resource names, or the post-dismiss DOM - viewer RBAC: read-only UI plus a server-side 403 on a session+CSRF mutation probe - optimistic lock: stale-version PUT -> 409 - apply failure/retry: helper-less gate panel fails applies honestly; retry creates a NEW job for the same revision (old record immutable) - subscription token lifecycle: issue -> live URL 200 -> revoke -> 404 (oracle-safe) - backup create/verify/restore: runs against a dedicated helper-backed panel (veil system user, /var/lib/veil state, root helper) that CI now starts; rollback probed via state.json-resident panel users - WebBasePath: a second CI panel with --web-base-path; SPA boots under the prefix and survives a hard refresh on a deep link Local verification: 15/15 (8 existing + 7 new) against the real binary.
mikkelchokolate
force-pushed
the
feat/arch-rework
branch
from
July 21, 2026 15:47
fbd555b to
565853a
Compare
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
…ode, helper probe The arch branch had never run CI before its first PR; several pre-existing breakages surfaced together with one new-env issue: - web/web.go embeds all:dist, but the test/e2e/privilege-boundary/ package-smoke jobs and the Docker image built ./cmd/veil without ever building web/dist. Added the pnpm web build to those jobs and a webbuilder stage (node 24 + pnpm 11.10.0) to the Dockerfile; dockerignore web/node_modules and web/dist (built in-image). - go.mod was not tidy (uuid/sqlite direct vs indirect) since the arch rework; tidied. - staticcheck U1000: CredentialStore.insert and nextVersionFor became dead when the mutation orchestration moved callers to the querier-based shared functions; removed. gofmt drift in service.go and openapi_routes_contract_test.go fixed. - browser-e2e: the helper-backed panel step now probes a real backup create and dumps the helper/panel logs on failure, and the diagnostics artifact includes all panel/helper logs.
… chgrp - docker-build: pnpm install failed inside the image on ignored build scripts (esbuild, msw) because web/.npmrc was not copied before install. - lint: staticcheck compiles web/web.go; give it a minimal dist stub (analysis only, no binary produced). - test: go vet needs the embed too — build web/dist before all Go steps. - browser-e2e helper step: GH runner sudoers is (ALL), not (ALL:ALL) — sudo -g veil prompted for a password and the helper never started. Start as plain root, then chgrp veil the socket for the panel.
staticcheck flagged branch debt the first CI run surfaced: - stopTrafficSubsystem and currentRevisionView have had no callers since the mutation orchestration rework; removed. - render_normalized_test: dead first assignments of repo/creds (SA4006). The OpenAPI enum/rename work on this branch was never propagated to the generated Go SDK; verify-sdk failed on the drift. Regenerated via go generate ./sdk/go.
Two defects the first race-mode CI run exposed:
- Test harnesses newApplyTrackedRouter / newApplyTrackedRouterWithState
stubbed service actions and health checks but NOT the firewall applier:
apply's sync-firewall step shells out to ufw, which fails as non-root
(CI) and would mutate the host ruleset as root. Apply outcomes depended
on the test environment. Stub firewallApplierInstance with the existing
fake in both harnesses.
- The pre-migration backup directory used a second-granularity timestamp;
two migrations within the same second (boot 2 right after a state
restore) collided on VACUUM INTO ('output file already exists') and the
migration aborted before touching data. Nanosecond + random suffix.
Verified: full internal/api suite as root AND as an unprivileged user.
…eQL) ?limit= was parsed as int64 and narrowed to int with no bounds check (go/incorrect-integer-conversion, 2 high-severity alerts). Clamp to [1, max] while still int64 (history: 5000, top: 500); non-positive values fall back to the default.
…rded CodeQL still flagged the conversion: clamping in sibling branches is not a dominating guard. The int(n) now sits inside the n>=1 && n<=max branch.
CodeQL's incorrect-integer-conversion query only accepts bounds proven against constants — parameter-derived bounds (int64(max)) are rejected. The conversion now sits in a branch guarded by a literal ceiling (10000, fits int32); the caller max is applied to the already-safe int.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Follow-up fixes on the client/traffic model arc. The previous blockers (atomic client mutations with revision/snapshot, quota reconciliation through the same orchestration, separated OpenAPI/Orval outputs, typed route search, browser CI building the React SPA) are untouched. This PR fixes the remaining issues and extends the real browser gate.
1. Startup legacy migration through the mutation machinery
StartupMigrateLegacyLockedpreviously committed normalized clients via a rawclientRepo.WithTx— no desired revision, no immutable snapshot, no apply job, so the panel could reportsyncedwhile migrated normalized state differed from runtime. The migration now runs throughcommitClientMutationLocked(migration + verification + marker + revision + snapshot in ONE transaction) and triggers exactly one apply job for the new revision.2. Per-boot fingerprint instead of a blind marker skip
The version marker is no longer a skip gate. Every startup fingerprints the CURRENT legacy profile set (
MissingInboundProfiles: stable derived client ID + binding representation) and idempotently migrates only unrepresented profiles — a restored older state file with profiles the marker never covered is now handled. The marker degrades to an audit record carrying the set fingerprint.3.
quotaBytesprecision, end to endint64 quotas cross the wire as JSON numbers;
Number()/Number.parseInt()in the SPA lose precision above 2^53-1. The contract now caps quotas atNumber.MAX_SAFE_INTEGEReverywhere:maximum: 9007199254740991(+ regenerated Orval outputs)client.validate, bulkset_quota) viaclient.MaxQuotaBytes9007199254740993is rejected;Number()can't tell it from the cap)4. AST-based i18n leak gate
The regex leak scan excluded newlines and silently missed every multiline JSX text node. Replaced with an AST scanner (
web/scripts/i18n_leaks.mjs,@babel/parseras a declared devDependency): JSX text, user-facing attributes (placeholder/title/aria-label/alt/label), and string literals in conditional/logical branches rendered as element children (attribute-bound expressions likevariant={...}are prop values, not text). It found 14 real leaks; all extracted — most keys already existed in the catalogs, plus genuinely newshell.panelTitle/subTokens.urlLabel. Removed a dead|| "Logout"fallback.pnpm i18n:check= catalog parity (python) + AST leaks (node), both CLEAN.5. Extended real Playwright gate
Seven new flows against the real panel binary (no mocks):
--web-base-path; SPA boots under prefix and survives hard refresh on a deep linkCI starts all three panels (main, WebBasePath, helper-backed). Local: 15/15 (8 existing + 7 new).
Found while building the gate
0600; the unprivileged panel then couldn't re-read its own state and the restore job reportedfaileddespite correct on-disk data.stageRestoreFilenow preserves the original file's mode and (as root) owner/group. Unit-tested. This bit on any real host restore.veil.db(SQLite), which the backup archive does NOT include — a state.json restore cannot roll back API-created clients. The gate therefore probes rollback with state.json-resident state (panel users). Recommend extending the archive format to cover the DB in the next backup-arc pass.Verification (all local, real binary)
go test $(go list ./... | grep -v test/e2e)— cleani18n:check(parity + AST), vitest,vite build— clean