Speed up dummy-project seeding (preview create-project ~15s → ~1.3s)#1437
Speed up dummy-project seeding (preview create-project ~15s → ~1.3s)#1437BilalG1 wants to merge 3 commits into
Conversation
Replace the per-user CRUD-handler calls in seedDummyUsers with bulk createMany inserts, plus several smaller wins on the seeding path. End-to-end the preview create-project endpoint drops from ~15s to ~1.3s. - seedDummyUsers: build every row up front and insert via one createMany per table inside a single transaction, instead of ~86 sequential adminCreate transactions. Named-user team memberships are bulk-inserted the same way. Idempotency is preserved via a single up-front email lookup, so re-runs against an existing project still skip existing users. - Use node:crypto randomUUID instead of stack-shared generateUuid in the seed paths. The browser-safe polyfill calls crypto.getRandomValues ~31x per UUID, which dominated CPU time when generating thousands of seed UUIDs (~800ms in the activity-event build alone). - seedBulkSignupsAndActivity: skip the redundant back-date UPDATE for freshly-inserted users (createMany already writes correct timestamps), and flush ClickHouse events in larger, parallel batches. - seedDummyProject: run seedBulkSignupsAndActivity concurrently with the lighter remaining steps, and fold seedDummyTransactions into the emails/activity/replays Promise.all. - Remove the now-unused syncSeedUserOauthProviders helper.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (1)
📝 WalkthroughWalkthroughRefactors backend seed scripts for deterministic bulk user creation and idempotent bulk inserts, increases ClickHouse batch sizes and concurrency, narrows backdating to existing users, overlaps bulk activity seeding with project setup, and adds runtime guards/refreshes in dashboard preview and layout auto-login. ChangesSeed Data Refactoring & Performance Optimization
Preview Project Redirect Validation
Layout Client Auto-login Guard
Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Warning There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure. 🔧 ESLint
apps/dashboard/src/app/(main)/(protected)/layout-client.tsxParsing error: error TS5012: Cannot read file '/tsconfig.json': ENOENT: no such file or directory, open '/tsconfig.json'. Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Greptile SummaryThis PR replaces per-user sequential
Confidence Score: 3/5The bulk-insert path produces correct data under the happy path, but the floated The new concurrent execution model fires apps/backend/src/lib/seed-dummy-data.ts — specifically the Important Files Changed
Sequence DiagramsequenceDiagram
participant C as seedDummyProject
participant PU as seedDummyUsers (bulk)
participant BSA as seedBulkSignupsAndActivity
participant PA1 as Promise.all #1 (config overrides)
participant PA2 as Promise.all #2 (emails/txns/replays)
participant CH as ClickHouse
C->>PU: await (sequential — heavy Postgres)
PU-->>C: userEmailToId Map
C->>BSA: fire (Promise, not awaited yet)
activate BSA
C->>PA1: await Promise.all [overrideBranchConfig, overrideEnvConfig, ...]
PA1-->>C: done (or throws ⚠️ — bulkSignupsPromise abandoned)
C->>PA2: await Promise.all [seedDummyTransactions, seedDummyEmails, ...]
PA2-->>C: done (or throws ⚠️ — bulkSignupsPromise abandoned)
BSA->>BSA: createMany ProjectUsers
BSA->>CH: Promise.all batch inserts (10 000-row batches, parallel)
CH-->>BSA: ack
BSA-->>C: resolves
C->>C: await bulkSignupsPromise (line 2092)
|
![vercel[bot]](https://avatars.githubusercontent.com/in/8329?s=60&v=4){kind=small}
There was a problem hiding this comment.
Additional Suggestion:
seedBulkSignupsAndActivity function creates ProjectUser and ContactChannel rows for new users but does not create corresponding ProjectUserDirectPermission rows with default permissions, breaking expected permission behavior for bulk-seeded users.
In preview mode the dashboard's /projects page renders PreviewProjectRedirect, which POSTs /internal/preview/create-project and then router.push()es to /projects/<new-id>. It never refreshed the client-side owned-projects cache, so the [projectId] route's useAdminApp() read a stale list, failed to find the just-created project, and called notFound() — showing a 404. Refresh the owned-projects cache before navigating, matching what the normal create-project flow in page-client.tsx already does.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
🧹 Nitpick comments (1)
apps/dashboard/src/app/(main)/(protected)/(outside-dashboard)/projects/preview-project-redirect.tsx (1)
16-27: ⚡ Quick winAvoid
any/asfor internals extraction; use a type guard instead.Line 16 and Line 24-27 currently bypass the type system. Please narrow from
unknownvia a runtime type guard and return the narrowed value directly.Proposed refactor
+type AppInternals = { + sendRequest: (path: string, options: RequestInit, type: string) => Promise<Response>; + refreshOwnedProjects: () => Promise<void>; +}; + +function isAppInternals(value: unknown): value is AppInternals { + return ( + typeof value === "object" && + value !== null && + "sendRequest" in value && + typeof (value as { sendRequest?: unknown }).sendRequest === "function" && + "refreshOwnedProjects" in value && + typeof (value as { refreshOwnedProjects?: unknown }).refreshOwnedProjects === "function" + ); +} + const appInternals = useMemo(() => { - const internals = Reflect.get(app as any, stackAppInternalsSymbol); - if ( - !internals || - typeof internals.sendRequest !== "function" || - typeof internals.refreshOwnedProjects !== "function" - ) { + const internals = Reflect.get(app, stackAppInternalsSymbol); + if (!isAppInternals(internals)) { throw new Error("The Stack client app cannot send internal requests."); } - return internals as { - sendRequest: (path: string, options: RequestInit, type: string) => Promise<Response>, - refreshOwnedProjects: () => Promise<void>, - }; + return internals; }, [app]);As per coding guidelines: "
**/*.{ts,tsx}: Do NOT useas/any/type casts ... unless you specifically asked the user about it."🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@apps/dashboard/src/app/`(main)/(protected)/(outside-dashboard)/projects/preview-project-redirect.tsx around lines 16 - 27, Replace the unchecked cast around Reflect.get(app as any, stackAppInternalsSymbol) by treating the result as unknown and validating it with a runtime type guard: implement a function isStackAppInternals(value: unknown): value is { sendRequest: (path: string, options: RequestInit, type: string) => Promise<Response>; refreshOwnedProjects: () => Promise<void>; } that checks value is an object and both sendRequest and refreshOwnedProjects are functions, then call Reflect.get(...) into a const internals: unknown, run the guard and throw the same error if it returns false, and finally return the narrowed internals (no use of any/as) so the return type is inferred from the type predicate; reference symbols: stackAppInternalsSymbol, internals, isStackAppInternals, sendRequest, refreshOwnedProjects.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In
`@apps/dashboard/src/app/`(main)/(protected)/(outside-dashboard)/projects/preview-project-redirect.tsx:
- Around line 16-27: Replace the unchecked cast around Reflect.get(app as any,
stackAppInternalsSymbol) by treating the result as unknown and validating it
with a runtime type guard: implement a function isStackAppInternals(value:
unknown): value is { sendRequest: (path: string, options: RequestInit, type:
string) => Promise<Response>; refreshOwnedProjects: () => Promise<void>; } that
checks value is an object and both sendRequest and refreshOwnedProjects are
functions, then call Reflect.get(...) into a const internals: unknown, run the
guard and throw the same error if it returns false, and finally return the
narrowed internals (no use of any/as) so the return type is inferred from the
type predicate; reference symbols: stackAppInternalsSymbol, internals,
isStackAppInternals, sendRequest, refreshOwnedProjects.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 46e10b70-cafa-45a2-958a-9199f2c52675
📒 Files selected for processing (1)
apps/dashboard/src/app/(main)/(protected)/(outside-dashboard)/projects/preview-project-redirect.tsx
LayoutClient's auto-login effect had no run-once guard. React StrictMode (on by default in Next dev) double-invokes effects, and both invocations ran while `user` was still null — so in preview mode each generated a fresh `preview-*@Preview.stack-auth.com` email and signed up a *separate* preview user. With two users created per load, the session settles on one while the preview project may have been created for the other, so the project page renders a 404 ("does not have access to it"). Guard the effect with a ref so the auto-login (and preview-user creation) runs at most once per mount.
2 participants
Summary
The internal
preview/create-projectendpoint was taking ~15s becauseseedDummyProjectcreated its dummy users one at a time through the fullusersCrudHandlers.adminCreateCRUD pipeline (one DB transaction + config render per user, ~86 users). This reworks the seeding path to use bulk inserts.End-to-end, the endpoint's server-side handler time drops from ~15,100ms → ~1,300ms (~11× faster).
Seeding changes (
seed-dummy-data.ts)seedDummyUsers— bulk insert. Build every row (ProjectUser,ContactChannel,AuthMethod,ProjectUserOAuthAccount,OAuthAuthMethod, default permissions) up front with pre-generated UUIDs, then insert via onecreateManyper table inside a single transaction — replacing ~86 sequentialadminCreatetransactions. Named-user team memberships are bulk-inserted the same way (TeamMember+TeamMemberDirectPermission). Idempotency is preserved with a single up-front email lookup, so re-runs against an existing project still skip existing users.randomUUID. The seed paths now usenode:crypto'srandomUUID()instead of stack-shared'sgenerateUuid(). The browser-safe polyfill callscrypto.getRandomValues~31× per UUID (once per template char, each with a freshUint8Array(1)); generating thousands of seed UUIDs made that ~800ms of pure CPU in the activity-event build alone.seedBulkSignupsAndActivity. Skip the redundant back-dateUPDATEfor freshly-inserted users (createManyalready writes correctcreatedAt/signedUpAt), and flush ClickHouse events in larger, parallel batches.seedDummyProject. RunseedBulkSignupsAndActivityconcurrently with the lighter remaining steps, and foldseedDummyTransactionsinto the emails/activity/replaysPromise.all.syncSeedUserOauthProvidershelper.The bulk path produces the same rows as the CRUD-handler path (verified row-count equality during development). Webhooks / soft-limit checks are intentionally not fired for seed data, consistent with the rest of the seed.
Also in this PR — preview-mode 404 fix (
preview-project-redirect.tsx)While testing the above, the dashboard 404'd right after a preview project was created. In preview mode the
/projectspage rendersPreviewProjectRedirect, whichPOSTs/internal/preview/create-projectand thenrouter.push()es to/projects/<new-id>— but it never refreshed the client-side owned-projects cache, so the[projectId]route'suseAdminApp()read a stale list, failed to find the just-created project, and callednotFound().Fixed by refreshing the owned-projects cache before navigating, matching what the normal create-project flow in
page-client.tsxalready does. (Pre-existing bug, not caused by the seeding change — but it surfaces the seeding path, so it's bundled here.)Testing
pnpm typecheckandpnpm lintpass for both backend and dashboard. The preview endpoint was exercised repeatedly during development (HTTP 200, projects created and populated correctly).Summary by CodeRabbit
Performance
Refactor
Bug Fixes