add SSR HTML cache benchmarks and prehash experiment#40
add SSR HTML cache benchmarks and prehash experiment#40remorses wants to merge 3 commits intorsc-merge-mainfrom
Conversation
Deploying with
|
| Status | Name | Latest Commit | Updated (UTC) |
|---|---|---|---|
| ❌ Deployment failed View logs |
spiceflow-website-worker | 78d7cdd | Mar 18 2026, 02:18 PM |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e8ac354c94
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| const cached = ssrCache.get(digest) | ||
| if (cached) { | ||
| // Cancel unconsumed streams to free tee branch buffers | ||
| void htmlStream.cancel().catch(() => {}) | ||
| void flightStream2.cancel().catch(() => {}) | ||
| return new Response(cached.html, { |
There was a problem hiding this comment.
renderHtmlStreaming() builds appendToHead from MetaState({ baseUrl: request.url }), so the final HTML can vary by host/protocol even when the RSC flight bytes are identical. With this cache keyed only by digest, a multi-domain or proxied deployment can render https://a.example/foo first and then serve that cached HTML to https://b.example/foo, leaking the first host's og:url/og:image metadata into the second response.
Useful? React with 👍 / 👎.
| "peerDependencies": { | ||
| "react": "*", | ||
| "react-dom": "*", |
There was a problem hiding this comment.
This change makes react and react-dom required peers for the entire package even though the root entry now has a dedicated check-entry path for non-React usage. API-only consumers that just import spiceflow will now get peer-dependency failures/warnings (and, in stricter package-manager setups, blocked installs) unless they add React manually, which regresses the documented core-framework use case.
Useful? React with 👍 / 👎.
| if (isZodSchema(schema)) { | ||
| let jsonSchema = zodToJsonSchema(schema as any, { | ||
| removeAdditionalStrategy: 'strict', | ||
| }) as any | ||
| const { $schema, ...rest } = jsonSchema | ||
| return rest as any | ||
| throw new Error( | ||
| `cannot get json schema from Zod v3. update to latest Zod v4 to use openapi spiceflow plugin`, | ||
| ) |
There was a problem hiding this comment.
openapi()
Any existing app that still passes Zod 3 schemas into openapi() now throws at runtime as soon as schema generation reaches this branch. Before this commit the same path converted those schemas via zod-to-json-schema, so doc/SDK generation regresses from "works" to "hard failure" for Zod 3 users even though the rest of Spiceflow still explicitly distinguishes isZodSchema from isZod4.
Useful? React with 👍 / 👎.
Cache the final HTML output for fast GET/HEAD pages using an LRU cache keyed by an MD5 hash of the RSC flight stream. How it works: - A TransformStream wraps the flight-for-SSR stream, hashing each chunk as React consumes it during SSR (zero extra tees, zero extra races) - After the existing allReady vs 50ms race, if allReady won, the hash digest is available via getDigest() - Cache hit: return cached HTML bytes immediately (cancel unconsumed streams) - Cache miss: collect full HTML via collectStream, store in LRU, return - Timeout (>50ms): stream normally, no caching Safety: - Disabled in dev (import.meta.env.DEV) - Only GET/HEAD requests with 2xx RSC response status - Responses with Set-Cookie or other uncacheable headers skip cache entirely - Cache hit uses current request's computed headers, not stale cached ones - Headers stored as [string, string][] tuples to preserve multi-value semantics - LRU bounded to 5MB total byte size with oldest-first eviction Files: - lru.ts: generic byte-bounded LRU cache (reusable) - react/ssr-cache.ts: SSR-specific cache entry, hash transform, utilities - react/entry.ssr.tsx: caching logic integrated into renderHtml - react/ssr-cache.test.ts: 16 tests covering LRU, hash transform, utilities
Fix the hash finalization race by awaiting a digest promise that resolves when the hash transform flushes, instead of reading a synchronous getter after SSR allReady. This makes the cache key deterministic even when the transform closes on a later microtask. Also add so the cache can be disabled at runtime for benchmarking and debugging, and document the behavior in tests. The release note is included via a changeset for the public package.
Add an experimental path that hashes the RSC flight stream before SSR so cache hits can skip HTML rendering entirely. The implementation keeps the existing 50ms streaming budget by handing the remaining time to the normal allReady race on timeout. Also make the node benchmark route much heavier in the Spiceflow, Next.js, and Hono examples so SSR and HTML generation costs are easier to compare under load. This makes the current post-SSR cache win more obvious and shows that the prehash path still does not pay off on this route.
remorses
force-pushed
the
ssr-cache-prehash-bench
branch
from
e8ac354 to
78d7cdd
Compare
1 participant
What changed
LRUCacheand SSR cache utilities, plus tests for hash timing, mode parsing, and prehash helpersSPICEFLOW_DISABLE_SSR_CACHEand an experimentalSPICEFLOW_SSR_CACHE_MODE=prehashpath so the cache strategies can be benchmarked directlynodejs-example/aboutroute much heavier in the Spiceflow, Next.js, and Hono baselines so SSR and HTML generation costs are easier to compare under loadWhy
The original
/aboutroute was too small to make the HTML-side optimization easy to evaluate. A heavier deterministic page makes the tradeoffs much clearer: the current post-SSR HTML cache helps on larger trees, while the prehash-before-SSR experiment still does not pay off on this benchmark route.Benchmark notes
On the heavier
/aboutroute in the Node benchmark:post:881.02 req/s,56.58msprehash:777.91 req/s,64.80msoff:789.96 req/s,63.85msThat keeps the current post-SSR cache as the best-performing mode in this benchmark, while the prehash path remains useful as an experiment for further profiling.