Skip to content

feat(code): render PNGs and image data URLs in previews#2143

Merged
pauldambra merged 3 commits into
mainfrom
posthog-code/render-png-and-data-url-previews
May 15, 2026
Merged

feat(code): render PNGs and image data URLs in previews#2143
pauldambra merged 3 commits into
mainfrom
posthog-code/render-png-and-data-url-previews

Conversation

@pauldambra
Copy link
Copy Markdown
Member

Summary

  • PNG files read through the Read tool no longer get dumped into CodeMirror as binary garbage — the agent returns image content blocks and ReadToolView now renders them inline.
  • File previews and inline code previews now detect when content is a single image data URL (data:image/png;base64,…, etc.) and render it as an image instead of as raw text.
  • All rendering goes through a new SafeImagePreview that validates the mime type against an allowlist (no SVG, since SVG can carry script payloads), caps the payload at 20 MB, and falls back to a friendly message via an ErrorBoundary + <img onError> so a malformed or malicious value can't crash the preview.

Changes

  • apps/code/src/shared/utils/imageDataUrl.ts — strict data-URL parser with mime allowlist + size guard, plus unit tests.
  • apps/code/src/renderer/components/ui/SafeImagePreview.tsx — safe <img> wrapper with onError + ErrorBoundary fallback.
  • apps/code/src/renderer/features/sessions/components/session-update/toolCallUtils.tsx — new getContentImage helper to extract ACP image content blocks.
  • apps/code/src/renderer/features/sessions/components/session-update/ReadToolView.tsx — renders image content blocks via SafeImagePreview when present.
  • apps/code/src/renderer/features/sessions/components/session-update/CodePreview.tsx — renders a data-URL image when the content is a single image data URL.
  • apps/code/src/renderer/features/code-editor/components/CodeEditorPanel.tsx — uses SafeImagePreview for both PNG files and files whose content is a data URL.

Test plan

  • pnpm --filter code test --run — all 1200 tests pass, including 16 new parser tests.
  • pnpm --filter code typecheck — clean.
  • pnpm lint — clean.
  • Manual: open a .png file in the file viewer → renders as image.
  • Manual: open a text file whose only content is `data:image/png;base64,…` → renders as image.
  • Manual: have the agent run `Read` on a PNG → expanded tool view shows the image.
  • Manual: corrupt a data URL (bad base64 / disallowed mime / SVG) → falls back gracefully, no crash.

Created with PostHog Code

PNG files read through the Read tool, and file contents that are a
single image data URL, were being dumped into CodeMirror as binary
garbage. Detect both cases and route them through a new SafeImagePreview
that validates the mime type allowlist (no SVG), caps the payload size,
and falls back to a friendly message via an ErrorBoundary + onError so a
malformed or malicious value can't crash the preview.

Generated-By: PostHog Code
Task-Id: 2f630f6e-b0b5-4193-85d3-d6ca39eccb28
@pauldambra pauldambra marked this pull request as ready for review May 13, 2026 22:42
@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps Bot commented May 13, 2026

Prompt To Fix All With AI
Fix the following 2 code review issues. Work through them one at a time, proposing concise fixes.

---

### Issue 1 of 2
apps/code/src/renderer/features/sessions/components/session-update/ReadToolView.tsx:61
The `" in"` suffix is now unconditional, so when `lineCount` is `null` (no text content in the tool response) the title becomes `"Read in"` instead of the original `"Read"`. The `" in"` should remain inside the conditional to preserve the prior behaviour.

```suggestion
            : `Read${lineCount !== null ? ` ${lineCount} lines in` : ""}`}
```

### Issue 2 of 2
apps/code/src/shared/utils/imageDataUrl.test.ts:26-52
**Tests should be parameterised**

Several tests assert multiple independent inputs inside a single `it` block ("accepts other allowed mime types", "rejects non-image mime types", "rejects empty or non-data-URL strings", "rejects malformed data URLs", and the `isAllowedImageMimeType` tests). The team's simplicity rules say to always prefer parameterised tests. Using `it.each` would make each case independently named and reported, so a failure identifies the specific value without having to mentally untangle multi-`expect` blocks.

Reviews (1): Last reviewed commit: "feat(code): render PNGs and image data U..." | Re-trigger Greptile

Comment thread apps/code/src/shared/utils/imageDataUrl.test.ts Outdated
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 3f311c33e1

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread apps/code/src/renderer/components/ui/SafeImagePreview.tsx Outdated
Copy link
Copy Markdown
Member Author

@pauldambra pauldambra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note

🤖 Automated comment by QA Swarm — not written by a human

QA Swarm review complete. See inline comments. Top-level summary follows in a separate comment.

Comment thread apps/code/src/renderer/features/code-editor/components/CodeEditorPanel.tsx Outdated
Comment thread apps/code/src/renderer/components/ui/SafeImagePreview.tsx
Comment thread apps/code/src/shared/utils/imageDataUrl.ts Outdated
Comment thread apps/code/src/renderer/components/ui/SafeImagePreview.tsx Outdated
Comment thread apps/code/src/renderer/features/code-editor/components/CodeEditorPanel.tsx Outdated
Comment thread apps/code/src/shared/utils/imageDataUrl.ts Outdated
@pauldambra
Copy link
Copy Markdown
Member Author

Note

🤖 Automated comment by QA Swarm — not written by a human

Multi-perspective review: qa-team (security, frontend, performance, copy, two generalists), paul-reviewer, xp-reviewer

Verdict: ⚠️ REQUEST CHANGES

Security posture is solid (SVG explicitly blocked, mime allowlist, length cap, no javascript: injection path, no ReDoS). The blockers are user-visible behavioural bugs introduced by the change: SVG file previews regress, and SafeImagePreview's error state doesn't reset across image swaps. The HIGH from xp is straight-line in-file duplication that's worth fixing while you're touching this code anyway.

Key findings

🟠 HIGH

  • CodeEditorPanel.tsx lines 150–170 vs 205–225 are nearly byte-for-byte identical — extract a FilePanelImagePreview helper.

🟡 MEDIUM

  • SVG file previews regress to "Failed to render image" because shared/constants/image.ts still maps .svg to image/svg+xml, which the new allowlist rejects.
  • SafeImagePreview hasError state doesn't reset when base64/mimeType change — stale failure can stick to a new image.
  • Naming drift: ImageContentData.data vs ParsedImageDataUrl.base64 for the same idea.
  • ReadToolView coordinates two getters with a primitive flag (imageContent ? undefined : getReadToolContent(...)) — discriminated union would express this once.

🟢 LOW / ⚪ NIT

  • Copy regression: "Read in" (no line count) where it used to be "Read".
  • MAX_BASE64_LENGTH is checked against the whole trimmed string, not the base64 length — misnamed.
  • parseImageDataUrl runs on every render of CodeEditorPanel and CodePreview — memoise.
  • ErrorBoundary around <img> doesn't catch decode failures (those use onError) — likely dead defence.
  • All failure paths in parseImageDataUrl return null silently — at least one logger.debug would help diagnosis.

Convergence (independently flagged by 2+ reviewers — highest confidence)

Finding Reviewers
SVG file preview regression qa-team/generalist-a + xp
base64 vs data naming drift paul + xp
"Read in" copy regression qa-team/copy + paul
MAX_BASE64_LENGTH misnamed qa-team/performance + xp
ErrorBoundary likely redundant qa-team/generalist-b + paul

Reviewer summaries

Reviewer Assessment
🔍 qa-team Solid security primitive; functional regressions for SVG files and hasError stickiness need fixing; perf is fine on the common path but parse is unmemoised and length cap is over-generous.
👤 paul Nice clean addition with thorough tests; mostly questions about silent error paths and whether ErrorBoundary around <img> is actually catching anything — approve with caveats.
📐 xp Good security primitive and good tests; main work left is removing in-file duplication in CodeEditorPanel, unifying the two image-payload shapes, and reconciling the two sources of truth for "is this an image?".

Automated by QA Swarm — not a human review

- Restore "Read" / "Read N lines in" title and add "Read image in" for
  image content blocks, fixing the "Read in" copy regression.
- Drop svg from shared IMAGE_MIME_TYPES so .svg files fall through to
  the text path instead of regressing to "Failed to render image".
- Reset hasError in SafeImagePreview when base64/mimeType change, using
  the render-time setState pattern; drop the ErrorBoundary wrapper since
  <img> errors fire onError, not exceptions.
- Validate base64 length against MAX_IMAGE_BASE64_LENGTH inside
  SafeImagePreview so tool-provided image data hits the same cap as
  user-pasted data URLs.
- Extract FilePanelImagePreview helper to remove the duplicated <Flex>
  wrapper between the PNG-file and data-URL branches of
  CodeEditorPanel.
- Unify ImageContentData.data -> base64 so producers and consumers share
  one field name.
- Rename MAX_BASE64_LENGTH -> MAX_DATA_URL_LENGTH (matches what is
  measured) and skip the trim() allocation when the value doesn't look
  like a data URL.
- Memoise parseImageDataUrl calls in CodeEditorPanel and CodePreview.
- Parameterise the multi-input parser tests with it.each.

Generated-By: PostHog Code
Task-Id: 2f630f6e-b0b5-4193-85d3-d6ca39eccb28
@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps Bot commented May 13, 2026

Prompt To Fix All With AI
Fix the following 1 code review issue. Work through them one at a time, proposing concise fixes.

---

### Issue 1 of 1
apps/code/src/shared/utils/imageDataUrl.ts:26
**Early prefix check silently caps leading-whitespace tolerance**

The fast-path guard slices only the first 32 characters before testing `/^\s*data:/`. Because `"data:"` is 5 chars, any string with 28 or more leading whitespace characters before the payload will return `null` here — before `trim()` is ever called. The full regex applied to the trimmed string would have matched correctly. The existing "trims surrounding whitespace" test only exercises 2-char indentation, so this edge is untested. Low real-world impact (file content is unlikely to have 28+ leading spaces before a data URL), but the function documents and tests that it trims surrounding whitespace without mentioning this limit.

Reviews (2): Last reviewed commit: "fix(code): address PR review feedback on..." | Re-trigger Greptile

@pauldambra
Copy link
Copy Markdown
Member Author

Note

🤖 Automated comment by PR Shepherd — not written by a human

Updated verdict: ✅ APPROVE

All actionable findings from the original QA Swarm review have been resolved by 456b2992:

Finding Severity Status
CodeEditorPanel duplication 🟠 HIGH FilePanelImagePreview extracted
SVG file preview regression 🟡 MEDIUM svg dropped from IMAGE_MIME_TYPES
hasError state stickiness 🟡 MEDIUM ✅ Reset via render-time setState
data vs base64 naming drift 🟡 MEDIUM ✅ Renamed to base64
ReadToolView discriminated union 🟡 MEDIUM ↪️ Resolved out-of-scope
"Read in" copy regression 🟢 LOW ✅ Title restructured
MAX_BASE64_LENGTH misnamed 🟢 LOW ✅ Renamed to MAX_DATA_URL_LENGTH
parseImageDataUrl unmemoised 🟢 LOW useMemo added
Silent fall-through logging 🟢 LOW ↪️ Resolved out-of-scope
ErrorBoundary likely redundant ⚪ NIT ✅ Removed

CI is green except integration-test (still running). The shepherd will apply the stamphog label once that finishes.


Automated by PR Shepherd — not a human review

@pauldambra pauldambra added the Stamphog This will request an autostamp by stamphog on small changes label May 13, 2026
…arser

Greptile flagged that the 32-char slice in the fast-path guard
silently rejected data URLs with 28+ leading whitespace characters,
even when the value would have parsed cleanly after trim().

Drop the slice, check the size limit up front (cheap, O(1)), and let
the prefix regex tolerate up to 1024 leading whitespace characters —
generous for realistic indented payloads while still bounding the
scan so a pathological all-whitespace input can't burn CPU.

Generated-By: PostHog Code
Task-Id: 2f630f6e-b0b5-4193-85d3-d6ca39eccb28
@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps Bot commented May 13, 2026

Reviews (3): Last reviewed commit: "fix(code): widen fast-path leading-white..." | Re-trigger Greptile

@pauldambra pauldambra added Stamphog This will request an autostamp by stamphog on small changes and removed Stamphog This will request an autostamp by stamphog on small changes labels May 13, 2026
@pauldambra pauldambra requested a review from a team May 14, 2026 01:55
@pauldambra pauldambra merged commit d66fa3a into main May 15, 2026
15 checks passed
@pauldambra pauldambra deleted the posthog-code/render-png-and-data-url-previews branch May 15, 2026 20:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Stamphog This will request an autostamp by stamphog on small changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants