Convert model output image file to png then save/moderate#72108

Open

fisher-alice wants to merge 2 commits intoalice/scale-image-moderationfrom

alice/png-conversion

Contributor

fisher-alice commented Apr 13, 2026 •

edited

Loading

This PR converts model-generated images to PNG files to avoid the following reported error:

AzureAiContentSafety::RequestFailed: Request to Azure AI Content Safety failed with status 400
The image format is not supported. Please double check the API definition.

We use Vercel AI SDK for client implementation of generating assistant responses when Gemini Flash Image 2.5 model is selected. Currently, the media type is included in the returned generated file, but seems to not always match the actual file type as we return early if the file extension is not supported at:

code-dot-org/apps/src/util/moderateImage.ts

Lines 49 to 54 in 382b958

    
           const fileExtension = mimeToExtension(file.type) || ''; 
        
           if ( 
        
             !LABS_WITH_IMAGE_MODERATION.includes(appName ?? '') || 
        
             !ALLOWED_IMAGE_FILE_EXTENSIONS.includes(fileExtension) 
        
           ) { 
        
             return 'error';

But we are still reporting errors that 'the image format is not supported'.

So this PR converts browser-renderable images to PNG files before saving to project assets and moderating.

Links

Testing story

Tested locally on aichat levels with Gemini 2.5 Flash Image model selected. No change for user.

Deployment notes

Privacy and security


          convert model output image file to png then save/moderate

457287b

fisher-alice marked this pull request as ready for review

April 13, 2026 22:13

fisher-alice requested a review from Copilot

April 13, 2026 22:13

Copilot started reviewing on behalf of fisher-alice

April 13, 2026 22:14

Copilot AI reviewed

View reviewed changes

Contributor

Copilot AI left a comment

Pull request overview

This PR aims to prevent Azure AI Content Safety “image format not supported” 400s by normalizing model-generated images to PNG before uploading them as project assets and sending them to moderation.

Changes:

Convert Vercel AI SDK GeneratedFile images to a browser File in PNG format via canvas.
Update asset upload helper to accept a browser File (instead of GeneratedFile) and upload using file.type.
Simplify image moderation helper to accept a browser File directly.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 4 comments.

File	Description
apps/src/aichat/api/client/generateChatResponse.ts	Adds client-side PNG conversion and uses the converted `File` for upload + moderation.
apps/src/aichat/api/client/helpers/fileHelpers.ts	Changes generated-file upload to accept a browser `File` and removes `prepareGeneratedFile`.
apps/src/aichat/api/client/helpers/safetyHelpers.ts	Changes moderation helper to accept a browser `File` directly.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

apps/src/aichat/api/client/generateChatResponse.ts Outdated

Comment on lines +29 to +32

+                new Promise((resolve, reject) => {
+                  const blob = new Blob([new Uint8Array(file.uint8Array)], {
+                    type: file.mediaType,
+                  });

Copilot AI Apr 13, 2026

convertToPng sets the Blob MIME type from file.mediaType, but the PR description notes this value can be inaccurate. If the type is wrong, new Image() may fail to decode the object URL and the whole request will throw. Consider omitting the Blob type (or using a safe fallback) so the browser can sniff the bytes, and handle decode failures gracefully.

Copilot uses AI. Check for mistakes.

Contributor Author

fisher-alice Apr 14, 2026

Good feedback - will omit mediaType.

apps/src/aichat/api/client/generateChatResponse.ts Outdated

+                    const canvas = document.createElement('canvas');
+                    canvas.width = img.naturalWidth;
+                    canvas.height = img.naturalHeight;
+                    canvas.getContext('2d')?.drawImage(img, 0, 0);

Copilot AI Apr 13, 2026

convertToPng draws with canvas.getContext('2d')?.drawImage(...) but doesn’t check whether the 2D context was actually created. If getContext('2d') returns null, this will silently produce a blank PNG that still gets uploaded and moderated. It’d be safer to reject (or throw) when the context is missing.

Suggested change

      
                  canvas.getContext('2d')?.drawImage(img, 0, 0);
          
                  const context = canvas.getContext('2d');
          
                  if (!context) {
          
                    URL.revokeObjectURL(url);
          
                    reject(new Error('canvas 2d context creation failed'));
          
                    return;
          
                  }
          
                  context.drawImage(img, 0, 0);

Copilot uses AI. Check for mistakes.

Contributor Author

fisher-alice Apr 14, 2026

makes sense - will update

apps/src/aichat/api/client/generateChatResponse.ts

Comment on lines +124 to +128

+                    // Gemini API output format is not configurable, so we convert to PNG for consistent asset storage and moderation.
+                    const fileToSave = await convertToPng(file);
+                    let asset: ChatAsset;
+                    try {

Copilot AI Apr 13, 2026

await convertToPng(file) is outside the existing try/catch. If image decode or canvas.toBlob fails, this rejection will throw out of generateChatResponse, and performClientApiChatCompletion doesn’t catch it—so the user may lose the text response. Consider wrapping the PNG conversion in the same error-handling path as asset upload (log + continue) to preserve the current “return text even if file handling fails” behavior.

Suggested change

      
                  // Gemini API output format is not configurable, so we convert to PNG for consistent asset storage and moderation.
          
                  const fileToSave = await convertToPng(file);
          
                  let asset: ChatAsset;
          
                  try {
          
                  let fileToSave: GeneratedFile;
          
                  let asset: ChatAsset;
          
                  try {
          
                    // Gemini API output format is not configurable, so we convert to PNG for consistent asset storage and moderation.
          
                    fileToSave = await convertToPng(file);

Copilot uses AI. Check for mistakes.

Contributor Author

fisher-alice Apr 14, 2026

Adding another try-catch for possible conversion error (but not combining) in order to log specific error.

apps/src/aichat/api/client/generateChatResponse.ts Outdated

+              // Also, Vercel AI SDK does not always report media type accurately (seen from HoneyBadger Azure error reports).
+              const convertToPng = (file: GeneratedFile): Promise<File> =>
+                new Promise((resolve, reject) => {
+                  const blob = new Blob([new Uint8Array(file.uint8Array)], {

Copilot AI Apr 13, 2026

new Uint8Array(file.uint8Array) copies the image bytes before creating the Blob. Since file.uint8Array is already a Uint8Array, you can pass it directly to Blob to avoid an extra allocation/copy (which can be noticeable for large images).

Suggested change

      
                const blob = new Blob([new Uint8Array(file.uint8Array)], {
          
                const blob = new Blob([file.uint8Array], {

Copilot uses AI. Check for mistakes.

Contributor Author

fisher-alice Apr 14, 2026

TS error otherwise.


          omit mimetype and add check and catch for canvas context

c130d2e

fisher-alice mentioned this pull request

Scale down large images before sending to Azure moderation service #72099

Open

fisher-alice requested a review from a team

April 14, 2026 15:06

sanchitmalhotra126 reviewed

View reviewed changes

apps/src/aichat/api/client/generateChatResponse.ts

+              // Converts any browser-renderable image to PNG via canvas.
+              // Used with model-generated images since Vercel AI SDK doesn't currently expose output format configuration.
+              // Also, Vercel AI SDK does not always report media type accurately (seen from HoneyBadger Azure error reports).
+              const convertToPng = (file: GeneratedFile): Promise<File> =>

Contributor

sanchitmalhotra126 Apr 14, 2026

I wonder, should do this on the gateway side? cc @edcodedotorg

And/or should we split this into two operations: 1) check actual file type (using magic bytes etc) and report discrepancies and 2) convert unsupported file types to a supported file type (PNG in this case)?

Contributor

edcodedotorg Apr 14, 2026

@sanchitmalhotra126 see questions below

Contributor

edcodedotorg commented Apr 14, 2026 •

edited

Loading

This PR converts model-generated images to PNG files to avoid the following reported error:
AzureAiContentSafety::RequestFailed: Request to Azure AI Content Safety failed with status 400
The image format is not supported. Please double check the API definition.
Example report

We use Vercel AI SDK for client implementation of generating assistant responses when Gemini Flash Image 2.5 model is selected. Currently, the media type is included in the returned generated file, but seems to not always match the actual file type as we return early if the file extension is not supported at:

code-dot-org/apps/src/util/moderateImage.ts

Lines 49 to 54 in 382b958

const fileExtension = mimeToExtension(file.type) || '';

if (

!LABS_WITH_IMAGE_MODERATION.includes(appName ?? '') ||

!ALLOWED_IMAGE_FILE_EXTENSIONS.includes(fileExtension)

) {

return 'error';

But we are still reporting errors that 'the image format is not supported'.

I am a bit confused by the above.

How does the "return early" code snippet tie into media type not matching actual file type? (as in but seems to not always match the actual file type **as** we return early if the file extension seems to imply it ties in)
And if this is the cause why don't we fix that?
What is meant by But we are still reporting errors that 'the image format is not supported'. - I don't understand the but here, are we reporting not supported when it actually is supported?

So this PR converts browser-renderable images to PNG files before saving to project assets and moderating.

@sanchitmalhotra126 I think our moderation (generally and image specifically) being a concern of the gateway is a real possibility but we need to get crisp about what our story is..

Do we have a list of file types that Gemini is currently returning that are not supported by the moderation layer?
Do we have a set of image types we want supported for specific user use-cases?
Is there no way to limit the generation to formats that are supported or that are otherwise intended (based on user use case not just what the moderation layer requires - these can be different - see below)?
Is the plan/implementation here to always convert to our supported formats (seems PNG currently) JUST for moderation purposes but returning the original file for the user?

Contributor

sanchitmalhotra126 commented Apr 14, 2026

My current understanding of the situation is that the declared mediaType doesn't match the actual media type of the file (based on its byte signature). So it passes our file type checks but fails when Azure tries to process the image.

@sanchitmalhotra126 I think our moderation (generally and image specifically) being a concern of the gateway is a real possibility but we need to get crisp about what our story is..

100%. Re:

Is there no way to limit the generation to formats that are supported or that are otherwise intended (based on user use case not just what the moderation layer requires - these can be different - see below)?

As far as I can tell, the Gemini API does not allow us to specify an output format for image types (based on some forum comments it seems to default to JPEG for speed).

Whether or not we absorb this into the gateway right away, I think our code somewhere needs to answer the following questions:

Does the file's mediaType match the actual file contents (by checking magic bytes)?
Is the declared media type and actual file media type part of the file types we accept (e.g. for moderation purposes, and/or if we just want to limit the types of file we store on our system)?

After which, we can decide to report on any discrepancies and/or convert to a known format (or just discard if we want to treat this as a generation error).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet