feat: prompt AI to prefer ReadMediaFile over Python frame extraction for video analysis#816
Conversation
…for video analysis When users upload video files for analysis, the AI was writing Python scripts to extract frames instead of using the built-in ReadMediaFile tool. This change adds explicit prompt-level guidance in both tool descriptions: - read-media.md: new tip reminding the model to prefer ReadMediaFile directly for video analysis rather than writing Python scripts to extract frames - read.md: reinforces the same guidance when refusing binary/video files, explicitly telling the model not to use Python scripts for frame extraction Fixes the feedback from Q-0266: "Kimi CLI 视频分析希望默认调用 ReadMediaFile 而不是写 Python 切帧".
|
commit: |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 548aa7985f
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| - If the file doesn't exist or path is invalid, an error will be returned. | ||
| - The maximum size that can be read is {{ MAX_MEDIA_MEGABYTES }}MB. An error will be returned if the file is larger than this limit. | ||
| - The media content will be returned in a form that you can directly view and understand. | ||
| - When analyzing video files, prefer using ReadMediaFile directly rather than writing Python scripts to extract frames. The model can directly view and understand the video content returned by this tool. |
There was a problem hiding this comment.
When the current model has image_in but not video_in (a supported configuration in the capability registry and covered by the read-media tests), ReadMediaFile is still exposed for images, but ReadMediaFileTool.execution rejects video files with “current model does not support video input.” This new unconditional guidance tells the model not to extract frames and to use ReadMediaFile instead, so local video-analysis requests on image-only models are steered into a tool call that must fail instead of the existing frame-extraction fallback. Please make this guidance conditional on video_in or mention the fallback when videos are unsupported.
Useful? React with 👍 / 👎.
1 participant
Problem
When users upload video files for analysis, the AI was writing Python scripts to extract frames instead of using the built-in ReadMediaFile tool. This behavior was reported in feedback Q-0266.
Solution
Add explicit prompt-level guidance in both tool descriptions to steer the model toward using ReadMediaFile directly for video content:
Changes
read-media.md — Added a new tip in the Tips section:
read.md — Reinforced the guidance when describing binary file handling:
Why this approach
Test Results
Related