Skip to content

feat(scene): add scene_view screenshot capture source and fix viewport handling#872

Closed
jiajunfeng wants to merge 5 commits intoCoplayDev:betafrom
jiajunfeng:feat/scene-view-screenshot-capture
Closed

feat(scene): add scene_view screenshot capture source and fix viewport handling#872
jiajunfeng wants to merge 5 commits intoCoplayDev:betafrom
jiajunfeng:feat/scene-view-screenshot-capture

Conversation

@jiajunfeng
Copy link

@jiajunfeng jiajunfeng commented Mar 6, 2026

Adds capture_source to manage_scene screenshot, including support for scene_view.

  • adds scene_view_target to frame a target before Scene View capture
  • fixes Scene View viewport handling to use consistent viewport-local coordinates
  • rejects capture_source='scene_view' with super_size > 1 to avoid unsupported metadata/behavior mismatch

Validation:

  • cd Server && uv run python -m pytest tests/integration/test_manage_scene_screenshot_params.py -q
  • Result: 16 passed in 0.20s
  • ran targeted Unity EditMode regression for Screenshot_SceneViewRejectsSupersizeAboveOne

Summary by Sourcery

Add support for capturing screenshots from the Unity Scene View, extend object reference handling for sprites, and update tooling, docs, and tests accordingly.

New Features:

  • Introduce a capture_source option for screenshots, including Scene View capture with optional scene_view_target framing and CLI/agent wiring.
  • Add an editor utility to grab Scene View viewport pixels for accurate Scene View-based screenshots.

Bug Fixes:

  • Ensure Scene View screenshot capture uses consistent viewport-local coordinates to avoid incorrect framing.

Enhancements:

  • Improve Scene View framing by centralizing bounds calculation across RectTransforms, renderers, and colliders.
  • Extend component property setting to prefer SerializedProperty resolution for complex UnityEngine.Object JSON references, including spriteName/fileID selection for sprites.
  • Add long coercion helpers for JSON parameters and support sprite lookup by GUID+spriteName or GUID+fileID.
  • Record captureSource metadata in screenshot responses and enforce constraints such as disallowing Scene View capture in batch mode or with supersize > 1.
  • Update Roslyn installer dependencies to include System.Runtime.CompilerServices.Unsafe for netstandard2.0 compatibility.
  • Add a high-level AGENTS.md guide describing repository structure, workflows, and contribution practices.

Build:

  • Add uv.lock to the Server project to lock Python dependencies.

Documentation:

  • Document new screenshot capture_source options and Scene View limitations in the CLI usage guide.

Tests:

  • Add integration tests for capture_source and scene_view_target forwarding and restrictions, and a Unity EditMode regression test for Scene View supersize rejection.

Summary by CodeRabbit

Release Notes

  • New Features

    • Added Scene View screenshot capture capability with optional image inclusion and resolution scaling
    • Enhanced sprite reference resolution supporting atlas lookup and fileID matching
    • New --capture-source and --scene-view-target CLI options for controlling scene screenshot capture
  • Documentation

    • Added comprehensive repository guidelines
    • Updated screenshot command usage documentation
  • Tests

    • Added Scene View capture validation tests

@sourcery-ai
Copy link
Contributor

sourcery-ai bot commented Mar 6, 2026

Reviewer's Guide

Adds support for capturing screenshots from the Unity Scene View (alongside existing Game View capture), introduces a shared viewport-based Scene View capture utility, tightens parameter validation/forwarding across CLI, Python tool, and Unity editor layers, enhances object reference handling for sprites, and updates Roslyn dependencies and documentation.

Sequence diagram for screenshot capture with capture_source

sequenceDiagram
    actor User
    participant CLI_scene as CLI_scene_command
    participant Py_manage as Python_manage_scene_tool
    participant Unity_IPC as Unity_manage_scene_IPC
    participant ManageScene as ManageScene
    participant EditorSS as EditorWindowScreenshotUtility

    User->>CLI_scene: unity-mcp scene screenshot --capture-source scene_view --scene-view-target "Canvas" --include-image
    CLI_scene->>Py_manage: build params (captureSource, sceneViewTarget, includeImage, etc.)
    Py_manage->>Unity_IPC: run_command action=screenshot params
    Unity_IPC->>ManageScene: CaptureScreenshot(SceneCommand)

    Note over ManageScene: Normalize and validate captureSource
    ManageScene->>ManageScene: captureSource = game_view or scene_view
    alt invalid capture_source
        ManageScene-->>Unity_IPC: ErrorResponse (invalid capture_source)
        Unity_IPC-->>Py_manage: error
        Py_manage-->>CLI_scene: tool error
        CLI_scene-->>User: show error
    else capture_source == scene_view
        ManageScene->>ManageScene: validate superSize, batch, lookAt, viewPosition, viewRotation, camera
        alt params invalid
            ManageScene-->>Unity_IPC: ErrorResponse if invalid combo
            Unity_IPC-->>Py_manage: error
            Py_manage-->>CLI_scene: tool error
            CLI_scene-->>User: show error
        else params valid
            ManageScene->>ManageScene: CaptureSceneViewScreenshot(cmd, fileName,...)
            ManageScene->>ManageScene: FrameSceneView(SceneCommand{sceneViewTarget}) opt sceneViewTarget
            ManageScene->>EditorSS: CaptureSceneViewViewportToAssets(sceneView,...)
            EditorSS->>EditorSS: FocusAndRepaint(sceneView)
            EditorSS->>EditorSS: GetSceneViewViewportPixelRect(sceneView)
            EditorSS->>EditorSS: CaptureViewRect(sceneView, viewportRect)
            EditorSS-->>ManageScene: ScreenshotCaptureResult (imageBase64?, viewportWidth/Height)
            ManageScene-->>Unity_IPC: SuccessResponse data{captureSource=scene_view, captureMode=scene_view_viewport,...}
            Unity_IPC-->>Py_manage: success + image metadata
            Py_manage-->>CLI_scene: ToolResult (image extracted if include_image)
            CLI_scene-->>User: path, inline image, metadata
        end
    else capture_source == game_view
        ManageScene->>ManageScene: existing camera-based capture paths
        ManageScene-->>Unity_IPC: SuccessResponse data{captureSource=game_view,...}
        Unity_IPC-->>Py_manage: success
        Py_manage-->>CLI_scene: ToolResult
        CLI_scene-->>User: output
    end
Loading

Updated class diagram for screenshot capture utilities

classDiagram
    class SceneCommand {
        +string camera
        +string captureSource
        +bool? includeImage
        +int? maxResolution
        +string batch
        +JToken sceneViewTarget
        +JToken lookAt
        +int? superSize
        +Vector3? viewPosition
        +Quaternion? viewRotation
        +string fileName
    }

    class ManageScene {
        -static object CaptureScreenshot(SceneCommand cmd)
        -static object CaptureSceneViewScreenshot(SceneCommand cmd, string fileName, int resolvedSuperSize, bool includeImage, int maxResolution)
        -static object FrameSceneView(SceneCommand cmd)
        -static Bounds CalculateFrameBounds(GameObject target)
        -static bool TryGetRectTransformBounds(GameObject target, out Bounds bounds)
        -static bool TryGetRendererBounds(GameObject target, out Bounds bounds)
        -static bool TryGetColliderBounds(GameObject target, out Bounds bounds)
    }

    class EditorWindowScreenshotUtility {
        +const string ScreenshotsFolderName
        +const int RepaintSettlingDelayMs
        +static ScreenshotCaptureResult CaptureSceneViewViewportToAssets(SceneView sceneView, string fileName, int superSize, bool ensureUniqueFileName, bool includeImage, int maxResolution, out int viewportWidth, out int viewportHeight)
        -static void FocusAndRepaint(SceneView sceneView)
        -static Rect GetSceneViewViewportPixelRect(SceneView sceneView)
        -static Rect GetViewportLocalRectPoints(SceneView sceneView, float pixelsPerPoint)
        -static Texture2D CaptureViewRect(SceneView sceneView, Rect viewportRectPixels)
        -static object GetHostView(EditorWindow window)
        -static Rect? GetRectProperty(object instance, string propertyName)
        -static void InvokeMethodIfExists(object instance, string methodName)
        -static void FlipTextureVertically(Texture2D texture)
        -static ScreenshotCaptureResult PrepareCaptureResult(string fileName, int superSize, bool ensureUniqueFileName)
        -static string BuildFileName(string fileName)
        -static string EnsureUnique(string fullPath)
        -static void DestroyTexture(Texture2D texture)
    }

    class ScreenshotCaptureResult {
        +string FullPath
        +string AssetsRelativePath
        +int SuperSize
        +bool IsAsync
        +string ImageBase64
        +int ImageWidth
        +int ImageHeight
    }

    class ScreenshotUtility {
        +static Texture2D DownscaleTexture(Texture2D source, int maxResolution)
    }

    class SceneView {
        +Camera camera
        +GUIContent titleContent
        +static SceneView lastActiveSceneView
        +void Frame(Bounds bounds, bool instant)
    }

    class SuccessResponse {
        +SuccessResponse(string message, object data)
    }

    class ErrorResponse {
        +ErrorResponse(string message)
    }

    ManageScene ..> SceneCommand : uses
    ManageScene ..> EditorWindowScreenshotUtility : uses
    ManageScene ..> ScreenshotCaptureResult : returns
    ManageScene ..> SuccessResponse : returns
    ManageScene ..> ErrorResponse : returns
    EditorWindowScreenshotUtility ..> ScreenshotCaptureResult : creates
    EditorWindowScreenshotUtility ..> ScreenshotUtility : calls
    EditorWindowScreenshotUtility ..> SceneView : uses
    EditorWindowScreenshotUtility ..> Texture2D : uses
    EditorWindowScreenshotUtility ..> RenderTexture : uses
    EditorWindowScreenshotUtility ..> McpLog : logs
    EditorWindowScreenshotUtility ..> EditorWindow : uses
    EditorWindowScreenshotUtility ..> Rect : uses
Loading

Updated class diagram for ComponentOps and ParamCoercion sprite/object handling

classDiagram
    class ComponentOps {
        -static bool TrySetViaReflection(object component, Type type, string propertyName, string normalizedName, JToken value, BindingFlags flags, out string error)
        -static bool SetObjectReference(SerializedProperty prop, JToken value, out string error)
        -static bool SetEnum(SerializedProperty prop, JToken value, out string error)
        -static long GetSpriteFileId(Sprite sprite)
    }

    class ParamCoercion {
        +static int CoerceInt(JToken token, int defaultValue)
        +static int? CoerceIntNullable(JToken token)
        +static bool? CoerceBoolNullable(JToken token)
        +static long CoerceLong(JToken token, long defaultValue)
        +static long? CoerceLongNullable(JToken token)
    }

    class SerializedProperty {
        +UnityEngine.Object objectReferenceValue
        +string propertyPath
    }

    class Sprite {
        +string name
    }

    class AssetDatabase {
        +static UnityEngine.Object LoadAssetAtPath~UnityEngine.Object~(string path)
        +static UnityEngine.Object[] LoadAllAssetsAtPath(string path)
    }

    class GlobalObjectId {
        +static GlobalObjectId GetGlobalObjectIdSlow(UnityEngine.Object obj)
        +long targetObjectId
    }

    ComponentOps ..> ParamCoercion : uses CoerceLong
    ComponentOps ..> SerializedProperty : sets
    ComponentOps ..> AssetDatabase : loads assets
    ComponentOps ..> Sprite : matches by name
    ComponentOps ..> GlobalObjectId : uses for fileID
    ParamCoercion ..> JToken : consumes
Loading

File-Level Changes

Change Details Files
Add capture_source support with new scene_view capture path and parameter validation across Unity editor, Python tool, and CLI.
  • Extend SceneCommand and parameter mapping to accept captureSource/capture_source and optional sceneViewTarget.
  • Route capture_source='scene_view' requests through a new CaptureSceneViewScreenshot path with early validation rejecting unsupported combinations (batch, lookAt/view_position/view_rotation, camera, super_size>1, batch mode).
  • Ensure game_view captures include captureSource='game_view' in responses and add tests for scene_view forwarding and invalid capture_source usage for non-screenshot actions.
  • Expose --capture-source and --scene-view-target flags in the CLI screenshot command and forward them into manage_scene parameters.
  • Document scene_view capture usage and supersize limitations in CLI usage docs.
MCPForUnity/Editor/Tools/ManageScene.cs
Server/tests/integration/test_manage_scene_screenshot_params.py
Server/src/cli/commands/scene.py
Server/src/services/tools/manage_scene.py
Server/src/cli/CLI_USAGE_GUIDE.md
Implement robust Scene View viewport screenshot capture utility and improve framing bounds calculation for Scene View targets.
  • Introduce EditorWindowScreenshotUtility to capture the active Scene View’s visible viewport using the editor host view’s GrabPixels, handling focus/repaint, pixel-rect resolution, texture flipping, downscaling, and unique file naming.
  • Use EditorWindowScreenshotUtility from CaptureSceneViewScreenshot to generate Scene View screenshots with metadata including captureSource, captureMode, viewport dimensions, sceneViewName, and optional inline image.
  • Refactor FrameSceneView to use a new CalculateFrameBounds helper that prefers RectTransform bounds, then renderers, then colliders, with dedicated helpers for each to handle inactive children and tiny bounds.
MCPForUnity/Editor/Tools/ManageScene.cs
MCPForUnity/Editor/Helpers/EditorWindowScreenshotUtility.cs
Improve serialized object reference handling for UnityEngine.Object fields set via JSON, including sprite lookups by GUID, spriteName, or fileID.
  • Short-circuit reflection-based property/field setting when a JObject value targets a UnityEngine.Object, deferring to SerializedProperty-based resolution instead.
  • Extend SetObjectReference to resolve sprites inside atlases by spriteName or fileID when a GUID is provided, falling back to the main asset if no nested sprite match is found.
  • Add a CoerceLong/CoerceLongNullable helper for robust long parsing from JTokens and use it for fileID coercion, plus a GetSpriteFileId helper that uses GlobalObjectId to match sprite file IDs safely.
MCPForUnity/Editor/Helpers/ComponentOps.cs
MCPForUnity/Editor/Helpers/ParamCoercion.cs
Update Roslyn installer dependencies to include required Unsafe runtime DLL and add repo-level contributor guidelines for agents.
  • Add system.runtime.compilerservices.unsafe package to the RoslynInstaller’s DLL manifest to satisfy System.Memory dependencies for System.Reflection.Metadata on netstandard2.0.
  • Introduce AGENTS.md documenting repository layout, dev workflows, coding style, testing, PR conventions, and security tips for automated agents and contributors.
MCPForUnity/Editor/Setup/RoslynInstaller.cs
AGENTS.md
Expand Unity EditMode coverage for new Scene View screenshot validation behavior.
  • Add an EditMode test that verifies Scene View screenshot requests with superSize>1 are rejected with an appropriate error message from ManageScene.HandleCommand.
TestProjects/UnityMCPTests/Assets/Tests/EditMode/Tools/ManageSceneHierarchyPagingTests.cs

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@coderabbitai
Copy link
Contributor

coderabbitai bot commented Mar 6, 2026

📝 Walkthrough

Walkthrough

This PR introduces Scene View screenshot capture capability to complement existing Game View capture. It adds a new EditorWindowScreenshotUtility for viewport rendering, enhances ComponentOps for sprite reference resolution via JSON, extends the CLI with capture-source options, and adds corresponding server-side parameter handling with validation constraints.

Changes

Cohort / File(s) Summary
Scene View Screenshot Infrastructure
MCPForUnity/Editor/Helpers/EditorWindowScreenshotUtility.cs, MCPForUnity/Editor/Helpers/EditorWindowScreenshotUtility.cs.meta
New utility class for capturing SceneView viewport to PNG via reflection-based pixel grab; includes viewport resolution, texture encoding, Base64 encoding, and downscaling to max resolution with robust error handling.
Scene Management Enhancements
MCPForUnity/Editor/Tools/ManageScene.cs
Added captureSource field to SceneCommand; routes scene_view captures through new CaptureSceneViewScreenshot flow; enforced constraints (no super_size > 1, batch mode, look_at, or custom camera for scene_view); introduced CalculateFrameBounds and related helpers for consistent bounds calculation.
Sprite/Component Reference Handling
MCPForUnity/Editor/Helpers/ComponentOps.cs
Enhanced reflection path to defer UnityEngine.Object assignments to SerializedProperty; expanded SetObjectReference to resolve sprites by spriteName within atlas or by fileID; added GetSpriteFileId helper using GlobalObjectId for stable sprite identification.
Parameter Coercion & Infrastructure
MCPForUnity/Editor/Helpers/ParamCoercion.cs, MCPForUnity/Editor/Setup/RoslynInstaller.cs
Added CoerceLong and CoerceLongNullable methods mirroring existing int coercion patterns; added System.Runtime.CompilerServices.Unsafe (v4.5.3) NuGet entry as transitive dependency.
CLI & Server-side Integration
Server/src/cli/commands/scene.py, Server/src/services/tools/manage_scene.py, Server/src/cli/CLI_USAGE_GUIDE.md
Added --capture-source and --scene-view-target CLI options to screenshot command; extended manage_scene function with capture_source parameter; added validation to reject capture_source for non-screenshot actions; updated documentation with new flags and supersize limitation note.
Tests & Documentation
Server/tests/integration/test_manage_scene_screenshot_params.py, TestProjects/UnityMCPTests/Assets/Tests/EditMode/Tools/ManageSceneHierarchyPagingTests.cs, AGENTS.md
Added integration tests for capture_source parameter forwarding and error handling; added test validating super_size rejection for scene_view capture; introduced AGENTS.md with comprehensive repository guidelines (structure, build/test commands, coding style, testing, commits, security).

Sequence Diagram(s)

sequenceDiagram
    participant Client as CLI Client
    participant Server as Python Server
    participant Editor as Unity Editor
    participant FileSystem as File System
    participant SceneView as Scene View

    Client->>Server: screenshot<br/>capture_source=scene_view
    Server->>Server: Validate capture_source<br/>with action=screenshot
    Server->>Editor: Request screenshot<br/>captureSource=scene_view
    Editor->>Editor: Validate constraints<br/>(no super_size > 1, etc.)
    Editor->>SceneView: Focus & repaint SceneView
    SceneView->>SceneView: Render viewport
    Editor->>Editor: Capture viewport pixels<br/>via GrabPixels reflection
    Editor->>Editor: Flip texture vertically<br/>Encode to PNG
    Editor->>FileSystem: Save PNG to<br/>Assets/Screenshots
    Editor->>Editor: Compute viewport dimensions<br/>Optional Base64 encode
    Editor->>Server: Return metadata<br/>(path, dimensions, image)
    Server->>Client: Response with file path<br/>& optional image payload
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Possibly related PRs

Suggested labels

codex

Poem

🐰 A hop through the viewport we go,
Scene View now captured, a pixelated show!
Sprites dance by fileID or name,
Bounds calculated with careful frame.
CLI options spring forth with glee,
Screenshot capture now wild and free!

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 17.50% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (2 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly describes the main feature addition (scene_view screenshot capture source) and references a key fix (viewport handling).
Description check ✅ Passed The description covers the main changes, type of change (new feature), validation performed, and provides clear context. However, the template's structured sections are not explicitly followed with proper formatting.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment

Tip

Try Coding Plans. Let us write the prompt for your AI agent so you can ship faster (with fewer bugs).
Share your feedback on Discord.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Contributor

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've left some high level feedback:

  • The CoerceLongNullable helper in ParamCoercion is currently unused; consider removing it or wiring it into call sites to avoid dead code.
  • The checks in ComponentOps.TrySetViaReflection for JObject values targeting UnityEngine.Object fields/properties are duplicated across three branches; consider extracting this into a small helper to reduce repetition and keep the intent in one place.
  • In EditorWindowScreenshotUtility.FocusAndRepaint, the Thread.Sleep on the editor main thread may briefly stall the editor; if you hit responsiveness issues, consider an async/EditorApplication.delayCall-based approach to wait for repaint instead of blocking.
Prompt for AI Agents
Please address the comments from this code review:

## Overall Comments
- The `CoerceLongNullable` helper in `ParamCoercion` is currently unused; consider removing it or wiring it into call sites to avoid dead code.
- The checks in `ComponentOps.TrySetViaReflection` for `JObject` values targeting `UnityEngine.Object` fields/properties are duplicated across three branches; consider extracting this into a small helper to reduce repetition and keep the intent in one place.
- In `EditorWindowScreenshotUtility.FocusAndRepaint`, the `Thread.Sleep` on the editor main thread may briefly stall the editor; if you hit responsiveness issues, consider an async/`EditorApplication.delayCall`-based approach to wait for repaint instead of blocking.

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (2)
Server/src/services/tools/manage_scene.py (1)

186-199: ⚠️ Potential issue | 🟠 Major

Enforce the Scene View supersize restriction in the tool layer.

Line 197 only checks the action, so direct manage_scene(..., action="screenshot", capture_source="scene_view", screenshot_super_size=2) calls still pass through. That bypasses the new unsupported-combination guard and can reintroduce the metadata/behavior mismatch this PR is meant to prevent.

Suggested guard
         if coerced_max_resolution is not None and coerced_max_resolution <= 0:
             return {"success": False, "message": "max_resolution must be a positive integer greater than zero."}
@@
         if coerced_max_resolution is not None:
             params["maxResolution"] = coerced_max_resolution
         if capture_source is not None:
             if action != "screenshot":
                 return {"success": False, "message": "capture_source is only valid for action='screenshot'."}
+            if capture_source == "scene_view" and coerced_super_size not in (None, 1):
+                return {
+                    "success": False,
+                    "message": "capture_source='scene_view' only supports screenshot_super_size=1.",
+                }
             params["captureSource"] = capture_source
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@Server/src/services/tools/manage_scene.py` around lines 186 - 199,
manage_scene currently only checks action before accepting capture_source,
allowing calls like manage_scene(..., action="screenshot",
capture_source="scene_view", screenshot_super_size=2) to bypass the new
restriction; add a guard in the screenshot handling block that if capture_source
== "scene_view" then reject requests with a supersize > 1 (check
coerced_super_size or screenshot_super_size) and return a failure message
(similar to the existing capture_source/action check). Target the capture_source
handling inside manage_scene and validate coerced_super_size before setting
params["captureSource"].
Server/src/cli/commands/scene.py (1)

315-327: ⚠️ Potential issue | 🟡 Minor

Reject --scene-view-target unless Scene View capture is selected.

Line 361 forwards sceneViewTarget even when Line 327 keeps the default game_view. That makes unity-mcp scene screenshot --scene-view-target ... silently do the wrong thing instead of failing fast.

Suggested guard
 def screenshot(filename: Optional[str], supersize: int, camera: Optional[str],
                include_image: bool, max_resolution: Optional[int],
                capture_source: str,
                batch: Optional[str], look_at: Optional[str],
                view_position: Optional[str], view_rotation: Optional[str],
                orbit_angles: Optional[int], orbit_elevations: Optional[str],
                orbit_distance: Optional[float], orbit_fov: Optional[float],
                output_dir: Optional[str], scene_view_target: Optional[str]):
@@
     config = get_config()
+    normalized_capture_source = capture_source.lower()
+    if scene_view_target and normalized_capture_source != "scene_view":
+        print_error("--scene-view-target requires --capture-source scene_view")
+        sys.exit(1)
 
     params: dict[str, Any] = {"action": "screenshot"}
@@
-    if capture_source:
-        params["captureSource"] = capture_source.lower()
+    params["captureSource"] = normalized_capture_source
@@
     if scene_view_target:
         params["sceneViewTarget"] = scene_view_target

Also applies to: 361-362

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@Server/src/cli/commands/scene.py` around lines 315 - 327, The current params
building sends sceneViewTarget even when captureSource is the default game view;
update the CLI validation so that when a user supplies scene_view_target
(sceneViewTarget) we validate capture_source (captureSource) is set to
'scene_view' before adding it to params — if capture_source is not 'scene_view'
and scene_view_target is present, raise a user-facing error/exit early;
otherwise, add params["sceneViewTarget"] = scene_view_target only when
capture_source.lower() == "scene_view". Ensure this check is performed near
where params and capture_source are handled (the existing params dict
construction and the code that currently adds sceneViewTarget).
🧹 Nitpick comments (2)
MCPForUnity/Editor/Helpers/EditorWindowScreenshotUtility.cs (1)

104-117: Consider documenting the 75ms delay rationale.

The Thread.Sleep(RepaintSettlingDelayMs) blocks the editor thread. While acceptable for screenshot capture, a brief comment explaining why this delay is necessary (e.g., waiting for GPU rendering to complete) would help future maintainers.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@MCPForUnity/Editor/Helpers/EditorWindowScreenshotUtility.cs` around lines 104
- 117, Add a brief inline comment above the Thread.Sleep(RepaintSettlingDelayMs)
call in EditorWindowScreenshotUtility (the repaint block that calls
sceneView.Repaint(), InvokeMethodIfExists(sceneView, "RepaintImmediately"),
SceneView.RepaintAll(),
UnityEditorInternal.InternalEditorUtility.RepaintAllViews(),
EditorApplication.QueuePlayerLoopUpdate()) that explains the rationale for the
75ms delay (e.g., to allow the editor repaint and GPU/framebuffer swap to settle
before capturing a screenshot) and note that it intentionally blocks the editor
thread for a short period; include the constant name RepaintSettlingDelayMs in
the comment so future maintainers understand why this specific delay is used and
where to adjust it if needed.
MCPForUnity/Editor/Helpers/ComponentOps.cs (1)

835-849: Potential overflow when casting targetObjectId to long.

GlobalObjectId.targetObjectId is a ulong, and casting directly to long can produce negative values for IDs exceeding long.MaxValue. While rare, this could cause matching failures if the fileID comparison involves such large values.

Defensive approach using unchecked cast
         private static long GetSpriteFileId(Sprite sprite)
         {
             if (sprite == null)
                 return 0;
 
             try
             {
                 var globalId = GlobalObjectId.GetGlobalObjectIdSlow(sprite);
-                return (long)globalId.targetObjectId;
+                return unchecked((long)globalId.targetObjectId);
             }
             catch
             {
                 return 0;
             }
         }

This makes the overflow explicit, though you should ensure the caller (CoerceLong) also handles negative values consistently.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@MCPForUnity/Editor/Helpers/ComponentOps.cs` around lines 835 - 849, The cast
of GlobalObjectId.targetObjectId (ulong) to long in GetSpriteFileId can overflow
and produce negative IDs; update the cast to be explicit about overflow by using
an unchecked cast (e.g., unchecked((long)globalId.targetObjectId)) so the
wraparound is intentional, and keep the existing try/catch around
GlobalObjectId.GetGlobalObjectIdSlow; also verify that the caller CoerceLong
(and any code that compares file IDs) treats negative values consistently or
adds normalization/clamping if needed.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@MCPForUnity/Editor/Helpers/ComponentOps.cs`:
- Around line 643-666: The code currently falls through and calls
AssetDatabase.LoadAssetAtPath when fileID was present but no matching sprite was
found; change the behavior so that when jObj["fileID"] exists and
ParamCoercion.CoerceLong yields a non‑zero targetFileId but no sprite in
AssetDatabase.LoadAllAssetsAtPath matches GetSpriteFileId, the method should not
load the main asset — instead set prop.objectReferenceValue = null (or leave
unchanged) and return false and optionally log a clear error including the
targetFileId and path; update the logic around fileIdToken/targetFileId and the
foreach that uses GetSpriteFileId to detect the “not found” case and
early-return false rather than falling through to AssetDatabase.LoadAssetAtPath.

In `@MCPForUnity/Editor/Helpers/ParamCoercion.cs`:
- Around line 105-106: ParamCoercion.CoerceLong currently accepts a float parsed
by double.TryParse and truncates it to a long, which turns non-integral inputs
like "12.9" into 12; change the double.parse branches (the one around the
current TryParse at the top and the similar one around lines ~139-140) to only
succeed when the parsed double is mathematically integral and fits in Int64
(e.g., check that Math.Truncate(d) == d and d is between long.MinValue and
long.MaxValue) and otherwise treat it as a failed coercion; this prevents
CoerceLong from silently converting fractional values used for fileID (used by
ComponentOps.CoerceLong caller for fileID matching) into incorrect identifiers.

---

Outside diff comments:
In `@Server/src/cli/commands/scene.py`:
- Around line 315-327: The current params building sends sceneViewTarget even
when captureSource is the default game view; update the CLI validation so that
when a user supplies scene_view_target (sceneViewTarget) we validate
capture_source (captureSource) is set to 'scene_view' before adding it to params
— if capture_source is not 'scene_view' and scene_view_target is present, raise
a user-facing error/exit early; otherwise, add params["sceneViewTarget"] =
scene_view_target only when capture_source.lower() == "scene_view". Ensure this
check is performed near where params and capture_source are handled (the
existing params dict construction and the code that currently adds
sceneViewTarget).

In `@Server/src/services/tools/manage_scene.py`:
- Around line 186-199: manage_scene currently only checks action before
accepting capture_source, allowing calls like manage_scene(...,
action="screenshot", capture_source="scene_view", screenshot_super_size=2) to
bypass the new restriction; add a guard in the screenshot handling block that if
capture_source == "scene_view" then reject requests with a supersize > 1 (check
coerced_super_size or screenshot_super_size) and return a failure message
(similar to the existing capture_source/action check). Target the capture_source
handling inside manage_scene and validate coerced_super_size before setting
params["captureSource"].

---

Nitpick comments:
In `@MCPForUnity/Editor/Helpers/ComponentOps.cs`:
- Around line 835-849: The cast of GlobalObjectId.targetObjectId (ulong) to long
in GetSpriteFileId can overflow and produce negative IDs; update the cast to be
explicit about overflow by using an unchecked cast (e.g.,
unchecked((long)globalId.targetObjectId)) so the wraparound is intentional, and
keep the existing try/catch around GlobalObjectId.GetGlobalObjectIdSlow; also
verify that the caller CoerceLong (and any code that compares file IDs) treats
negative values consistently or adds normalization/clamping if needed.

In `@MCPForUnity/Editor/Helpers/EditorWindowScreenshotUtility.cs`:
- Around line 104-117: Add a brief inline comment above the
Thread.Sleep(RepaintSettlingDelayMs) call in EditorWindowScreenshotUtility (the
repaint block that calls sceneView.Repaint(), InvokeMethodIfExists(sceneView,
"RepaintImmediately"), SceneView.RepaintAll(),
UnityEditorInternal.InternalEditorUtility.RepaintAllViews(),
EditorApplication.QueuePlayerLoopUpdate()) that explains the rationale for the
75ms delay (e.g., to allow the editor repaint and GPU/framebuffer swap to settle
before capturing a screenshot) and note that it intentionally blocks the editor
thread for a short period; include the constant name RepaintSettlingDelayMs in
the comment so future maintainers understand why this specific delay is used and
where to adjust it if needed.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: c850cfbd-970b-4755-bb09-3cd0d8f39a16

📥 Commits

Reviewing files that changed from the base of the PR and between 7615dd3 and 7153272.

⛔ Files ignored due to path filters (1)
  • Server/uv.lock is excluded by !**/*.lock
📒 Files selected for processing (12)
  • AGENTS.md
  • MCPForUnity/Editor/Helpers/ComponentOps.cs
  • MCPForUnity/Editor/Helpers/EditorWindowScreenshotUtility.cs
  • MCPForUnity/Editor/Helpers/EditorWindowScreenshotUtility.cs.meta
  • MCPForUnity/Editor/Helpers/ParamCoercion.cs
  • MCPForUnity/Editor/Setup/RoslynInstaller.cs
  • MCPForUnity/Editor/Tools/ManageScene.cs
  • Server/src/cli/CLI_USAGE_GUIDE.md
  • Server/src/cli/commands/scene.py
  • Server/src/services/tools/manage_scene.py
  • Server/tests/integration/test_manage_scene_screenshot_params.py
  • TestProjects/UnityMCPTests/Assets/Tests/EditMode/Tools/ManageSceneHierarchyPagingTests.cs

Comment on lines +643 to 666
var fileIdToken = jObj["fileID"];
if (fileIdToken != null)
{
long targetFileId = ParamCoercion.CoerceLong(fileIdToken, 0);
if (targetFileId != 0)
{
var allAssets = AssetDatabase.LoadAllAssetsAtPath(path);
foreach (var asset in allAssets)
{
if (asset is Sprite sprite)
{
long spriteFileId = GetSpriteFileId(sprite);
if (spriteFileId == targetFileId)
{
prop.objectReferenceValue = sprite;
return true;
}
}
}
}
}

prop.objectReferenceValue = AssetDatabase.LoadAssetAtPath<UnityEngine.Object>(path);
return true;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Silent fallthrough when fileID doesn't match any sprite.

When fileID is provided but no sprite in the atlas matches, the code falls through to load the main asset at the path (line 665). This may return an unexpected asset type (e.g., the texture atlas itself) instead of failing with a clear error.

Consider returning an error when fileID is provided but not found:

Proposed fix
                     var fileIdToken = jObj["fileID"];
                     if (fileIdToken != null)
                     {
                         long targetFileId = ParamCoercion.CoerceLong(fileIdToken, 0);
                         if (targetFileId != 0)
                         {
                             var allAssets = AssetDatabase.LoadAllAssetsAtPath(path);
                             foreach (var asset in allAssets)
                             {
                                 if (asset is Sprite sprite)
                                 {
                                     long spriteFileId = GetSpriteFileId(sprite);
                                     if (spriteFileId == targetFileId)
                                     {
                                         prop.objectReferenceValue = sprite;
                                         return true;
                                     }
                                 }
                             }
+                            error = $"Sprite with fileID {targetFileId} not found in atlas '{path}'.";
+                            return false;
                         }
                     }
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@MCPForUnity/Editor/Helpers/ComponentOps.cs` around lines 643 - 666, The code
currently falls through and calls AssetDatabase.LoadAssetAtPath when fileID was
present but no matching sprite was found; change the behavior so that when
jObj["fileID"] exists and ParamCoercion.CoerceLong yields a non‑zero
targetFileId but no sprite in AssetDatabase.LoadAllAssetsAtPath matches
GetSpriteFileId, the method should not load the main asset — instead set
prop.objectReferenceValue = null (or leave unchanged) and return false and
optionally log a clear error including the targetFileId and path; update the
logic around fileIdToken/targetFileId and the foreach that uses GetSpriteFileId
to detect the “not found” case and early-return false rather than falling
through to AssetDatabase.LoadAssetAtPath.

Comment on lines +105 to +106
if (double.TryParse(s, NumberStyles.Float, CultureInfo.InvariantCulture, out var d))
return (long)d;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Reject fractional long inputs instead of truncating them.

These branches turn non-integral values into a different identifier ("12.9"12). The first caller uses CoerceLong for fileID matching in MCPForUnity/Editor/Helpers/ComponentOps.cs:643-663, so malformed input can resolve the wrong asset instead of being treated as invalid. Only convert when the parsed value is exactly integral.

Proposed fix
-                if (double.TryParse(s, NumberStyles.Float, CultureInfo.InvariantCulture, out var d))
-                    return (long)d;
+                if (double.TryParse(s, NumberStyles.Float, CultureInfo.InvariantCulture, out var d) &&
+                    !double.IsNaN(d) &&
+                    !double.IsInfinity(d) &&
+                    d >= long.MinValue &&
+                    d <= long.MaxValue &&
+                    d == Math.Truncate(d))
+                {
+                    return (long)d;
+                }
-                if (double.TryParse(s, NumberStyles.Float, CultureInfo.InvariantCulture, out var d))
-                    return (long)d;
+                if (double.TryParse(s, NumberStyles.Float, CultureInfo.InvariantCulture, out var d) &&
+                    !double.IsNaN(d) &&
+                    !double.IsInfinity(d) &&
+                    d >= long.MinValue &&
+                    d <= long.MaxValue &&
+                    d == Math.Truncate(d))
+                {
+                    return (long)d;
+                }

Also applies to: 139-140

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@MCPForUnity/Editor/Helpers/ParamCoercion.cs` around lines 105 - 106,
ParamCoercion.CoerceLong currently accepts a float parsed by double.TryParse and
truncates it to a long, which turns non-integral inputs like "12.9" into 12;
change the double.parse branches (the one around the current TryParse at the top
and the similar one around lines ~139-140) to only succeed when the parsed
double is mathematically integral and fits in Int64 (e.g., check that
Math.Truncate(d) == d and d is between long.MinValue and long.MaxValue) and
otherwise treat it as a failed coercion; this prevents CoerceLong from silently
converting fractional values used for fileID (used by ComponentOps.CoerceLong
caller for fileID matching) into incorrect identifiers.

@jiajunfeng
Copy link
Author

Superseded by #927.

Closing this PR because the original branch drifted significantly from current beta and the screenshot flow has since moved onto the manage_camera path. #927 rebases the Scene View screenshot feature onto the current architecture and drops unrelated commits from this branch.

@jiajunfeng jiajunfeng closed this Mar 12, 2026
@jiajunfeng jiajunfeng deleted the feat/scene-view-screenshot-capture branch March 17, 2026 07:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant