Skip to content

input: allow drag drop-point biasing within target bounding box#2159

Open
innocentnortey wants to merge 1 commit into
ChromeDevTools:mainfrom
innocentnortey:drag-offset-biasing
Open

input: allow drag drop-point biasing within target bounding box#2159
innocentnortey wants to merge 1 commit into
ChromeDevTools:mainfrom
innocentnortey:drag-offset-biasing

Conversation

@innocentnortey
Copy link
Copy Markdown

@innocentnortey innocentnortey commented May 31, 2026

Summary

Extend the existing drag tool with four optional, backwards-compatible parameters that let callers bias the drop point inside the target element's bounding rectangle instead of always landing at the geometric centre:

  • to_offset_x — CSS-pixel offset from target bounding-box top-left
  • to_offset_y — same, vertical
  • to_fraction_x — fractional x position in [0, 1]
  • to_fraction_y — fractional y position in [0, 1]

When omitted, drag(from_uid, to_uid) behaves exactly as it does today.

Motivation

Some drag-and-drop UIs expose a single visible drop-zone element whose geometric centre is not the safest or correct drop hotspot.

A concrete repro is Zoho Analytics Pivot View construction. The Pivot editor has three drop shelves stacked vertically (Columns / Rows / Data). Dropping a measure onto the visible Data shelf container can land in the neighbouring Rows shelf when the shelf's bounding-box centre sits close to the Rows region. Dropping onto a child element located deeper inside the intended Data shelf — e.g., an existing chip body or its remove-button — works reliably, which points at target-point geometry rather than source selection or timing as the failure mode.

Today chrome-devtools-mcp's drag tool uses Puppeteer's default ElementHandle.drag(toHandle) / toHandle.drop(fromHandle) path, which computes the drop point as the element's bounding-rect centroid via clickablePoint(). That makes deceptive or overlapping drop-zone layouts hard to automate reliably.

What changed

New optional drag parameters

Mutually exclusive per axis (offset OR fraction, not both).

Examples:

// Bias near the top interior of the target
{ "from_uid": "...", "to_uid": "...", "to_offset_y": 12 }

// Bias toward the lower-right interior of the target
{ "from_uid": "...", "to_uid": "...", "to_fraction_x": 0.85, "to_fraction_y": 0.9 }

Backwards compatibility

If none of the new parameters are provided, the tool keeps the existing implementation verbatim:

  • fromHandle.drag(toHandle)
  • fixed 50 ms pause
  • toHandle.drop(fromHandle)

This preserves behaviour for every existing caller.

Custom-target code path

If any of the new parameters is present, the tool:

  1. Scrolls source and target into view.
  2. Computes the drop point from the target element's bounding box (offset / fraction / centroid).
  3. Hovers the source, presses the mouse, moves the mouse to the computed drop point, waits 150 ms for dragover handlers, releases the mouse.

This mirrors Puppeteer's existing HTML5-drag emulation path (the non-intercepted branch of ElementHandle.drag) but lets the caller specify the target point. It intentionally avoids setDragInterception(true) — enabling drag interception would make ElementHandle.drag(toHandle) (used by the default path) hang waiting for Input.dragIntercepted, breaking every existing call site.

The 150 ms delay is intentional for HTML5 dragover-based UIs that need processing time between move and release.

Tests

Added a regression test in tests/tools/input.test.ts that verifies drag with to_offset_y: 10 lands in the top region of a 120×120 drop target (i.e. the bias actually takes effect).

Existing drag test continues to pass unchanged.

Generated artefacts

Regenerated via npm run gen:

  • docs/tool-reference.md — new params documented
  • src/bin/chrome-devtools-cli-options.ts — new CLI options entries
  • src/telemetry/tool_call_metrics.json — new arg-type metrics entries

Verification

npm install
npm run typecheck   # passes
npm run build       # passes
npm run gen         # regenerates docs/cli/metrics; format clean
npm run check-format  # passes
node scripts/test.mjs build/tests/tools/input.test.js  # all 46 input tests pass

References

  • chrome-devtools-mcp current drag implementation: src/tools/input.ts
  • Puppeteer ElementHandle.drag non-intercepted branch: node_modules/puppeteer-core/lib/puppeteer/api/ElementHandle.js
  • Zoho Analytics Pivot View repro discussed during a real automation session

@innocentnortey
input: allow drag drop-point biasing within target bounding box
06c6124 
Extend the `drag` tool with optional `to_offset_x` / `to_offset_y` /
`to_fraction_x` / `to_fraction_y` so callers can bias the drop point
inside the target element's bounding rectangle instead of always
dropping at the geometric center.

When none of the new parameters are provided, the tool keeps the
existing centroid-to-centroid behaviour
(`fromHandle.drag(toHandle)` + 50 ms pause + `toHandle.drop(fromHandle)`)
so every current caller is unaffected.

When any of the new parameters are provided, the tool computes a
custom point inside the target's bounding box and replicates the
existing HTML5-drag emulation path (hover source → mouse.down →
mouse.move to custom point → 150 ms pause → mouse.up). This avoids
needing `setDragInterception(true)`, which would conflict with the
default code path used by every other call site.

Motivation: some drag-and-drop UIs expose a single visible drop zone
whose geometric centre is not the safest or correct drop hotspot
(e.g. Zoho Analytics Pivot View construction, where dropping a measure
onto the visible Data shelf container can land in a neighbouring Rows
shelf because the shelf centre overlaps it). In those layouts,
dropping onto a child element deeper inside the intended shelf works
reliably, which points at target-point geometry rather than source
selection or timing as the failure mode.

Adds a regression test that verifies the offset path lands in the
expected sub-region of the target. Regenerated tool reference docs,
CLI options, and tool-call metrics via `npm run gen`.
@google-cla
Copy link
Copy Markdown

google-cla Bot commented May 31, 2026

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

@ranieleda-ui
Copy link
Copy Markdown

ranieleda-ui commented Jun 2, 2026 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@innocentnortey @ranieleda-ui