Skip to content

chore(testdata): upstream-config drift detected (cursor)#109

Merged
paultyng merged 2 commits into
mainfrom
chore/upstream-drift-cursor-20260526
May 26, 2026
Merged

chore(testdata): upstream-config drift detected (cursor)#109
paultyng merged 2 commits into
mainfrom
chore/upstream-drift-cursor-20260526

Conversation

@github-actions
Copy link
Copy Markdown
Contributor

Upstream config drift detected — cursor

Reconciling testagent's corpus with current upstream docs.

Added (7)

Net-new examples found in docs that pass testagent cursor validate --strict.

Retired (5)

Existing fixtures whose exact bytes are no longer present at the
.source URL we recorded. Only fixtures whose source page was
fetched this run are retired (guards against vendor docs URL
restructures).

Skipped — fail --strict

  • 6 examples failed testagent cursor validate --strict; tracked separately in update-compatibility.

How to evaluate

Each Added fixture passed testagent cursor validate --strict in the
workflow. That's necessary but not sufficient — runtime behavior of
the actual vendor binary (claude / codex / cursor) is the
canonical source of truth, not the docs page
. testagent's job is to
match what the real vendor accepts, even when testagent's own internal
runtime doesn't model every field.

Triage rules:

  1. Added: spot-check each fixture matches what the real vendor
    binary would accept. If our validator accepts a fixture but the
    doc example looks stale or wrong (typo, deprecated key), drop it
    from this PR and file an upstream doc bug — don't widen our
    validator to match a wrong doc
    .
  2. Retired: a fixture's exact bytes are no longer in upstream
    docs at the URL we recorded. Default to accepting the retirement.
    Push back only if the fixture exercises a real-world shape the
    vendor still accepts at runtime — in which case keep it locally
    and update the .source URL or add a comment explaining why.
  3. Skipped — fail --strict: doc examples we couldn't accept.
    Sometimes the doc is right and we're missing schema coverage
    (→ extend the allowlist via update-compatibility). Sometimes
    the doc is wrong (→ leave it skipped).

When uncertain, run the real cursor binary against the fixture in
a sandbox before merging.

🤖 Generated by .github/workflows/upstream-drift.yml

@paultyng paultyng marked this pull request as ready for review May 26, 2026 21:58
@paultyng paultyng merged commit eb79ebe into main May 26, 2026
3 checks passed
@paultyng paultyng deleted the chore/upstream-drift-cursor-20260526 branch May 26, 2026 21:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant