ci(spec-sdk-tests-vs-release): pin spec-sdk-tests/ alongside SDK on sdk_version override#929
ci(spec-sdk-tests-vs-release): pin spec-sdk-tests/ alongside SDK on sdk_version override#929leggetter wants to merge 2 commits into
Conversation
…dk_version override
Without this, dispatching with sdk_version=v1.3.0 pinned the SDK to its
release tag but left main's newer tests in place, causing TS compile
errors when the tests referenced fields the pinned SDK didn't have
('type does not exist in type DestinationUpdate'). Verified empirically
on the prior dispatch run.
Now when sdk_version is set, we checkout both sdks/outpost-typescript/
AND spec-sdk-tests/ from the same SDK release tag. The tag is namespaced
but points at a full repo commit, so spec-sdk-tests/ at that commit is
the test suite as it shipped with that SDK release — guaranteed aligned.
Bot regen PR case (the workflow's primary purpose) is unchanged:
sdk_version stays empty, working tree's SDK + tests are both used,
they're already aligned by the bot's regen.
There was a problem hiding this comment.
Pull request overview
This PR updates the spec-sdk-tests-vs-release GitHub Actions workflow so that when a workflow_dispatch run pins the TypeScript SDK to a released tag via sdk_version, it also pins spec-sdk-tests/ to the same tag, keeping the test suite aligned with the SDK’s historical shape.
Changes:
- Update workflow_dispatch documentation/inputs text to clarify that
sdk_versionpins both the SDK andspec-sdk-tests/. - Modify the
sdk_versionoverride step togit checkoutspec-sdk-tests/from the same SDK release tag assdks/outpost-typescript/.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| ver="${SDK_VERSION_INPUT#v}" | ||
| full_tag="sdks/outpost-typescript/v${ver}" | ||
| echo "Pinning SDK to $full_tag" | ||
| echo "Pinning SDK + tests to $full_tag" | ||
| git fetch --depth=1 origin "refs/tags/$full_tag:refs/tags/$full_tag" |
There was a problem hiding this comment.
Added validation in c642a5fe — both sdk_version and outpost_version now reject anything that doesn't match strict X.Y.Z (after stripping any leading v) with an explicit ::error:: naming the offending input. Same regex shape as the GH releases filter we already use elsewhere in the workflow, applied symmetrically to both inputs for consistency.
… X.Y.Z Spotted by Copilot review on #929. Typos in sdk_version or outpost_version otherwise produced cryptic git errors (fatal: couldn't find remote ref). Reject anything that isn't a strict semver triple before constructing the git refspec, with a clear ::error:: message naming the offending input.
Why
workflow_dispatchruns withsdk_versionset were failing at TS compile. Verified empirically on a dispatch withsdk_version=1.3.0:Root cause: the override pinned
sdks/outpost-typescript/to the v1.3.0 release tag but leftspec-sdk-tests/onmain's current shape. Tests onmainreference fields the pinned SDK doesn't have.Fix
When
sdk_versionis set, alsogit checkout sdks/outpost-typescript/v<x> -- spec-sdk-tests/so tests come from the same point in time as the SDK. SDK release tags are namespaced but point at full repo commits, so this works.Bot regen PR case (the workflow's primary purpose) is unchanged:
sdk_versionstays empty, the PR's own SDK and tests are both used, alignment guaranteed by the bot's regen.Also adds strict
X.Y.Zvalidation for bothsdk_versionandoutpost_versioninputs (Copilot review feedback) — typos now produce a clear::error::instead of cryptic git refspec failures.What we learned about historical SDK releases
Walked every TS SDK release tag and checked tests-at-tag vs SDK-at-tag alignment:
events.list()return typePromise<EventPaginatedResult>(flat)response.modelsPromise<PageIterator<{result: EventPaginatedResult}>>response.modelsThe break at v0.9.0 was a Speakeasy CLI bump introducing
PageIteratorwithout anything updating tests to match. Six SDK releases (v0.9.0, v0.9.1, v0.9.2, v0.10.0, v0.10.1, v1.0.0+ through v1.3.0) shipped with tests that couldn't compile against them. Manual AT runs at release time stopped catching this around the same point. This PR + #925 close the loop: no SDK release can merge withoutspec-sdk-tests.ymlpassing against its regen'd SDK.Test plan
sdk_version=1.3.0+outpost_version=1.0.3— reproduces the predicted v1.3.0 alignment bug (run 26649985084). Confirms the fix doesn't hide existing breakage.sdk_version=0.8.0+outpost_version=0.17.2— 135 passing / 0 failing / 15 pending (run 26655324437). First green run of this workflow ever, validates the mechanism end-to-end against a known-aligned historical pairing.::error::sdk_version must be a strict X.Y.Z version (got: ...)instead of cryptic git errors.🤖 Generated with Claude Code