[pilot] Migrate OperationsApiSchemaUsesCommonTypes to a native TypeSpec lint - #5095
Draft
msyyc wants to merge 1 commit into
Draft
Conversation
Pilot migration of the ARM validator rule OperationsApiSchemaUsesCommonTypes (RPC-Operations-V1-01) to a native TypeSpec lint, run end-to-end through the documented migration process to measure the real per-rule cost. - Add rule operations-api-schema-uses-common-types and register it. - Add two violating fixtures. The pre-existing custom-operations-schema fixture used route /customOperations, which can never match the validator selector /providers/<ns>/operations$, so it proved nothing; it is kept as the compliance case with reviewed ambient diagnostics. - legacy-operations-custom-response is the clean gap: Legacy.Operations<Response> with a service-defined response satisfies missing-operations-endpoint and previously produced no TypeSpec diagnostics at all. - Declare harness dependencies that were missing from package.json (openapi-validator-core, lodash, yaml, typespec-azure-rulesets, typespec-client-generator-core) so the validate harness runs. Note: diagnostics targeting an operation that comes from a template instantiation are silently dropped because the operation node lives in library source, so the rule targets the user-declared interface instead. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 5895e2fa-70fa-42e0-9443-b35905ca584c
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this is
A cost pilot, not a production change. It runs one bucket-C rule (validator fires, no TypeSpec lint mapping exists) end-to-end through the documented migration process so we can compare the measured effort against the 0.875–1.25 dev-day/rule estimate in the migration plan.
Rule migrated:
OperationsApiSchemaUsesCommonTypes(ARM, severityerror, RPC-Operations-V1-01, 84 firings across 450 compiled projects in the ARM coverage breakdown).Harness result:
Covered (TSP lint): 2 · Remaining unresolved gaps: 0 · exit 0. Spot-checkedLroErrorContentfor regressions — still green.Changes
tsp-lintdiff-local-linter/operations-api-schema-uses-common-types(~65 LOC) + registration inlinter.ts. Any ARMGETwhose route ends in/providers/<...>/operationsmust return a model that is namedOperationListResultand marked as an ARM common type (isArmCommonType).custom-operations-schemafixture used route/customOperations, which can never match the validator selector/providers/\w+\.\w+/operations$— it produced zero validator diagnostics and proved nothing. It is retained as the compliance case with reviewed ambient diagnostics.custom-operations-list-result— hand-written/providers/<ns>/operationsroute returning a service-defined list result. TypeSpec already reportsmissing-operations-endpointhere, so this shape was arguably covered already.legacy-operations-custom-response—Azure.ResourceManager.Legacy.Operations<ArmResponse<CustomOpListResult>>. This satisfiesmissing-operations-endpointand produced no TypeSpec diagnostics at all before this change. This is the clean gap that justifies the native rule.packages/typespec-lintdiffwas missing@microsoft.azure/openapi-validator-core,lodash,yaml,@azure-tools/typespec-azure-rulesets, and@azure-tools/typespec-client-generator-core; the validate harness could not start without them. The lockfile had no entry for this package at all, hence the largepnpm-lock.yamldiff.rule.mdrewritten with source-of-truth notes, both authorable shapes, deliberate parity deltas, and the test-case table.Finding worth discussing
context.reportDiagnostic({ target: operation })is silently dropped when the operation comes from a template instantiation (interface Operations extends Azure.ResourceManager.Legacy.Operations<...>), because the operation's node lives in library source. The rule was firing internally and emitting nothing; targeting the user-declared interface fixes it.This may be a systemic cause of the very low real-world parity on several already-migrated rules —
XmsResourceInPutResponse(391 validator firings vs 2 lint firings, 0.5%),PaginationResponse(1.2%),TagsAreNotAllowedForProxyResources(5.1%),ConsistentPatchProperties(8.3%) — all concern shapes that normally arrive via ARM templates. Worth auditing before re-estimating the "already migrated" bucket; it could be a cheap bulk fix rather than ~25 re-implementations.Measured effort
rule.mdonly; website doc + ruleset promotion outstandingPhases 1–4 ≈ 2.1 h ≈ 0.26 dev-day of hands-on work. This does not refute the 0.875–1.25 dev-day figure: the untested residual is website docs, promotion out of the staging package, PR review rounds and ARM sign-off — and two real costs sat entirely outside the estimate model (environment bring-up, compiler-behaviour debugging).
Other observations
azure-openapi-validatorviaLINTDIFF_VALIDATOR_ROOT; this is not documented.docs/ONBOARDING.mdis stale: it referencestests/,scripts/validate.ts, andRULE_ID_MAP/TSP_DIAGNOSTIC_MAP, none of which exist any more (nowtest/fixtures/,test/harness/validate.ts, andrule.mdfrontmatter).Not done
Website documentation, promotion into an official ruleset, rule category assignment, and release notes — i.e. the plan's phase 5 and "Review and Integration" steps.