Add CFSClean enforcement#9144
Conversation
|
Azure Pipelines: 22 pipeline(s) were filtered out due to trigger conditions. There may be pipelines that require an authorized user to comment /azp run to run. |
jongio
left a comment
There was a problem hiding this comment.
Verified the multi-value syntax against 1ES usage across the org - Permissive, CFSClean is the standard comma-separated policy format. The prior golang internalModuleProxy setup lines up with enforcing CFSClean, so the network isolation tightening has the internal mirror in place. Scoped and correct.
d5757b2 to
2519f48
Compare
jongio
left a comment
There was a problem hiding this comment.
Re-checked after the demo extension version bump to 0.7.2. version.txt, extension.yaml, and the CHANGELOG entry all line up, and it's a no-op test release to exercise the CFSClean pipeline change. Still good.
|
/check-enforcer override |
jongio
left a comment
There was a problem hiding this comment.
The two new templates added under eng/common/pipelines/templates/steps/ (maven-authenticate.yml and pypi-auth-dev-feed.yml) sit in the directory that is mirrored from azure-sdk-tools. eng/common/README.md warns that changes here get overwritten by the common-engineering sync.
I checked azure-sdk-tools/eng/common/pipelines/templates/steps and neither file exists upstream (the closest is a differently-named auth-dev-feed.yml). So the next sync will delete both files, and the callers that reference them, eng/pipelines/templates/jobs/build-cli.yml and eng/pipelines/templates/stages/build-and-test.yml, will then fail with a missing-template error.
Move both templates to eng/pipelines/templates/steps/ where the other repo-specific steps live (e.g. install-ms-openjdk.yml, setup-go.yml) and update the two template: references, or land them upstream in azure-sdk-tools first and consume them through the normal common-engineering update.
nuget.config and eng/settings.xml at the repo root look fine.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 10 out of 10 changed files in this pull request and generated 2 comments.
Comments suppressed due to low confidence (2)
eng/common/pipelines/templates/steps/pypi-auth-dev-feed.yml:1
- azd-code-reviewer:
eng/commonis synchronized fromAzure/azure-sdk-tools, andeng/common/README.md:3says local changes here will be overwritten; the workflow enforcer also rejects them (eng-common-workflow-enforcer.yml:12-19). Move this repo-specific template undereng/pipelines/templates/stepsand update its call site, or bring the canonical template in through the eng/common sync workflow.
parameters:
eng/common/pipelines/templates/steps/maven-authenticate.yml:1
- azd-code-reviewer:
eng/commonis synchronized fromAzure/azure-sdk-tools, andeng/common/README.md:3says local changes here will be overwritten; the workflow enforcer also rejects them (eng-common-workflow-enforcer.yml:12-19). Move this repo-specific template undereng/pipelines/templates/stepsand update its call site, or add it via the eng/common sync workflow.
parameters:
jongio
left a comment
There was a problem hiding this comment.
Re-reviewed the latest commit. Dropping the root nuget.config and scoping the restore with feedsToUse: select and vstsFeed: 'public/azure-sdk-for-net' in build-msi.yml keeps the windows.sln restore (the only NuGet restore in the repo) on the ADO mirror under CFSClean, so that change is self-contained and introduces no new issues.
One blocker from the earlier pass is still open: maven-authenticate.yml and pypi-auth-dev-feed.yml live under eng/common, which is overwritten by the azure-sdk-tools sync (see eng/common/README.md:3), and neither file exists upstream in azure-sdk-tools. The next common-engineering sync will delete both and break the template references in build-cli.yml and build-and-test.yml. Move them under eng/pipelines/templates/steps/ (next to setup-go.yml and install-ms-openjdk.yml), or land them upstream in azure-sdk-tools first, before merging.
677e3e9 to
e00e270
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 12 out of 12 changed files in this pull request and generated 2 comments.
Comments suppressed due to low confidence (1)
eng/pipelines/templates/stages/1es-redirect.yml:35
- [azd-code-reviewer] This enables CFSClean for every pipeline that extends this shared redirect, but only the CLI build jobs were migrated to authenticated package feeds. For example,
eval-unit.yml:50,58,eval-waza.yml:53,56,vscode-build.yml:37-38, andazuredevops-build.yml:38-39still install directly with npm/pip and will be blocked once CFSClean is enforced. Add the npm/PyPI authentication setup to every affected job, or scope the policy until those jobs are migrated.
networkIsolationPolicy: Permissive, CFSClean
jongio
left a comment
There was a problem hiding this comment.
One item from earlier passes is still open, now narrowed to a single file.
maven-authenticate.yml has since landed in azure-sdk-tools upstream, so that half is resolved. pypi-auth-dev-feed.yml still doesn't exist upstream (the closest match is the differently-named auth-dev-feed.yml). Since eng/common is overwritten by the common-engineering sync (eng/common/README.md), the next sync will delete pypi-auth-dev-feed.yml and break the template: reference to it in build-and-test.yml. Land it upstream in azure-sdk-tools first (as was done for maven), or move it under eng/pipelines/templates/steps/ and update the reference.
da649ec to
44940c5
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 12 out of 12 changed files in this pull request and generated 4 comments.
Comments suppressed due to low confidence (1)
eng/common/pipelines/templates/steps/pypi-auth-dev-feed.yml:40
- [azd-code-reviewer]
PIP_INDEX_URLis credential-bearing, as the new comment notes, but this block prints it and copies it into a non-secretUV_DEFAULT_INDEXvariable. That can expose the feed token in logs or later task output. Do not log the URL, and preserve secret handling when passing it to uv by explicitly mapping a secret variable only into uv-consuming steps.
Write-Host "Found pip index URL: $($env:PIP_INDEX_URL)"
# UV_DEFAULT_INDEX is the canonical replacement for the deprecated UV_INDEX_URL (uv 0.4.23+).
# PIP_INDEX_URL is set by PipAuthenticate@1 and contains embedded credentials, which uv
# will use for Basic auth against the ADO feed (and its PyPI upstream) per astral-sh/uv#12651.
Write-Host "##vso[task.setvariable variable=UV_DEFAULT_INDEX]$($env:PIP_INDEX_URL)"
jongio
left a comment
There was a problem hiding this comment.
Following up on the new NuGet/PyPI auth commit: it adds a second reference to pypi-auth-dev-feed.yml, which still isn't landed upstream. Inline note has the fix.
|
/azp run azure-dev - cli |
|
Azure Pipelines: 1 pipeline(s) were filtered out due to trigger conditions. |
|
/azp run azure-dev - ext - microsoft.azd.demo |
|
Azure Pipelines: Successfully started running 1 pipeline(s). |
44940c5 to
5906b98
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 11 out of 11 changed files in this pull request and generated no new comments.
Comments suppressed due to low confidence (3)
eng/common/pipelines/templates/steps/maven-authenticate.yml:17
- azd-code-reviewer: This unqualified feed name is resolved in the pipeline's current Azure DevOps project (
internal), while the mirror ineng/settings.xml:9targets thepublicproject.MavenAuthenticatecan therefore authenticate the wrong or nonexistent feed, leaving Maven unable to access the enforced mirror. Use the cross-project feed identifier, consistent with the NuGet and PyPI configuration in this PR.
artifactsFeeds: "azure-sdk-for-java"
eng/common/pipelines/templates/steps/pypi-auth-dev-feed.yml:36
- azd-code-reviewer:
PIP_INDEX_URLcontains embedded credentials (as noted below), so printing the complete value creates an avoidable credential-disclosure path if task masking is absent or changes. Log only that the variable was found; the URL itself is not needed for diagnostics.
Write-Host "Found pip index URL: $($env:PIP_INDEX_URL)"
cli/azd/extensions/microsoft.azd.demo/CHANGELOG.md:5
- azd-code-reviewer: The PR description says this only updates
networkIsolationPolicy, but the diff also adds package-manager authentication/mirror infrastructure and cuts demo extension version 0.7.2. Update the description to cover these material changes so reviewers and release history accurately reflect the scope.
## 0.7.2 (2026-07-16)
- Simple test release of extension, with no changes.
jongio
left a comment
There was a problem hiding this comment.
azure-dev - cli can't run on this PR at all, so most of what this changes is unexercised.
Your /azp run azure-dev - cli at 00:58 came back with "1 pipeline(s) were filtered out due to trigger conditions". eng/pipelines/release-cli.yml is why:
pr:
paths:
include:
- go.mod
- cli/
- eng/pipelines/release-cli.yml
- eng/pipelines/templates/steps/publish-cli.yml
exclude:
- cli/azd/extensions/**Everything this PR touches is either under eng/pipelines/templates/{jobs,stages,steps}, which isn't in the include list, or under cli/azd/extensions/microsoft.azd.demo, which is explicitly excluded. No changed file matches, so the pipeline never queues. The CD trigger: block right above it already lists /eng/pipelines/templates/jobs/build-cli.yml; the pr: block just never got the same entries.
azure-dev - vscode has the same gap. release-vscode.yml only includes ext/vscode and eng/pipelines/release-vscode.yml, neither of which this PR touches.
That leaves azure-dev - ext - microsoft.azd.demo as the only one of the three @danieljurek listed that actually ran, and it goes through release-azd-extension.yml, which never pulls in build-cli.yml, build-and-test.yml, or build-msi.yml. So the NuGet, npm, Maven and pip changes have zero coverage here, and the first real execution is post-merge on main with CFSClean already turned on.
Adding these to the pr: include lists in both pipelines would let you validate before merging:
- eng/pipelines/templates/jobs/build-cli.yml
- eng/pipelines/templates/stages/build-and-test.yml
- eng/pipelines/templates/steps/build-msi.yml
- eng/pipelines/templates/stages/1es-redirect.ymlTwo inline comments below: the eng/common sync problem from my last pass (I tracked down the upstream file it should be named after) and a dead env var in build-cli.yml.
| @@ -0,0 +1,49 @@ | |||
| parameters: | |||
There was a problem hiding this comment.
Still open from my last pass, and I found the specific fix this time. This template already exists upstream as eng/common/pipelines/templates/steps/auth-dev-feed.yml in azure-sdk-tools. Same parameters, same steps, same uv block; only quote style and indentation differ. There is no pypi-auth-dev-feed.yml upstream.
The eng/common sync isn't additive. It clears the destination directory and re-copies from azure-sdk-tools, so downstream-only files get deleted (that's what removed files in #5413). Once the next sync lands, this file is gone and both - template: /eng/common/pipelines/templates/steps/pypi-auth-dev-feed.yml references, build-cli.yml:115 and build-and-test.yml:161, fail during template expansion, before the run starts.
Rename it to auth-dev-feed.yml using the upstream bytes and point both references at that name. Then the sync overwrites it harmlessly instead of deleting it.
Related, so nobody "fixes" the wrong thing: create-authenticated-npmrc.yml and maven-authenticate.yml in this PR are already byte-identical to upstream (blobs 3fa699b387d and 1b84e984add match azure-sdk-tools exactly). Don't apply local edits to those two, including the suggestion to switch artifactsFeeds to public/azure-sdk-for-java. Anything like that has to land upstream first or the next sync reverts it.
| GOTESTSUM_JUNITFILE: junitTestReport.xml | ||
| SYSTEM_ACCESSTOKEN: $(System.AccessToken) | ||
| PIP_INDEX_URL: $(PIP_INDEX_URL) | ||
| PIP_EXTRA_INDEX_URL: $(PIP_EXTRA_INDEX_URL) |
There was a problem hiding this comment.
PIP_EXTRA_INDEX_URL never gets set, so this hands literal text to the test process.
The template gives PipAuthenticate@1 a single feed with onlyAddExtraIndex: false. The task shifts that one endpoint into PIP_INDEX_URL and only sets PIP_EXTRA_INDEX_URL when endpoints remain, which they don't (Tasks/PipAuthenticateV1/pipauthenticatemain.ts). Azure Pipelines leaves unresolved $(...) macros as-is, so go test ends up with PIP_EXTRA_INDEX_URL=$(PIP_EXTRA_INDEX_URL).
pip warns on a non-URL index and ignores it rather than failing, so nothing breaks. It's just log noise and a misleading env var. Drop the line.
jongio
left a comment
There was a problem hiding this comment.
Re-checked 5906b98. Two things still stand, and one of the open questions resolves.
pypi-auth-dev-feed.yml gets deleted by the next common-engineering sync
I pulled the upstream blob SHAs to work out which of the three eng/common files are actually safe:
| file | upstream in azure-sdk-tools |
this PR |
|---|---|---|
create-authenticated-npmrc.yml |
3fa699b387d |
3fa699b387d, identical |
maven-authenticate.yml |
1b84e984add |
1b84e984add, identical |
pypi-auth-dev-feed.yml |
404, doesn't exist | fc2396c8448 |
The first two match byte for byte because they were landed upstream first. pypi-auth-dev-feed.yml wasn't, and its content already exists upstream under a different name: eng/common/pipelines/templates/steps/auth-dev-feed.yml (cd8fa23e67e). Same four parameters, same Setup DevOpsFeedName step, same Twine, Pip and uv blocks, same UV_PYTHON_INSTALL_MIRROR value. Only quoting and indentation differ.
eng/common/README.md says updates here "end up getting overwritten with future updates", and the sync clears the destination rather than merging, so the file disappears and both of these break at template expansion:
eng/pipelines/templates/jobs/build-cli.yml:115eng/pipelines/templates/stages/build-and-test.yml:161
Dropping the new file and repointing both references is the whole fix:
- template: /eng/common/pipelines/templates/steps/auth-dev-feed.ymlStill 1 of the 3 pipelines @danieljurek asked for
Checks on 5906b98 only show azure-dev - ext - microsoft.azd.demo. azure-dev - cli and azure-dev - vscode still don't queue, because the pr: path filters in release-cli.yml and release-vscode.yml don't list any file this PR touches. So the NuGet, npm, Maven and pip changes stay unexercised until this is on main with CFSClean already enabled.
On the build-msi.yml feed switch
WixToolset.DUtil and WixToolset.WcaUtil 4.0.0 both resolve on public/azure-sdk-for-net through its nuget.org upstream, so vstsFeed points at a feed that carries what cli/installer/windows.sln restores. The open item there is just the includeNuGetOrg default, which is already flagged inline.
jongio
left a comment
There was a problem hiding this comment.
Re-checked 5906b98 today. No new commits since my last pass, so the two open items still stand, and I re-verified the main one against upstream just now.
pypi-auth-dev-feed.yml still gets deleted by the next eng/common sync. As of today it still doesn't exist in azure-sdk-tools (404), while the equivalent auth-dev-feed.yml does, with the same parameters, the same steps, and the same uv block. eng/common is overwritten by the common-engineering sync, so once it runs this file disappears and both - template: /eng/common/pipelines/templates/steps/pypi-auth-dev-feed.yml references fail at expansion time: build-cli.yml:115 and build-and-test.yml:161. The create-authenticated-npmrc.yml and maven-authenticate.yml blobs in this PR are byte-identical to upstream (3fa699b387d and 1b84e984add), so those two are safe to leave as-is. The fix is to drop the new file and point both references at auth-dev-feed.yml, or land pypi-auth-dev-feed.yml upstream first the way maven was. Full details are in the inline thread.
PIP_EXTRA_INDEX_URL: $(PIP_EXTRA_INDEX_URL) in build-cli.yml is still a dead macro. PipAuthenticate@1 runs with a single feed and onlyAddExtraIndex: false, so it only populates PIP_INDEX_URL. PIP_EXTRA_INDEX_URL never gets set, and the test process receives the literal $(PIP_EXTRA_INDEX_URL). pip ignores a non-URL index so nothing breaks, but it's misleading log noise. Drop the line. Inline thread has it.
One process note, not blocking on its own: only azure-dev - ext - microsoft.azd.demo actually runs on this PR. azure-dev - cli and azure-dev - vscode don't queue, because their pr: path filters don't list any file this PR touches, so the NuGet, npm, Maven, and pip wiring stays unexercised until this lands on main with CFSClean already on. Adding the touched template paths to those pr: include lists would let you validate before merge.
jongio
left a comment
There was a problem hiding this comment.
Re-verified 5906b98 against azure-sdk-tools upstream just now. No new commits since my last pass, so the same two items are still open, and the first is still a merge blocker.
pypi-auth-dev-feed.yml gets deleted by the next eng/common sync. Checked the upstream blobs again today: pypi-auth-dev-feed.yml still returns 404 in azure-sdk-tools, so it doesn't exist upstream. The equivalent auth-dev-feed.yml does exist (cd8fa23e67e) with the same parameters, the same steps, and the same uv block. eng/common is overwritten by the common-engineering sync (eng/common/README.md), and the sync clears the destination rather than merging, so a downstream-only file gets removed. Once it runs, pypi-auth-dev-feed.yml disappears and both template: references fail at expansion time, before the run starts: build-cli.yml:115 and build-and-test.yml:161. Drop the new file and point both references at auth-dev-feed.yml, or land pypi-auth-dev-feed.yml upstream first the way maven-authenticate.yml was. For contrast, create-authenticated-npmrc.yml (3fa699b387d) and maven-authenticate.yml (1b84e984add) in this PR are already byte-identical to upstream, so those two are safe to leave as-is.
PIP_EXTRA_INDEX_URL: $(PIP_EXTRA_INDEX_URL) in build-cli.yml is a dead macro. PipAuthenticate@1 runs with a single feed and onlyAddExtraIndex: false, so it only populates PIP_INDEX_URL. PIP_EXTRA_INDEX_URL never gets set, and the test process receives the literal $(PIP_EXTRA_INDEX_URL). pip ignores a non-URL index so nothing breaks, but it's misleading log noise. Drop the line.
One process note, not blocking on its own: only azure-dev - ext - microsoft.azd.demo runs on this PR. azure-dev - cli and azure-dev - vscode don't queue, because their pr: path filters don't list any file this PR touches, so the NuGet, npm, Maven, and pip wiring stays unexercised until this lands on main with CFSClean already on. Adding the touched template paths to those pr: include lists would let you validate before merge. Details are in the existing inline threads.
This pull request makes a small change to the pipeline configuration by updating the
networkIsolationPolicysetting. The policy is now set to include bothPermissiveandCFSClean, which may affect how network isolation is enforced during pipeline runs.networkIsolationPolicyineng/pipelines/templates/stages/1es-redirect.ymlto includeCFSCleanin addition toPermissive.azure-dev - ext - microsoft.azd.demo
azure-dev - cli
Resolves #9221