Skip to content

Add CFSClean enforcement#9144

Open
chidozieononiwu wants to merge 8 commits into
Azure:mainfrom
chidozieononiwu:enforceCFSClean
Open

Add CFSClean enforcement#9144
chidozieononiwu wants to merge 8 commits into
Azure:mainfrom
chidozieononiwu:enforceCFSClean

Conversation

@chidozieononiwu

@chidozieononiwu chidozieononiwu commented Jul 14, 2026

Copy link
Copy Markdown
Member

This pull request makes a small change to the pipeline configuration by updating the networkIsolationPolicy setting. The policy is now set to include both Permissive and CFSClean, which may affect how network isolation is enforced during pipeline runs.

  • Pipeline configuration:
    • Updated the networkIsolationPolicy in eng/pipelines/templates/stages/1es-redirect.yml to include CFSClean in addition to Permissive.

azure-dev - ext - microsoft.azd.demo
azure-dev - cli

Resolves #9221

Copilot AI review requested due to automatic review settings July 14, 2026 23:56
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
22 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Enables CFSClean enforcement across pipelines using the shared 1ES redirect template.

Changes:

  • Adds CFSClean to the network isolation policy.

@jongio jongio left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified the multi-value syntax against 1ES usage across the org - Permissive, CFSClean is the standard comma-separated policy format. The prior golang internalModuleProxy setup lines up with enforcing CFSClean, so the network isolation tightening has the internal mirror in place. Scoped and correct.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

Comment thread cli/azd/extensions/microsoft.azd.demo/version.txt
@github-actions github-actions Bot added the ext-demo microsoft.azd.demo extension label Jul 16, 2026

@jongio jongio left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-checked after the demo extension version bump to 0.7.2. version.txt, extension.yaml, and the CHANGELOG entry all line up, and it's a no-op test release to exercise the CFSClean pipeline change. Still good.

@chidozieononiwu

Copy link
Copy Markdown
Member Author

/check-enforcer override

Copilot AI review requested due to automatic review settings July 22, 2026 01:44

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated 2 comments.

Comment thread eng/common/pipelines/templates/steps/pypi-auth-dev-feed.yml
Comment thread eng/common/pipelines/templates/steps/maven-authenticate.yml

@jongio jongio left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The two new templates added under eng/common/pipelines/templates/steps/ (maven-authenticate.yml and pypi-auth-dev-feed.yml) sit in the directory that is mirrored from azure-sdk-tools. eng/common/README.md warns that changes here get overwritten by the common-engineering sync.

I checked azure-sdk-tools/eng/common/pipelines/templates/steps and neither file exists upstream (the closest is a differently-named auth-dev-feed.yml). So the next sync will delete both files, and the callers that reference them, eng/pipelines/templates/jobs/build-cli.yml and eng/pipelines/templates/stages/build-and-test.yml, will then fail with a missing-template error.

Move both templates to eng/pipelines/templates/steps/ where the other repo-specific steps live (e.g. install-ms-openjdk.yml, setup-go.yml) and update the two template: references, or land them upstream in azure-sdk-tools first and consume them through the normal common-engineering update.

nuget.config and eng/settings.xml at the repo root look fine.

Copilot AI review requested due to automatic review settings July 22, 2026 02:32

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated 2 comments.

Comments suppressed due to low confidence (2)

eng/common/pipelines/templates/steps/pypi-auth-dev-feed.yml:1

  • azd-code-reviewer: eng/common is synchronized from Azure/azure-sdk-tools, and eng/common/README.md:3 says local changes here will be overwritten; the workflow enforcer also rejects them (eng-common-workflow-enforcer.yml:12-19). Move this repo-specific template under eng/pipelines/templates/steps and update its call site, or bring the canonical template in through the eng/common sync workflow.
parameters:

eng/common/pipelines/templates/steps/maven-authenticate.yml:1

  • azd-code-reviewer: eng/common is synchronized from Azure/azure-sdk-tools, and eng/common/README.md:3 says local changes here will be overwritten; the workflow enforcer also rejects them (eng-common-workflow-enforcer.yml:12-19). Move this repo-specific template under eng/pipelines/templates/steps and update its call site, or add it via the eng/common sync workflow.
parameters:

Comment thread eng/pipelines/templates/stages/1es-redirect.yml
Comment thread cli/azd/extensions/microsoft.azd.demo/extension.yaml

@jongio jongio left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewed the latest commit. Dropping the root nuget.config and scoping the restore with feedsToUse: select and vstsFeed: 'public/azure-sdk-for-net' in build-msi.yml keeps the windows.sln restore (the only NuGet restore in the repo) on the ADO mirror under CFSClean, so that change is self-contained and introduces no new issues.

One blocker from the earlier pass is still open: maven-authenticate.yml and pypi-auth-dev-feed.yml live under eng/common, which is overwritten by the azure-sdk-tools sync (see eng/common/README.md:3), and neither file exists upstream in azure-sdk-tools. The next common-engineering sync will delete both and break the template references in build-cli.yml and build-and-test.yml. Move them under eng/pipelines/templates/steps/ (next to setup-go.yml and install-ms-openjdk.yml), or land them upstream in azure-sdk-tools first, before merging.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 12 out of 12 changed files in this pull request and generated 2 comments.

Comments suppressed due to low confidence (1)

eng/pipelines/templates/stages/1es-redirect.yml:35

  • [azd-code-reviewer] This enables CFSClean for every pipeline that extends this shared redirect, but only the CLI build jobs were migrated to authenticated package feeds. For example, eval-unit.yml:50,58, eval-waza.yml:53,56, vscode-build.yml:37-38, and azuredevops-build.yml:38-39 still install directly with npm/pip and will be blocked once CFSClean is enforced. Add the npm/PyPI authentication setup to every affected job, or scope the policy until those jobs are migrated.
      networkIsolationPolicy: Permissive, CFSClean

Comment thread eng/pipelines/templates/jobs/build-cli.yml Outdated
Comment thread eng/pipelines/templates/jobs/build-cli.yml

@jongio jongio left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One item from earlier passes is still open, now narrowed to a single file.

maven-authenticate.yml has since landed in azure-sdk-tools upstream, so that half is resolved. pypi-auth-dev-feed.yml still doesn't exist upstream (the closest match is the differently-named auth-dev-feed.yml). Since eng/common is overwritten by the common-engineering sync (eng/common/README.md), the next sync will delete pypi-auth-dev-feed.yml and break the template: reference to it in build-and-test.yml. Land it upstream in azure-sdk-tools first (as was done for maven), or move it under eng/pipelines/templates/steps/ and update the reference.

Comment thread eng/common/pipelines/templates/steps/pypi-auth-dev-feed.yml
Copilot AI review requested due to automatic review settings July 23, 2026 22:52
@chidozieononiwu
chidozieononiwu force-pushed the enforceCFSClean branch 2 times, most recently from da649ec to 44940c5 Compare July 23, 2026 22:56

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 12 out of 12 changed files in this pull request and generated 4 comments.

Comments suppressed due to low confidence (1)

eng/common/pipelines/templates/steps/pypi-auth-dev-feed.yml:40

  • [azd-code-reviewer] PIP_INDEX_URL is credential-bearing, as the new comment notes, but this block prints it and copies it into a non-secret UV_DEFAULT_INDEX variable. That can expose the feed token in logs or later task output. Do not log the URL, and preserve secret handling when passing it to uv by explicitly mapping a secret variable only into uv-consuming steps.
            Write-Host "Found pip index URL: $($env:PIP_INDEX_URL)"
            # UV_DEFAULT_INDEX is the canonical replacement for the deprecated UV_INDEX_URL (uv 0.4.23+).
            # PIP_INDEX_URL is set by PipAuthenticate@1 and contains embedded credentials, which uv
            # will use for Basic auth against the ADO feed (and its PyPI upstream) per astral-sh/uv#12651.
            Write-Host "##vso[task.setvariable variable=UV_DEFAULT_INDEX]$($env:PIP_INDEX_URL)"

Comment thread eng/common/pipelines/templates/steps/maven-authenticate.yml
Comment thread eng/pipelines/templates/steps/build-msi.yml
Comment thread eng/pipelines/templates/jobs/build-cli.yml
Comment thread eng/pipelines/templates/stages/1es-redirect.yml

@jongio jongio left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Following up on the new NuGet/PyPI auth commit: it adds a second reference to pypi-auth-dev-feed.yml, which still isn't landed upstream. Inline note has the fix.

Comment thread eng/pipelines/templates/jobs/build-cli.yml
@chidozieononiwu

Copy link
Copy Markdown
Member Author

/azp run azure-dev - cli

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
1 pipeline(s) were filtered out due to trigger conditions.

@chidozieononiwu

Copy link
Copy Markdown
Member Author

/azp run azure-dev - ext - microsoft.azd.demo

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).

Copilot AI review requested due to automatic review settings July 24, 2026 18:37

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (3)

eng/common/pipelines/templates/steps/maven-authenticate.yml:17

  • azd-code-reviewer: This unqualified feed name is resolved in the pipeline's current Azure DevOps project (internal), while the mirror in eng/settings.xml:9 targets the public project. MavenAuthenticate can therefore authenticate the wrong or nonexistent feed, leaving Maven unable to access the enforced mirror. Use the cross-project feed identifier, consistent with the NuGet and PyPI configuration in this PR.
      artifactsFeeds: "azure-sdk-for-java"

eng/common/pipelines/templates/steps/pypi-auth-dev-feed.yml:36

  • azd-code-reviewer: PIP_INDEX_URL contains embedded credentials (as noted below), so printing the complete value creates an avoidable credential-disclosure path if task masking is absent or changes. Log only that the variable was found; the URL itself is not needed for diagnostics.
            Write-Host "Found pip index URL: $($env:PIP_INDEX_URL)"

cli/azd/extensions/microsoft.azd.demo/CHANGELOG.md:5

  • azd-code-reviewer: The PR description says this only updates networkIsolationPolicy, but the diff also adds package-manager authentication/mirror infrastructure and cuts demo extension version 0.7.2. Update the description to cover these material changes so reviewers and release history accurately reflect the scope.
## 0.7.2 (2026-07-16)

- Simple test release of extension, with no changes.

@jongio jongio left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

azure-dev - cli can't run on this PR at all, so most of what this changes is unexercised.

Your /azp run azure-dev - cli at 00:58 came back with "1 pipeline(s) were filtered out due to trigger conditions". eng/pipelines/release-cli.yml is why:

pr:
  paths:
    include:
      - go.mod
      - cli/
      - eng/pipelines/release-cli.yml
      - eng/pipelines/templates/steps/publish-cli.yml
    exclude:
      - cli/azd/extensions/**

Everything this PR touches is either under eng/pipelines/templates/{jobs,stages,steps}, which isn't in the include list, or under cli/azd/extensions/microsoft.azd.demo, which is explicitly excluded. No changed file matches, so the pipeline never queues. The CD trigger: block right above it already lists /eng/pipelines/templates/jobs/build-cli.yml; the pr: block just never got the same entries.

azure-dev - vscode has the same gap. release-vscode.yml only includes ext/vscode and eng/pipelines/release-vscode.yml, neither of which this PR touches.

That leaves azure-dev - ext - microsoft.azd.demo as the only one of the three @danieljurek listed that actually ran, and it goes through release-azd-extension.yml, which never pulls in build-cli.yml, build-and-test.yml, or build-msi.yml. So the NuGet, npm, Maven and pip changes have zero coverage here, and the first real execution is post-merge on main with CFSClean already turned on.

Adding these to the pr: include lists in both pipelines would let you validate before merging:

      - eng/pipelines/templates/jobs/build-cli.yml
      - eng/pipelines/templates/stages/build-and-test.yml
      - eng/pipelines/templates/steps/build-msi.yml
      - eng/pipelines/templates/stages/1es-redirect.yml

Two inline comments below: the eng/common sync problem from my last pass (I tracked down the upstream file it should be named after) and a dead env var in build-cli.yml.

@@ -0,0 +1,49 @@
parameters:

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still open from my last pass, and I found the specific fix this time. This template already exists upstream as eng/common/pipelines/templates/steps/auth-dev-feed.yml in azure-sdk-tools. Same parameters, same steps, same uv block; only quote style and indentation differ. There is no pypi-auth-dev-feed.yml upstream.

The eng/common sync isn't additive. It clears the destination directory and re-copies from azure-sdk-tools, so downstream-only files get deleted (that's what removed files in #5413). Once the next sync lands, this file is gone and both - template: /eng/common/pipelines/templates/steps/pypi-auth-dev-feed.yml references, build-cli.yml:115 and build-and-test.yml:161, fail during template expansion, before the run starts.

Rename it to auth-dev-feed.yml using the upstream bytes and point both references at that name. Then the sync overwrites it harmlessly instead of deleting it.

Related, so nobody "fixes" the wrong thing: create-authenticated-npmrc.yml and maven-authenticate.yml in this PR are already byte-identical to upstream (blobs 3fa699b387d and 1b84e984add match azure-sdk-tools exactly). Don't apply local edits to those two, including the suggestion to switch artifactsFeeds to public/azure-sdk-for-java. Anything like that has to land upstream first or the next sync reverts it.

GOTESTSUM_JUNITFILE: junitTestReport.xml
SYSTEM_ACCESSTOKEN: $(System.AccessToken)
PIP_INDEX_URL: $(PIP_INDEX_URL)
PIP_EXTRA_INDEX_URL: $(PIP_EXTRA_INDEX_URL)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PIP_EXTRA_INDEX_URL never gets set, so this hands literal text to the test process.

The template gives PipAuthenticate@1 a single feed with onlyAddExtraIndex: false. The task shifts that one endpoint into PIP_INDEX_URL and only sets PIP_EXTRA_INDEX_URL when endpoints remain, which they don't (Tasks/PipAuthenticateV1/pipauthenticatemain.ts). Azure Pipelines leaves unresolved $(...) macros as-is, so go test ends up with PIP_EXTRA_INDEX_URL=$(PIP_EXTRA_INDEX_URL).

pip warns on a non-URL index and ignores it rather than failing, so nothing breaks. It's just log noise and a misleading env var. Drop the line.

@jongio jongio left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-checked 5906b98. Two things still stand, and one of the open questions resolves.

pypi-auth-dev-feed.yml gets deleted by the next common-engineering sync

I pulled the upstream blob SHAs to work out which of the three eng/common files are actually safe:

file upstream in azure-sdk-tools this PR
create-authenticated-npmrc.yml 3fa699b387d 3fa699b387d, identical
maven-authenticate.yml 1b84e984add 1b84e984add, identical
pypi-auth-dev-feed.yml 404, doesn't exist fc2396c8448

The first two match byte for byte because they were landed upstream first. pypi-auth-dev-feed.yml wasn't, and its content already exists upstream under a different name: eng/common/pipelines/templates/steps/auth-dev-feed.yml (cd8fa23e67e). Same four parameters, same Setup DevOpsFeedName step, same Twine, Pip and uv blocks, same UV_PYTHON_INSTALL_MIRROR value. Only quoting and indentation differ.

eng/common/README.md says updates here "end up getting overwritten with future updates", and the sync clears the destination rather than merging, so the file disappears and both of these break at template expansion:

  • eng/pipelines/templates/jobs/build-cli.yml:115
  • eng/pipelines/templates/stages/build-and-test.yml:161

Dropping the new file and repointing both references is the whole fix:

- template: /eng/common/pipelines/templates/steps/auth-dev-feed.yml

Still 1 of the 3 pipelines @danieljurek asked for

Checks on 5906b98 only show azure-dev - ext - microsoft.azd.demo. azure-dev - cli and azure-dev - vscode still don't queue, because the pr: path filters in release-cli.yml and release-vscode.yml don't list any file this PR touches. So the NuGet, npm, Maven and pip changes stay unexercised until this is on main with CFSClean already enabled.

On the build-msi.yml feed switch

WixToolset.DUtil and WixToolset.WcaUtil 4.0.0 both resolve on public/azure-sdk-for-net through its nuget.org upstream, so vstsFeed points at a feed that carries what cli/installer/windows.sln restores. The open item there is just the includeNuGetOrg default, which is already flagged inline.

@jongio jongio left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-checked 5906b98 today. No new commits since my last pass, so the two open items still stand, and I re-verified the main one against upstream just now.

pypi-auth-dev-feed.yml still gets deleted by the next eng/common sync. As of today it still doesn't exist in azure-sdk-tools (404), while the equivalent auth-dev-feed.yml does, with the same parameters, the same steps, and the same uv block. eng/common is overwritten by the common-engineering sync, so once it runs this file disappears and both - template: /eng/common/pipelines/templates/steps/pypi-auth-dev-feed.yml references fail at expansion time: build-cli.yml:115 and build-and-test.yml:161. The create-authenticated-npmrc.yml and maven-authenticate.yml blobs in this PR are byte-identical to upstream (3fa699b387d and 1b84e984add), so those two are safe to leave as-is. The fix is to drop the new file and point both references at auth-dev-feed.yml, or land pypi-auth-dev-feed.yml upstream first the way maven was. Full details are in the inline thread.

PIP_EXTRA_INDEX_URL: $(PIP_EXTRA_INDEX_URL) in build-cli.yml is still a dead macro. PipAuthenticate@1 runs with a single feed and onlyAddExtraIndex: false, so it only populates PIP_INDEX_URL. PIP_EXTRA_INDEX_URL never gets set, and the test process receives the literal $(PIP_EXTRA_INDEX_URL). pip ignores a non-URL index so nothing breaks, but it's misleading log noise. Drop the line. Inline thread has it.

One process note, not blocking on its own: only azure-dev - ext - microsoft.azd.demo actually runs on this PR. azure-dev - cli and azure-dev - vscode don't queue, because their pr: path filters don't list any file this PR touches, so the NuGet, npm, Maven, and pip wiring stays unexercised until this lands on main with CFSClean already on. Adding the touched template paths to those pr: include lists would let you validate before merge.

@jongio jongio left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-verified 5906b98 against azure-sdk-tools upstream just now. No new commits since my last pass, so the same two items are still open, and the first is still a merge blocker.

pypi-auth-dev-feed.yml gets deleted by the next eng/common sync. Checked the upstream blobs again today: pypi-auth-dev-feed.yml still returns 404 in azure-sdk-tools, so it doesn't exist upstream. The equivalent auth-dev-feed.yml does exist (cd8fa23e67e) with the same parameters, the same steps, and the same uv block. eng/common is overwritten by the common-engineering sync (eng/common/README.md), and the sync clears the destination rather than merging, so a downstream-only file gets removed. Once it runs, pypi-auth-dev-feed.yml disappears and both template: references fail at expansion time, before the run starts: build-cli.yml:115 and build-and-test.yml:161. Drop the new file and point both references at auth-dev-feed.yml, or land pypi-auth-dev-feed.yml upstream first the way maven-authenticate.yml was. For contrast, create-authenticated-npmrc.yml (3fa699b387d) and maven-authenticate.yml (1b84e984add) in this PR are already byte-identical to upstream, so those two are safe to leave as-is.

PIP_EXTRA_INDEX_URL: $(PIP_EXTRA_INDEX_URL) in build-cli.yml is a dead macro. PipAuthenticate@1 runs with a single feed and onlyAddExtraIndex: false, so it only populates PIP_INDEX_URL. PIP_EXTRA_INDEX_URL never gets set, and the test process receives the literal $(PIP_EXTRA_INDEX_URL). pip ignores a non-URL index so nothing breaks, but it's misleading log noise. Drop the line.

One process note, not blocking on its own: only azure-dev - ext - microsoft.azd.demo runs on this PR. azure-dev - cli and azure-dev - vscode don't queue, because their pr: path filters don't list any file this PR touches, so the NuGet, npm, Maven, and pip wiring stays unexercised until this lands on main with CFSClean already on. Adding the touched template paths to those pr: include lists would let you validate before merge. Details are in the existing inline threads.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ext-demo microsoft.azd.demo extension

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Enforce CFSClean

6 participants