Skip to content

Update CLI command descriptions and testing prompts#2131

Open
vcolin7 wants to merge 5 commits intomainfrom
vcolin7/fix-cli-extension-descriptions
Open

Update CLI command descriptions and testing prompts#2131
vcolin7 wants to merge 5 commits intomainfrom
vcolin7/fix-cli-extension-descriptions

Conversation

@vcolin7
Copy link
Contributor

@vcolin7 vcolin7 commented Mar 20, 2026

What does this PR do?

Enhance clarity of CLI extension command descriptions and update prompts for manual testing. Revise azure-rules.txt to disambiguate when to use the extension tools for Azure CLI commands from bestpractices.

GitHub issue number?

Fixes: #1987

Pre-merge Checklist

  • Required for All PRs
    • Read contribution guidelines
    • PR title clearly describes the change
    • Commit history is clean with descriptive messages (cleanup guide)
    • Added comprehensive tests for new/modified functionality
    • Updated servers/Azure.Mcp.Server/CHANGELOG.md and/or servers/Fabric.Mcp.Server/CHANGELOG.md for product changes (features, bug fixes, UI/UX, updated dependencies)
  • For MCP tool changes:
    • One tool per PR: This PR adds or modifies only one MCP tool for faster review cycles
    • Updated servers/Azure.Mcp.Server/README.md and/or servers/Fabric.Mcp.Server/README.md documentation
    • Validate README.md changes using script at eng/scripts/Process-PackageReadMe.ps1. See Package README
    • Updated command list in /servers/Azure.Mcp.Server/docs/azmcp-commands.md and/or /docs/fabric-commands.md
    • Run .\eng\scripts\Update-AzCommandsMetadata.ps1 to update tool metadata in azmcp-commands.md (required for CI)
    • For new or modified tool descriptions, ran ToolDescriptionEvaluator and obtained a score of 0.4 or more and a top 3 ranking for all related test prompts
    • For tools with new names, including new tools or renamed tools, update consolidated-tools.json
    • For renamed tools, follow the Tool Rename Checklist and tag the PR with the breaking-change label
    • For new tools associated with Azure services or publicly available tools/APIs/products, add URL to documentation in the PR description
  • Extra steps for Azure MCP Server tool changes:
    • Updated test prompts in /servers/Azure.Mcp.Server/docs/e2eTestPrompts.md
    • 👉 For Community (non-Microsoft team member) PRs:
      • Security review: Reviewed code for security vulnerabilities, malicious code, or suspicious activities before running tests (crypto mining, spam, data exfiltration, etc.)
      • Manual tests run: added comment /azp run mcp - pullrequest - live to run Live Test Pipeline

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the Azure MCP Server’s “extension/CLI” tool descriptions and related guidance/prompts to better steer tool selection toward extension_cli_generate / extension_cli_install, addressing issue #1987.

Changes:

  • Refines extension area and CLI command descriptions to more explicitly mention az/azd/func/azqr.
  • Updates azure-rules.txt to prioritize extension tools for CLI-related user requests.
  • Adjusts Azure CLI e2e test prompts to better trigger the intended extension tools.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 6 comments.

Show a summary per file
File Description
tools/Azure.Mcp.Tools.Extension/src/ExtensionSetup.cs Updates the extension/cli command group descriptions for improved clarity and routing.
tools/Azure.Mcp.Tools.Extension/src/Commands/CliInstallCommand.cs Revises tool description text for CLI install guidance.
tools/Azure.Mcp.Tools.Extension/src/Commands/CliGenerateCommand.cs Revises tool description text for Azure CLI command generation guidance.
tools/Azure.Mcp.Tools.Extension/src/Commands/AzqrCommand.cs Condenses/rewrites azqr tool description text.
servers/Azure.Mcp.Server/src/Resources/azure-rules.txt Updates server rules to route CLI-related intents to extension tools first.
servers/Azure.Mcp.Server/docs/e2eTestPrompts.md Updates e2e prompts for Azure CLI to better trigger extension tools.

@vcolin7
Copy link
Contributor Author

vcolin7 commented Mar 20, 2026

I've successfully tested all of the manual prompts reported in the original issue:


Screenshot 2026-03-19 215809
image
image
image
image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

tools-Azd Azure Developer CLI related

Projects

Status: Untriaged

Development

Successfully merging this pull request may close these issues.

[All & Namespace mode] Test prompts do not trigger extension_cli_generate / extension_cli_install tool

2 participants