feat: Add NL2Query quality test runner

[languy]

Introduce a test runner that takes test spec, schema as input and produces a quality report md file as output.
Test runs the same function calls used in NL2Query for eatch test spec.
The report includes statistics on time measurement, token count (input and output) and grading scores.
The test spec specifies the purpose of the test, prompt and expected query and we use AI to grade the actual query against the expected result.
The runner is designed to be lightweight and generic (user provides test spec).
The runner needs to be executed manually at runtime by the user who must be signed in github, in order to use gh copilot.

• Can be invoked with runNl2QueryQualityTest command. Command is only available in debug mode.
• User selects LLM model to test
• User selects LLM model that grades the output
• User selects number of test runs

[github-actions]

🎉 Build Summary

🔗 Source

• Commit: f14fbde
• Pull Request: #3116 feat: Add NL2Query quality test runner

📦 Package Information

• Version: 0.35.1
• Preview: True
• VSIX File: vscode-cosmosdb-0.35.1.vsix
• VSIX Size: 3.34 MB
• Artifact: dev-languy-add-nl2query-quality-test-0.35.1-f14fbde.zip

🧪 Test Results

• Unit Tests: ✅ success
• Integration Tests: ✅ success

✅ Build Status

All checks completed successfully!

[github-actions]

🎭 E2E Tests (Playwright + VS Code)

Commit: 2202ab3
Pull Request: #3116 feat: Add NL2Query quality test runner

🧪 Result

• E2E Tests: ✅ success

📥 Artifacts (run)

• e2e-results-1 — 202 B
• e2e-html-report-1 — 198.7 KB

Tip: the HTML report artifact contains a self-contained Playwright report.
Download the zip, extract, and open index.html — or run
npx playwright show-report <extracted-dir> for the interactive view.

[github-actions]

🔨 Build, Lint & Test

🔗 Source

• Commit: 2202ab3
• Pull Request: #3116 feat: Add NL2Query quality test runner

📦 Package Information

• Version: 0.35.1
• Preview: True
• VSIX File: vscode-cosmosdb-0.35.1.vsix
• VSIX Size: 3.35 MB
• Artifact: dev-languy-add-nl2query-quality-test-0.35.1-2202ab3.zip

🧪 Test Results

• Unit Tests: ✅ success
• Integration Tests (extension host): ✅ success

📥 Artifacts (run)

• dev-languy-add-nl2query-quality-test-0.35.1-2202ab3 — 3.35 MB

✅ Build Status

Build and local tests passed. See sibling comments below for E2E and NoSQL integration results.

[Copilot]

Pull request overview

Adds a manual NL2Query quality test runner to help evaluate generateQuery behavior across a user-provided test spec and schema, producing a Markdown report with timing/token/grade statistics. This fits as a dev-only workflow tool for validating NL2Query prompt/pipeline quality during development.

Changes:

• Adds a dev-only VS Code command (cosmosDB.dev.runNl2QueryQualityTest) to run NL2Query quality tests and generate a Markdown report.
• Adds documentation and gitignore scaffolding for the quality test suite under test/quality/nl2query/.
• Wires command visibility via a dev-mode context key (cosmosDB.devMode) and package contributions.

Reviewed changes

Copilot reviewed 5 out of 6 changed files in this pull request and generated 9 comments.

Show a summary per file

File	Description
test/quality/nl2query/README.md	Adds instructions and structure for running the NL2Query quality suite.
test/quality/nl2query/.gitignore	Ignores generated reports under results/.
src/extension.ts	Registers a dev-only context + dynamically imports the quality test command in Development mode.
src/commands/nl2queryQualityTest.ts	Implements the interactive runner, batch grading, and Markdown report generation.
package.json	Contributes the new dev command and gates it in the Command Palette with cosmosDB.devMode.
package-lock.json	Lockfile update (platform metadata normalization).