-
Notifications
You must be signed in to change notification settings - Fork 0
feat(qa): targeted test command templates + full test discovery in project index #402
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,79 @@ | ||
| #!/usr/bin/env python3 | ||
| """ | ||
| Tests for ServiceAnalyzer testing detection. | ||
|
|
||
| Verifies that service analysis (which feeds project_index.json consumed by | ||
| coder/QA prompts) surfaces test commands discovered by TestDiscovery for | ||
| all supported ecosystems, not just JS/Python. | ||
| """ | ||
|
|
||
| import sys | ||
| import tempfile | ||
| from pathlib import Path | ||
|
|
||
| import pytest | ||
|
|
||
| sys.path.insert(0, str(Path(__file__).parent.parent / "apps" / "backend")) | ||
|
|
||
| from analysis.analyzers import ServiceAnalyzer | ||
|
|
||
|
|
||
| @pytest.fixture | ||
| def temp_dir(): | ||
| """Create a temporary directory for tests.""" | ||
| with tempfile.TemporaryDirectory() as tmpdir: | ||
| yield Path(tmpdir) | ||
|
|
||
|
|
||
| class TestServiceAnalyzerTesting: | ||
| """Tests for _detect_testing via TestDiscovery.""" | ||
|
|
||
| def test_python_service_gets_test_commands(self, temp_dir): | ||
| """Python service surfaces pytest command and targeted template.""" | ||
| (temp_dir / "requirements.txt").write_text("pytest\n") | ||
| (temp_dir / "tests").mkdir() | ||
|
|
||
| analyzer = ServiceAnalyzer(temp_dir, "backend") | ||
| analyzer._detect_testing() | ||
|
|
||
| assert analyzer.analysis["testing"] == "pytest" | ||
| assert analyzer.analysis["test_command"] == "pytest" | ||
| assert analyzer.analysis["targeted_test_command"] == "pytest {target}" | ||
| assert analyzer.analysis["test_directory"] == "tests" | ||
|
|
||
| def test_gradle_service_gets_test_commands(self, temp_dir): | ||
| """JVM/Gradle service surfaces gradle test commands.""" | ||
| (temp_dir / "build.gradle.kts").write_text('plugins { kotlin("jvm") }') | ||
| (temp_dir / "gradlew").write_text("#!/bin/sh") | ||
|
|
||
| analyzer = ServiceAnalyzer(temp_dir, "api") | ||
| analyzer._detect_testing() | ||
|
|
||
| assert analyzer.analysis["testing"] == "gradle" | ||
| assert analyzer.analysis["test_command"] == "./gradlew test" | ||
| assert ( | ||
| analyzer.analysis["targeted_test_command"] | ||
| == "./gradlew test --tests {target}" | ||
| ) | ||
|
|
||
| def test_js_service_separates_unit_and_e2e(self, temp_dir): | ||
| """JS service reports unit and e2e frameworks separately.""" | ||
| (temp_dir / "package.json").write_text( | ||
| '{"devDependencies": {"vitest": "^1.0.0", "cypress": "^13.0.0"}}' | ||
| ) | ||
|
|
||
| analyzer = ServiceAnalyzer(temp_dir, "frontend") | ||
| analyzer._detect_testing() | ||
|
|
||
| assert analyzer.analysis["testing"] == "vitest" | ||
| assert analyzer.analysis["e2e_testing"] == "cypress" | ||
|
|
||
| def test_service_without_tests_has_no_test_command(self, temp_dir): | ||
| """Service without test setup gets no test keys.""" | ||
| (temp_dir / "main.zig").write_text("pub fn main() void {}") | ||
|
|
||
| analyzer = ServiceAnalyzer(temp_dir, "tool") | ||
| analyzer._detect_testing() | ||
|
|
||
| assert "test_command" not in analyzer.analysis | ||
| assert "targeted_test_command" not in analyzer.analysis |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🩺 Stability & Availability | 🔵 Trivial | ⚡ Quick win
Consider broadening the exception guard for robustness.
Only
ImportErroris caught, butTestDiscovery().discover(self.path)could raise other exceptions (e.g.,OSErrorfromPath.resolve()orglob()on restricted paths). This would crash the entireanalyze()call rather than falling back to minimal detection. The discovery code handles most file/JSON errors internally, so the risk is low, but wrapping thediscover()call in a broader guard (or addingOSError) would improve resilience.As per path instructions, check for proper error handling and security considerations.
♻️ Proposed refactor
try: from analysis.test_discovery import TestDiscovery discovery = TestDiscovery().discover(self.path) - except ImportError: + except (ImportError, OSError): # Fall back to the previous minimal detection📝 Committable suggestion
🤖 Prompt for AI Agents
Source: Path instructions