Skip to content

docs: define testing approach for agents #7

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
sergdort merged 1 commit into from
Mar 1, 2026
Merged

docs: define testing approach for agents #7

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 34 additions & 0 deletions AGENTS.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -343,6 +343,40 @@ For destructive actions:
- Run smoke queries (`category list`, `expense list`).
- Validate row counts and recent timestamps.

## Testing Approach (Required for Agents)

When implementing or reviewing features, follow these rules:

1. **Write tests for humans first.**
- Keep tests readable and intentional (clear setup, action, assertion flow).
- Prefer explicit expected values over opaque helper magic.

2. **Test requirements and business rules, not implementation trivia.**
- Feature tests should validate behavior users/business care about.
- Keep lower-level/unit tests for mechanics, but feature coverage must stay requirement-led.

3. **Do not “game” tests.**
Copy link

Copilot AI Mar 1, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This line introduces a curly quote character (“ ”). The rest of AGENTS.md appears to use plain ASCII punctuation, so keeping straight quotes avoids encoding/grep/copy-paste issues in terminals and tooling.

Suggested change
3. **Do not “game” tests.**
3. **Do not "game" tests.**

Copilot uses AI. Check for mistakes.
- Making tests green is not the only goal.
- When tests fail, first verify whether requirements are correct and still desired.
- Never patch tests with hacks that hide real regressions.

4. **Default to red → green → refactor.**
Copy link

Copilot AI Mar 1, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This heading uses a Unicode arrow (→). Consider using ASCII (e.g., "->" or words like "then") to keep the document consistent and avoid display issues in some environments.

Suggested change
4. **Default to red → green → refactor.**
4. **Default to red -> green -> refactor.**

Copilot uses AI. Check for mistakes.
- Add/adjust a failing test first, then implement behavior.
- If infra is missing, add an explicit pending/placeholder test tied to requirement text and finish it in the same feature stream.
- Pending/placeholder tests are acceptable on feature branches, but must be resolved before merge unless a human explicitly approves carrying them forward.

5. **Treat flaky tests as defects.**
- Report flakiness and investigate root cause (time, async races, shared state, environment coupling).
- Prefer refactoring for determinism over retries/timeouts as a workaround.
- Do not ship retry-based or timeout-inflation hacks as the final fix; if root-cause work cannot be completed immediately, open a tracked follow-up with repro notes.

6. **Integration tests must be hermetic locally and in CI.**
- Never rely on developer machine state (real user DB, real env tokens, existing external integrations).
- Use isolated temp DB/env per test or test file, and clean up after each run.

7. **If behavior changes, update tests and requirement docs together.**
- If a previous expectation is no longer valid, document why and update tests intentionally.

## Agent Behavior Rules

- Prefer read operations before writes.
Expand Down