Document agent pitfalls in AGENTS.md: test coverage and docs spell check#12580
Merged
Conversation
The CI test runs use `--cov=aiohttp/ --cov=tests/`, so uncovered lines in `tests/` show up in the codecov patch report alongside uncovered lines in `aiohttp/`. Spell out the common shapes that trip agents up (defensive `raise` inside a monkeypatched stub, one-sided cleanup branches behind `if had_own_attr:`) and the patterns that keep tests fully covered, with a pointer to yarl#1687 as the canonical example of the failure mode.
psf-chronographer
Bot
added
the
bot:chronographer:provided
CI runs `make doc-spelling` under `-W --keep-going` and treats unknown words as hard failures, including every `CHANGES/*.rst` fragment in the build. Spell out the pre-push step and the wordlist policy so a missing technical word does not burn a CI run. Adds `codecov` to `docs/spelling_wordlist.txt` for the fragment from the previous commit.
bdraco
changed the title
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #12580 +/- ##
=======================================
Coverage 98.94% 98.94%
=======================================
Files 131 131
Lines 46683 46683
Branches 2421 2421
=======================================
Hits 46189 46189
Misses 369 369
Partials 125 125
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
Merging this PR will improve performance by 12.45%
|
| Benchmark | BASE |
HEAD |
Efficiency | |
|---|---|---|---|---|
| ⚡ | test_resolve_root_route_with_many_fixed_routes |
1,078.4 µs | 951.8 µs | +13.3% |
| ⚡ | test_resolve_dynamic_resource_url_with_many_dynamic_routes |
4 ms | 3.7 ms | +8.56% |
| ⚡ | test_resolve_dynamic_resource_url_with_many_dynamic_routes_with_common_prefix |
293.3 ms | 252.9 ms | +16% |
| ⚡ | test_resolve_dynamic_resource_url_with_many_static_routes |
4 ms | 3.7 ms | +8.15% |
| ⚡ | test_resolve_gitapi |
338.3 ms | 301.7 ms | +12.12% |
| ⚡ | test_resolve_gitapi_root |
2.5 ms | 2.2 ms | +14.63% |
| ⚡ | test_resolve_gitapi_subapps |
340.9 ms | 302.8 ms | +12.59% |
| ⚡ | test_resolve_multiple_fixed_url_with_many_routes |
2.7 ms | 2.4 ms | +13.49% |
| ⚡ | test_resolve_multiple_level_fixed_url_with_many_routes |
10.6 ms | 9.2 ms | +14.76% |
| ⚡ | test_resolve_prefix_resources_many_prefix_many_plain |
5.4 ms | 4.9 ms | +9.38% |
| ⚡ | test_resolve_root_route |
1,075.5 µs | 942.7 µs | +14.09% |
| ⚡ | test_resolve_single_fixed_url_with_many_routes |
1,081.6 µs | 955.9 µs | +13.15% |
| ⚡ | test_resolve_static_root_route |
1.2 ms | 1.1 ms | +11.9% |
Tip
Curious why this is faster? Comment @codspeedbot explain why this is faster on this PR, or directly use the CodSpeed MCP with your agent.
Comparing bdraco:agents-md-test-coverage (ed635cd) with master (5edd76f)
Footnotes
-
69 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What do these changes do?
Two AGENTS.md additions for failure modes agents keep hitting in
this repo:
covered section under Tests, explaining that the CI test
runs use
--cov=aiohttp/ --cov=tests/so uncovered lines intests/show up in the codecov patch report alongsideuncovered lines in
aiohttp/. Lists the shapes agentscommonly leave uncovered (defensive
raiseinside amonkeypatched stub, one-sided cleanup branches behind
if had_own_attr:) and the patterns that keep tests fullycovered. References
aio-libs/yarl#1687
and the corresponding
aio-libs/yarl#1689.
check before pushing, documenting that the
lintCI jobruns
make doc-spellingwith-W --keep-goingand readsevery
CHANGES/*.rstfragment, so an unknown technical wordin a news fragment fails CI before a human sees the PR.
Adds
codecovtodocs/spelling_wordlist.txtfor the newfragment. Mirrors the equivalent
aio-libs/multidict#1345
addition.
Renumbers the surrounding PR rule (Commit hygiene becomes #7)
and extends Things-not-to-do with matching bullets.
Are there changes in behavior for the user?
No. Docs-only change to the agent-facing contributor notes,
plus one wordlist entry.
Is it a substantial burden for the maintainers to support this?
No. Both rules are already enforced (by the codecov patch
report and by the docs spelling step in CI); this just writes
them down where agents will read them.
Related issue number
N/A.
Checklist
CONTRIBUTORS.txt(N/A, already listed)CHANGES/folderDrafted with Claude Opus 4.7; reviewed by @bdraco.
Agent run details (optional, for reviewers)
Docs-only change, no test suite to run. Verification:
pre-commit run --files AGENTS.md CHANGES/12580.contrib.rst docs/spelling_wordlist.txtpassed.make doc-spellingflaggedcodecovbefore the wordlist entry was added; after adding it, the only remaining warnings are two pre-existing words (accessorfromCHANGES/10665.feature.rst,flakeyfromCHANGES/12571.contrib.rst), unrelated to this PR.