Add tests/ to CI mypy invocation and fix test type errors#110
Conversation
There was a problem hiding this comment.
Pull request overview
Expands CI type checking to include the tests/ tree and resolves the resulting mypy strictness issues in test code, ensuring the tests.* mypy override is exercised and test typing regressions are caught in CI.
Changes:
- Update CI mypy invocation to run against both
chainweaver/andtests/. - Fix
[type-arg]issues in tests by replacing baredictreturn annotations withdict[str, Any]. - Suppress intentional
[return-value]mismatches in@tool-decorated test functions where the annotated output schema is aBaseModel, but the implementation returns adict(per documented usage).
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
.github/workflows/ci.yml |
Runs mypy over tests/ in CI to catch test typing issues. |
tests/helpers.py |
Tightens helper return annotations to dict[str, Any] for mypy strictness. |
tests/test_flow_execution.py |
Updates local tool fn return annotations to dict[str, Any] to satisfy mypy. |
tests/test_decorators.py |
Adds targeted # type: ignore[return-value] for known/intentional decorator typing mismatch in tests. |
dgenio
left a comment
There was a problem hiding this comment.
Review: REQUEST_CHANGES
Code changes are correct and CI passes (3.10–3.13 all green, ruff + mypy clean). One must-fix item before merging:
🔴 Must-fix — canonical docs show stale mypy command (see inline comment on ci.yml)
AGENTS.md § 7 and docs/agent-context/workflows.md § Validation commands both still list python -m mypy chainweaver/. After this PR merges, anyone following the authoritative sequence will skip tests/ — silently undoing the type-safety gains the PR introduces. Details and the exact one-line fix are in the inline comment on .github/workflows/ci.yml line 37.
🔶 Follow-up — dead module = "tests.*" mypy override in pyproject.toml
This override is never applied:
[[tool.mypy.overrides]]
module = "tests.*"
disallow_untyped_defs = falseWhen mypy is invoked with filesystem paths (python -m mypy chainweaver/ tests/), test files are processed as standalone scripts — not as tests.* package members — because tests/ has no __init__.py. The glob never matches, and mypy confirms it:
pyproject.toml: note: unused section(s): module = ['tests.*']
This was already dead before this PR; the expanded invocation makes it visible. Tests pass under full strict = true regardless (the PR's type fixes are correct and sufficient). Two options for a follow-up:
- Remove the override entirely — tests now have proper annotations; the relaxation is no longer needed.
- Add
tests/__init__.pyto maketestsa package so the glob resolves correctly.
Suggested follow-up issue: Fix or remove dead module="tests.*" mypy override in pyproject.toml — priority:low, complexity:simple.
ℹ️ Pre-existing failure (not introduced by this PR, informational only)
TestToolZeroDivisionError.test_zero_division_error_wrapped fails locally on Python 3.14.3. CPython 3.14 changed the ZeroDivisionError message for // from "integer division or modulo by zero" to "division by zero". Not a blocker (CI covers 3.10–3.13 and passes), but worth fixing before Python 3.14 is added to the matrix:
# Fragile (fails on 3.14+):
assert "integer division or modulo by zero" in str(record.error)
# Resilient across all versions:
assert "division by zero" in str(record.error)| - name: Type check with mypy | ||
| if: ${{ matrix.python-version == '3.10' }} | ||
| run: python -m mypy chainweaver/ | ||
| run: python -m mypy chainweaver/ tests/ |
There was a problem hiding this comment.
Blocker — doc governance trigger missed
This CI step now runs python -m mypy chainweaver/ tests/, but both canonical docs still show the old command:
AGENTS.md § 7 Validation commandsdocs/agent-context/workflows.md § Validation commands
Per AGENTS.md § 10 Update policy:
Every PR: check whether AGENTS.md or any
docs/agent-context/file is stale with respect to the change. Update in the same PR.
These documents are the source of truth for every contributor and agent on this repo. After this PR merges, anyone following the canonical sequence will type-check chainweaver/ only — silently skipping the type-safety gains this PR introduces.
Required in this PR — two one-line edits:
-
AGENTS.md § 7— update the mypy line:-python -m mypy chainweaver/ +python -m mypy chainweaver/ tests/
-
docs/agent-context/workflows.md § Validation commands— same change in the# 3. Type checkblock. Optionally also expand the CI pipeline table row to"mypy chainweaver/ tests/ (Python 3.10 only)"for precision.
There was a problem hiding this comment.
Done in commit 1d21e0b. Updated both AGENTS.md § 7 and docs/agent-context/workflows.md (Validation commands block + CI pipeline table row) to reflect python -m mypy chainweaver/ tests/.
| def double(number: int) -> ValueOutput: | ||
| return {"value": number * 2} | ||
| return {"value": number * 2} # type: ignore[return-value] | ||
|
|
There was a problem hiding this comment.
Major — 22 suppressions accepted; follow-up issue requested
The # type: ignore[return-value] on every return statement is the correct pragmatic fix. The decorator requires -> SomeBaseModel for output schema inference, but bodies return dict — that mismatch can't be resolved without changing the decorator API, as the PR description accurately explains.
Accepting this approach. Please open a follow-up issue to track the root-cause fix:
Suggested issue:
- Title:
Improve @tool decorator to accept dict returns without type: ignore[return-value] - Scope: Explore overloading the decorator return type annotation to accept
dict[str, Any] | BaseModel, so test (and user) functions can write the natural pattern without suppression. - Labels:
priority:low,complexity:average - Note: There are 22 occurrences here, and every future
@tooltest will need the same boilerplate unless this is addressed.
Addressed in commit
The dead |
Agent-Logs-Url: https://github.com/dgenio/ChainWeaver/sessions/5d7749b4-0843-4b34-9bfc-1914f9ba38cb Co-authored-by: dgenio <12731907+dgenio@users.noreply.github.com>
… test assertion Agent-Logs-Url: https://github.com/dgenio/ChainWeaver/sessions/be9a4941-f16f-4f73-9f84-5aba5b9c3ee7 Co-authored-by: dgenio <12731907+dgenio@users.noreply.github.com>
1d21e0b to
0d4e0f7
Compare
Conflict in tests/test_flow_execution.py between this branch's string-containment assertion and main's typed record.error.detail set-membership check (landed via PR #110 and #109). Taking main's version — it checks the typed attribute directly and explicitly enumerates both CPython 3.13 and 3.14+ message strings.
CI ran
mypy chainweaver/only, leaving thetests.*override inpyproject.tomlunexercised and test type errors undetected.Changes
CI
ci.yml: expand mypy step topython -m mypy chainweaver/ tests/[type-arg]fixes (tests/helpers.py,tests/test_flow_execution.py)from typing import Any; replaced 10 baredictreturn annotations withdict[str, Any][return-value]fixes (tests/test_decorators.py)# type: ignore[return-value]to 22 return statements in@tool-decorated test functionsThe
@tooldecorator requires-> SomeBaseModelon the wrapped function for output schema inference, but the function bodies returndict— matching the documented usage pattern. This is an intentional type mismatch that can't be resolved without changing the decorator API: