Skip to content

Add tests/ to CI mypy invocation and fix test type errors#110

Merged
dgenio merged 3 commits intomainfrom
copilot/add-tests-to-ci-mypy-invocation
Apr 13, 2026
Merged

Add tests/ to CI mypy invocation and fix test type errors#110
dgenio merged 3 commits intomainfrom
copilot/add-tests-to-ci-mypy-invocation

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Apr 12, 2026

CI ran mypy chainweaver/ only, leaving the tests.* override in pyproject.toml unexercised and test type errors undetected.

Changes

CI

  • ci.yml: expand mypy step to python -m mypy chainweaver/ tests/

[type-arg] fixes (tests/helpers.py, tests/test_flow_execution.py)

  • Added from typing import Any; replaced 10 bare dict return annotations with dict[str, Any]

[return-value] fixes (tests/test_decorators.py)

  • Added # type: ignore[return-value] to 22 return statements in @tool-decorated test functions

The @tool decorator requires -> SomeBaseModel on the wrapped function for output schema inference, but the function bodies return dict — matching the documented usage pattern. This is an intentional type mismatch that can't be resolved without changing the decorator API:

@tool(description="Doubles a number.")
def double(number: int) -> ValueOutput:
    return {"value": number * 2}  # type: ignore[return-value]

Copilot AI linked an issue Apr 12, 2026 that may be closed by this pull request
Copilot AI changed the title [WIP] Add tests to CI mypy invocation and fix test type errors Add tests/ to CI mypy invocation and fix test type errors Apr 12, 2026
Copilot AI requested a review from dgenio April 12, 2026 14:40
@dgenio dgenio marked this pull request as ready for review April 12, 2026 14:40
@dgenio dgenio requested a review from Copilot April 12, 2026 14:41
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Expands CI type checking to include the tests/ tree and resolves the resulting mypy strictness issues in test code, ensuring the tests.* mypy override is exercised and test typing regressions are caught in CI.

Changes:

  • Update CI mypy invocation to run against both chainweaver/ and tests/.
  • Fix [type-arg] issues in tests by replacing bare dict return annotations with dict[str, Any].
  • Suppress intentional [return-value] mismatches in @tool-decorated test functions where the annotated output schema is a BaseModel, but the implementation returns a dict (per documented usage).

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated no comments.

File Description
.github/workflows/ci.yml Runs mypy over tests/ in CI to catch test typing issues.
tests/helpers.py Tightens helper return annotations to dict[str, Any] for mypy strictness.
tests/test_flow_execution.py Updates local tool fn return annotations to dict[str, Any] to satisfy mypy.
tests/test_decorators.py Adds targeted # type: ignore[return-value] for known/intentional decorator typing mismatch in tests.

Copy link
Copy Markdown
Owner

@dgenio dgenio left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review: REQUEST_CHANGES

Code changes are correct and CI passes (3.10–3.13 all green, ruff + mypy clean). One must-fix item before merging:


🔴 Must-fix — canonical docs show stale mypy command (see inline comment on ci.yml)

AGENTS.md § 7 and docs/agent-context/workflows.md § Validation commands both still list python -m mypy chainweaver/. After this PR merges, anyone following the authoritative sequence will skip tests/ — silently undoing the type-safety gains the PR introduces. Details and the exact one-line fix are in the inline comment on .github/workflows/ci.yml line 37.


🔶 Follow-up — dead module = "tests.*" mypy override in pyproject.toml

This override is never applied:

[[tool.mypy.overrides]]
module = "tests.*"
disallow_untyped_defs = false

When mypy is invoked with filesystem paths (python -m mypy chainweaver/ tests/), test files are processed as standalone scripts — not as tests.* package members — because tests/ has no __init__.py. The glob never matches, and mypy confirms it:

pyproject.toml: note: unused section(s): module = ['tests.*']

This was already dead before this PR; the expanded invocation makes it visible. Tests pass under full strict = true regardless (the PR's type fixes are correct and sufficient). Two options for a follow-up:

  1. Remove the override entirely — tests now have proper annotations; the relaxation is no longer needed.
  2. Add tests/__init__.py to make tests a package so the glob resolves correctly.

Suggested follow-up issue: Fix or remove dead module="tests.*" mypy override in pyproject.tomlpriority:low, complexity:simple.


ℹ️ Pre-existing failure (not introduced by this PR, informational only)

TestToolZeroDivisionError.test_zero_division_error_wrapped fails locally on Python 3.14.3. CPython 3.14 changed the ZeroDivisionError message for // from "integer division or modulo by zero" to "division by zero". Not a blocker (CI covers 3.10–3.13 and passes), but worth fixing before Python 3.14 is added to the matrix:

# Fragile (fails on 3.14+):
assert "integer division or modulo by zero" in str(record.error)

# Resilient across all versions:
assert "division by zero" in str(record.error)

Comment thread .github/workflows/ci.yml
- name: Type check with mypy
if: ${{ matrix.python-version == '3.10' }}
run: python -m mypy chainweaver/
run: python -m mypy chainweaver/ tests/
Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocker — doc governance trigger missed

This CI step now runs python -m mypy chainweaver/ tests/, but both canonical docs still show the old command:

  • AGENTS.md § 7 Validation commands
  • docs/agent-context/workflows.md § Validation commands

Per AGENTS.md § 10 Update policy:

Every PR: check whether AGENTS.md or any docs/agent-context/ file is stale with respect to the change. Update in the same PR.

These documents are the source of truth for every contributor and agent on this repo. After this PR merges, anyone following the canonical sequence will type-check chainweaver/ only — silently skipping the type-safety gains this PR introduces.

Required in this PR — two one-line edits:

  1. AGENTS.md § 7 — update the mypy line:

    -python -m mypy chainweaver/
    +python -m mypy chainweaver/ tests/
  2. docs/agent-context/workflows.md § Validation commands — same change in the # 3. Type check block. Optionally also expand the CI pipeline table row to "mypy chainweaver/ tests/ (Python 3.10 only)" for precision.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in commit 1d21e0b. Updated both AGENTS.md § 7 and docs/agent-context/workflows.md (Validation commands block + CI pipeline table row) to reflect python -m mypy chainweaver/ tests/.

Comment thread tests/test_decorators.py
def double(number: int) -> ValueOutput:
return {"value": number * 2}
return {"value": number * 2} # type: ignore[return-value]

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Major — 22 suppressions accepted; follow-up issue requested

The # type: ignore[return-value] on every return statement is the correct pragmatic fix. The decorator requires -> SomeBaseModel for output schema inference, but bodies return dict — that mismatch can't be resolved without changing the decorator API, as the PR description accurately explains.

Accepting this approach. Please open a follow-up issue to track the root-cause fix:

Suggested issue:

  • Title: Improve @tool decorator to accept dict returns without type: ignore[return-value]
  • Scope: Explore overloading the decorator return type annotation to accept dict[str, Any] | BaseModel, so test (and user) functions can write the natural pattern without suppression.
  • Labels: priority:low, complexity:average
  • Note: There are 22 occurrences here, and every future @tool test will need the same boilerplate unless this is addressed.

Copy link
Copy Markdown
Contributor Author

Copilot AI commented Apr 13, 2026

Review: REQUEST_CHANGES

Code changes are correct and CI passes (3.10–3.13 all green, ruff + mypy clean). One must-fix item before merging:...

Addressed in commit 1d21e0b:

  • AGENTS.md and docs/agent-context/workflows.md now show python -m mypy chainweaver/ tests/
  • Fragile ZeroDivisionError assertion fixed: changed "integer division or modulo by zero" to "by zero", which is a common suffix present in both Python ≤3.13 ("integer division or modulo by zero") and 3.14+ ("division by zero")

The dead module = "tests.*" override is a pre-existing issue. Since tests currently pass clean under strict = true (the type: ignore comments added in the previous commit handle the intentional mismatches), removing the override is straightforward if you'd like a follow-up PR to clean it up.

@dgenio dgenio force-pushed the copilot/add-tests-to-ci-mypy-invocation branch from 1d21e0b to 0d4e0f7 Compare April 13, 2026 10:55
@dgenio dgenio merged commit f880a5a into main Apr 13, 2026
@dgenio dgenio deleted the copilot/add-tests-to-ci-mypy-invocation branch April 13, 2026 10:56
dgenio added a commit that referenced this pull request Apr 13, 2026
Conflict in tests/test_flow_execution.py between this branch's
string-containment assertion and main's typed record.error.detail
set-membership check (landed via PR #110 and #109). Taking main's
version — it checks the typed attribute directly and explicitly
enumerates both CPython 3.13 and 3.14+ message strings.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add tests/ to CI mypy invocation and fix test type errors

3 participants