refactor: Add LogExpectation TypedDict for better type safety

.beads/issues.jsonl

@@ -39,14 +39,14 @@
 {"id":"vgi-python-k7x","title":"Use Mapping instead of dict in extract_argument_specs signature","description":"The arg_types parameter in extract_argument_specs() is typed as dict[str, pa.DataType]. Using Mapping[str, pa.DataType] from collections.abc would be more flexible, accepting any mapping type.","status":"closed","priority":4,"issue_type":"task","created_at":"2026-01-05T11:51:21.021496-05:00","created_by":"rusty","updated_at":"2026-01-05T12:03:51.771301-05:00","closed_at":"2026-01-05T12:03:51.771301-05:00","close_reason":"Closed"}
 {"id":"vgi-python-kz4","title":"Rename TableInOutGeneratorFunction to TableInOutGenerator for consistency","description":"Naming inconsistency: TableFunctionGenerator uses *Generator suffix, but TableInOutGeneratorFunction uses *GeneratorFunction suffix. Rename TableInOutGeneratorFunction to TableInOutGenerator for consistency. Also consider renaming ScalarFunctionGenerator if needed.","status":"closed","priority":3,"issue_type":"task","created_at":"2026-01-04T20:06:41.581028-05:00","created_by":"rusty","updated_at":"2026-01-04T21:43:58.141038-05:00","closed_at":"2026-01-04T21:43:58.141038-05:00","close_reason":"PR #7 created: https://github.com/Query-farm/vgi-python/pull/7"}
 {"id":"vgi-python-l1u","title":"Consider custom __repr__ for ArgumentSpec","description":"The default dataclass __repr__ includes the full Arrow type repr which can be verbose. Consider a custom __repr__ that's more concise for debugging, e.g., 'ArgumentSpec(name=\"count\", pos=0, type=int64)' instead of showing the full pa.DataType object.","status":"open","priority":4,"issue_type":"task","created_at":"2026-01-05T11:51:21.415976-05:00","created_by":"rusty","updated_at":"2026-01-05T11:51:21.415976-05:00"}
-{"id":"vgi-python-lec","title":"Add test coverage for testing.py helper edge cases","notes":"Coverage: 89% in vgi/testing.py. Missing tests for:\n- Lines 421-422, 450-451: StopIteration handling in _process_batch\n- Lines 468-472: FINISHED status during data phase\n- Lines 485-486, 502-503: _finalize edge cases\n\nLow priority since these are test helpers.","status":"open","priority":4,"issue_type":"task","created_at":"2026-01-04T22:15:34.006563-05:00","created_by":"rusty","updated_at":"2026-01-04T22:16:18.592782-05:00"}
+{"id":"vgi-python-lec","title":"Add test coverage for testing.py helper edge cases","notes":"Coverage: 89% in vgi/testing.py. Missing tests for:\n- Lines 421-422, 450-451: StopIteration handling in _process_batch\n- Lines 468-472: FINISHED status during data phase\n- Lines 485-486, 502-503: _finalize edge cases\n\nLow priority since these are test helpers.","status":"closed","priority":4,"issue_type":"task","created_at":"2026-01-04T22:15:34.006563-05:00","created_by":"rusty","updated_at":"2026-01-05T12:07:48.026786-05:00","closed_at":"2026-01-05T12:07:48.026786-05:00","close_reason":"Closed"}
 {"id":"vgi-python-lzc","title":"Extract duplicated sort_key function in argument_spec","description":"The sort_key function is duplicated at lines 139-142 and 309-312 in argument_spec.py. Extract it to a module-level function to follow DRY principles.","status":"closed","priority":3,"issue_type":"task","created_at":"2026-01-05T11:51:19.141041-05:00","created_by":"rusty","updated_at":"2026-01-05T11:55:22.535816-05:00","closed_at":"2026-01-05T11:55:22.535816-05:00","close_reason":"Closed"}
 {"id":"vgi-python-m45","title":"Create tests/test_argument_spec.py","description":"## Overview\n\nCreate comprehensive tests for the argument specification serialization module.\n\n## File Location\n\n`tests/test_argument_spec.py`\n\n## Test Classes and Cases\n\n### TestArgumentSpecToSchema\n\nTest converting ArgumentSpec objects to Arrow schema.\n\n#### test_positional_arguments_preserve_order\n- Create specs with positions 0, 1, 2\n- Convert to schema\n- Verify field order matches position order\n- Verify field types are preserved\n\n#### test_named_arguments_have_metadata\n- Create spec with position='key' (named)\n- Convert to schema\n- Verify field has `vgi_arg=named` metadata\n\n#### test_mixed_positional_and_named\n- Create mix of positional (0, 1) and named ('format', 'verbose') specs\n- Convert to schema\n- Verify positional come first, then named\n- Verify named have correct metadata\n\n#### test_table_input_uses_null_type\n- Create spec with is_table_input=True\n- Convert to schema\n- Verify field type is pa.null()\n- Verify field has `vgi_type=table` metadata\n\n#### test_any_type_uses_null_type\n- Create spec with is_any_type=True\n- Convert to schema\n- Verify field type is pa.null()\n- Verify field has `vgi_type=any` metadata\n\n#### test_varargs_has_metadata\n- Create spec with is_varargs=True and arrow_type=pa.int64()\n- Convert to schema\n- Verify field type is pa.int64() (element type preserved)\n- Verify field has `vgi_varargs=true` metadata\n\n### TestSchemaToArgumentSpecs\n\nTest converting Arrow schema back to ArgumentSpec objects.\n\n#### test_positional_arguments_from_schema\n- Create schema with 3 fields (no metadata)\n- Convert to specs\n- Verify positions are 0, 1, 2\n\n#### test_named_arguments_from_metadata\n- Create schema with `vgi_arg=named` metadata on fields\n- Convert to specs\n- Verify position is field name string\n\n#### test_table_input_detected\n- Create schema with `vgi_type=table` metadata\n- Convert to specs\n- Verify is_table_input=True\n\n#### test_any_type_detected\n- Create schema with `vgi_type=any` metadata\n- Convert to specs\n- Verify is_any_type=True\n\n#### test_varargs_detected\n- Create schema with `vgi_varargs=true` metadata\n- Convert to specs\n- Verify is_varargs=True\n\n### TestRoundTrip\n\nTest that specs survive serialization round-trip.\n\n#### test_complex_arrow_types_preserved\nTest each of these types round-trips correctly:\n- pa.int64(), pa.float32(), pa.utf8()\n- pa.list_(pa.float64())\n- pa.struct([pa.field('a', pa.int32()), pa.field('b', pa.string())])\n- pa.map_(pa.string(), pa.int64())\n- pa.decimal128(10, 2)\n- pa.timestamp('us', tz='UTC')\n\n#### test_full_function_signature_roundtrip\n- Create specs matching a realistic function:\n  - count: int, position 0\n  - data: TableInput, position 1\n  - extra: float varargs, position 2\n  - format: str, named 'format'\n- Convert to schema, serialize to bytes, deserialize, convert back to specs\n- Verify all specs match original\n\n### TestExtractArgumentSpecs\n\nTest extracting specs from function classes.\n\n#### test_extract_from_simple_function\n- Define function class with Arg descriptors\n- Call extract_argument_specs with arg_types dict\n- Verify specs match descriptors\n\n#### test_extract_table_input\n- Define function with Arg[TableInput]\n- Extract specs\n- Verify is_table_input=True\n\n#### test_extract_any_arrow\n- Define function with Arg[AnyArrow]\n- Extract specs\n- Verify is_any_type=True\n\n#### test_extract_varargs\n- Define function with Arg[int](2, varargs=True)\n- Extract specs\n- Verify is_varargs=True\n\n### TestEdgeCases\n\n#### test_empty_schema\n- Convert empty list of specs to schema\n- Verify empty schema works\n- Convert back, verify empty list\n\n#### test_only_named_arguments\n- Create specs with only named arguments (no positional)\n- Round-trip and verify\n\n#### test_only_positional_arguments\n- Create specs with only positional arguments (no named)\n- Round-trip and verify\n\n## Test Utilities\n\nConsider creating fixtures for common patterns:\n- `make_spec()` helper for creating ArgumentSpec\n- Sample function classes for extraction tests","status":"closed","priority":2,"issue_type":"task","created_at":"2026-01-05T11:18:53.312911-05:00","created_by":"rusty","updated_at":"2026-01-05T11:32:35.580879-05:00","closed_at":"2026-01-05T11:32:35.580879-05:00","close_reason":"Created comprehensive tests with 43 passing test cases","dependencies":[{"issue_id":"vgi-python-m45","depends_on_id":"vgi-python-cd0","type":"blocks","created_at":"2026-01-05T11:19:30.779207-05:00","created_by":"rusty"}]}
 {"id":"vgi-python-odi","title":"Change max_processes from method to property in Function hierarchy","description":"Refactor max_processes from a method to a property across the Function class hierarchy (Function, ScalarFunction, TableFunctionGenerator, TableInOutFunction, etc.). This makes the API more consistent since max_processes is effectively a constant per function class and properties are more idiomatic for such values.","status":"closed","priority":2,"issue_type":"task","created_at":"2026-01-04T11:25:29.750648-05:00","created_by":"rusty","updated_at":"2026-01-04T11:50:57.566545-05:00","closed_at":"2026-01-04T11:50:57.566545-05:00","close_reason":"Closed"}
 {"id":"vgi-python-p91","title":"Move exception classes from function.py to own file","description":"Move InitIdentifierError and SchemaValidationError from vgi/function.py to a new vgi/exceptions.py file. Update imports in function.py and any other files that reference these exceptions.","status":"closed","priority":2,"issue_type":"task","created_at":"2026-01-04T09:12:28.058227-05:00","created_by":"rusty","updated_at":"2026-01-04T09:17:52.477661-05:00","closed_at":"2026-01-04T09:17:52.477661-05:00","close_reason":"Closed"}
 {"id":"vgi-python-qud","title":"Test FunctionStorageSqlite: global_delete, global_exists, queue_clear","notes":"Coverage: 83% in vgi/function_storage.py. Missing tests for:\n- Line 266: KeyError path in global_get (key not found)\n- Lines 273-278: global_delete method\n- Lines 282-290: global_exists method  \n- Line 337: queue_push with empty list\n- Lines 376-385: queue_clear method\n\nThese storage operations need direct unit tests to ensure correctness.","status":"closed","priority":3,"issue_type":"task","created_at":"2026-01-04T22:15:25.982124-05:00","created_by":"rusty","updated_at":"2026-01-04T22:30:05.625934-05:00","closed_at":"2026-01-04T22:30:05.625934-05:00","close_reason":"Added comprehensive tests for FunctionStorageSqlite. Coverage improved from 83% to 98%."}
 {"id":"vgi-python-r3t","title":"Consolidate test client infrastructure in testing.py","description":"testing.py has three test client classes (FunctionTestClient, TableFunctionTestClient, ScalarFunctionTestClient) with shared infrastructure patterns. Extend _BaseTestClient pattern to reduce code duplication. Consider using a single unified client with method dispatch based on function type.","status":"closed","priority":3,"issue_type":"task","created_at":"2026-01-04T20:06:53.913912-05:00","created_by":"rusty","updated_at":"2026-01-04T22:02:51.368907-05:00","closed_at":"2026-01-04T22:02:51.368907-05:00","close_reason":"Not warranted - _BaseTestClient already provides shared infrastructure (context manager, log capture, logging). The three clients handle genuinely different protocols (TableInOut with finalize, TableFunction with no input, Scalar with different protocol). Unifying would add type detection complexity without real benefit."}
-{"id":"vgi-python-set","title":"Improve type annotations in testing.py test helpers","notes":"92.61% type coverage (70 Anys) in vgi/testing.py\n\nMain opportunities:\n- Lines 136-137, 641-642, 685-686, etc: `args: tuple[Any, ...]` and `kwargs: dict[str, Any]`\n  Could use ParamSpec or more specific signatures\n- Lines 151-152: `positional: tuple[pa.Scalar[Any], ...]` - unavoidable (PyArrow)\n- Lines 761, 843: Log expectation dicts - could use TypedDict\n\nLower priority since these are test helpers and flexibility is intentional.","status":"open","priority":4,"issue_type":"task","created_at":"2026-01-04T22:19:50.204524-05:00","created_by":"rusty","updated_at":"2026-01-04T22:20:05.418044-05:00"}
+{"id":"vgi-python-set","title":"Improve type annotations in testing.py test helpers","notes":"92.61% type coverage (70 Anys) in vgi/testing.py\n\nMain opportunities:\n- Lines 136-137, 641-642, 685-686, etc: `args: tuple[Any, ...]` and `kwargs: dict[str, Any]`\n  Could use ParamSpec or more specific signatures\n- Lines 151-152: `positional: tuple[pa.Scalar[Any], ...]` - unavoidable (PyArrow)\n- Lines 761, 843: Log expectation dicts - could use TypedDict\n\nLower priority since these are test helpers and flexibility is intentional.","status":"closed","priority":4,"issue_type":"task","created_at":"2026-01-04T22:19:50.204524-05:00","created_by":"rusty","updated_at":"2026-01-05T12:09:36.813123-05:00","closed_at":"2026-01-05T12:09:36.813123-05:00","close_reason":"Closed"}
 {"id":"vgi-python-uq8","title":"Add validation for missing arg_types in extract_argument_specs","description":"In extract_argument_specs(), if an argument name is missing from arg_types dict, it silently defaults to pa.null(). This could mask bugs where the caller forgot to provide a type. Consider raising an error or logging a warning when a type mapping is missing.","status":"closed","priority":2,"issue_type":"task","created_at":"2026-01-05T11:51:19.501577-05:00","created_by":"rusty","updated_at":"2026-01-05T11:53:52.594504-05:00","closed_at":"2026-01-05T11:53:52.594504-05:00","close_reason":"Closed"}
 {"id":"vgi-python-vir","title":"Add tests for varargs in Arg descriptor","description":"In tests/test_arguments.py:\n- Basic varargs: receives multiple values as tuple\n- Single value varargs: works with exactly 1 value  \n- Empty varargs: raises error when 0 values\n- Type validation: each element validated\n- Varargs with constraints (ge, le, etc.): validates each element\n- Varargs with choices: validates each element against choices","status":"closed","priority":2,"issue_type":"task","created_at":"2026-01-05T10:49:20.285269-05:00","created_by":"rusty","updated_at":"2026-01-05T11:00:06.124186-05:00","closed_at":"2026-01-05T11:00:06.124186-05:00","close_reason":"Added comprehensive tests for varargs in Arg descriptor and Arguments.get_varargs()","dependencies":[{"issue_id":"vgi-python-vir","depends_on_id":"vgi-python-jrf","type":"blocks","created_at":"2026-01-05T10:49:26.454614-05:00","created_by":"rusty"}]}
 {"id":"vgi-python-vzg","title":"Update documentation for DuckDB settings feature","description":"Update documentation to cover DuckDB settings feature.\n\nFiles to update:\n1. docs/protocol.md - Add settings to protocol flow diagrams and Invocation fields\n2. docs/metadata.md - Document required_settings in Meta class\n3. CLAUDE.md - Add example showing settings usage pattern\n4. docs/lifecycle.md - Mention settings availability during bind\n\nInclude:\n- When settings are available (bind phase and later)\n- How to declare required settings\n- How to access settings in function code\n- Example patterns for settings-dependent output schemas","status":"closed","priority":3,"issue_type":"task","created_at":"2026-01-04T13:05:48.795757-05:00","created_by":"rusty","updated_at":"2026-01-04T13:28:21.005166-05:00","closed_at":"2026-01-04T13:28:21.005166-05:00","close_reason":"Documentation updated in protocol.md and CLAUDE.md","dependencies":[{"issue_id":"vgi-python-vzg","depends_on_id":"vgi-python-67w","type":"blocks","created_at":"2026-01-04T13:06:14.089469-05:00","created_by":"rusty"}]}

vgi/testing.py

@@ -76,7 +76,7 @@
 
 import uuid
 from collections.abc import Callable, Generator, Iterator
-from typing import Any, Self, cast
+from typing import Any, Self, TypedDict, cast
 
 import pyarrow as pa
 import structlog
@@ -112,6 +112,7 @@
     "TableInOutFunctionTestClientError",
     "TableFunctionTestClient",
     "ScalarFunctionTestClient",
+    "LogExpectation",
     "batch",
     "assert_function_output",
     "assert_function_logs",
@@ -123,6 +124,29 @@
 ]
 
 
+class LogExpectation(TypedDict, total=False):
+    """Type definition for log message expectations in assert_function_logs.
+
+    All fields are optional. Use any combination to match log messages:
+    - level: Match exact log level (Level enum)
+    - message: Match exact message string
+    - message_contains: Match if message contains this substring
+    - message_startswith: Match if message starts with this prefix
+
+    Example:
+        expected_logs: list[LogExpectation] = [
+            {"level": Level.INFO, "message_contains": "Processing"},
+            {"message_startswith": "Completed"},
+        ]
+
+    """
+
+    level: Level
+    message: str
+    message_contains: str
+    message_startswith: str
+
+
 class TableInOutFunctionTestClientError(Exception):
     """Error raised by TableInOutFunctionTestClient operations."""
 
@@ -758,7 +782,7 @@ def assert_function_output(
 def assert_function_logs(
     function: type[TableInOutGenerator] | type[TableInOutFunction],
     input: list[pa.RecordBatch],
-    expected_logs: list[dict[str, Any]],
+    expected_logs: list[LogExpectation],
     args: tuple[Any, ...] | None = None,
     kwargs: dict[str, Any] | None = None,
     msg: str | None = None,
@@ -840,7 +864,7 @@ def assert_function_logs(
     return outputs
 
 
-def _log_matches(log: Message, expectation: dict[str, Any]) -> bool:
+def _log_matches(log: Message, expectation: LogExpectation) -> bool:
     """Check if a log message matches an expectation dict."""
     return not (
         ("level" in expectation and log.level != expectation["level"])