feat(py): MiddlewareV2 implementation + implement listValues API in reflection server

[huangjeff5]

Description

Implements middleware v2 for Python Genkit. Middleware can now intercept both the generate call (high-level, before request construction) and the raw model call, enabling composable pre/post-processing hooks.

What's new

Middleware architecture

• New genkit._core._middleware package with a BaseMiddleware abstract base class exposing two hooks: wrap_generate (full generate pipeline) and wrap_model (raw model dispatch)
• Built-in middleware: SimulateSystemPrompt, AugmentWithContext, ValidateSupport, DownloadRequestMedia, Retry, Fallback
• Public genkit.middleware module exports all built-in middleware

Generate loop

• generate_action now accepts a middleware list and dispatches through dispatch_generate / dispatch_model chains
• _generate.py restructured around inner run_one_iteration function wrapped by middleware hooks
• Old _middleware.py removed; logic moved into the new package

Registry / reflection fixes

• Registry.list_values(kind) now returns a dict[str, object] (name → value map)
• Registry.initialize_all_plugins() added to ensure plugins register their values before listing
• Genkit registers the default model via register_value('defaultModel', ...) on startup
• /api/values endpoint: 400 for missing or unsupported type params ('type' X is not supported. Only 'defaultModel' is supported)
• /api/actions and /api/values wrapped in try/except with safe JSON serialization (default=str)

Testing

• generate_test.py and middleware_test.py updated for new dispatch signatures
• reflection_test.py extended with listValues and initialize_all_plugins coverage
• Sample py/samples/middleware/src/main.py updated to demonstrate v2 API

[gemini-code-assist]

Code Review

This pull request introduces a comprehensive middleware system for Genkit, including new implementations for retries, fallbacks, media downloading, and system prompt simulation. The core generation logic has been refactored to support these hooks across model calls and tool executions. Feedback highlights several areas for improvement: the media downloader should raise errors for oversized content rather than truncating it and should reuse HTTP clients for efficiency; the retry logic requires better jitter management and removal of unreachable code; and the system prompt simulation should handle multiple system messages. Additionally, internal recursive calls in the generation action should be updated to prevent redundant telemetry spans.

[huangjeff5]

/gemini review

[gemini-code-assist]

Code Review

This pull request refactors the Genkit middleware system from function-based to a more structured class-based approach, introducing a BaseMiddleware class and several new middleware implementations for functionalities like context augmentation, media download, retry, fallback, system prompt simulation, and request validation. The core generation logic (_ai/_generate.py) and prompt definitions (_ai/_aio.py, _ai/_prompt.py) have been updated to integrate with this new middleware architecture. Additionally, the reflection endpoints (_core/_reflection.py) and registry (_core/_registry.py) have been enhanced to support the new middleware and improve robustness, including changes to how /api/values handles different type parameters. Test files and sample code have been refactored to align with these changes. Feedback includes addressing a potential side effect in _generate_action where the middleware list is mutated before being copied, optimizing _DownloadRequestMediaMiddleware to avoid resource leaks by using a shared httpx.AsyncClient instance, and aligning the /api/values endpoint's type parameter handling with the PR description to return an empty dictionary for unknown types instead of a 400 error.

[huangjeff5]

/gemini review

[huangjeff5]

2026 not 2025

[gemini-code-assist]

Code Review

This pull request introduces a class-based middleware architecture for Genkit, replacing the previous functional approach with BaseMiddleware hooks for generation, model calls, and tool execution. It includes several new built-in middleware components and enhances the reflection server's plugin initialization and value reporting. Review feedback suggests refactoring the reflection server to avoid hardcoded strings for supported types and recommends restoring informative error handling in the middleware sample to prevent suppressing diagnostic information.

[gemini-code-assist]

To improve maintainability, consider defining the supported value types in a collection rather than hardcoding 'defaultModel'. This will make it easier to add more supported types in the future and makes the error message dynamic.

        supported_types = ('defaultModel',)
        if type_param not in supported_types:
            return JSONResponse(
                {'error': f"'type' {type_param} is not supported. Only {', '.join(supported_types)} are supported"},
                status_code=400,
                headers={'x-genkit-version': version},
            )

[gemini-code-assist]

The broad except Exception: pass can hide important errors from users running this sample. The previous implementation, which caught the exception and printed a helpful message about the GEMINI_API_KEY, was more user-friendly. Please consider restoring a similar error handling mechanism to improve the developer experience.

Suggested change

	except Exception: # noqa: S110
	pass
	except Exception as e:
	logger.error("Sample failed to run. If you see an auth error, check your GEMINI_API_KEY.", exc_info=e)