feat(ai-agents): support custom structured output schemas for AI Agent responses#3186
Open
feat(ai-agents): support custom structured output schemas for AI Agent responses#3186
Conversation
|
Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits. |
…mas for the AI Agent
…ut to be able to specify the AI Agent to return a specific Flow Builder content + handle the AI Agent response to fetch this content from Flow Builder with all its followups
f6ee699 to
88a52ee
Compare
Test Results42 tests 42 ✅ 1m 21s ⏱️ Results for commit 88a52ee. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Allow the AI Agent to return custom structured output message types beyond the built-in ones (text, carousel, etc.). This enables the Flow Builder plugin to instruct the AI Agent to respond with a specific Flow Builder content ID, which is then resolved into its full content with all follow-ups.
Context
Currently, the AI Agent can only return a fixed set of message types. There are scenarios where the AI Agent needs to respond by referencing existing Flow Builder content (e.g. a specific node in the flow) instead of generating a free-form response. This PR makes the output schema extensible so that each consumer can inject its own message types.
Approach taken / Explain the design
1. Generic core interfaces (
botonic-core)BaseMessage,OutputMessage,AgenticOutputMessage,RunResult, andInferenceResponsegeneric with anExtratype parameter, so they can carry additional message types while remaining backward-compatible (defaults tonever).outputMessagesSchemastoAiAgentArgsto pass extra Zod schemas to the agent.2. Dynamic output schema (
botonic-plugin-ai-agents)getOutputSchema(externalSchemas)that merges them with any externally provided Zod schemas at runtime.AIAgentBuildernow receives and forwardsoutputMessagesSchemas.3. Flow Builder content message (
botonic-plugin-flow-builder)FlowBuilderContentMessagetype and its correspondingFlowBuilderContentSchemaZod object, representing a message with acontentIdpointing to a Flow Builder node.getContentsByAiAgentto:FlowBuilderContentSchemato the AI Agent plugin.flowBuilderContentmessages.flowBuilderContentmessage by fetching the referenced node from the CMS and returning its full contents.To document / Usage example
To add a custom output message type, define a Zod schema and pass it in
outputMessagesSchemas:The AI Agent will then be able to return messages of that type alongside the built-in ones.
As for the new
FlowBuilderContentSchema, in the AI Agent prompt you could do something like this:"If the user is asking for the status of an order, display the Flow Builder content 'OPEN_WISMO_WEBVIEW'"
This will result in the AI Agent returning as a response the contents of the Flow Builder content ID OPEN_WISMO_WEBVIEW directly.
Testing
The pull request has unit tests.