Skip to content

feat(ai-agents): support custom structured output schemas for AI Agent responses#3186

Open
asastre wants to merge 5 commits intomasterfrom
ai-agents/display-flow-builder-content
Open

feat(ai-agents): support custom structured output schemas for AI Agent responses#3186
asastre wants to merge 5 commits intomasterfrom
ai-agents/display-flow-builder-content

Conversation

@asastre
Copy link
Contributor

@asastre asastre commented Mar 17, 2026

Description

Allow the AI Agent to return custom structured output message types beyond the built-in ones (text, carousel, etc.). This enables the Flow Builder plugin to instruct the AI Agent to respond with a specific Flow Builder content ID, which is then resolved into its full content with all follow-ups.

Context

Currently, the AI Agent can only return a fixed set of message types. There are scenarios where the AI Agent needs to respond by referencing existing Flow Builder content (e.g. a specific node in the flow) instead of generating a free-form response. This PR makes the output schema extensible so that each consumer can inject its own message types.

Approach taken / Explain the design

1. Generic core interfaces (botonic-core)

  • Made BaseMessage, OutputMessage, AgenticOutputMessage, RunResult, and InferenceResponse generic with an Extra type parameter, so they can carry additional message types while remaining backward-compatible (defaults to never).
  • Added outputMessagesSchemas to AiAgentArgs to pass extra Zod schemas to the agent.

2. Dynamic output schema (botonic-plugin-ai-agents)

  • Extracted base message schemas into a constant array and introduced getOutputSchema(externalSchemas) that merges them with any externally provided Zod schemas at runtime.
  • AIAgentBuilder now receives and forwards outputMessagesSchemas.

3. Flow Builder content message (botonic-plugin-flow-builder)

  • Created FlowBuilderContentMessage type and its corresponding FlowBuilderContentSchema Zod object, representing a message with a contentId pointing to a Flow Builder node.
  • Updated getContentsByAiAgent to:
    • Pass FlowBuilderContentSchema to the AI Agent plugin.
    • Separate the AI Agent response into regular messages and flowBuilderContent messages.
    • Resolve each flowBuilderContent message by fetching the referenced node from the CMS and returning its full contents.

To document / Usage example

To add a custom output message type, define a Zod schema and pass it in outputMessagesSchemas:

const MyCustomSchema = z.object({
  type: z.enum(['myCustomType']),
  someField: z.string(),
})

// When calling the AI Agent plugin
aiAgentArgs.outputMessagesSchemas = [MyCustomSchema]

The AI Agent will then be able to return messages of that type alongside the built-in ones.

As for the new FlowBuilderContentSchema, in the AI Agent prompt you could do something like this:
"If the user is asking for the status of an order, display the Flow Builder content 'OPEN_WISMO_WEBVIEW'"

This will result in the AI Agent returning as a response the contents of the Flow Builder content ID OPEN_WISMO_WEBVIEW directly.

Testing

The pull request has unit tests.

@chatgpt-codex-connector
Copy link

Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.
Credits must be used to enable repository wide code reviews.

@asastre asastre force-pushed the ai-agents/display-flow-builder-content branch from f6ee699 to 88a52ee Compare March 18, 2026 12:35
@github-actions
Copy link

Test Results

42 tests   42 ✅  1m 21s ⏱️
 5 suites   0 💤
 1 files     0 ❌

Results for commit 88a52ee.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant