Bug: onToolStart/onToolEnd callbacks never fire for built-in web_search
Observed
When using fillForm() with enableWebSearch: true and FillCallbacks with
onToolStart/onToolEnd, only custom additionalTools produce per-call callbacks.
Built-in provider web search tools (e.g. web_search) appear only in the aggregated
turnStats.toolCalls summary (e.g. web_search(17)) — no individual
onToolStart/onToolEnd callbacks fire.
Expected
onToolStart and onToolEnd should fire for every tool call, including built-in
provider web search tools, so consumers get per-call visibility (name, input, output,
duration).
Root Cause
In liveAgent.ts, wrapToolsWithCallbacks() only wraps tools that have an execute
function:
const execute = (tool as any).execute;
if (typeof execute === 'function') {
wrapped[name] = wrapTool(name, tool, execute, callbacks);
} else {
// Pass through declarative tools unchanged
wrapped[name] = tool;
}
Provider web search tools (loaded via loadWebSearchTools()) are declarative AI SDK
tools — they have a schema but no execute function. The AI SDK handles their
execution internally during generateText(), bypassing the callback wrapper entirely.
The aggregated turnStats.toolCalls count works because it comes from the AI SDK
response metadata, not from the callback wrapper.
Suggested Fix
Option A (recommended): Hook into the AI SDK's onStepFinish or tool result
metadata to emit synthetic onToolStart/onToolEnd callbacks for declarative tools
after each generateText() call.
Option B: Convert provider web search tools from declarative to executable wrappers
that call the provider API directly, so wrapToolsWithCallbacks can intercept them.
Option A is less invasive and preserves the AI SDK's native handling.
Reproduction
import { fillForm } from 'markform';
const result = await fillForm({
form: myFormMarkdown,
model: 'openai/gpt-5-mini',
enableWebSearch: true,
additionalTools: { my_tool: myTool },
callbacks: {
onToolStart: ({ name, input }) => console.log(`tool start: ${name}`),
onToolEnd: ({ name }) => console.log(`tool end: ${name}`),
},
});
// Logs: "tool start: my_tool", "tool end: my_tool"
// Does NOT log: "tool start: web_search" (despite web_search appearing in turnStats)
Affected Version
markform 0.1.17
Bug: onToolStart/onToolEnd callbacks never fire for built-in web_search
Observed
When using
fillForm()withenableWebSearch: trueandFillCallbackswithonToolStart/onToolEnd, only customadditionalToolsproduce per-call callbacks.Built-in provider web search tools (e.g.
web_search) appear only in the aggregatedturnStats.toolCallssummary (e.g.web_search(17)) — no individualonToolStart/onToolEndcallbacks fire.Expected
onToolStartandonToolEndshould fire for every tool call, including built-inprovider web search tools, so consumers get per-call visibility (name, input, output,
duration).
Root Cause
In
liveAgent.ts,wrapToolsWithCallbacks()only wraps tools that have anexecutefunction:
Provider web search tools (loaded via
loadWebSearchTools()) are declarative AI SDKtools — they have a schema but no
executefunction. The AI SDK handles theirexecution internally during
generateText(), bypassing the callback wrapper entirely.The aggregated
turnStats.toolCallscount works because it comes from the AI SDKresponse metadata, not from the callback wrapper.
Suggested Fix
Option A (recommended): Hook into the AI SDK's
onStepFinishor tool resultmetadata to emit synthetic
onToolStart/onToolEndcallbacks for declarative toolsafter each
generateText()call.Option B: Convert provider web search tools from declarative to executable wrappers
that call the provider API directly, so
wrapToolsWithCallbackscan intercept them.Option A is less invasive and preserves the AI SDK's native handling.
Reproduction
Affected Version
markform 0.1.17