-
Notifications
You must be signed in to change notification settings - Fork 4
Open
Description
Priority: P3 — Connects the direct executor to the streaming endpoint
File: src/llm_orchestration_service.py
Method: stream_orchestration_response()
Type: Control flow addition
Same logic as Task-07 but for the streaming generator. The check must be inserted before the query validation yield block, after language detection:
# STEP 0.1: Multi-step service prefix check (bypass NLU pipeline)
if request.message.startswith(SERVICE_STEP_PREFIXES):
logger.info(f"[{request.chatId}] #service prefix detected - direct step stream")
step_stream = await self.service_workflow_executor.execute_direct_step_streaming(
request=request,
time_metric=time_metric,
)
if step_stream is not None:
async for chunk in step_stream:
yield chunk
stream_ctx.mark_completed()
return
# If None, fall through to normal pipeline
Work:
Add the prefix check block immediately after language detection inside the streaming generator.
Verify stream timeout wrapper in the API layer still applies (it wraps the entire generator so it applies automatically — no change needed in the API file).
Add integration test: send #service payload to the streaming path and verify SSE stream delivers content + buttons + END.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels
Type
Projects
Status
Todo