Skip to content

Add #service Prefix Short-Circuit to stream_orchestration_response #361

@nuwangeek

Description

@nuwangeek

Priority: P3 — Connects the direct executor to the streaming endpoint
File: src/llm_orchestration_service.py
Method: stream_orchestration_response()
Type: Control flow addition
Same logic as Task-07 but for the streaming generator. The check must be inserted before the query validation yield block, after language detection:

# STEP 0.1: Multi-step service prefix check (bypass NLU pipeline)
if request.message.startswith(SERVICE_STEP_PREFIXES):
    logger.info(f"[{request.chatId}] #service prefix detected - direct step stream")
    step_stream = await self.service_workflow_executor.execute_direct_step_streaming(
        request=request,
        time_metric=time_metric,
    )
    if step_stream is not None:
        async for chunk in step_stream:
            yield chunk
        stream_ctx.mark_completed()
        return
    # If None, fall through to normal pipeline

Work:
Add the prefix check block immediately after language detection inside the streaming generator.
Verify stream timeout wrapper in the API layer still applies (it wraps the entire generator so it applies automatically — no change needed in the API file).
Add integration test: send #service payload to the streaming path and verify SSE stream delivers content + buttons + END.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

Status

Todo

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions