Skip to content

fix: shutdown trace manager, not the executor, between jobs#1800

Open
andreiancuta-uipath wants to merge 3 commits into
mainfrom
fix/trace-shutdown
Open

fix: shutdown trace manager, not the executor, between jobs#1800
andreiancuta-uipath wants to merge 3 commits into
mainfrom
fix/trace-shutdown

Conversation

@andreiancuta-uipath

@andreiancuta-uipath andreiancuta-uipath commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Regression introduced here.

LiveTrackingSpanProcessor.shutdown() only shuts down its internal ThreadPoolExecutor. It does not remove the processor from OpenTelemetry’s process-wide processor list. After that, the processor is dead but still subscribed to span start/end events, so the next job’s span is still delivered to job 1’s processor. Since that is shutdown, we get an error:
RuntimeError: cannot schedule new futures after shutdown

sequenceDiagram
    participant Server as cli_server.handle_start
    participant Run as cli_run.run
    participant TM as UiPathTraceManager
    participant DSP as _DelegatingSpanProcessor
    participant Live1 as job 1 LiveTrackingSpanProcessor
    participant Exec1 as job 1 ThreadPoolExecutor
    participant Live2 as job 2 LiveTrackingSpanProcessor
    participant Exec2 as job 2 ThreadPoolExecutor

    Server->>Run: job 1 via asyncio.to_thread(cmd.main, ...)
    Run->>TM: create trace manager
    Run->>Live1: create live tracking processor
    Run->>TM: add_span_processor(live1)
    TM->>DSP: add(live1)
    Run->>DSP: traced job 1 span starts
    DSP->>Live1: on_start(span)
    Live1->>Exec1: submit(upsert_span)
    Run->>Live1: live1.shutdown()
    Live1->>Exec1: shutdown(wait=True)
    Note over DSP,Live1: live1 is shut down but still registered

    Server->>Run: job 2 in the same process
    Run->>TM: create new trace manager
    Run->>Live2: create live tracking processor
    Run->>TM: add_span_processor(live2)
    TM->>DSP: add(live2)
    Run->>DSP: traced job 2 first span starts
    DSP->>Live1: on_start(span)
    Live1->>Exec1: submit(upsert_span)
    Exec1-->>Live1: RuntimeError: cannot schedule new futures after shutdown
    Live1-->>DSP: exception propagates
    DSP-->>Run: span start fails before job 2 code runs
    Run-->>Server: result file is written as faulted
    Note over Live2,Exec2: live2 would be able to send job 2 spans, but live1 crashes first
Loading

Fix:

  • Job-local span processors no longer leak into later jobs in the same uipath server process.
  • Other trace processors registered by that job are also cleaned up, not just live tracking.
  • Spans emitted during runtime/factory disposal still have a live processor available.
  • New: also shutdown the LlmHttpOpsExporter from the LiveTrackingProcessor
  • Reproduced issue locally, with uipath server and 2 agent jobs, and validated the fix the same way.
  • Added an integration test; verified it catches the original regression.

Copilot AI review requested due to automatic review settings July 7, 2026 14:00
@github-actions github-actions Bot added test:uipath-langchain Triggers tests in the uipath-langchain-python repository test:uipath-runtime test:uipath-integrations labels Jul 7, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adjusts the UiPath CLI’s tracing lifecycle so per-job tracing resources are shut down via UiPathTraceManager.shutdown() (instead of manually shutting down only the live-tracking processor), and bumps the uipath package version.

Changes:

  • Update uipath run and uipath debug to call trace_manager.shutdown() in a finally block after runtime disposal.
  • Bump uipath version to 2.13.4 (and update lockfile accordingly).
  • Add a new planning/review markdown document (currently appears unrelated to this PR’s title/scope).

Reviewed changes

Copilot reviewed 3 out of 4 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
plan/review-import-cycle-duplication-plan.md Adds a review-plan document (appears unrelated to trace-manager shutdown change).
packages/uipath/src/uipath/_cli/cli_run.py Switches teardown to trace_manager.shutdown() and inlines live span processor construction.
packages/uipath/src/uipath/_cli/cli_debug.py Same teardown change as cli_run.py for debug execution.
packages/uipath/pyproject.toml Bumps package version to 2.13.4.
packages/uipath/uv.lock Updates locked uipath version to 2.13.4.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread packages/uipath/src/uipath/_cli/cli_run.py
Comment thread packages/uipath/src/uipath/_cli/cli_debug.py
@andreiancuta-uipath andreiancuta-uipath force-pushed the fix/trace-shutdown branch 3 times, most recently from 4c08721 to c7f0af6 Compare July 7, 2026 15:13
@sonarqubecloud

sonarqubecloud Bot commented Jul 8, 2026

Copy link
Copy Markdown

Quality Gate Failed Quality Gate failed

Failed conditions
83.3% Coverage on New Code (required ≥ 90%)

See analysis details on SonarQube Cloud

@github-actions

github-actions Bot commented Jul 8, 2026

Copy link
Copy Markdown

🚨 Heads up: uipath-langchain cross-tests are FAILING 🚨

Your changes may break the uipath-langchain-python integration.

⚠️ These checks are NOT enforced by branch protection rules. Please review the failures before merging.

🔍 Inspect the failed run →

@andreiancuta-uipath

Copy link
Copy Markdown
Contributor Author

Sonar coverage complains about an exception branch that logs. Not an essential path

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

test:uipath-integrations test:uipath-langchain Triggers tests in the uipath-langchain-python repository test:uipath-runtime

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants