fix(eval): stop stripping -community-agents model suffix before calling LLM Gateway

[Chibionos]

Summary

LLM-judge evaluators (both current-gen and legacy) strip the -community-agents
suffix off the configured model name before calling UiPathLlmChatService. That
suffix is not cosmetic — Community/EU tenants' LLM Gateway routing rules are keyed
on the suffixed model id, the same id AgentHub sends unmodified when it runs
the agent itself. Stripping it before the eval call sends a bare model id the
Gateway has no EU routing rule for:

Status Code: 417
Message: "No llm routing rule found for product agentsplaygroundfallback in EU
using model gpt-5.4-2026-03-05 - check data residency configuration."

...even though the agent using the identical gpt-5.4-2026-03-05-community-agents
id runs fine, and the evaluator's own config has the identical suffixed id.

Confirmed via production trace: the "Autonomous Agent" span completes successfully
while the llm-judge-trajectory-eval-001 evaluator span 417s on the very same
model string minus the suffix.

The strip was introduced from-scratch in #483 (Jul 2025) with no recorded
rationale (no PR description/review comment references it), and was copy-pasted
into 3 more evaluator files as the eval framework grew. It never accounted for
Community/EU Gateway routing.

Changes

• llm_as_judge_evaluator.py (LLMJudgeMixin._get_llm_response) — covers both
  current-gen uipath-llm-judge-output-semantic-similarity and
  uipath-llm-judge-trajectory-similarity evaluators (this is the exact path hit
  in the production incident).
• legacy_llm_as_judge_evaluator.py, legacy_trajectory_evaluator.py — same strip,
  legacy evaluator classes.
• legacy_evaluator_utils.py — removed the now-dead clean_model_name() helper
  and its two call sites in legacy_context_precision_evaluator.py /
  legacy_faithfulness_evaluator.py.

The COMMUNITY_agents_SUFFIX constant itself is left in place in
uipath.platform.constants (still covered by the backward-compat shim test) —
only its use inside these six evaluator call sites is removed.

Test plan

• Added tests/evaluators/test_llm_judge_model_suffix.py — one test per fixed
  code path, mocking the LLM service and asserting the model id reaching
  chat_completions(...) matches the configured (suffixed) value exactly.
  Watched all 6 fail against the pre-fix code (bare model name), then pass
  after the fix.
• Full uipath package test suite passes (uv run pytest, no regressions).
• ruff check, ruff format --check, mypy clean on all touched files.

Co-Authored-By: Claude Sonnet 5 noreply@anthropic.com

https://claude.ai/code/session_01P5DrPWsvEMA5UdzVtnsvuM

[Copilot]

Pull request overview

This PR fixes LLM-judge evaluators to pass the configured model ID to the LLM Gateway unchanged, specifically preserving the -community-agents suffix required for Community/EU routing rules.

Changes:

• Removed -community-agents suffix stripping in both current-gen and legacy LLM-judge evaluator call paths.
• Deleted the legacy clean_model_name() helper and updated its prior call sites to use the model value as-is.
• Added regression tests to assert the exact model string is forwarded into chat_completions(...) for each affected evaluator path.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
packages/uipath/src/uipath/eval/evaluators/llm_as_judge_evaluator.py	Stop stripping -community-agents in LLMJudgeMixin._get_llm_response so the Gateway receives the configured model ID.
packages/uipath/src/uipath/eval/evaluators/legacy_llm_as_judge_evaluator.py	Stop stripping -community-agents in the legacy LLM-as-a-judge evaluator request path.
packages/uipath/src/uipath/eval/evaluators/legacy_trajectory_evaluator.py	Stop stripping -community-agents in the legacy trajectory evaluator request path.
packages/uipath/src/uipath/eval/evaluators/legacy_context_precision_evaluator.py	Remove use of clean_model_name() and forward self.model directly.
packages/uipath/src/uipath/eval/evaluators/legacy_faithfulness_evaluator.py	Remove use of clean_model_name() and forward self.model directly.
packages/uipath/src/uipath/eval/evaluators/legacy_evaluator_utils.py	Remove now-dead clean_model_name() helper.
packages/uipath/tests/evaluators/test_llm_judge_model_suffix.py	Add regression tests asserting the configured (suffixed) model ID is what reaches the LLM call for each fixed code path.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
100.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[github-actions]

🚨 Heads up: uipath-langchain cross-tests are FAILING 🚨

Your changes may break the uipath-langchain-python integration.

⚠️ These checks are NOT enforced by branch protection rules. Please review the failures before merging.

🔍 Inspect the failed run →