Skip to content

refactor(translation): normalize Anthropic requests through IR - #183

Open
nachiketb-nvidia wants to merge 4 commits into
mainfrom
nachiketb/anthropic-request-normalization
Open

refactor(translation): normalize Anthropic requests through IR#183
nachiketb-nvidia wants to merge 4 commits into
mainfrom
nachiketb/anthropic-request-normalization

Conversation

@nachiketb-nvidia

@nachiketb-nvidia nachiketb-nvidia commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

What

  • Normalize outbound Anthropic requests through the typed LlmRequest IR.
  • Remove backend-owned JSON rewriting for system roles, reasoning, tool IDs, and thinking blocks.
  • Scope provider extensions to their decoded source format so Anthropic-native fields do not leak across formats.
  • Preserve exact native requests only when nested cache_control annotations require it.

Why

Request semantics should be normalized once by source decoding and target encoding. The previous implementation moved a large serde_json::Value post-processor into the translation crate, but still duplicated behavior already represented by the IR and made provider ownership unclear.

How

  • Record the decoded source_format on LlmRequest.
  • Decode Anthropic message-level system/developer turns into typed instructions.
  • Decode OpenAI-style reasoning_effort and encode it as Anthropic thinking plus output_config.
  • Re-emit top-level extensions only when their source format is Anthropic; retain native context_management and drop cross-provider extensions.
  • Sanitize and collision-disambiguate tool IDs while encoding typed tool calls/results.
  • Omit unsigned thinking and messages left empty after typed encoding.
  • Keep the translation crate exact-round-trip contract unchanged; the native backend requests canonical encoding except for nested Anthropic cache markers that the current IR cannot represent.

What to review

  • LlmRequest::source_format and provider-extension ownership.
  • Anthropic decode/encode behavior for instructions, reasoning, tools, and thinking.
  • The narrow exact-replay exception for native cache_control blocks.
  • Removal of the duplicate backend normalizer.

Validation

  • cargo fmt --all --check
  • cargo clippy --workspace --all-targets -- -D warnings
  • cargo test --workspace
  • cargo test -p switchyard-translation
  • cargo test -p switchyard-components --test adversarial_native_backends

Signed-off-by: nachiketb <nachiketb@nvidia.com>
Signed-off-by: nachiketb <nachiketb@nvidia.com>
@nachiketb-nvidia
nachiketb-nvidia requested a review from a team as a code owner July 29, 2026 18:00
@coderabbitai

coderabbitai Bot commented Jul 29, 2026

Copy link
Copy Markdown

Review Change Stack

Walkthrough

Anthropic outbound requests now pass through shared translation and buffered normalization. The normalization handles endpoint-compatible fields, system content, tool-use IDs, and unsigned thinking blocks. Translation and native-backend tests verify the resulting request shape and replay behavior.

Changes

Anthropic outbound request flow

Layer / File(s) Summary
Buffered outbound normalization
crates/switchyard-translation/src/codecs/anthropic/buffered.rs
Preserved and newly encoded Anthropic requests are normalized by removing incompatible fields, promoting system/developer content, sanitizing tool-use IDs, and filtering unsigned thinking blocks.
Native backend translation routing
crates/switchyard-components/src/backends/anthropic.rs
The backend always uses shared request translation, then applies the model override and extra body fields; local normalization helpers were removed.
Normalization behavior validation
crates/switchyard-translation/tests/request_translation.rs, crates/switchyard-components/tests/adversarial_native_backends.rs
Tests verify field normalization, extension preservation, tool ID sanitization, thinking-block filtering, and replay message counts.

Estimated code review effort: 4 (Complex) | ~45 minutes

Poem

I’m a rabbit with packets to pack,
Anthropic requests hop on the track.
Thinking blocks shed, tool IDs grow neat,
Shared translators make flows complete.
Tests thump their paws: “The shape is right!”

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title is concise and accurately reflects the main change: moving Anthropic request normalization into the translation layer.

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@crates/switchyard-translation/src/codecs/anthropic/buffered.rs`:
- Around line 488-497: Update normalize_outbound_request and its callers to
accept mutable diagnostics and policy, using push_lossy to report every dropped
message or non-text block. In
crates/switchyard-translation/src/codecs/anthropic/buffered.rs lines 488-497,
report removed messages and avoid emitting an empty messages array; in lines
375-386, report non-text blocks discarded when lifting system/developer turns.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 495958e0-d442-4cd2-a1d4-0fb611cdef29

📥 Commits

Reviewing files that changed from the base of the PR and between 36b9636 and db602bc.

📒 Files selected for processing (4)
  • crates/switchyard-components/src/backends/anthropic.rs
  • crates/switchyard-components/tests/adversarial_native_backends.rs
  • crates/switchyard-translation/src/codecs/anthropic/buffered.rs
  • crates/switchyard-translation/tests/request_translation.rs

Comment on lines +488 to +497
let kept = blocks
.into_iter()
.filter(|block| !is_unsigned_thinking_block(block))
.collect::<Vec<_>>();
if kept.is_empty() {
return None;
}
message.insert("content".to_string(), Value::Array(kept));
Some(Value::Object(message))
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick win

Lossy normalization has no diagnostics path. normalize_outbound_request takes and returns a bare Value, so every drop it performs is invisible to callers even though the rest of this codec reports losses via push_lossy. Threading &mut Vec<Diagnostic> + policy through would cover both sites and give a place to guard the all-messages-removed case.

  • crates/switchyard-translation/src/codecs/anthropic/buffered.rs#L488-L497: report dropped messages and avoid emitting an empty messages array.
  • crates/switchyard-translation/src/codecs/anthropic/buffered.rs#L375-L386: report non-text blocks discarded when a system/developer turn is lifted.
📍 Affects 1 file
  • crates/switchyard-translation/src/codecs/anthropic/buffered.rs#L488-L497 (this comment)
  • crates/switchyard-translation/src/codecs/anthropic/buffered.rs#L375-L386
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@crates/switchyard-translation/src/codecs/anthropic/buffered.rs` around lines
488 - 497, Update normalize_outbound_request and its callers to accept mutable
diagnostics and policy, using push_lossy to report every dropped message or
non-text block. In
crates/switchyard-translation/src/codecs/anthropic/buffered.rs lines 488-497,
report removed messages and avoid emitting an empty messages array; in lines
375-386, report non-text blocks discarded when lifting system/developer turns.

Comment thread crates/switchyard-translation/src/codecs/anthropic/buffered.rs Outdated
Signed-off-by: nachiketb <nachiketb@nvidia.com>
@nachiketb-nvidia nachiketb-nvidia changed the title refactor(translation): own Anthropic request normalization refactor(translation): normalize Anthropic requests through IR Jul 29, 2026
Signed-off-by: nachiketb <nachiketb@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants