fix(observability): classify OpenHuman/Embedding/streaming backend 'Invalid token' 401 as SessionExpired (TAURI-RUST-4P0 + 4K5 + 1EE)#2786
Open
CodeGhost21 wants to merge 3 commits into
Conversation
… SessionExpired
The OpenHuman backend rejects an expired/revoked JWT with the envelope
`{"success":false,"error":"Invalid token"}` (vs. the explicit
`"Session expired. Please log in again."` body that the existing
classifier already catches). Same emit site
(`providers::ops::api_error` → `web_channel.run_chat_task`), same
wrapping, same expected user state — just a different body substring
chosen by the backend's JWT-validity branch.
Issue tinyhumansai#2286 deliberately stopped matching bare `"Invalid token"` as
session-expired because that string also surfaces from Discord /
OAuth provider rejections, which are actionable scoped errors that
must reach Sentry. We preserve that contract with a conjunctive
matcher: BOTH the OpenHuman-scoped `"OpenHuman API error (401"`
prefix AND the envelope-shaped `"\"error\":\"Invalid token\""` must
be present.
tinyhumansai#2286 cases still route to Sentry (verified by the existing
`does_not_classify_byo_key_provider_401_as_session_expired` test
staying green):
- `"Invalid token"` → None ✓
- `"got an invalid token here"` → None ✓
- `"OpenAI API error (401 Unauthorized): invalid_api_key"` → None ✓
- `"Anthropic API error (401 Unauthorized): ..."` → None ✓
Targets Sentry OPENHUMAN-TAURI-4P0 (issue 5332): low volume so far
(1 event) but the wire shape is durable — every OpenHuman user with
a stale JWT will hit this on the next agent turn, so quietly
demoting it to a `warn!` log keeps the noise from compounding.
Contributor
📝 WalkthroughWalkthroughAdds conjunctive matching and docs in ChangesOpenHuman 401 Invalid Token Classification
Sequence Diagram(s)(omitted — change is a targeted classifier update and tests; no multi-component sequential flow requires visualization) Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Possibly related PRs
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
graycyrus
reviewed
May 27, 2026
Contributor
graycyrus
left a comment
graycyrus
left a comment
There was a problem hiding this comment.
@CodeGhost21 hey! the code looks good to me — the conjunctive anchor approach is exactly right here. Requiring both the "OpenHuman API error (401" prefix and the envelope-shaped "\"error\":\"Invalid token\"" body together is the correct way to preserve the #2286 contract while catching this specific backend branch, and the two new test cases pin both the verbatim Sentry wire shape and the unwrapped emit shape cleanly.
CI still has a few checks pending (Windows E2E, core image build, Rust core coverage). Once those come back green I'll come back and approve this. Let me know if anything comes up in the meantime.
…sionExpired (TAURI-RUST-4K5)
TAURI-RUST-4K5 (~118 events, escalating on 0.56.0,
domain=embeddings operation=openai_embed status=401) carries the same
OpenHuman backend `{"success":false,"error":"Invalid token"}` envelope
as 4P0, but the embedding client at
`src/openhuman/embeddings/openai.rs:139` wraps it with the
`"Embedding API error"` prefix instead of `"OpenHuman API error"`. The
breadcrumb `[scheduler_gate] signed_out false -> true` immediately
preceding the 401 in the event payload confirms it's the same
session-expired cause, just emitted at the embedding layer.
The conjunctive `"OpenHuman API error (401"` anchor added in the
previous commit catches the chat-API path; this commit adds a parallel
`"Embedding API error (401"` anchor so the embedding path also routes
to SessionExpired. The envelope-shaped `"\"error\":\"Invalid token\""`
gate stays the same, so third-party BYO-key embedding 401s (OpenAI /
Voyage / Cohere rejecting the user's own API key) continue to escalate
as actionable misconfiguration — covered by the new
`does_not_classify_embedding_byo_key_401_as_session_expired`
polarity guard.
## Test plan
- [x] `cargo test classifies_embedding_api_invalid_token` — passes (new)
- [x] `cargo test does_not_classify_embedding_byo_key` — passes (new polarity guard)
- [x] `cargo test classifies_openhuman_invalid_token` — passes (4P0, unchanged)
- [x] `cargo test does_not_classify_byo_key_provider` — passes (tinyhumansai#2286 BYO-key contract preserved)
- [x] `cargo test core::observability` — 91 tests pass, 0 regressions
- [x] `cargo check --bin openhuman-core` — passes
- [x] `cargo fmt --check` — clean
CodeGhost21
changed the title
11 tasks
…as SessionExpired (TAURI-RUST-1EE)
Third emit-site prefix for the same OpenHuman backend
`{"success":false,"error":"Invalid token"}` 401 envelope this PR
already classifies for non-streaming chat (4P0) and embeddings (4K5).
TAURI-RUST-1EE (Sentry issue 1807, 110 events, 109 on
openhuman@0.56.0, domain=llm_provider operation=streaming_chat
status=401 provider=OpenHuman) is the streaming-chat path: the body is
wrapped at `inference/provider/compatible.rs:949` with the
`"OpenHuman streaming API error"` prefix. The `streaming` token between
`OpenHuman` and `API error` means the 4P0 anchor
(`"OpenHuman API error (401"`) does not match it, so it needs its own
prefix arm.
Same conjunctive-anchor pattern as the existing arms — the
OpenHuman-scoped streaming prefix gates the match so a third-party
BYO-key streaming 401 (`"OpenAI streaming API error (401):
invalid_api_key"`) stays actionable in Sentry.
Tests:
- `classifies_openhuman_streaming_invalid_token_401_as_session_expired`
— verbatim 1EE wire shape (direct + caller-wrapped).
- `does_not_classify_streaming_byo_key_401_as_session_expired` —
polarity guard for the streaming prefix.
## Test plan
- [x] `cargo test classifies_openhuman_streaming_invalid_token` — passes
- [x] `cargo test does_not_classify_streaming_byo_key` — passes (polarity)
- [x] `cargo test core::observability` — 93 tests pass, 0 regressions
- [x] `cargo check --bin openhuman-core` — passes
- [x] `cargo fmt --check` — clean
CodeGhost21
changed the title
12 tasks
graycyrus
reviewed
May 28, 2026
Contributor
graycyrus
left a comment
graycyrus
left a comment
There was a problem hiding this comment.
@CodeGhost21 the two follow-on commits look clean — the 4K5 (Embedding API) and 1EE (streaming) arms follow the same conjunctive-anchor pattern as the original 4P0 arm, the polarity guards for BYO-key embedding and streaming 401s are solid, and the test coverage (direct + caller-wrapped shapes for each arm) is thorough.
One CI check is still pending (Windows / Appium Chromium E2E). Once that clears, I'll come back and approve this.
oxoxDev
pushed a commit
to CodeGhost21/openhuman
that referenced
this pull request
May 28, 2026
`reliable::format_failure_aggregate` (no-configured-fallbacks branch) wraps every exhausted `reliable_chat_with_system` turn with: "The model `<name>` may not be available on your provider. Configure a fallback chain via `reliability.model_fallbacks` in your OpenHuman config, or change your default model in Settings → AI.\n\nAll providers/models failed. Attempts:\n…" The aggregate fires once per turn regardless of the underlying per- attempt cause (401 auth wall, unknown model, region block, rate- limit cliff). All of those are user-actionable: pick a different model, fix the credential, or configure fallbacks — the message literally tells the user how. Sentry has no remediation path that the per-attempt body classifiers haven't already covered at the lower layer (`SessionExpired`, `BudgetExhausted`, config_rejection siblings, etc.). Adds `"reliability.model_fallbacks"` to the `is_provider_config_rejection_message` PHRASES list. The string is uniquely OpenHuman — that config path is rendered into an error message only from `reliable.rs:332-334`, verified via grep across `src/`. A stray "may not be available" log line elsewhere will not collide. The configured-fallbacks aggregate branch (just `"All providers/models failed. Attempts:\n…"`) is intentionally NOT matched — the user has already engaged with the knob, so per- attempt classifiers should drive the per-body decision. Targets Sentry OPENHUMAN-TAURI-4JS (issue 5215): 25 events on v0.56.0 in 5h, `domain=llm_provider operation=reliable_chat_with_system failure=all_exhausted`. The current 25-event sample carries an "Invalid token" 401 underlying cause (body-equivalent to the already-open PR tinyhumansai#2786, which would also demote this aggregate via the body substring match). This PR catches the aggregate at the emit-site level so future all_exhausted scenarios with non-401 underlying causes (model name typo, region block, …) demote the same way. Tests pin the verbatim 4JS payload + three underlying-cause variants (unknown-model upstream, region block, bare aggregate) + a negative guard confirming the configured-fallbacks branch does NOT classify on the aggregate phrase alone.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Extend
is_session_expired_messageinsrc/core/observability.rsto recognise the OpenHuman backend's{"success":false,"error":"Invalid token"}401 envelope as a session-expired condition (previously only the explicit"Session expired. Please log in again."body was recognised). The same upstream cause — backend rejecting the bearer JWT as invalid — surfaces under three emit-site prefixes depending on the call path, each producing its own Sentry fingerprint:OpenHuman API error (401 …)run_chat_task)Embedding API error (401 …)embeddings/openai.rs:139OpenHuman streaming API error (401 …)compatible.rs:949)All three are typically preceded by a
[scheduler_gate] signed_out false -> truebreadcrumb. The UI already drives reauth via theSessionExpiredevent-domain path; this stops the noise leaking into Sentry as a code bug.Why three arms, not one
The matcher uses conjunctive anchors per arm —
"<emit-site prefix> (401"AND the envelope-shaped"\"error\":\"Invalid token\"". Anchoring on each OpenHuman-scoped prefix is what preserves the #2286 BYO-key contract:"OpenAI API error (401 Unauthorized): invalid_api_key"(user's own OpenAI key revoked) must NOT match."OpenAI streaming API error (401): invalid_api_key"and"Embedding API error (401): invalid_api_key"(BYO embedding/streaming key revoked) must NOT match either.Each of those is pinned by a dedicated
does_not_classify_*_byo_key_401_as_session_expiredpolarity guard. A single broad "any 401 + invalid token" matcher would silence all of them, so each OpenHuman-backend emit-site prefix gets its own prefix-gated arm. Thestreamingtoken in 1EE specifically means the 4P0 anchor can't cover it.Tests added (
observability::tests)classifies_openhuman_invalid_token_401_as_session_expired— 4P0 (wrapped + unwrapped).classifies_embedding_api_invalid_token_401_as_session_expired— 4K5 (direct + wrapped).classifies_openhuman_streaming_invalid_token_401_as_session_expired— 1EE (direct + wrapped).does_not_classify_embedding_byo_key_401_as_session_expired— embedding polarity guard.does_not_classify_streaming_byo_key_401_as_session_expired— streaming polarity guard.does_not_classify_byo_key_provider_401_as_session_expired— SessionExpired clears the session for unrelated backend 401s #2286 chat-path contract, still green.Test plan
cargo test classifies_openhuman_invalid_token— passes (4P0)cargo test classifies_embedding_api_invalid_token— passes (4K5)cargo test classifies_openhuman_streaming_invalid_token— passes (1EE)cargo test does_not_classify_embedding_byo_key/does_not_classify_streaming_byo_key/does_not_classify_byo_key_provider— pass (polarity)cargo test core::observability— 93 tests pass, 0 regressionscargo check --bin openhuman-core— passescargo fmt --check— clean