Skip to content

feat: add T-I-F reliability helpers (TifScore, evaluateTif)#145

Merged
ferhimedamine merged 4 commits into
mainfrom
feat/tif-reliability-helpers
Jun 13, 2026
Merged

feat: add T-I-F reliability helpers (TifScore, evaluateTif)#145
ferhimedamine merged 4 commits into
mainfrom
feat/tif-reliability-helpers

Conversation

@ferhimedamine

@ferhimedamine ferhimedamine commented Jun 12, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Summary

Part of T-I-F RFC Phase 3 — adds type-safe T-I-F reliability helpers so developers don't have to hand-roll metadata.reliability parsing.

Changes:

  • TifScore interface and TifClassification type in types.ts
  • computeTifScore(history) and tifScoreFromMetadata(data) pure functions
  • evaluateTif(memoryId) method on DakeraClient
  • All exported from index.ts
  • 15 unit tests in src/tif.test.ts

T-I-F computation: upvote/positive→truth, downvote/negative→falsity, flag→indeterminacy. Classification: falsity≥0.5→surface_contradiction, indeterminacy≥0.5→ask_clarification, truth≥0.7→confident_reuse, else→verify_before_use.

Related PRs (all 4 SDKs batch)

  • dakera-py: feat/tif-reliability-helpers
  • dakera-js (this PR)
  • dakera-rs: feat/tif-reliability-helpers
  • dakera-go: feat/tif-reliability-helpers

🤖 Generated with Claude Code

Reviewed-by: Jean-Sébastien Beaulieu (@SeCuReDmE-main-dev) — T-I-F contract parity review

Dakera CTO and others added 3 commits June 12, 2026 23:28
@claude
feat: add T-I-F reliability helpers (TifScore, evaluateTif)
eda05c3 
Implements Phase 3 of the T-I-F RFC (dakera-deploy#161).

- Add TifScore interface, TifClassification type, computeTifScore() and
  tifScoreFromMetadata() helpers to types.ts
- Add evaluateTif() method to DakeraClient
- Export new types/functions from index.ts
- Add 15 unit tests covering computation, classification, and metadata parsing

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
fix: resolve TypeScript import-type error and update docs for TifScore
3c306ae 
- Fix TS1361: split computeTifScore out of `import type` into a regular
  value import — it is a function, not a type alias
- Update CHANGELOG.md and README.md with TifScore / evaluateTif documentation

Part of T-I-F RFC Phase 3 (DAK-6562).
@Paperclip-Paperclip
ci: re-trigger CI checks after TS1361 fix
21032d3 
Co-Authored-By: Paperclip <noreply@paperclip.ing>
@ferhimedamine ferhimedamine force-pushed the feat/tif-reliability-helpers branch from feb5888 to 21032d3 Compare June 12, 2026 23:29
@SeCuReDmE-main-dev

SeCuReDmE-main-dev commented Jun 12, 2026

Copy link
Copy Markdown

Phase 3 SDK review note from the RFC side.

The TypeScript surface looks good: typed TifScore, explicit classification union, computeTifScore(), tifScoreFromMetadata(), and evaluateTif() are the right ergonomic layer.

Before this leaves draft, I would align it with the canonical Phase 3 T-I-F math across MCP and the other SDKs. Current risk: this implementation appears to use raw feedback proportions only, while dakera-mcp#123 adds thin-evidence base indeterminacy for feedback_count < 3. No-feedback behavior should also be identical across all implementations.

Suggested gate: add or reference shared golden vectors so Python/JS/Rust/Go/MCP all return the same truth, indeterminacy, falsity, classification, and count for the same feedback histories. That will prevent agents from getting different reliability classifications depending on whether they enter through MCP or an SDK.

Once parity is locked, this looks like the right JS/TS API shape.

@claude
fix(tif): add thin-evidence base indeterminacy + golden vectors (DAK-…
6588bf6 
…6566)

Aligns JS/TS SDK with MCP canonical T-I-F v1 contract:
- Inject base indeterminacy when feedback_count < 3 to prevent
  false confidence from sparse signals
- Normalise T+I+F to 1.0 after adding base indeterminacy
- Add 8 golden vector tests matching MCP/Python/Rust/Go
- Add 3 thin-evidence unit tests

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@ferhimedamine ferhimedamine mentioned this pull request Jun 13, 2026
Merged
@SeCuReDmE-main-dev

SeCuReDmE-main-dev commented Jun 13, 2026

Copy link
Copy Markdown

@ferhimedamine final Phase 3 review complete from my side for the JS SDK PR.

I rechecked the current PR state and the DAK-6566 parity fixes. The previous blockers are resolved:

  • CI is green.
  • PR is mergeable.
  • no-feedback now maps to ask_clarification.
  • thin-evidence base indeterminacy is present.
  • the 8 golden vectors are present and cover the canonical T-I-F v1 contract.
  • 3 downvote + 3 flag correctly prioritizes surface_contradiction.
  • metadata reliability parsing remains backward compatible with Phase 1 / Phase 2.

This is review-ready from my side. No further requested changes from me.

@SeCuReDmE-main-dev SeCuReDmE-main-dev mentioned this pull request Jun 13, 2026
Closed
@ferhimedamine ferhimedamine marked this pull request as ready for review June 13, 2026 02:46
@ferhimedamine ferhimedamine added the auto-merge Auto-merge when CI passes label Jun 13, 2026
@ferhimedamine ferhimedamine merged commit eb1c867 into main Jun 13, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

auto-merge Auto-merge when CI passes

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@ferhimedamine @SeCuReDmE-main-dev