feat(rag): bilingual dora kb — labeled french working translation (#424 phase 2)#431
Merged
Conversation
… p2) french users now get regulation snippets in french, not just a french answer over english source text. the kb is seeded in both languages and retrieval prefers the user's locale, falling back to english when a french chunk is missing. - translateComplianceKb: generates dora-articles.fr.json (working llm translation), flagged official:false with an eur-lex disclaimer + source url - seedComplianceKb: loads en + fr files, tags each chunk metadata.lang / official - complianceKbRetriever: language-filtered search (fr -> en -> unfiltered fallback), carries metadata.official through to the source object - contextFormatter.formatSources: prefer metadata.url over the 'regulation' category tag (this also makes phase 1's eur-lex link actually reach the ui) + expose official - frontend: Source.official type + a "working translation" badge on unofficial chunks, en/fr i18n strings - unit tests: lang filter + en fallback, url precedence, official flag pass-through the french text is a labeled non-official working translation; the authoritative text stays the official eur-lex french version, linked on every citation.
|
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
#424 Phase 2 — French DORA knowledge base
Phase 1 (#429) gave French users a French answer + a link to the official EUR-Lex French text. Phase 2 seeds the KB itself in French, so retrieved regulation snippets are in the user's language too.
What changed
scripts/translateComplianceKb.js→ generatesdora-articles.fr.json, a working LLM translation of the 58 DORA entries. Flaggedofficial: falsewith an EUR-Lex disclaimer + source URL.seedComplianceKb.js→ loadsen+frfiles, tags every chunkmetadata.lang/metadata.official. Smart-sync re-seeds when the point count changes (58 → 116).complianceKbRetriever.js→ language-filtered search: prefers the user's locale (fr), falls back toen, then unfiltered as a last resort. Carriesmetadata.officialthrough.contextFormatter.formatSources→ now prefersmetadata.urlover the'regulation'category tag. This also fixes Phase 1: the EUR-Lex link was being shadowed bymetadata.source = 'regulation'and never reached the UI.Source.officialtype + a "working translation" badge on unofficial chunks (en/fr i18n).Trust / compliance
The French text is a labeled, non-official working translation. The authoritative text stays the official EUR-Lex French version, linked on every citation. We do not present a machine translation as the regulation.
Tests
metadata.urlprecedence over the category tag,officialpass-throughDeploy note
Prod
compliance_kbneeds a re-seed to pick up the French partition (point count 58 → 116 triggers smart-sync automatically on the seed step).