Skip to content

[Design Doc] Muhammad Maaz — Section 3: Technical Approachadd draft technical approach section for factual_grounding, relevance…#1

Closed
Maazkorejo wants to merge 2 commits into
mainfrom
feat/maaz-technical-approach-draft
Closed

[Design Doc] Muhammad Maaz — Section 3: Technical Approachadd draft technical approach section for factual_grounding, relevance…#1
Maazkorejo wants to merge 2 commits into
mainfrom
feat/maaz-technical-approach-draft

Conversation

@Maazkorejo

Copy link
Copy Markdown
Collaborator

What does this PR do?

Adds a draft of Section 3 (Technical Approach) for the design document, covering the technical approach for all four evaluation criteria I own: factual_grounding, relevance, refusal_check, and completeness.

Which module(s) does it touch?

docs/section3_technical_approach.md — new file, no existing code touched. This is a design document draft, not an implementation PR; no source modules exist yet per the Week 1 no-code rule.

How did you test it?

No automated tests apply since this is documentation, not code. The claims in this section are backed by manual verification: I ran sentence-transformers (all-MiniLM-L6-v2) locally against two hand-picked (response, context) pairs — a paraphrase and a factual contradiction — and included the actual cosine similarity scores (0.982 and 0.822) in the appendix, rather than assuming the model's behavior.

Edge cases handled:

The document itself identifies edge cases the eventual implementation will need to handle: numeric/factual contradictions that still score high on cosine similarity (verified with real data), refusal-adjacent phrases like "I cannot stress enough how important this is" that shouldn't trigger refusal_check, and the more open-ended limitations of prompt-decomposition for completeness. These are documented as known risks, not yet solved in code.

Checklist:

  • Tests added or updated — N/A, documentation only
  • Docstrings on all new public functions — N/A, no functions in this PR
  • No unused imports — N/A
  • README updated if public API changed — N/A, no API changes
  • Lead Engineer has reviewed and approved before submitting to director — pending, this is a draft for team review before the Day 7 design doc submission

Corrected formatting and grammatical inconsistencies in the technical approach documentation, ensuring clarity and consistency in the presentation of methods and implications.
@Maazkorejo

Copy link
Copy Markdown
Collaborator Author

Closing this in favor of the combined design doc — team decided to consolidate all sections into one file instead of separate PRs per section. Section 3 content has been carried over intact: see #2 (or whatever PR number the combined one gets).

@Maazkorejo Maazkorejo closed this Jul 23, 2026
@Maazkorejo
Maazkorejo deleted the feat/maaz-technical-approach-draft branch July 23, 2026 16:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant