[FEAT] : Improves extraction post-processing quality by introducing a dedicated normalization pipeline for noisy conversational outputs

[Aryama-srivastav]

Description

Raw model responses can contain repeated values, ambiguous phrases, inconsistent plural formatting, and missing placeholders. This PR makes extraction handling deterministic, reviewable, and consistent before downstream form filling.

What changed:

Added quality processor in extraction_quality.py

• Duplicate entity detection + deterministic merge strategy
• Ambiguity flags for low-confidence terms
• Standardized plural normalization (;-separated values → deduplicated ordered list)
• Consistent missing-value sentinel policy (MISSING)
• Per-run quality report generation

Integrated post-processing into llm.py

• add_response_to_json() now routes all extracted values through the quality processor
• Added get_quality_report() for downstream access
• main_loop() now logs extraction quality report alongside extracted JSON

Added focused tests in test_extraction_quality_controls.py

• Missing sentinel consistency
• Plural normalization and deduplication
• Deterministic duplicate merge behavior
• Ambiguity flagging
• LLM integration path through add_response_to_json()

Type of change
Feature (non-breaking enhancement)
Bug fix (non-breaking reliability improvement)

How Has This Been Tested?

Test A (focused feature suite):
Ran:
python -m pytest test_extraction_quality_controls.py -q

Verified output:
5 passed

Acceptance criteria verification:
Duplicate keys/values handled deterministically: ✅
Plural outputs normalized to documented format: ✅
Ambiguous fields marked for review: ✅
Missing values use one consistent sentinel across modules: ✅

Test Configuration:

Firmware version: N/A
Hardware: Local development machine (Windows)
SDK: N/A
Python: 3.13
Shell: PowerShell

Checklist:

• My code follows the style guidelines of this project
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes
• Any dependent changes have been merged and published in downstream modules

[Copilot]

Pull request overview

This PR introduces a dedicated extraction normalization pipeline to make LLM extraction outputs deterministic and consistent before downstream form-filling, and integrates that pipeline into the LLM JSON accumulation flow.

Changes:

• Added ExtractionQualityProcessor to normalize missing values, plural outputs, duplicates, and ambiguity signals, plus a per-run quality report.
• Integrated the quality processor into LLM.add_response_to_json() and added LLM.get_quality_report(), with report logging in main_loop().
• Added focused pytest coverage for sentinel handling, plural normalization/dedup, duplicate merging, ambiguity flagging, and the LLM integration path.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

File	Description
src/extraction_quality.py	Implements normalization/merge logic and generates a structured quality report.
src/llm.py	Routes extracted values through the quality processor and exposes/logs the quality report.
src/test/test_extraction_quality_controls.py	Adds targeted tests validating normalization behavior and LLM integration.
src/inputs/input.txt	Updates the sample input text used for local runs/examples.

Comments suppressed due to low confidence (1)

src/llm.py:113

• handle_plural_values() appears to be unused now that add_response_to_json() routes normalization through ExtractionQualityProcessor (no other references found in the repo). Consider removing this method (and its logging) to reduce dead code, or keeping it but delegating to the quality processor to avoid divergence.

    def handle_plural_values(self, plural_value):
        """
        This method handles plural values.
        Takes in strings of the form 'value1; value2; value3; ...; valueN'
        returns a list with the respective values -> [value1, value2, value3, ..., valueN]
        """

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.