-
Notifications
You must be signed in to change notification settings - Fork 130
Description
📝 Description
When the AI fills out a form, it currently just drops the answer into the PDF without explaining where it got the information. We need a way to see the AI's "homework." This feature makes the AI extract the exact quote from the audio transcript that it used to find the answer. It then creates a separate Audit Trail text file alongside the filled PDF so humans can easily verify the AI's work.
💡 Rationale
FireForm handles high-stakes emergency reporting. If an incident report is ever questioned in court or by a fire chief, officials must be able to see the raw text that justified the AI's decision. We cannot have a "black box" where data magically appears.
🛠️ Proposed Solution
We will force the LLM to output its answers as a strict dictionary containing both the final answer and the source quote. Then, we will generate a clean .txt file logging this data.
- Logic change in
src/llm.py: Updatebuild_promptto demand a strict JSON format with"value"and"quote"keys. - Logic change in
src/llm.py: Add a safety check inadd_response_to_jsonto handle cases where the LLM mistakenly returns a Pythonlistinstead of a string (joining lists with a;to prevent crashes). - Logic change in
src/filler.py: Extract only the"value"to fill the PDF fields, then write a formatted_audit.txtfile that lists every field alongside its AI value and transcript source quote.
✅ Acceptance Criteria
- Prompt Adherence: The LLM successfully returns both the value and the quote for every field.
- Crash Prevention: If the LLM hallucinates a JSON array (e.g.,
["John", "Jane"]), the system catches it and converts it safely without throwing anAttributeError. - Audit File Generation: A clean
_audit.txtfile is saved in the same directory as the output PDF, containing readable timestamps, fields, values, and quotes.
📌 Additional Context
We specifically chose to generate a .txt file instead of drawing a new page inside the PDF. This avoids adding heavy new dependencies (like reportlab) to our Docker image, keeping the application lightweight, fast, and completely free of external library bloat. Furthermore, a .txt file is much easier for a fire department's database system to parse later compared to extracting text from a PDF graphic layer.