DAAF-Contribution-Community · brhkim · May 4, 2026 · Apr 12, 2026 · Apr 15, 2026 · Apr 21, 2026
@@ -23,12 +23,12 @@ All agents in this directory MUST follow the canonical template at `agent_refere
 
 **All agents returning output to the orchestrator MUST respect these universal constraints:**
 
-1. **Hard cap: 1000 words maximum** for Agent return output (exception: `data-ingest` agent uses a 2500-word cap because profiling findings feed directly into skill authoring and must be comprehensive)
+1. **Hard cap: 2000 words maximum** for Agent return output (exception: `data-ingest` agent uses a 3500-word cap because profiling findings feed directly into skill authoring and must be comprehensive)
 2. **Do NOT include:** Raw execution logs, data samples, Polars/pandas table displays, full checkpoint output, QA script code, or multi-paragraph explanations in any section
 3. **Script files are the archive; the Agent return is the signal.** Execution logs are already appended to script files by `run_with_capture.sh`. Reference files by path — do not reproduce their contents.
 4. **Summarize, don't echo.** "CP1 PASSED: 2,528 rows, 12 cols, 0.3% missing" — not the full stdout.
 
-**Why this matters:** The orchestrator context window is shared across the entire pipeline. A single verbose subagent return (2,000+ words) consumes ~4,000 tokens. Over 10 subagent round-trips in a stage, that's 40,000 tokens — 20% of the orchestrator's total capacity — consumed by output alone.
+**Why this matters:** The orchestrator context window is shared across the entire pipeline. A single verbose subagent return (4,000+ words) consumes ~8,000 tokens. Over 10 subagent round-trips in a stage, that's 80,000 tokens — a meaningful share of the orchestrator's capacity — consumed by output alone.
 
 ---
 
@@ -248,6 +248,8 @@ Shows which agents produce output consumed by other agents:
 | **search-agent** | framework-engineer | Scoping findings (current state, affected files, patterns) | Framework Development Mode |
 | **search-agent** | Orchestrator | Flexible findings report with confidence assessment | Any mode/stage |
 
+**Preliminary Notes Persistence:** Discovery and profiling agents' full returns are persisted to `output/preliminary_notes/` as lossless markdown files by the orchestrator. Downstream agents reference these files by path for full-fidelity access, while the orchestrator retains compressed summaries for its own coordination. This applies to search-agent (Stage 2), source-researcher (Stage 3), research-synthesizer (Stage 3.5), and data-ingest (DI-3 through DI-6) returns. See orchestrator SKILL.md for the complete protocol.
+
 ---
 
 ## When to Use Each Agent

@@ -530,7 +530,7 @@ When reviewing visualization scripts that produce PNG output, use the **Read too
 
 ## Output Format
 
-**Hard cap: 1000 words maximum.** The orchestrator has limited context. Your output is a *verdict*, not a *transcript* — the cr/ script files contain the full investigation evidence.
+**Hard cap: 2000 words maximum.** The orchestrator has limited context. Your output is a *verdict*, not a *transcript* — the cr/ script files contain the full investigation evidence.
 
 **Do NOT include in your output:**
 - Raw execution logs or captured stdout/stderr from QA scripts (these are appended to the cr/ files)

@@ -143,11 +143,11 @@ Read `agent_reference/SCRIPT_EXECUTION_REFERENCE.md` before writing any scripts.
 | Output | Destination | Constraint | Purpose |
 |--------|-------------|------------|---------|
 | **Execution log** (stdout from script) | Appended to script file on disk | **No size limit** — the log is an archival artifact | Primary source material for DI-7 skill authoring |
-| **Subagent return** (your response to orchestrator) | Orchestrator context window | **2500-word hard cap** | Signal for orchestrator decision-making and STATE.md |
+| **Subagent return** (your response to orchestrator) | Orchestrator context window | **3500-word hard cap** | Signal for orchestrator decision-making and STATE.md |
 
 **Scripts should print comprehensive, complete results to stdout.** For datasets under 100 columns, print EVERY column's full profile. The execution log costs nothing in context — it lives on disk. The DI-7 skill authoring subagent reads these logs as its primary source for building reference files. Thin execution logs produce thin skills.
 
-**Your return to the orchestrator summarizes the key findings** within the 2500-word cap. The orchestrator does not need per-column detail — it needs status, key observations, confidence, and issues.
+**Your return to the orchestrator summarizes the key findings** within the 3500-word cap. The orchestrator does not need per-column detail — it needs status, key observations, confidence, and issues.
 
 ### 5. Part-Scoped Execution
 
@@ -179,7 +179,7 @@ Part C writes: 07a_key-integrity.py, 07b_cross-level-linkage.py,
                09a_quality-anomaly.py, 09b_quality-anomaly.py
 ```
 
-**Invocation model:** The orchestrator invokes data-ingest ONCE per part, passing ALL file paths. The agent writes all per-file scripts and any cross-file scripts in that single invocation. The 2500-word return cap applies to the combined output summary.
+**Invocation model:** The orchestrator invokes data-ingest ONCE per part, passing ALL file paths. The agent writes all per-file scripts and any cross-file scripts in that single invocation. The 3500-word return cap applies to the combined output summary.
 
 **Key type comparison for 07b:** Before attempting join simulations, compare the declared linking key's type across files and flag mismatches as a BLOCKER (e.g., `leaid` stored as Int64 in file 1 but String in file 2).
 
@@ -211,7 +211,7 @@ When invoked, check the `profiling_part` parameter and execute the corresponding
 
 3. **STOP — do NOT execute the script.** Return the script path and API findings to the orchestrator. The orchestrator presents the script to the user for approval before executing it, because DI-0 makes external network calls.
 
-**DI-0 Output (return to orchestrator — 2500-word cap):** API findings (base URL, auth method, rate limits, pagination, complexity assessment), acquisition script path, expected output path, confidence assessment, issues encountered. Note: no download details yet — the script has not been executed.
+**DI-0 Output (return to orchestrator — 3500-word cap):** API findings (base URL, auth method, rate limits, pagination, complexity assessment), acquisition script path, expected output path, confidence assessment, issues encountered. Note: no download details yet — the script has not been executed.
 
 **When data was acquired via API:** Note the acquisition script path in your Part D interpretations for provenance. The script documents the exact API call, parameters, and download date.
 
@@ -249,7 +249,7 @@ When invoked, check the `profiling_part` parameter and execute the corresponding
 - Script 03 MUST print coded value indicators (columns with negative values, sentinel values like 999/9999)
 - For datasets under 100 columns, print EVERY column's profile — no abbreviation
 
-**Part A Output (return to orchestrator — 2500-word cap):** Summarize schema, column type distribution, key observations, and conditional script recommendations. The execution logs contain the complete detail.
+**Part A Output (return to orchestrator — 3500-word cap):** Summarize schema, column type distribution, key observations, and conditional script recommendations. The execution logs contain the complete detail.
 
 ### Part B: Statistical Deep Dive (Scripts 04-06)
 
@@ -272,7 +272,7 @@ When invoked, check the `profiling_part` parameter and execute the corresponding
 - Script 05 (if run) MUST print the complete temporal coverage table with record counts per time period
 - Script 06 (if run) MUST print full entity coverage results including coverage rate vs. known universe
 
-**Part B Output (return to orchestrator — 2500-word cap):** Summarize distribution patterns, notable outliers, temporal/entity coverage highlights. The execution logs contain the complete detail.
+**Part B Output (return to orchestrator — 3500-word cap):** Summarize distribution patterns, notable outliers, temporal/entity coverage highlights. The execution logs contain the complete detail.
 
 ### Part C: Relational Analysis (Scripts 07-09)
 
@@ -304,7 +304,7 @@ When invoked, check the `profiling_part` parameter and execute the corresponding
 - Script 09 MUST print the COMPLETE coded value scan results: every sentinel value found, in which columns, with counts
 - Script 09 MUST print the full anomaly catalog with severity classifications
 
-**Part C Output (return to orchestrator — 2500-word cap):** Summarize recommended keys, dependency highlights, top anomalies. The execution logs contain the complete detail.
+**Part C Output (return to orchestrator — 3500-word cap):** Summarize recommended keys, dependency highlights, top anomalies. The execution logs contain the complete detail.
 
 ### Part D: Interpretation & Reconciliation (Scripts 10-11)
 
@@ -326,7 +326,7 @@ When invoked, check the `profiling_part` parameter and execute the corresponding
 - Script 10 MUST print semantic family groupings (identifiers, outcomes, demographics, etc.) covering all columns
 - Script 11 (if run) MUST print the COMPLETE discrepancy report: every doc claim checked, with observed vs documented values
 
-**Part D Output (return to orchestrator — 2500-word cap):** Summarize interpretation count, confidence distribution, key discrepancies. Include the full interpretation table for ALL columns (this is critical for PSU-DI2 user review). The execution logs contain additional detail.
+**Part D Output (return to orchestrator — 3500-word cap):** Summarize interpretation count, confidence distribution, key discrepancies. Include the full interpretation table for ALL columns (this is critical for PSU-DI2 user review). The execution logs contain additional detail.
 
 ### Decision Points
 
@@ -342,7 +342,7 @@ When invoked, check the `profiling_part` parameter and execute the corresponding
 
 ## Output Format
 
-Return part-specific findings in this structure (max 2500 words):
+Return part-specific findings in this structure (max 3500 words):
 
 ### Part Summary
 **Status:** [COMPLETE | COMPLETE_WITH_WARNINGS | BLOCKED]
@@ -416,6 +416,8 @@ Categories: Access | Data | Method | Perf | Process
 | COMPLETE_WITH_WARNINGS | Log warnings; proceed with caution; may request user review |
 | BLOCKED | Present STOP condition; await user resolution before re-invoking |
 
+**Preliminary Notes Persistence:** The orchestrator writes the full, unmodified data-ingest return for each profiling part to `output/preliminary_notes/{date}_part{X}_{descriptor}.md` before extracting a summary. Subsequent profiling parts and the DI-7 skill authoring agent read these files directly for full-fidelity access to prior profiling findings.
+
 </downstream_consumer>
 
 ---
@@ -429,7 +431,7 @@ Categories: Access | Data | Method | Perf | Process
 - Include complete discrepancy report with evidence
 - Archive all profiling scripts in the project's scripts directory
 - Execute only the assigned profiling part
-- Return findings within the 2500-word output cap
+- Return findings within the 3500-word output cap
 - Include conditional script recommendations in Part A output
 
 ### Ask First Before
@@ -549,7 +551,7 @@ Awaiting guidance before proceeding.
 - Discrepancies are noted without evidence
 - Preliminary interpretations are not marked as `[PRELIMINARY]`
 - Conditional script decisions are not documented (Part A)
-- Output exceeds 2500-word cap
+- Output exceeds 3500-word cap
 
 ### Self-Check
 
@@ -563,7 +565,7 @@ Before returning output, verify:
 | 4 | Are ALL semantic interpretations marked `[PRELIMINARY]`? | Add markers to every interpretation |
 | 5 | Does the output include evidence for every discrepancy? | Add observed vs documented evidence |
 | 6 | Are conditional script recommendations included (Part A)? | Add recommendations with rationale |
-| 7 | Is the output within the 2500-word cap? | Compress findings tables; keep all columns represented but condense prose |
+| 7 | Is the output within the 3500-word cap? | Compress findings tables; keep all columns represented but condense prose |
 | 8 | Are all scripts written to the correct part subdirectory? | Move scripts to correct paths |
 
 ---

@@ -240,7 +240,7 @@ When you receive a revision request due to QA BLOCKER:
 
 ## Output Format
 
-**Hard cap: 1000 words maximum.** The orchestrator has limited context. Your output is a *signal*, not an *archive* — the script files themselves are the audit trail.
+**Hard cap: 2000 words maximum.** The orchestrator has limited context. Your output is a *signal*, not an *archive* — the script files themselves are the audit trail.
 
 **Do NOT include in your output:**
 - Raw execution logs or captured stdout/stderr (these are already appended to the script file)

@@ -383,6 +383,8 @@ Your synthesis is consumed by **data-planner** to create Plan.md and Plan_Tasks.
 
 **Be opinionated, not wishy-washy.** The planner needs clear direction, not "consider either X or Y." Make recommendations with rationale. If two approaches are genuinely equivalent, pick one and state why.
 
+**Preliminary Notes Persistence:** The orchestrator writes the full, unmodified research-synthesizer return to `output/preliminary_notes/{date}_stage3.5_research-synthesis.md` before extracting a summary. Downstream agents (data-planner, plan-checker, data-verifier) read this file directly for full-fidelity access to the synthesis findings.
+
 </downstream_consumer>
 
 ---

@@ -333,6 +333,8 @@ Based on this source research:
 - Source-specific coded values must be documented with handling actions
 - Any Truth Hierarchy discrepancies must be explicitly documented
 
+**Preliminary Notes Persistence:** The orchestrator writes the full, unmodified source-researcher return to `output/preliminary_notes/{date}_stage3_{source-name}_source-research.md` before extracting a summary for its own working memory. Downstream agents (research-synthesizer, data-planner, plan-checker, data-verifier) read these files directly for full-fidelity access to source research findings.
+
 </downstream_consumer>
 
 ---

@@ -1,4 +1,4 @@
-#!/bin/bash
+#!/usr/bin/env bash
 # Claude Code Session Archiver
 # Archives complete session transcripts on session end
 #

@@ -1,4 +1,4 @@
-#!/bin/bash
+#!/usr/bin/env bash
 # audit-log.sh — PostToolUse hook that logs every tool invocation
 #
 # Creates an append-only JSONL audit trail at .claude/logs/audit.jsonl.
@@ -20,6 +20,10 @@
 # Hook event: PostToolUse (matcher: "")
 # Registered in: .claude/settings.json
 
+# -u: catch unset variable typos. Deliberately omit -e: this hook must
+# never block execution — all error paths exit 0.
+set -u
+
 INPUT=$(cat)
 
 # Parse fields from the hook JSON

@@ -1,4 +1,4 @@
-#!/bin/bash
+#!/usr/bin/env bash
 # bash-safety.sh — PreToolUse hook that blocks dangerous Bash commands
 #
 # This is the primary safety guardrail for the DAAF environment. It reads
@@ -15,13 +15,21 @@
 #   rewrites remote history and is always blocked. Similarly, `curl <url>`
 #   is fine, but `curl <url> | bash` is arbitrary code execution.
 #
-# Hook event: PreToolUse (matcher: "Bash")                                                 
+# Hook event: PreToolUse (matcher: "Bash")
 # Registered in: .claude/settings.json
 
 # Fail CLOSED: if anything unexpected goes wrong, block the command.
 # This is a security hook — ambiguous failures must not silently allow execution.
 trap 'echo "BLOCKED by bash-safety hook: unexpected error in safety check" >&2; exit 2' ERR
 
+# --- Dependency check (fail-closed) ---
+# Without jq, we cannot inspect the tool invocation JSON. Failing open here
+# would silently bypass ALL safety checks, so we must block.
+if ! command -v jq &>/dev/null; then
+    echo "BLOCKED by bash-safety hook: jq is not installed (required for hook)" >&2
+    exit 2
+fi
+
 INPUT=$(cat)
 
 # Only inspect Bash tool calls

@@ -1,4 +1,4 @@
-#!/bin/bash
+#!/usr/bin/env bash
 # context-reporter.sh — Multi-event context utilization & timestamp hook
 #
 # Injects context window utilization and a current timestamp into Claude's
@@ -31,6 +31,10 @@
 #   0 = success (stdout/JSON processed by Claude Code)
 #   All error paths exit 0 to never block tool execution.
 
+# -u: catch unset variable typos. Deliberately omit -e: this hook must
+# never block tool execution — all error paths exit 0.
+set -u
+
 INPUT=$(cat)
 HOOK_EVENT=$(echo "$INPUT" | jq -r '.hook_event_name // empty' 2>/dev/null) || HOOK_EVENT=""
 SESSION_ID=$(echo "$INPUT" | jq -r '.session_id // "default"' 2>/dev/null) || SESSION_ID="default"
@@ -49,7 +53,7 @@ if [[ -f "$CTX_CACHE" ]]; then
     MAX_CONTEXT=$(cat "$CTX_CACHE" 2>/dev/null)
 else
     LATEST_CTX=$(ls -t /tmp/claude-ctx-window-* 2>/dev/null | head -1)
-    if [[ -n "$LATEST_CTX" ]]; then
+    if [[ -n "${LATEST_CTX:-}" ]]; then
         MAX_CONTEXT=$(cat "$LATEST_CTX" 2>/dev/null)
     fi
 fi
@@ -118,7 +122,7 @@ cache_model() {
         select(.message.model) | .message.model
     ' 2>/dev/null | head -1)
 
-    [[ -n "$model" ]] && echo "$model" > "$cache" 2>/dev/null
+    [[ -n "${model:-}" ]] && echo "$model" > "$cache" 2>/dev/null
 }
 
 # ---------------------------------------------------------------------------
@@ -137,7 +141,7 @@ fi
 
 # Interval elapsed — calculate and emit
 MSG=$(calculate "$TRANSCRIPT_PATH")
-[[ -z "$MSG" ]] && exit 0
+[[ -z "${MSG:-}" ]] && exit 0
 
 # Update the shared timestamp gate
 echo "$NOW" > "$LAST_INJECT_FILE" 2>/dev/null

@@ -1,4 +1,4 @@
-#!/bin/bash
+#!/usr/bin/env bash
 # deny-claude-code-guide.sh
 # Blocks the claude-code-guide built-in agent from being spawned.
 # This is a built-in agent with an opaque system prompt that doesn't

@@ -1,4 +1,4 @@
-#!/bin/bash
+#!/usr/bin/env bash
 # enforce-explore-model.sh
 # Prevents Explore-type subagents from being launched with the haiku model.
 # Explore agents need frontier-tier reasoning for thorough codebase analysis.

@@ -1,4 +1,4 @@
-#!/bin/bash
+#!/usr/bin/env bash
 # enforce-file-first.sh — PreToolUse hook that blocks direct python execution
 #
 # Enforces the file-first execution protocol: all Python scripts must be

@@ -1,4 +1,4 @@
-#!/bin/bash
+#!/usr/bin/env bash
 # enforce-foreground-agents.sh
 # Prevents agents from being launched in the background.
 # Background agents cannot prompt for user permissions, which causes

@@ -31,6 +31,7 @@ DAAF generates transparent, auditable code and documentation so you CAN verify e
 5. WHAT THIS MEANS IN PRACTICE:
 - Always review core products from DAAF: early and often (e.g., preliminary scoping notes, research plan drafts)
 - Scrutinize results with your domain knowledge, especially anything that implies causation
+- Pay close attention to when DAAF loads reference files and Skills to inform and ground its work -- and make sure it's not taking shortcuts or going off of vibes
 - Treat DAAF's output as a high-quality first draft requiring expert review, not as a finished product
 - Know that DAAF's automated quality checks are strong but not infallible — they are one layer of defense, not the only one
 - Review the References section in reports: DAAF does its best to track and attribute citations for data sources, methods, and tools used, but citations can be wrong, incomplete, or missing entirely — you should verify that the right things are credited and that the citations themselves are accurate

@@ -1,4 +1,4 @@
-#!/bin/bash
+#!/usr/bin/env bash
 # flag-orchestrator-loaded.sh — Set flag when daaf-orchestrator skill loads
 #
 # Hook: PostToolUse (matcher: "Skill")

@@ -1,4 +1,4 @@
-#!/bin/bash
+#!/usr/bin/env bash
 # output-scanner.sh — PostToolUse hook that scans tool output for leaked secrets
 #
 # Checks tool_response content for patterns matching common credential formats:
-Original file line number
+Diff line change
@@ Expand Up @@
     **Be opinionated, not wishy-washy.** The planner needs clear direction, not "consider either X or Y." Make recommendations with rationale. If two approaches are genuinely equivalent, pick one and state why.
+    **Preliminary Notes Persistence:** The orchestrator writes the full, unmodified research-synthesizer return to `output/preliminary_notes/{date}_stage3.5_research-synthesis.md` before extracting a summary. Downstream agents (data-planner, plan-checker, data-verifier) read this file directly for full-fidelity access to the synthesis findings.
     </downstream_consumer>
     ---
@@ Expand Down @@