Skip to content

Separate signed requirements from issue acceptance criteria#565

Merged
wwind123 merged 3 commits into
mainfrom
issue-561-separate-human-requirements
Jul 14, 2026
Merged

Separate signed requirements from issue acceptance criteria#565
wwind123 merged 3 commits into
mainfrom
issue-561-separate-human-requirements

Conversation

@wwind123

Copy link
Copy Markdown
Owner

Fixes #561

Summary

  • render explicit signed-human-requirements context, including the empty state, across planning, implementation, and PR-review prompts
  • clearly separate ordinary issue acceptance criteria from protocol requirement IDs
  • pass authoritative reviewer requirement IDs into generic schema repair and remove fabricated empty-context acknowledgements
  • add prompt and repair regressions

Tests

  • python3 -m pytest tests/test_prompts.py tests/test_repair.py tests/test_agent_loop.py
  • python3 -m pytest (1541 passed)

@wwind123

Copy link
Copy Markdown
Owner Author

{
"schema_version": 1,
"kind": "coder_followup",
"state": "blocking",
"summary": "Implemented the approved plan for issue #561 and opened pull request #565. Prompts now distinguish ordinary issue acceptance criteria from the authoritative signed human requirements set, and generic reviewer schema repair receives that set and removes fabricated acknowledgements when it is empty.",
"addressed_items": [],
"remaining_items": [],
"human_requirements": {
"addressed_ids": [],
"checked_discussion_directly": false
},
"tests_run": [
"python3 -m pytest tests/test_prompts.py tests/test_repair.py tests/test_agent_loop.py (486 passed)",
"python3 -m pytest (1541 passed)"
],
"pr_number": 565
}

-- OpenAI Codex: gpt-5.6-luna

@wwind123

Copy link
Copy Markdown
Owner Author

Review verdict: Blocking

Good fix overall, but the non-compact plan-review prompt (build_plan_review_prompt) still embeds an unconditional 'Requirement 1' example in its human_requirement_dispositions JSON, which reproduces the exact fabrication pattern issue #561 is meant to eliminate; the compact version of the same prompt was fixed but this path was missed and lacks regression coverage.

Blocking issues

  • src/coding_review_agent_loop/prompts.py:1135-1137 — build_plan_review_prompt (the non-compact plan-review prompt, used when --planning-context-mode full) still shows the JSON response example with a populated "human_requirement_dispositions": [{"requirement_id": "Requirement 1", ...}] unconditionally, regardless of whether any signed requirements were actually surfaced. The parallel compact-mode schema in _plan_review_schema_and_rules() (prompts.py:732) was correctly changed to [] in this same PR, but this non-compact copy was left stale. Since this concrete example appears directly in the JSON schema block (not gated by the surfaced/none-surfaced guidance that appears later in the prompt), an agent following the example literally can still fabricate a 'Requirement 1' disposition when none were surfaced — the same failure mode issue Separate issue acceptance criteria from signed human requirements in agent prompts #561 reports, just via the full-context path instead of the compact one. The new regression test (test_empty_signed_requirements_are_separate_from_issue_acceptance_criteria) only checks build_issue_plan_prompt's schema string for '"human_requirement_dispositions": []', not build_plan_review_prompt's (compact or full), so this gap has no coverage.

Same-PR follow-ups

  • src/coding_review_agent_loop/prompts.py:410-415 — in render_coder_human_requirements_prompt_context, format_human_requirements now always returns non-empty text (including the new 'no signed human requirements were surfaced' branch), so block can never be falsy and the if not block: return CoderHumanRequirementsPromptContext(block="", ...) branch is now dead code. Remove it or restore a real short-circuit.

-- Anthropic Claude: claude-sonnet-5

@wwind123

Copy link
Copy Markdown
Owner Author

Review verdict: Approved

The PR successfully separates signed human requirements from ordinary issue acceptance criteria in prompts and the repair logic, correctly handling cases where no signed requirements are surfaced.

-- Google Antigravity: Gemini 3.1 Pro (High)

@wwind123

Copy link
Copy Markdown
Owner Author

Coder follow-up

Implemented both blocking prompt fixes and pushed them to the existing PR branch.

Addressed items

  • item-1: Blocking issue from Anthropic Claude: claude-sonnet-5, round 1: src/coding_review_agent_loop/prompts.py:1135-1137 — build_plan_review_prompt (the non-compact pla...
    • Resolution: Changed the full-context build_plan_review_prompt JSON example to use an empty human_requirement_dispositions array, and extended regression coverage to full and compact plan-review prompts when no signed requirements are surfaced.
  • item-2: Same-PR follow-up from Anthropic Claude: claude-sonnet-5, round 1: src/coding_review_agent_loop/prompts.py:410-415 — in `render_coder_human_requirements_prompt_cont...
    • Resolution: Removed the unreachable empty-block branch from render_coder_human_requirements_prompt_context because format_human_requirements always returns text, including the explicit no-requirements guidance.

Remaining items

  • None.

Tests run

  • python3 -m pytest tests/test_prompts.py (passed: 105 tests)
  • python3 -m pytest (passed: 1541 tests)

-- OpenAI Codex: gpt-5.6-luna

@wwind123

Copy link
Copy Markdown
Owner Author

Review verdict: Approved

Round 2 fixes both prior items: the full (non-compact) plan-review prompt's JSON schema now shows an empty human_requirement_dispositions example, matching the compact prompt, with new regression assertions covering both; and the dead if-not-block branch in render_coder_human_requirements_prompt_context was removed. No new issues found.

Prior unresolved item dispositions

  • [item-1] Blocking issue from Anthropic Claude: claude-sonnet-5, round 1: src/coding_review_agent_loop/prompts.py:1135-1137 — build_plan_review_prompt (the non-compact pla... -> resolved
  • [item-2] Same-PR follow-up from Anthropic Claude: claude-sonnet-5, round 1: src/coding_review_agent_loop/prompts.py:410-415 — in `render_coder_human_requirements_prompt_cont... -> resolved

-- Anthropic Claude: claude-sonnet-5

@wwind123

Copy link
Copy Markdown
Owner Author

Review verdict: Blocking

The PR successfully separates signed requirements from acceptance criteria and resolves the prior items, but introduces a contradictory repair prompt example.

Blocking issues

  • In src/coding_review_agent_loop/repair.py, Valid Format J was updated to show "human_requirement_dispositions": [], but the <!-- HUMAN_REQUIREMENTS_ADDRESSED --> marker and ### Human requirements section were left in the example. This contradicts the dynamic instruction which explicitly forbids those markdown markers when there are no surfaced requirements. Remove the marker and section from Valid Format J so the example correctly demonstrates the empty state.

Prior unresolved item dispositions

  • [item-1] Blocking issue from Anthropic Claude: claude-sonnet-5, round 1: src/coding_review_agent_loop/prompts.py:1135-1137 — build_plan_review_prompt (the non-compact pla... -> resolved
  • [item-2] Same-PR follow-up from Anthropic Claude: claude-sonnet-5, round 1: src/coding_review_agent_loop/prompts.py:410-415 — in `render_coder_human_requirements_prompt_cont... -> resolved

-- Google Antigravity: Gemini 3.1 Pro (High)

@wwind123

Copy link
Copy Markdown
Owner Author

Coder follow-up

Removed the invalid human-requirements marker and section from Valid Format J and added regression coverage.

Addressed items

  • item-3: Blocking issue from Google Antigravity: Gemini 3.1 Pro (High), round 2: In src/coding_review_agent_loop/repair.py, Valid Format J was updated to show `"human_require...
    • Resolution: Valid Format J now demonstrates the empty human_requirement_dispositions state without HUMAN_REQUIREMENTS_ADDRESSED or a Human requirements section.

Remaining items

  • None.

Tests run

  • python3 -m pytest tests/test_repair.py -k 'repair_prompt_empty_initial_plan_state or repair_prompt_plan_revision_preserves_human_requirements_acknowledgement' (passed: 2)
  • python3 -m pytest (passed: 1542)

-- OpenAI Codex: gpt-5.6-luna

@wwind123

Copy link
Copy Markdown
Owner Author

Review verdict: Approved

Round 3 removes the stray HUMAN_REQUIREMENTS_ADDRESSED marker and Human requirements section from repair.py's empty-state 'Valid Format J' example, matching the dynamic instruction that forbids those markers when no requirements are surfaced, with a targeted regression test. No new issues found.

Prior unresolved item dispositions

  • [item-3] Blocking issue from Google Antigravity: Gemini 3.1 Pro (High), round 2: In src/coding_review_agent_loop/repair.py, Valid Format J was updated to show `"human_require... -> resolved

-- Anthropic Claude: claude-sonnet-5

@wwind123

Copy link
Copy Markdown
Owner Author

Review verdict: Approved

The PR successfully separates signed human requirements from ordinary issue acceptance criteria in prompts, correctly generating an empty human_requirement_dispositions array when no signed requirements are surfaced. [item-3] is also resolved by correctly removing the markdown markers in Valid Format J's empty state example. Note: GitHub PR tests are currently pending.

Prior unresolved item dispositions

  • [item-3] Blocking issue from Google Antigravity: Gemini 3.1 Pro (High), round 2: In src/coding_review_agent_loop/repair.py, Valid Format J was updated to show `"human_require... -> resolved

-- Google Antigravity: Gemini 3.1 Pro (High)

@wwind123
wwind123 merged commit 98a68e3 into main Jul 14, 2026
1 check passed
@wwind123
wwind123 deleted the issue-561-separate-human-requirements branch July 14, 2026 05:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Separate issue acceptance criteria from signed human requirements in agent prompts

1 participant