Skip to content

refactor: rewrite commit-extract + commit-semantic to JSONL pipeline#9

Open
feng-y wants to merge 1 commit into
mainfrom
worktree-commit-extract-rewrite
Open

refactor: rewrite commit-extract + commit-semantic to JSONL pipeline#9
feng-y wants to merge 1 commit into
mainfrom
worktree-commit-extract-rewrite

Conversation

@feng-y

@feng-y feng-y commented Mar 22, 2026

Copy link
Copy Markdown
Owner

Summary

  • commit-extract: Replace heuristic with Team Agent orchestrator — adaptive batching by patch weight, workers use docs/generate_commit.md prompt, output YYYY-MM.jsonl
  • commit-semantic: Simplify 5-stage to 4-stage JSONL consumer (ingest → aggregate → distill → export)
  • io_utils: Add append_jsonl(); git_utils: Add no_merges param
  • Delete old heuristic, keyword classification, ExportSummary dataclass usage

Test Plan

  • 756 tests passing (52 new, 9 updated for JSONL format)
  • E2E: commit-extract manifest generation, commit-semantic 4-stage pipeline
  • Unit: adaptive batching, stat parsing, incremental resume, ingest fault tolerance

Design Spec

docs/superpowers/specs/2026-03-22-commit-extract-rewrite-design.md

DO NOT MERGE — pending E2E validation

commit-extract: replace heuristic with Team Agent orchestrator pattern
- Adaptive batching by patch weight (budget 3000, max 15/batch)
- Workers use docs/generate_commit.md prompt via Task tool
- Output: YYYY-MM.jsonl (not YAML), resume via SHA scanning
- Delete old heuristic, prompts/generate_commit_log.md

commit-semantic: simplify 5-stage to 4-stage JSONL consumer
- ingest: expand sections/items into semantic units
- aggregate: group by theme, compute op/importance distributions
- distill: score patterns, rank canonical demands
- export: summary.json with stats
- Remove keyword classification, scoring heuristics, ExportSummary

io_utils: add append_jsonl(); git_utils: add no_merges param
Tests: 756 passed (52 new, 9 updated for JSONL format)
@feng-y feng-y force-pushed the worktree-commit-extract-rewrite branch from 0d55ec0 to 7b101b0 Compare March 22, 2026 23:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant