doc-pipeline-engine

Modular document processing engine with contract-gated pipeline stages. Standalone module — usable independently or as a component in larger systems (e.g. polyforge, office-polyforge).

What

• Turns documents (PDF / Office / images / email / text) into structured summaries through contract-gated pipeline stages.
• Two interchangeable legs: local (offline, no API key — spaCy + Jinja) and anthropic_sdk (cloud LLM, vendor-configurable via base_url for Bedrock / Vertex / gateways).
• Every stage boundary is validated against Pydantic v2 contracts (ADR-0001).
• Renders results to Markdown, DOCX, and PDF.
• Embeddable data-plane engine — orchestrator-agnostic, no control-plane lock-in (ADR-0007).
• Runs fully air-gapped on the offline leg for privacy-sensitive documents.

How

make install                                                           # uv sync
make run_local SAMPLE=samples/legal/us/us-open-government-act-2007.pdf  # offline leg, no API key

Full run surface (both legs, CLI switches, ANTHROPIC_API_KEY / ANTHROPIC_BASE_URL), the devcontainer, and on-demand installs → CONTRIBUTING.md.

Why

Incumbent document pipelines tend to couple extraction to one cloud vendor or a heavyweight hosted service. doc-pipeline-engine keeps the engine embeddable and contract-gated instead: the same Pydantic contracts drive the offline and cloud legs interchangeably, so you can start air-gapped and add a cloud leg later without rewiring. See docs/architecture.md.

Refs

• Architecture
• Roadmap
• Decisions (ADRs)
• Landscape: ingest · process · output · E2E systems · domain extraction
• Scraping landscape
• Changelog
• Contributing

License

Apache-2.0 (SPDX: Apache-2.0). Bundled third-party sample content is attributed in NOTICE.