Skip to content

Add real-world examples: data pipeline, MCP search, and naive vs compiled comparison#114

Merged
dgenio merged 5 commits intomainfrom
copilot/add-real-world-examples
Apr 14, 2026
Merged

Add real-world examples: data pipeline, MCP search, and naive vs compiled comparison#114
dgenio merged 5 commits intomainfrom
copilot/add-real-world-examples

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Apr 12, 2026

The repo had only one trivial arithmetic example, leaving users without concrete evidence of real-world applicability or the latency/cost argument for adopting ChainWeaver.

New examples

  • examples/etl_flow.py — 5-step ETL flow (fetch_data → validate_records → normalize_fields → enrich_records → store_records) over an inventory mock dataset. Demonstrates field normalization, type coercion, and invalid-record dropping.

  • examples/mcp_search_flow.py — MCP-style retrieval flow (search_knowledge_base → extract_relevant_fields → format_response) over a mock knowledge base. Illustrates how ChainWeaver eliminates intermediate LLM calls from fully deterministic search/extract/format patterns.

  • examples/naive_vs_compiled.py — Side-by-side timing comparison of a 5-step record-enrichment flow. The naive path inserts time.sleep(0.3) between each step (simulated LLM round-trips); the compiled path runs the same flow via a registered Flow. Prints a table:

    ┌─────────────────────────┬───────────────┬───────────────┐
    │ Metric                 │        Naive │     Compiled │
    ├─────────────────────────┼───────────────┼───────────────┤
    │ Wall time (ms)         │       1200.7 │          0.3 │
    │ LLM calls              │            4 │            0 │
    │ Speedup                │         1.0x │      4447.3x │
    │ Time saved (ms)        │              │       1200.4 │
    │ LLM calls saved        │              │            4 │
    └─────────────────────────┴───────────────┴───────────────┘
    

README

Updated Quick Start and Development sections to list all four examples with one-line descriptions.

Notes

Copilot AI linked an issue Apr 12, 2026 that may be closed by this pull request
14 tasks
Copilot AI changed the title [WIP] Add real-world examples for MCP-like and data pipeline scenarios Add real-world examples: data pipeline, MCP search, and naive vs compiled comparison Apr 12, 2026
Copilot AI requested a review from dgenio April 12, 2026 14:40
@dgenio dgenio marked this pull request as ready for review April 12, 2026 14:41
@dgenio dgenio requested a review from Copilot April 12, 2026 14:41
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds multiple runnable, real-world-focused example scripts to demonstrate ChainWeaver’s deterministic execution benefits (including a latency/cost comparison), and updates the README to reference them.

Changes:

  • Added three new example scripts: ETL-style data flow, MCP-style search flow, and naive-vs-compiled timing comparison.
  • Updated README Quick Start and Development sections to list all available examples and how to run them.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 9 comments.

File Description
README.md Lists additional example scripts in Quick Start and Development sections.
examples/data_pipeline_flow.py New 5-step ETL-style deterministic flow example over mock inventory data.
examples/mcp_search_flow.py New 3-step MCP-style search → extract → format deterministic flow example.
examples/naive_vs_compiled.py New timing comparison example contrasting simulated naive LLM step transitions vs FlowExecutor execution.

Comment thread README.md Outdated
Comment thread README.md Outdated
Comment thread examples/mcp_search_flow.py Outdated
Comment thread examples/mcp_search_flow.py Outdated
Comment thread examples/data_pipeline_flow.py Outdated
Comment thread examples/naive_vs_compiled.py Outdated
Comment thread examples/naive_vs_compiled.py
Comment thread examples/naive_vs_compiled.py
Comment thread examples/mcp_search_flow.py
…ck in examples

- Replace all 'chain'/'pipeline' terminology in examples and README with
  canonical 'flow' vocabulary per AGENTS.md domain vocabulary rules
- Rename examples/data_pipeline_flow.py -> examples/etl_flow.py; rename
  internal identifiers pipeline_flow -> etl_flow and flow name
  'data_pipeline' -> 'data_etl' to eliminate vocabulary violations
- Fix total_hits bug in mcp_search_flow.py: compute total match count
  before applying top_k slice so downstream steps see true hit count
- Add result.success / execution_log check in naive_vs_compiled.py
  run_compiled() using real ExecutionResult API; surface failing step
  name and error before output comparison on flow failure
- Update README Development section: bare 'pytest' -> 'python -m pytest
  tests/ -v' to match authoritative validation commands in AGENTS.md
…mples

- etl_flow.py: module docstring 'Data pipeline flow' -> 'ETL flow'
- naive_vs_compiled.py: '# Number of pipeline steps' -> '# Number of flow steps'
- mcp_search_flow.py: mock-data body 'tool-chain execution' -> 'flow execution'

All per AGENTS.md domain vocabulary: 'flow' is canonical; 'chain' and
'pipeline' are explicitly forbidden synonyms.
@dgenio dgenio merged commit d6e87a6 into main Apr 14, 2026
4 checks passed
@dgenio dgenio deleted the copilot/add-real-world-examples branch April 14, 2026 05:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add real-world examples with MCP-like and data pipeline scenarios

3 participants