Smart Grid Transformer Integration & Automated Scenario Generation Pipeline#288
Draft
Rohith-Kanathur wants to merge 30 commits into
Draft
Smart Grid Transformer Integration & Automated Scenario Generation Pipeline#288Rohith-Kanathur wants to merge 30 commits into
Rohith-Kanathur wants to merge 30 commits into
Conversation
…ing, and terminal output refinements
…ine for benchmark scenario creation Introduces a fully automated 4-phase scenario generation pipeline driven by LiteLLM: PHASE 1 — Asset Profile Construction - LLM generates targeted ArXiv search queries from the asset's canonical academic name - Fetches PDFs via ArXiv API (up to 2 per query, first 5 pages extracted via pypdf) - Synthesises sensor mappings, failure modes, ISO standards, and relevant tool mappings into an AssetProfile (Pydantic model); fatal if unparseable PHASE 2 — Scenario Budget Allocation - LLM dynamically distributes the total scenario count across 5 subagent categories (iot, fmsr, tsfm, wo, multiagent) based on the AssetProfile - Multiagent capped at 50% of total; fatal if allocation is unparseable PHASE 3 — Individual Agent Generation & Validation (iot / fmsr / tsfm / wo) - Fetches 2 typed few-shot examples from ibm-research/AssetOpsBench on HuggingFace - SCENARIO_GENERATOR_PROMPT produces raw scenario dicts per subagent - VALIDATE_REPAIR_PROMPT corrects schema, tool alignment, and characteristic_form quality - Validation diffs (before/after) written to numbered log files when --log is active PHASE 4 — Multi-Agent Combiner - MULTIAGENT_COMBINER_PROMPT seeds from up to 10 single-agent scenarios to produce complex cross-subagent orchestration scenarios (e.g. IoT → FMSR → WO) CLI (python -m scenarios.generator): --num-scenarios N Total scenarios to generate (default: 50) --output PATH Output JSON path (default: generated_scenarios.json) --model-id MODEL LiteLLM model override --show-workflow Granular phase-by-phase terminal output with diffs --log Dump all raw prompts + responses to logs/<asset>_<ts>/ Supporting additions: - models.py: AssetProfile, ScenarioBudget, Scenario Pydantic models - prompts.py: 6 prompt templates (PROFILE_BUILDER, SCENARIO_GENERATOR, VALIDATE_REPAIR, MULTIAGENT_COMBINER, RESEARCH_QUERY_GENERATOR, BUDGET_ALLOCATOR) - utils.py: fetch_arxiv_studies() with multi-query dedup + PDF extraction; fetch_hf_fewshot() with type-filtered HuggingFace loading + mock fallback - Log header includes ArXiv paper titles and PDF URLs for full traceability - src/scenarios/README.md: full usage docs, pipeline breakdown, output schema, troubleshooting table, and log file structure reference
This reverts commit 64f0ae6.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overview
This PR delivers two major contributions to AssetOpsBench:
ScenarioGeneratorAgent) that scales benchmark creation to new asset classes without manual authoring1. Smart Grid Transformer Integration
CouchDB Data
Four New FMSR Tools
predict_health_indexinterpret_dgaassess_winding_temperatureassess_load_profileTests
2. Scenario Generation Pipeline (
ScenarioGeneratorAgent)A three-phase automated pipeline that generates physically plausible, causally consistent, and tool-reachable benchmark scenarios for any onboarded asset class.
Phase 1: Asset Profiling
Discovers live asset instances, sensors, and failure mappings from CouchDB, retrieves and synthesizes domain literature from ArXiv or Semantic Scholar, and merges everything with MCP tool schemas into a single structured
AssetProfilethat grounds all downstream generation.Phase 2: Budget Allocation
Distributes the total scenario count across focusses (
iot,fmsr,tsfm,wo,vibration,multiagent) proportionally to the asset's available data modalities and tool coverage, withmultiagentcapped at 75% of total budget to preserve lane diversity.Phase 3: Scenario Generation & Validation
Generates per-focus scenarios conditioned on the asset profile, runs each candidate through an LLM-based repair step.
Output
Each run produces a timestamped directory with
scenarios.json. Each scenario object contains anid,type,text,category, andcharacteristic_form.CLI