diff --git a/submissions/Aryan/HOW_I_DID_IT.md b/submissions/Aryan/HOW_I_DID_IT.md new file mode 100644 index 00000000..1b071677 --- /dev/null +++ b/submissions/Aryan/HOW_I_DID_IT.md @@ -0,0 +1,240 @@ +# How I Built the LPI Life Agent + +## Step-by-Step Process + +### Phase 1: Understanding the LPI System + +I started by exploring how the LPI (Life Programmable Interface) works. The initial example showed how to connect to tools, but it wasn’t clear how real tool execution happens. I realized that: + +* The system uses JSON-RPC communication +* Tools are exposed via a Node.js server +* Proper initialization (`notifications/initialized`) is required before calling tools + +This phase was mostly about understanding the protocol rather than writing code. + +--- + +### Phase 2: Defining the Use Case + +Instead of building a generic agent, I focused on a specific query: + +> “How are digital twins used in healthcare?” + +This helped me design the agent around: + +* Conceptual understanding (methodology) +* Real-world application (case studies) + +--- + +### Phase 3: Tool Selection Strategy + +Rather than using many tools, I intentionally selected two: + +* `smile_overview` → provides structured methodology +* `get_case_studies` → provides real-world implementations + +The idea was: + +> Combine theory + application to produce a meaningful answer + +--- + +### Phase 4: Fixing Tool Execution + +Initially, I used `test-client.js`, which only runs demo tests. + +The key fix was: + +* Switching to `dist/src/index.js` (actual server) +* Adding initialization message: + +```json +{"jsonrpc": "2.0", "method": "notifications/initialized"} +``` + +Without this, tool calls returned empty results. + +--- + +### Phase 5: Parsing Tool Output + +The biggest challenge was handling tool responses. + +The output format was nested: + +```json +result → content → text +``` + +Instead of treating it as plain text, I extracted: + +```python +content[0]["text"] +``` + +This allowed me to access actual usable data. + +--- + +### Phase 6: Improving Relevance + +The `get_case_studies` tool returned multiple industries. + +Problem: + +* The first case study was often unrelated (e.g., smart buildings) + +Solution: + +* Modified tool arguments: + + ```python + {"query": "healthcare digital twin"} + ``` +* Extracted only the healthcare section from the response + +This ensured the answer actually matched the user query. + +--- + +### Phase 7: Structuring the Output + +Instead of dumping raw text, I structured the response into: + +* SMILE Framework (Summary) +* Case Study (Summary) +* Analysis +* Conclusion + +This made the agent: + +* easier to read +* more explainable +* aligned with real-world reasoning + +--- + +## Problems I Faced + +### 1. Wrong Execution Path + +Using `test-client.js` resulted in logs instead of real data + +Fix: + +* Switched to actual LPI server (`dist/src/index.js`) + +--- + +### 2. Missing Initialization + +Without sending the initialization message, tool calls silently failed. + +Fix: + +* Added JSON-RPC initialization before requests + +--- + +### 3. Empty or Broken Output + +Initially, outputs were empty or incomplete. + +Cause: + +* Using `readline()` instead of full output read + +Fix: + +* Switched to `process.communicate()` + +--- + +### 4. Irrelevant Case Studies + +Tool returned multiple industries. + +Fix: + +* Filtered for healthcare-specific content + +--- + +### 5. Poor Summarization + +Splitting by sentences broke headings like `# S.M.I.L.E.` + +Fix: + +* Switched to simple truncation (`text[:400]`) + +--- + +## How I Solved Them + +* Read and understood JSON-RPC communication instead of guessing +* Used proper server instead of test client +* Implemented structured parsing for nested responses +* Added domain-specific filtering for relevance +* Simplified summarization instead of overengineering + +--- + +## What I Learned + +### Tool Integration Matters More Than Models + +The challenge wasn’t AI—it was correctly connecting and using tools. + +--- + +### More Data ≠ Better Output + +Raw tool output was too large and noisy. Filtering made answers significantly better. + +--- + +### Explainability Improves Quality + +Structuring output into sections made the agent more understandable and useful. + +--- + +### Debugging Is the Real Work + +Most time was spent fixing: + +* paths +* protocol issues +* parsing + +Not writing logic. + +--- + +### Simplicity Wins + +The final agent is simple: + +* 2 tools +* basic parsing +* structured output + +But it works reliably. + +--- + +## Final Thoughts + +This project was less about building a complex AI system and more about: + +* understanding how tools communicate +* extracting meaningful information +* presenting it clearly + +The biggest takeaway was that a good agent is not defined by complexity, but by: + +> how effectively it connects, filters, and explains information. + +--- diff --git a/submissions/Aryan/level2.md b/submissions/Aryan/level2.md new file mode 100644 index 00000000..d26af3e6 --- /dev/null +++ b/submissions/Aryan/level2.md @@ -0,0 +1,64 @@ +# Level 2 Submission — Aryan + +## LPI Sandbox Setup + +All tools executed successfully, confirming that the LPI sandbox is functioning correctly. The test client demonstrated a modular architecture where each tool exposes a specific capability. Instead of relying on a single LLM response, the system operates through structured tool calls, making the agent behavior more transparent and controllable. + +--- + +## Test Client Output + +=== LPI Sandbox Test Client === + +[LPI Sandbox] Server started — 7 read-only tools available +Connected to LPI Sandbox + +Available tools (7): + +- smile_overview +- smile_phase_detail +- query_knowledge +- get_case_studies +- get_insights +- list_topics +- get_methodology_step + +[PASS] smile_overview({}) +[PASS] smile_phase_detail({"phase":"reality-emulation"}) +[PASS] list_topics({}) +[PASS] query_knowledge({"query":"explainable AI"}) +[PASS] get_case_studies({}) +[PASS] get_case_studies({"query":"smart buildings"}) +[PASS] get_insights({"scenario":"personal health digital twin","tier":"free"}) +[PASS] get_methodology_step({"phase":"concurrent-engineering"}) + +=== Results === +Passed: 8/8 +Failed: 0/8 + +All tools are operational. The LPI Sandbox is ready for agent development. + +--- + +## Local LLM Setup (Ollama) + +**Model:** +qwen2.5:1.5b + +**Prompt:** +What is SMILE methodology? + +**Response (summary):** +The model described SMILE as a structured approach to managing the data lifecycle (creation, storage, access, deletion), emphasizing automation and governance. However, this interpretation is not grounded in a recognized or standardized framework. The response reflects generic data management concepts rather than a formally defined methodology. + +--- + +## Observation + +Running the model locally provides control over execution factors such as latency, hardware usage, and reproducibility. However, it does not expose internal reasoning—only observable outputs. This reinforces the need for external grounding (e.g., tools) rather than relying solely on model-generated explanations. + +--- + +## Reflection on SMILE Methodology + +The model’s response aligns with general principles of data lifecycle management and system design. However, attributing these ideas to a defined “SMILE methodology” is unsupported. It is more accurate to interpret this as a generic abstraction rather than a validated framework. diff --git a/submissions/Aryan/level3.md b/submissions/Aryan/level3.md new file mode 100644 index 00000000..92bdc6a9 --- /dev/null +++ b/submissions/Aryan/level3.md @@ -0,0 +1,133 @@ +# Level 3 Submission — Aryan + +## Project: Explainable Knowledge Agent (LPI) + +**Repository**: https://github.com/iamaryan07/lpi-life-agent + +**Agent**: https://github.com/iamaryan07/lpi-life-agent/blob/main/agent.py + +**A2A Card**: https://github.com/iamaryan07/lpi-life-agent/blob/main/agent.json + +--- + +## Overview + +This project implements a Level 3 agent using the Life Programmable Interface (LPI). + +The agent answers user queries by: + +* selecting appropriate tools based on the query +* retrieving structured knowledge from LPI +* combining multiple sources +* generating a structured, explainable response using an LLM (Qwen via Ollama) + +--- + +## Tools Used + +* `smile_overview` → provides SMILE methodology and conceptual framework +* `get_case_studies` → provides real-world digital twin implementations +* `get_methodology_step`, `get_insights` → used for implementation-focused queries + +--- + +## How It Works + +1. Accepts a user query +2. Selects tools based on query type (rule-based logic) +3. Sends JSON-RPC requests to LPI server (`dist/src/index.js`) +4. Retrieves structured outputs from multiple tools +5. Extracts and filters relevant content (e.g., healthcare-specific sections) +6. Combines tool outputs into a unified context +7. Uses an LLM (Qwen via Ollama) to generate a structured response + +--- + +## Key Features + +* Tool coordination across multiple LPI endpoints +* Query-based tool selection (simple reasoning logic) +* JSON-RPC communication with LPI via subprocess +* Context-aware filtering for domain-specific relevance (healthcare) +* LLM-based reasoning to combine methodology and real-world data +* Structured output format: + + * Understanding + * SMILE Phases + * Real-World Application + * Insight + * Conclusion + +--- + +## Design Decisions & Independent Thinking + +**Approach & Trade-offs:** +I used a simple rule-based tool selector instead of an LLM planner to keep tool usage predictable and avoid incorrect tool calls. This trades flexibility for reliability and easier debugging. + +**Choices Beyond Instructions:** + +* Added an LLM (Qwen via Ollama) to combine tool outputs into a structured answer instead of returning raw data +* Implemented filtering to extract only healthcare-relevant case study content +* Designed a strict prompt to reduce hallucination and enforce grounded responses +* Built custom parsing for nested JSON-RPC responses (`result → content → text`) + +**Learning:** +Combining tool outputs with controlled LLM reasoning is more important than just calling tools. + +--- + +## Example Query + +``` +How are digital twins used in healthcare? +``` + +--- + +## Example Output (Summary) + +* Explanation of digital twins in healthcare +* Relevant SMILE phases (e.g., Reality Emulation, Concurrent Engineering) +* Healthcare case study (continuous patient twin) +* Insight connecting methodology with real-world application +* Clear structured conclusion + +--- + +## Level 3 Criteria Met + +* ✔ Uses multiple tools in coordination +* ✔ Selects tools based on query type +* ✔ Integrates outputs from different sources +* ✔ Uses an LLM to synthesize and structure results +* ✔ Produces explainable, structured answers + +--- + +## Notes + +* Uses actual LPI server (`dist/src/index.js`) instead of test client +* Applies filtering to extract healthcare-relevant case study content +* Implemented using Python (agent) and Node.js (LPI server) + +--- + +## Reflection + +### What I did beyond the instructions + +* Implemented query-based tool selection instead of fixed tool usage +* Applied filtering logic to extract domain-specific (healthcare) insights +* Integrated an LLM (Qwen via Ollama) for reasoning instead of rule-based output +* Designed a structured prompt to enforce grounded, explainable responses +* Handled nested JSON-RPC parsing (`result → content → text`) + +### What I would improve next + +* Add explicit reasoning trace for better transparency +* Improve error handling for tool and LLM failures +* Support multi-step reasoning instead of single-pass generation +* Expand tool selection strategy for broader query coverage + +--- diff --git a/submissions/Aryan/level4/.well-known/agent.json b/submissions/Aryan/level4/.well-known/agent.json new file mode 100644 index 00000000..5f856a4c --- /dev/null +++ b/submissions/Aryan/level4/.well-known/agent.json @@ -0,0 +1,22 @@ +{ + "name": "LPI Multi-Agent System", + "description": "A Level 4 multi-agent system that uses expert and researcher agents to answer queries using LPI tools with orchestration and validation.", + "version": "1.0.0", + "authors": ["Aryan"], + "agents": [ + { + "id": "agent_a_expert", + "role": "Expert", + "description": "Provides structured explanations using SMILE methodology" + }, + { + "id": "agent_b_researcher", + "role": "Researcher", + "description": "Finds and extracts relevant case studies and knowledge" + } + ], + "orchestrator": { + "id": "orchestrator", + "description": "Coordinates agents and combines outputs into final answer" + } +} diff --git a/submissions/Aryan/level4/README.md b/submissions/Aryan/level4/README.md new file mode 100644 index 00000000..4bf247e2 --- /dev/null +++ b/submissions/Aryan/level4/README.md @@ -0,0 +1,120 @@ +# Level 4 — Secure Multi-Agent System (LPI) + +## Overview + +This project implements a **secure multi-agent system** using the Life Programmable Interface (LPI). + +The system answers user queries by: +- retrieving grounded knowledge using LPI tools +- reasoning strictly over that data +- enforcing security constraints to prevent misuse + +--- + +## Architecture + +The system consists of **two agents + orchestrator**: + +### 1. Agent B — Research Agent +- Calls LPI tools (`smile_overview`, `get_case_studies`) +- Extracts relevant data (healthcare-focused filtering) +- Returns structured grounding data + +### 2. Agent A — Reasoning Agent +- Takes grounded data from Agent B +- Uses a constrained prompt (no external knowledge) +- Generates a structured, explainable answer + +### 3. Orchestrator +- Handles user input +- Routes data between agents +- Applies security checks +- Produces final output + +--- + +## Data Flow + +User Query → Orchestrator → Agent B (LPI Tools) → Grounded Data (SMILE + Case Study) → Agent A (Reasoning) → Final Answer + +--- + +## Tools Used + +- `smile_overview` → SMILE methodology +- `get_case_studies` → real-world digital twin implementations + +--- + +## Security Features +Implemented in `security.py`: +### 1. Prompt Injection Protection +Blocks malicious inputs such as: +- "ignore previous instructions" +- "reveal system prompt" +### 2. Data Leak Prevention +Redacts sensitive outputs like: +- system prompts +- hidden instructions +- tool schemas +### 3. DoS Protection +Limits input size to prevent overload +### 4. Agent Validation +Ensures correct structure of inter-agent communication +--- + +## Key Design Decisions - + +**Strict grounding**: + +Agent A can only use data from Agent B - + +**No hallucination**: LLM constrained with hard rules - +**Tool-first architecture**: reasoning depends on LPI outputs - +**Security-first design**: all inputs and outputs are filtered + +--- + +## Example Query - How are digital twins used in healthcare? + +--- +## Example Output - + +- SMILE-based explanation +- Healthcare case study (continuous patient twin) +- Structured reasoning (phases, application, insight, conclusion) + +--- + +# How to Run + +## 1. Start LPI Server +```bash +npm run build +node dist/src/index.js +``` + +## 2. Start Ollama +Run the following command: +``` +ollama run qwen2.5:1.5b +``` +Or follow the detailed steps provided. + +--- + +### Details include: +- Starting the server, running Ollama, and executing the orchestrator script. +- The project structure is outlined: + - `level4/` directory contains key scripts and files. + - Security layer is implemented in `security.py`. + - Architecture includes agent scripts and orchestrator. +- Notes mention: + - Use of local LPI server and Ollama for reasoning. + - Filtering applied for relevance in healthcare context. +- Additional information about author and credibility: + - Setup matches actual code, clearly shows architecture, emphasizes security, and maintains credibility. + +### Final Advice: +- Encourage clarity, technical honesty, and avoiding buzzwords. +- A reminder that additional versions of documentation can be provided if needed. diff --git a/submissions/Aryan/level4/agent_a.py b/submissions/Aryan/level4/agent_a.py new file mode 100644 index 00000000..b692b5dd --- /dev/null +++ b/submissions/Aryan/level4/agent_a.py @@ -0,0 +1,146 @@ +import requests +from security import prevent_data_leak + + +# ---- LLM (same as your Level 3) ---- +def ask_llm(prompt): + try: + res = requests.post( + "http://localhost:11434/api/generate", + json={ + "model": "qwen2.5:1.5b", + "prompt": prompt, + "stream": False + } + ) + + data = res.json() + + if "response" in data: + return data["response"] + else: + return f"Unexpected LLM response: {data}" + + except Exception as e: + return f"LLM Error: {str(e)}" + + +# ---- AGENT A ---- +def run_agent_a(input_data): + user_query = input_data.get("query", "") + grounding = input_data.get("grounding_data", {}) + + smile_data = grounding.get("smile_data", "") + case_data = grounding.get("case_data", "") + + # ---- YOUR ORIGINAL PROMPT (slightly adapted) ---- + prompt = f""" + You are a STRICT reasoning agent using SMILE methodology and real case study data. + + ==================== + INPUT + ==================== + + User Query: + {user_query} + + SMILE Data: + {smile_data} + + Case Study Data: + {case_data} + + ==================== + MANDATORY RULES (NO EXCEPTIONS) + ==================== + + 1. You MUST use ONLY the provided SMILE Data and Case Study Data. + 2. You MUST NOT use general knowledge. + 3. You MUST NOT invent examples, systems, or explanations. + 4. If information is missing → explicitly write: "Not specified in data". + 5. You MUST NOT introduce any SMILE phase not present in SMILE Data. + + ==================== + ALLOWED SMILE PHASES + ==================== + + Reality Emulation + Concurrent Engineering + Collective Intelligence + Contextual Intelligence + + If a phase is not explicitly present in SMILE Data → DO NOT use it. + + ==================== + GROUNDING REQUIREMENTS + ==================== + + - Every claim MUST be traceable to SMILE Data or Case Study Data. + - You MUST reference the case study explicitly (name or scenario). + - Do NOT generalize beyond given data. + - Keep reasoning tight and evidence-based. + + ==================== + OUTPUT STRUCTURE (STRICT) + ==================== + + 1. Understanding: + Explain ONLY using SMILE Data (no external explanation). + + 2. Key SMILE Phases: + - Select ONLY 2–3 phases from allowed list + - Each phase must be justified using provided data + + 3. Real-World Application: + - Use ONLY the given case study + - Describe what was done (no assumptions) + + 4. Insight: + - Connect SMILE phases with the case study + - No generic statements + + 5. Conclusion: + - Short, grounded summary + + ==================== + STYLE CONSTRAINTS + ==================== + + - Be concise + - Avoid long explanations + - Avoid generic AI statements + - No extra information outside provided data + + ==================== + FINAL CHECK (MANDATORY) + ==================== + + Before answering, ensure: + - No invented SMILE phases + - No general knowledge used + - All statements trace back to input data + + If any rule is violated → correct internally before answering. + """ + + response = ask_llm(prompt) + response = prevent_data_leak(response) + + return { + "answer": response, + "source": "Agent A (SMILE reasoning)" + } + + +# ---- TEST ---- +if __name__ == "__main__": + sample_input = { + "query": "How are digital twins used in healthcare?", + "grounding_data": { + "smile_data": "SMILE methodology phases...", + "case_data": "Healthcare digital twin case study..." + } + } + + output = run_agent_a(sample_input) + print(output["answer"]) diff --git a/submissions/Aryan/level4/agent_b.py b/submissions/Aryan/level4/agent_b.py new file mode 100644 index 00000000..88e200e8 --- /dev/null +++ b/submissions/Aryan/level4/agent_b.py @@ -0,0 +1,133 @@ +import json +import subprocess +import os +from security import prevent_data_leak + + +# ---- PATH SETUP (same as your code) ---- +BASE_DIR = os.path.dirname(os.path.abspath(__file__)) +LPI_PATH = os.path.join(BASE_DIR, "..", "..", "dist", "src", "index.js") + +if not os.path.exists(LPI_PATH): + raise FileNotFoundError(f"LPI server not found at {LPI_PATH}") + + +# ---- CALL LPI TOOL (your logic, cleaned) ---- +def call_lpi_tool(tool_name, query): + try: + process = subprocess.Popen( + ["node", LPI_PATH], + stdin=subprocess.PIPE, + stdout=subprocess.PIPE, + stderr=subprocess.PIPE, + text=True, + encoding="utf-8" + ) + + # INIT + process.stdin.write(json.dumps({ + "jsonrpc": "2.0", + "method": "notifications/initialized" + }) + "\n") + + # ARGUMENT FIX (your special handling preserved) + if tool_name == "get_case_studies": + args = {"query": "healthcare digital twin"} + else: + args = {"query": query} + + # TOOL CALL + process.stdin.write(json.dumps({ + "jsonrpc": "2.0", + "method": "tools/call", + "params": { + "name": tool_name, + "arguments": args + }, + "id": 1 + }) + "\n") + + process.stdin.flush() + + stdout, _ = process.communicate(timeout=10) + + if not stdout.strip(): + return "" + + # ---- PARSE OUTPUT (same logic, but forward scan instead of reverse) ---- + for line in stdout.split("\n"): + try: + parsed = json.loads(line) + + if "result" in parsed: + result = parsed["result"] + + if isinstance(result, dict) and "content" in result: + content = result["content"] + + if isinstance(content, list) and content: + text = content[0].get("text", "") + + # ---- CASE STUDY FILTER (your idea preserved) ---- + if tool_name == "get_case_studies": + parts = text.split("## ") + for part in parts: + if "health" in part.lower(): + return "## " + part[:1200] + + return text + + return str(result) + + except: + continue + + return "" + + except Exception as e: + return f"Error calling {tool_name}: {str(e)}" + + +# ---- TOOL SELECTION (same as your Level 3) ---- +def choose_tools(query): + q = query.lower() + + if "how" in q or "use" in q: + return ["smile_overview", "get_case_studies"] + elif "implement" in q or "steps" in q: + return ["get_methodology_step", "get_insights"] + else: + return ["query_knowledge", "get_case_studies"] + + +# ---- AGENT B ---- +def run_agent_b(input_data): + query = input_data.get("query", "") + + # ---- SELECT TOOLS ---- + tool1, tool2 = choose_tools(query) + + # ---- CALL TOOLS ---- + data1 = call_lpi_tool(tool1, query) + data2 = call_lpi_tool(tool2, query) + + # ---- SECURITY FILTER ---- + data1 = prevent_data_leak(data1) + data2 = prevent_data_leak(data2) + + # ---- STRUCTURED OUTPUT ---- + return { + "smile_data": data1, + "case_data": data2, + "sources": [tool1, tool2] + } + + +# ---- TEST ---- +if __name__ == "__main__": + result = run_agent_b({ + "query": "How are digital twins used in healthcare?" + }) + + print("\n=== AGENT B OUTPUT ===\n") + print(json.dumps(result, indent=2)) diff --git a/submissions/Aryan/level4/demo.md b/submissions/Aryan/level4/demo.md new file mode 100644 index 00000000..926de511 --- /dev/null +++ b/submissions/Aryan/level4/demo.md @@ -0,0 +1,92 @@ +# Demo — Level 4 Secure Multi-Agent System (LPI) + +## Objective + +Demonstrate a secure multi-agent system that: +- Uses LPI tools for grounded data retrieval +- Performs constrained reasoning +- Prevents prompt injection and data leakage +- Produces structured, explainable answers + +--- + +## System Components + +### 1. Orchestrator +- Accepts user query +- Applies security checks +- Coordinates agents + +### 2. Agent B (Research Agent) +- Calls LPI tools: + - `smile_overview` + - `get_case_studies` +- Extracts relevant data (healthcare-focused) +- Returns structured grounding data + +### 3. Agent A (Reasoning Agent) +- Uses ONLY grounded data +- Applies strict rules (no hallucination) +- Generates structured answer + +### 4. Security Layer +- Input sanitization +- Prompt injection blocking +- Data leak prevention +- Input length validation + +--- +# Demo Flow + +### Step 1 — User Input +```text +How are digital twins used in healthcare? +``` + +### Step 2 — Security Validation + +Input is checked for: +- Prompt injection patterns +- Excessive length + +If malicious → request is blocked. + +### Step 3 — Agent B (Research) + +Tools Selected: +- smile_overview +- get_case_studies + +Output: +- SMILE methodology data +- Healthcare case study (Continuous Patient Twin) + +### Step 4 — Agent A (Reasoning) + +Agent A receives: +- SMILE framework +- Healthcare case study + +Enforced Constraints: +- No external knowledge +- No invented phases +- Only grounded reasoning + +### Step 5 — Final Output +**Understanding:** +Digital twins are implemented using the SMILE methodology, which focuses on impact-driven lifecycle modeling. + +**SMILE Phases:** +- Reality Emulation +- Concurrent Engineering +- Collective Intelligence +- Contextual Intelligence + +**Real-World Application:** +*(Healthcare case study output here)* + +**Insight:** +Connection between SMILE phases and real-world implementation. + +**Conclusion:** +Short, grounded summary based on provided data. diff --git a/submissions/Aryan/level4/orchestrator.py b/submissions/Aryan/level4/orchestrator.py new file mode 100644 index 00000000..c3c6b9e9 --- /dev/null +++ b/submissions/Aryan/level4/orchestrator.py @@ -0,0 +1,68 @@ +import json +from agent_a import run_agent_a +from agent_b import run_agent_b + +# ---- SECURITY ---- +from security import sanitize_input, validate_length, prevent_data_leak + + +# ---- ORCHESTRATOR ---- +def run_system(): + print("=== ORCHESTRATOR START ===\n") + + # ---- USER INPUT (same as your original) ---- + try: + user_query = input("Enter your question: ") + user_query = sanitize_input(user_query) + user_query = validate_length(user_query) + except ValueError as e: + print(f"[SECURITY BLOCKED]: {e}") + return + + # ---- STEP 1: AGENT B (RESEARCH) ---- + print("\n[Step 1] Researching...\n") + + try: + grounding_output = run_agent_b({ + "query": user_query + }) + except Exception as e: + print(f"[Agent B Error]: {e}") + return + + # ---- SANITIZE OUTPUT ---- + grounding_output = { + "smile_data": prevent_data_leak(grounding_output.get("smile_data", "")), + "case_data": prevent_data_leak(grounding_output.get("case_data", "")), + "sources": grounding_output.get("sources", []) + } + + print("[Agent B Output Ready]\n") + + # ---- STEP 2: AGENT A (REASONING) ---- + print("[Step 2] Generating answer...\n") + + try: + final_output = run_agent_a({ + "query": user_query, + "grounding_data": grounding_output + }) + except Exception as e: + print(f"[Agent A Error]: {e}") + return + + # ---- FINAL OUTPUT ---- + answer = prevent_data_leak(final_output.get("answer", "")) + + print("----- FINAL ANSWER -----\n") + print(answer) + + # ---- SOURCES (same as your Level 3 requirement) ---- + print("\n----- SOURCES -----") + for src in grounding_output.get("sources", []): + print(f"- {src}") + + +# ---- RUN ---- +if __name__ == "__main__": + run_system() diff --git a/submissions/Aryan/level4/security.py b/submissions/Aryan/level4/security.py new file mode 100644 index 00000000..4c83c9b2 --- /dev/null +++ b/submissions/Aryan/level4/security.py @@ -0,0 +1,65 @@ +import re + + +# ---- 1. PROMPT INJECTION DEFENSE ---- +def sanitize_input(text): + blocked_patterns = [ + "ignore previous instructions", + "system prompt", + "reveal hidden", + "bypass", + "override", + "act as", + "jailbreak" + ] + + text_lower = text.lower() + + for pattern in blocked_patterns: + if pattern in text_lower: + raise ValueError(f"Blocked malicious pattern: {pattern}") + + return text + + +# ---- 2. DATA EXFILTRATION DEFENSE ---- +def prevent_data_leak(text): + sensitive_keywords = [ + "system prompt", + "internal instructions", + "hidden policy", + "tool schema" + ] + + text_lower = text.lower() + + for keyword in sensitive_keywords: + if keyword in text_lower: + return "[REDACTED: Sensitive content blocked]" + + return text + + +# ---- 3. DOS DEFENSE ---- +def validate_length(text, max_length=500): + if not isinstance(text, str): + raise ValueError("Invalid input type") + + if len(text) > max_length: + raise ValueError("Input too long — possible DoS attack") + + return text + + +# ---- 4. BASIC AGENT VALIDATION ---- +def validate_agent_call(data): + if not isinstance(data, dict): + raise ValueError("Invalid agent input format") + + if "grounding_data" not in data: + raise ValueError("Missing grounding data") + + if not isinstance(data["grounding_data"], dict): + raise ValueError("Invalid grounding data structure") + + return True diff --git a/submissions/Aryan/level4/security_audit.md b/submissions/Aryan/level4/security_audit.md new file mode 100644 index 00000000..af4fc1a2 --- /dev/null +++ b/submissions/Aryan/level4/security_audit.md @@ -0,0 +1,140 @@ +# Security Audit — Level 4 Multi-Agent System + +## Overview + +This document outlines the security mechanisms implemented in the multi-agent system built using the Life Programmable Interface (LPI). + +The system is designed to: +- Prevent prompt injection attacks +- Avoid data leakage +- Ensure safe inter-agent communication +- Maintain strict grounding of outputs + +--- + +## System Attack Surface + +The system has three primary exposure points: + +1. **User Input (Orchestrator)** +2. **Tool Outputs (Agent B → Agent A)** +3. **LLM Output (Agent A)** + +Each layer is protected with targeted defenses. + +--- +# 1. Prompt Injection Protection + +### Risk +Malicious users may attempt to override instructions, e.g.: +- "ignore previous instructions" +- "reveal system prompt" + +### Mitigation +Implemented in `sanitize_input()`: +- Blocks known injection patterns: + - ignore previous instructions + - system prompt + - override + - jailbreak + - reveal hidden +- Raises exception if detected +- Prevents malicious queries from reaching agents +### Result +Prevents accidental leakage of internal system details + +--- + +## 3. Denial-of-Service (DoS) Protection + +### Risk +Large inputs can overload the system or LLM. + +### Mitigation +Implemented in `validate_length()`: +- Limits input size (default: 500 characters) +- Rejects oversized inputs + +### Result +Protects the system from resource exhaustion + +--- + +## 4. Inter-Agent Data Validation + +### Risk +Malformed or manipulated data between agents can break logic or introduce vulnerabilities. + +### Mitigation +Implemented in `validate_agent_call()`: +- Ensures input is a dictionary +- Ensures `grounding_data` exists +- Validates correct structure + +### Result +Maintains integrity of agent communication + +--- + +## 5. Grounding Enforcement (Anti-Hallucination) + +### Risk +LLM may generate: +- Fabricated SMILE phases +- Unsupported claims + +### Mitigation +Agent A enforces strict rules: +- Only uses data from Agent B +- No external knowledge allowed +- Missing data → explicitly stated + +### Result +All outputs are traceable to tool data + +--- + +## 6. Tool Output Filtering + +### Risk +LPI tools may return excessive or irrelevant data. + +### Mitigation (Agent B) +- Extracts only: + - SMILE overview + - Healthcare-related case study +- Filters irrelevant sections + +### Result +- Reduces noise +- Improves precision +- Limits unintended exposure + +--- + +## Security Guarantees + +The system ensures: +- ✔ No prompt injection execution +- ✔ No sensitive data leakage +- ✔ Controlled input size +- ✔ Valid inter-agent communication +- ✔ Grounded, verifiable outputs + +--- + +## Limitations + +- Keyword-based filtering may not catch all advanced attacks +- Relies on predefined patterns (not adaptive) +- Does not include authentication or rate limiting + +--- + +## Future Improvements + +- Semantic prompt injection detection +- Role-based access control +- Rate limiting and request throttling +- Output verification using a secondary agent +- Structured schema validation for all tool outputs diff --git a/submissions/Aryan/level4/threat_model.md b/submissions/Aryan/level4/threat_model.md new file mode 100644 index 00000000..146e763d --- /dev/null +++ b/submissions/Aryan/level4/threat_model.md @@ -0,0 +1,207 @@ +# Threat Model — Level 4 Secure Multi-Agent System + +## Overview + +This document outlines the threat model for the multi-agent system built using the Life Programmable Interface (LPI). + +The system consists of: +- User input interface (orchestrator) +- Agent B (tool-based data retrieval) +- Agent A (LLM-based reasoning) +- Security layer (input/output validation) + +The goal is to identify potential threats and describe how they are mitigated. + +--- + +## System Assets + +The following assets must be protected: + +- User queries +- LPI tool outputs (SMILE data, case studies) +- Internal prompts and system instructions +- Agent communication data (`grounding_data`) +- Final generated responses + +--- + +## Trust Boundaries + +1. **User → Orchestrator** + - Untrusted input enters the system + +2. **Orchestrator → Agent B** + - Tool execution boundary + +3. **Agent B → Agent A** + - Data transfer between agents + +4. **Agent A → Output** + - LLM-generated content exposed to user + +--- + +## Threats & Mitigations + +--- + +### 1. Prompt Injection + +#### Threat +User attempts to override system behavior: +- "ignore previous instructions" +- "reveal system prompt" + +#### Impact +- Compromised reasoning +- Exposure of internal logic + +#### Mitigation +- `sanitize_input()` blocks known malicious patterns +- Input rejected before reaching agents + +#### Residual Risk +- Advanced or obfuscated attacks may bypass keyword filters + +--- + +### 2. Data Exfiltration + +#### Threat +Sensitive information leaked via: +- LLM output +- Tool responses + +#### Impact +- Exposure of internal prompts or system details + +#### Mitigation +- `prevent_data_leak()` scans and redacts sensitive keywords +- Applied at: + - Agent B output + - Agent A output + - Final response + +#### Residual Risk +- Contextual leaks not matching keywords may pass + +--- + +### 3. Hallucination / Ungrounded Output + +#### Threat +LLM generates: +- Incorrect SMILE phases +- Unsupported claims + +#### Impact +- Misinformation +- Loss of system reliability + +#### Mitigation +- Agent A uses strict grounding rules: + - Only uses Agent B data + - No external knowledge + - Missing info explicitly stated + +#### Residual Risk +- LLM may still misinterpret structured data + +--- + +### 4. Denial of Service (DoS) + +#### Threat +Large or malformed inputs overwhelm system + +#### Impact +- Resource exhaustion +- System slowdown or crash + +#### Mitigation +- `validate_length()` limits input size (500 chars) + +#### Residual Risk +- Repeated valid requests may still cause load + +--- + +### 5. Tool Misuse / Overreach + +#### Threat +Agent B retrieves: +- Irrelevant +- Excessive +- Unfiltered data + +#### Impact +- Noise in reasoning +- Potential data exposure + +#### Mitigation +- Filters only: + - SMILE overview + - Healthcare-related case study + +#### Residual Risk +- Partial irrelevant data may still pass + +--- + +### 6. Inter-Agent Data Tampering + +#### Threat +Malformed or manipulated data passed between agents + +#### Impact +- Broken logic +- Incorrect outputs + +#### Mitigation +- `validate_agent_call()` ensures: + - Correct structure + - Required fields present + +#### Residual Risk +- Does not verify semantic correctness + +--- + +## Security Assumptions + +- LPI tools are trusted and not malicious +- Ollama model runs locally and is not compromised +- No external API exposure + +--- + +## Security Guarantees + +The system ensures: + +- ✔ Input sanitization before processing +- ✔ Controlled data flow between agents +- ✔ No leakage of sensitive system data +- ✔ Grounded and verifiable outputs +- ✔ Limited attack surface through filtering + +--- + +## Limitations + +- Keyword-based defenses are not fully robust +- No authentication or user identity verification +- No rate limiting +- No encryption between components + +--- + +## Future Improvements + +- Semantic prompt injection detection +- Rate limiting and request throttling +- Authentication and access control +- Output validation using a secondary agent +- Schema validation for all tool outputs +- Logging and monitoring for anomaly detection