feat(agent-os): add MCP security primitives with OWASP coverage#822
feat(agent-os): add MCP security primitives with OWASP coverage#822jackbatzner wants to merge 1 commit intomicrosoft:mainfrom
Conversation
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
Welcome to the Agent Governance Toolkit! Thanks for your first pull request. |
🤖 AI Agent: breaking-change-detector — Summary🔍 API Compatibility ReportSummaryThis pull request introduces several new features and components to the Findings
Details🔵 New Public API
Migration GuideNo migration steps are necessary as no breaking changes were identified. Downstream users can adopt the new functionality without modifying existing code. Conclusion✅ No breaking changes detected. This pull request is safe to merge from an API compatibility perspective. |
🤖 AI Agent: docs-sync-checker — Issues Found📝 Documentation Sync ReportIssues Found
Suggestions
Additional Notes
Action RequiredPlease address the missing docstrings, update the README, and add a CHANGELOG entry to ensure documentation is in sync with the new functionality. |
There was a problem hiding this comment.
🤖 AI Agent: code-reviewer
Review of feat(agent-os): add MCP security primitives with OWASP coverage
This PR introduces a significant set of security primitives for the agent-os package, implementing a robust framework for MCP (Model Context Protocol) governance. The changes are extensive and touch on multiple aspects of the system, including policy enforcement, session management, message signing, input/output scanning, rate limiting, and credential redaction. Below is a detailed review of the PR, focusing on the specified areas of concern:
🔴 CRITICAL
-
Credential Redaction and Logging
- The
CredentialRedactorlogs the number of redactions applied (logger.info("Credential redaction applied to %s value(s)", redaction_count)). While this is useful for debugging, it could inadvertently leak sensitive information in logs if the redacted content is logged elsewhere. Ensure that logging is sanitized and does not expose sensitive data. - Action: Add a warning in the documentation and code comments to ensure that sensitive data is not logged elsewhere in the application.
- The
-
HMAC-SHA256 Key Management
- The
MCPMessageSigneruses HMAC-SHA256 for message signing. However, the PR does not include details about how the signing keys are generated, stored, or rotated. Improper key management can lead to severe security vulnerabilities. - Action: Ensure that signing keys are securely generated, stored, and rotated. Consider integrating with a secure key management service (e.g., Azure Key Vault).
- The
-
Fail-Closed Mechanism
- While the PR mentions that the system is designed to "fail-closed," there is no explicit test coverage to verify this behavior. For example, the
_evaluatemethod inMCPGatewayhas atry-exceptblock to handle unexpected errors, but the test suite does not seem to include cases that simulate such errors. - Action: Add test cases to explicitly verify that the system fails closed under various failure scenarios, such as exceptions during policy evaluation or rate-limiting checks.
- While the PR mentions that the system is designed to "fail-closed," there is no explicit test coverage to verify this behavior. For example, the
-
Thread Safety
- The PR mentions the use of
threading.Lockfor thread safety in the rate limiter and session authenticator. However, there is no evidence of tests that validate the thread safety of these components under concurrent execution. - Action: Add stress tests to simulate concurrent access to shared resources (e.g., rate limit counters, session stores) and verify that the locks are functioning as intended.
- The PR mentions the use of
-
Nonce Management
- The PR introduces
MCPNonceStorefor nonce management but does not provide details on how nonce uniqueness is enforced across distributed systems. This could lead to replay attacks if the same nonce is reused. - Action: Ensure that nonce generation and storage are designed to prevent collisions, even in distributed environments. Consider using a distributed key-value store like Redis for nonce storage.
- The PR introduces
🟡 WARNING
-
Backward Compatibility
- The PR adds a significant number of new classes and methods to the
agent-ospackage. While these additions are non-breaking, they increase the surface area of the public API. - Action: Clearly document all new public APIs and their intended usage. Consider marking new APIs as "experimental" if they are subject to change in future releases.
- The PR adds a significant number of new classes and methods to the
-
OpenTelemetry Integration
- The
MCPMetricsclass introduces optional OpenTelemetry integration. However, the PR does not include any tests to verify the behavior when OpenTelemetry is unavailable or misconfigured. - Action: Add test cases to validate the behavior of
MCPMetricsin environments where OpenTelemetry is not installed or fails to initialize.
- The
💡 SUGGESTIONS
-
Input Validation
- The
CredentialRedactorclass includes aPATTERNSattribute with predefined regex patterns for detecting credentials. While these patterns are comprehensive, they may not cover all possible formats of sensitive data. - Suggestion: Allow users to extend or override the default patterns via configuration or constructor arguments.
- The
-
Performance Optimization
- The
CredentialRedactorapplies all patterns sequentially, which could be inefficient for large inputs. While the current implementation is acceptable, consider optimizing the pattern matching process using a single combined regex or a more efficient algorithm. - Suggestion: Evaluate the performance of the redaction process with large inputs and optimize if necessary.
- The
-
Documentation
- The PR introduces several new classes and methods, but the accompanying documentation is minimal. For example, the
MCPGatewayclass has a docstring, but it does not explain all the configuration options in detail. - Suggestion: Expand the documentation to include examples and detailed explanations of each new class and method. This will help users understand how to integrate these features into their applications.
- The PR introduces several new classes and methods, but the accompanying documentation is minimal. For example, the
-
Test Coverage
- While the PR mentions that 2938 tests passed, it also notes that 86 tests were skipped. It's unclear why these tests were skipped.
- Suggestion: Investigate the skipped tests and ensure they are either fixed or explicitly marked as expected to be skipped with a clear explanation.
-
Error Handling
- The
_evaluatemethod inMCPGatewaycatches all exceptions and logs an error message. While this is a good fail-safe mechanism, it may mask underlying issues. - Suggestion: Log the specific exception message and type to aid in debugging.
- The
-
Standards Compliance
- The PR claims compliance with 11 of the 12 sections of the OWASP MCP Security Cheat Sheet. However, there is no evidence of a systematic mapping between the implemented features and the OWASP guidelines.
- Suggestion: Include a detailed mapping in the documentation to demonstrate how each section of the OWASP MCP Security Cheat Sheet is addressed.
Summary
This PR is a significant step forward in enhancing the security posture of the agent-os package. However, there are several critical areas that need to be addressed to ensure the robustness and reliability of the new features. The recommendations above should be implemented to mitigate potential security risks, ensure backward compatibility, and improve the overall quality of the codebase.
- Critical Issues: 5
- Warnings: 2
- Suggestions: 6
Please address the critical issues and warnings before merging this PR. The suggestions can be addressed in subsequent iterations if necessary.
🤖 AI Agent: security-scanner — Security Review of PR: `feat(agent-os): add MCP security primitives with OWASP coverage`Security Review of PR:
|
| Category | Risk | Description | Recommendation |
|---|---|---|---|
| Prompt Injection Defense Bypass | 🔴 CRITICAL | Regex patterns in CredentialRedactor may fail to detect obfuscated credentials or injections. |
Expand regex patterns, integrate prompt injection detection, or use a specialized library. |
| Policy Engine Circumvention | 🔴 CRITICAL | Lack of a "catch-all" check in _evaluate and potential for misconfigured GovernancePolicy. |
Add a catch-all check, validate GovernancePolicy, and consider integrity checks. |
| Trust Chain Weaknesses | 🟠 HIGH | No details on HMAC key management or rotation in MCPMessageSigner. |
Implement secure key management and key rotation. |
| Credential Exposure | 🔴 CRITICAL | Logging redaction counts could leak information about sensitive data. | Avoid logging redaction counts or secure log storage. |
| Sandbox Escape | 🔵 LOW | No explicit sandboxing mechanisms are mentioned. | Consider integrating sandboxing mechanisms. |
| Deserialization Attacks | 🟠 HIGH | Lack of input validation and potential unsafe deserialization. | Use safe deserialization libraries and validate inputs. |
| Race Conditions | 🟡 MEDIUM | Potential race conditions in _agent_call_counts dictionary. |
Use thread-safe data structures and add synchronization. |
| Supply Chain | 🟠 HIGH | New dependency (opentelemetry) not verified for integrity or vulnerabilities. |
Use dependency management tools and audit dependencies. |
General Recommendations
- Perform a comprehensive security review of the
GovernancePolicyobject to ensure it cannot be misconfigured or tampered with. - Add unit tests for edge cases, including malformed inputs, concurrent access, and policy misconfigurations.
- Document all security mechanisms, including key management, sandboxing, and dependency management, in the project documentation.
This PR introduces critical security features but also has several areas that require immediate attention to ensure the robustness of the MCP security primitives.
🤖 AI Agent: test-generator — `packages/agent-os/src/agent_os/__init__.py`🧪 Test Coverage Analysis
|
Description
Adds core MCP (Model Context Protocol) security primitives to the Python agent-os package, implementing coverage for 11 of 12 sections of the OWASP MCP Security Cheat Sheet.
MCP Security Primitives:
Enterprise-Grade Patterns:
mcp_protocols.py) for sessions, nonces, rate limits, and audit with in-memory defaults, swappable to Redis/DBtime.time()oruuid.uuid4(); all injected via constructor for deterministic testingthreading.Lockon mutable state in rate limiter and session authenticatorTests: 2938 passed, 86 skipped
OWASP MCP Security Cheat Sheet Coverage:
§1 Tool Poisoning ✅ | §2 Rug Pull ✅ | §3 Tool Shadowing ✅ | §4 Indirect Prompt Injection ✅ | §5 Server Compromise ✅ | §6 Credential Theft ✅ | §7 Resource Exhaustion ✅ | §8 Logging & Monitoring ✅ | §9 Privilege Escalation ✅ | §10 Consent Phishing ✅ | §11 Consent UI (N/A — server-side SDK) | §12 Standards Compliance ✅
Type of Change
Package(s) Affected
Checklist
Related Issues
Supersedes #774 (split for easier review)