[Bounty $8k] fix(runtime): fail closed when policy engine is unavailable (fixes #3592)

[Karry2019web]

Closes #3592

Summary

Add a PolicyEngine class that evaluates whether an operation should be allowed or denied. When the policy engine is unavailable (network failure, service down, etc.), all operations fail closed — denied by default — to maintain security posture.

Changes

• New src/common/policy.py — PolicyEngine class with:
  • is_available() — cached health check against policy backend
  • evaluate(action, resource, context) — returns PolicyDecision.ALLOW, DENY, or ERROR
  • Fail-closed: when the engine is unreachable, returns DENY
• Modified src/orchestrator/engine.py — OrchestrationEngine now:
  • Accepts an optional policy_engine parameter (defaults to a no-endpoint engine that allows everything)
  • Gated _execute_task through self.policy.evaluate() before any work begins
  • Policy deny triggers on_error hooks and returns immediately
• New tests/test_policy.py — 7 tests covering:
  • Default allow when no endpoint configured
  • Fail-closed deny when engine is unavailable
  • Health check caching behavior
  • Integration with OrchestrationEngine

Acceptance Criteria

• Tests cover denied execution and engine unavailability
• The worker records one durable terminal outcome and does not leave orphaned work
• Retry behavior is bounded (policy deny stops execution immediately)

Testing

pytest tests/test_policy.py -v