Skip to content

feat(guardion): Guardion middleware + replace x-portkey-* with x-guardion-* headers #1

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
rafaelsandroni wants to merge 1 commit into
base: 2.0.0
Choose a base branch
from
Open

feat(guardion): Guardion middleware + replace x-portkey-* with x-guardion-* headers #1

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 8 additions & 8 deletions Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,8 +29,8 @@ COPY --chown=node:node . .

# Build the application and clean up
RUN npm run build \
&& rm -rf node_modules \
&& npm install --omit=dev
&& rm -rf node_modules \
&& npm install --omit=dev

# Use the official Node.js runtime as a parent image
FROM node:24-alpine
Expand Down Expand Up @@ -64,13 +64,13 @@ RUN echo '#!/bin/sh' > /app/entrypoint.sh && \
USER node

# ENV variables
ENV SOURCE_SYNC_API_BASEPATH="https://api.portkey.ai/v1/sync"
ENV CONTROL_PLANE_BASEPATH="https://api.portkey.ai/v1"
ENV ALBUS_BASEPATH="https://albus.portkey.ai"
ENV SOURCE_SYNC_API_BASEPATH="https://api.guardion.ai/v1/sync"
ENV CONTROL_PLANE_BASEPATH="https://api.guardion.ai/v1"
ENV ALBUS_BASEPATH="https://api.guardion.ai"
ENV NODE_ENV="production"
ENV PORT=${PORT:-8787}
ENV MCP_PORT=${MCP_PORT:-8788}
ENV CONFIG_READER_PATH="https://api.portkey.ai/model-configs"
ENV PORT=${PORT:-8788}
ENV MCP_PORT=${MCP_PORT:-8789}
ENV CONFIG_READER_PATH="https://api.guardion.ai/model-configs"
ENV MODEL_CONFIGS_PROXY_FETCH_ENABLED="true"

# Expose the port your app runs on
Expand Down
198 changes: 198 additions & 0 deletions docs/PRD-DYNAMIC-CONFIG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,198 @@
# PRD: Dynamic Cloud Configuration & Guardrails for GuardionGateway

> **Last Updated**: 2026-03-03
> **Implementation Status**: Core MVP **complete** — re-validated against updated upstream Portkey structure.
> See [§6 Implementation Status](#6-implementation-status) for item-by-item tracking.

## 1. Overview

The GuardionGateway is a fork of the Portkey AI Gateway. It dynamically fetches per-tenant configurations, provider API keys, and guardrail policies from the Guard API (`/v1/config`), validates incoming tokens, and evaluates policies via a native Portkey Plugin — eliminating the need for end-users to send `x-portkey-config` headers.

## 2. Goals
- Validate `x-guardion-api-key` or `Authorization: Bearer <token>` on every `/v1/*` request.
- Fetch project/org configurations from `GET /v1/config` and translate them to Portkey's internal format.
- Bypass Portkey's `authN` middleware smoothly by spoofing a synthetic `OrganisationDetails` context object.
- Evaluate guardrail policies via a native plugin at `src/plugins/guardion/guardrails.ts`.
- Push telemetry logs asynchronously to the Guard API after each request.
- Remain easy to rebase onto the upstream Portkey main branch.

## 3. Updated Architecture (Post Upstream Sync)

### 3.1 Key Structural Changes from Upstream

The upstream Portkey repository has been significantly evolved. The following changes affect Guardion integration:

| Change | Impact on Guardion |
|---|---|
| `plugins/index.ts` removed | Plugins now live at `src/plugins/<provider>/` and are registered via `src/plugins/build.ts` |
| `src/middlewares/log/index.ts` removed | Logging now flows through the `portkey()` middleware and `WinkyLogObject` in `portkey/types.ts` |
| New `portkey()` middleware wraps the core pipeline | Guardrail hooks, logging, and context enrichment now happen inside `src/middlewares/portkey/` |
| `authNMiddleWare` + `authZMiddleWare` added | These validate Portkey API keys. Guardion bypasses them by injecting a synthetic `OrganisationDetails` via `setContext` |
| Unified `requestCache` service | Replaces `CacheService`. Guardion's auth middleware updated to use `requestCache(env(c))` |

### 3.2 Guardion File Locations (Current)

| File | Location | Role |
|---|---|---|
| Auth middleware | `src/middlewares/guardion/index.ts` | Token extraction, config fetch, cache, context injection |
| Config mapper | `src/middlewares/guardion/mapper.ts` | Guard API → Portkey schema; also creates spoofed org context |
| Telemetry | `src/middlewares/guardion/telemetry.ts` | Buffered async flush to Guard API — standalone, no upstream dependency |
| Guardrails plugin | `src/plugins/guardion/guardrails.ts` | Native plugin: session lock, eval, mode→status, tool policy |
| Session cache | `src/plugins/guardion/sessionCache.ts` | Per-session flag/block counters and lock state |
| Tool policy | `src/plugins/guardion/toolPolicy.ts` | Allowlist/denylist enforcement on tool definitions and invocations |

### 3.3 Upstream Touch Points (Minimal, 2 Lines)

| File | What we added | Risk |
|---|---|---|
| `src/index.ts` | `app.use('*', guardionAuth)` before `authNMiddleWare()` | 🟡 Medium |
| `src/plugins/build.ts` *(planned)* | Register `guardion.guardrails` in the plugin registry | 🟢 Low |

> Telemetry is now fully extracted to `src/middlewares/guardion/telemetry.ts` — zero lines in upstream logging files.

### 3.4 Core Request Flow

```
Client Request
↓
app.use('*', guardionAuth) ← [Guardion] fetch /v1/config, inject x-portkey-config, spoof OrgDetails
↓
app.use('*', authNMiddleWare()) ← [Portkey] reads OrgDetails from context — uses our spoofed object
↓
app.use('*', portkey()) ← [Portkey] reads x-portkey-config → activates hooks, cache, retry
↓
hooks (beforeRequestHook) ← [Guardion Plugin] session lock check, Guard API eval, tool policy
↓
LLM Provider (OpenAI, Anthropic …)
↓
hooks (afterRequestHook) ← [Guardion Plugin] output eval
↓
portkey() logs → pushTelemetryLog() ← [Guardion] async buffered flush to Guard API /v1/telemetry/logs
```

## 4. Guard API Config Schema

### 4.1 Request (`GET /v1/config`)
```http
Authorization: Bearer <API_KEY>
x-guardion-application: <APP_ID> # Optional — scopes config to a specific application
x-guardion-policy: <POLICY_ID> # Optional — scopes to a specific policy
```

### 4.2 Response (200 OK)
```json
{
"success": true,
"data": {
"organization_id": "org_abc",
"project_id": "prj_xyz",
"routing": {
"mode": "fallback",
"targets": [{ "provider": "openai", "api_key": "sk-...", "weight": 1 }]
},
"features": { "cache": true, "retry": { "attempts": 3 } },
"guardrails": {
"input": [{ "policy_id": "pol_abc1", "mode": "intercept", "action": "redact" }],
"output": [{ "policy_id": "pol_xyz1", "mode": "monitor", "action": "flag" }],
"rules": {
"flag_threshold": { "count": 5, "window_ms": 600000, "consequence": { "action": "block_session", "duration_ms": 3600000 } }
},
"tool_policy": { "mode": "allowlist", "allowed_tools": ["web_search"], "on_violation": "block" }
}
}
}
```

After mapping, each guardrail entry gets `provider: 'guardion'` and `type: 'guardrails'` injected, binding it to the native plugin.

## 5. Guardrails Plugin (`src/plugins/guardion/guardrails.ts`)

### 5.1 Policy Action Model

| Concept | Values | Description |
|---|---|---|
| **Policy Mode** (config) | `monitor` \| `intercept` | What the policy is allowed to do |
| **Status** (outcome / telemetry) | `flag` \| `block` \| `redact` | What actually happened |

**Decision logic:**
```
mode=monitor + flagged → status: 'flag', verdict: true
mode=intercept + flagged
+ correction available → status: 'redact', verdict: true (content mutated)
+ no correction → status: 'block', verdict: false
session locked → status: 'block', scope: 'session', verdict: false (skip eval)
Guard API down → fail open: verdict: true
```

### 5.2 Session Behavior Rules
- Per `(apiKey, applicationId)` state stored in `sessionCache.ts`.
- Tracks `flagEvents[]` and `blockEvents[]` as rolling timestamp windows.
- If `flag_threshold` or `block_threshold` is exceeded, session is locked for the configured `duration_ms`.
- All requests from a locked session return `verdict: false` immediately without calling the Guard API.

### 5.3 Tool Call Policy
- **`beforeRequestHook`**: Checks `request.json.tools[]` declarations against allowlist/denylist. Strips forbidden tools (allowlist mode) or blocks.
- **`afterRequestHook`**: Checks `choices[0].message.tool_calls[]` for violations.

## 6. Telemetry (`src/middlewares/guardion/telemetry.ts`)

- **Fully extracted** to its own file — no upstream file dependency.
- Receives the Portkey `WinkyLogObject` (from `portkey()` middleware) and enriches it with guardrail hook results.
- Buffers up to 50 logs, flushing every 5 seconds or on limit, to `POST {GUARD_API_URL}/v1/telemetry/logs`.
- Fire-and-forget: never blocks the main request.

Telemetry payload `guardrails` field includes:
```json
{
"status": "blocked",
"scope": "request",
"applied_policies": [{ "policy_id": "pol_abc1", "mode": "intercept", "status": "block" }],
"breakdown": [{ "detector": "pii-v1", "detected": true, "score": 0.94, "status": "block" }],
"session_lock": { "active": false },
"tool_policy": { "violations": [], "tools_called": ["web_search"] }
}
```

## 7. Caching

- Uses the upstream `requestCache(env(c))` service (works across Node, Cloudflare Workers, Bun).
- Cache entry format: `{ value: configData, createdAt: timestamp }`.
- Soft TTL: 60 seconds (re-fetches in background on staleness).
- Hard TTL: 24 hours (stale entry used as fallback if Guard API is down).
- 401/404 responses from Guard API invalidate the cache instantly.

## 8. Graceful Degradation

| Failure | Behavior |
|---|---|
| Guard API unreachable on fresh request | Warn and continue without config injection (fail open) |
| Guard API unreachable on stale cache | Serve stale config, log warning |
| Guard API 401/404 | Delete cache, continue without config |
| Guardrails Guard API timeout | `verdict: true` (fail open), log error |
| Session cache error | Log error, skip session check, proceed with eval |

## 9. Future Scope

| Feature | Status |
|---|---|
| `approval` action (request queue + webhook) | ⏸ Deferred |
| Telemetry DLQ (retry on Guard API downtime) | ⏸ Deferred |
| WebSocket / Realtime guardrails | ⏸ Deferred |
| Streaming (SSE) guardrails | ⏸ Deferred |
| Portkey Virtual Keys integration | ⏸ Deferred |

## 10. Implementation Status

| Feature | Status | File |
|---|---|---|
| `guardionAuth` middleware | ✅ Complete | `src/middlewares/guardion/index.ts` |
| Config mapper + spoofed org | ✅ Complete | `src/middlewares/guardion/mapper.ts` |
| Telemetry sidecar (extracted) | ✅ Complete | `src/middlewares/guardion/telemetry.ts` |
| Guardrails plugin (mode/status/session/tool) | ✅ Complete | `src/plugins/guardion/guardrails.ts` |
| Session behavior rules | ✅ Complete | `src/plugins/guardion/sessionCache.ts` |
| Tool call policy enforcement | ✅ Complete | `src/plugins/guardion/toolPolicy.ts` |
| `guardionAuth` wired in `src/index.ts` | ✅ Complete | `src/index.ts` (1 line) |
| Unit tests — guardrails plugin (9/9) | ✅ Complete | `src/plugins/guardion/guardrails.test.ts` |
| Integration tests — guardionAuth (11/11) | ✅ Complete | `tests/integration/guardionAuth.test.ts` |
| Plugin registration in `src/plugins/build.ts` | 🔜 Pending | `src/plugins/build.ts` |
| Redis config caching | 🔜 Pending (uses `requestCache`) | — |
66 changes: 66 additions & 0 deletions docs/PRD-FRONTEND-PLAYGROUND.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
# PRD: HTML Frontend Index & Interactive Playground

> **Last Updated**: 2026-03-03
> **Status**: Planned

## 1. Overview
As part of making the Guardion Gateway (Portkey fork) more accessible to developers, we need an HTML frontend on the root URL (`/`) instead of a simple 404 or text response. This page will serve as a welcome index, documentation, and an interactive testing playground.

## 2. Goals
- Provide a welcoming landing page on the Gateway's root `/` path.
- Include clear instructions and documentation on how to use the Gateway via `curl`.
- Provide a simple interactive web playground with a form to test the API.
- Include an interactive `curl` code editor displaying examples for various supported LLM providers (e.g., OpenAI, Anthropic).

## 3. Scope of Work

### 3.1 HTTP Route Updates
- Modify the Gateway server configuration to serve an HTML page (either static or templated) when a user hits `GET /`.
- Ensure this doesn't conflict with any `GET /v1/*` or other API routing.

### 3.2 HTML Page Structure
The frontend should be a single-page HTML application consisting of:
- **Header**: Guardion Gateway branding and title.
- **Documentation Section**: Explanation of what the gateway is and how to authenticate using `x-guardion-api-key` or `Authorization: Bearer <token>`.
- **Curl Examples Code Editor**:
- A mock code editor UI component displaying `curl` commands.
- Tabs or a dropdown to switch between providers (OpenAI, Anthropic, Google, etc.).
- Snippets showing how to use the unified `/v1/chat/completions` endpoint for each provider.
- **Interactive Playground Form**:
- Input fields for:
- API Key (`x-guardion-api-key` or `Bearer Token`).
- Provider Selection.
- Model Selection.
- System/User Prompt text areas.
- A "Submit" button to execute the request against the running gateway instance.
- A "Response" panel to display the JSON output, status code, and guardrail telemetry info.

## 4. Technical Implementation Details
- **Tech Stack**: Vanilla HTML/CSS/JS is preferred to keep the gateway lightweight. No React or heavy frontend build step should be required. Use a lightweight CSS framework (like Tailwind via CDN or minimal custom CSS) for styling.
- **Static Assets**: If necessary, static assets (JS/CSS) should be served directly by the gateway framework or embedded inline.
- **Integration Point**: In the core routing file (e.g., `src/index.ts` or similar router entry point), register a specific route for `GET /` to return the HTML content using standard content-type `text/html`.

## 5. Curl Code Snippet Examples

**OpenAI Example:**
```bash
curl "http://localhost:8787/v1/chat/completions"
-H "Authorization: Bearer <YOUR_GUARDION_API_KEY>"
-H "Content-Type: application/json"
-d '{
"model": "gpt-4",
"messages": [{"role": "user", "content": "Hello, world!"}]
}'
```

**Anthropic Example:**
```bash
curl "http://localhost:8787/v1/chat/completions"
-H "Authorization: Bearer <YOUR_GUARDION_API_KEY>"
-H "x-portkey-provider: anthropic"
-H "Content-Type: application/json"
-d '{
"model": "claude-3-opus-20240229",
"messages": [{"role": "user", "content": "Hello, world!"}]
}'
```
47 changes: 47 additions & 0 deletions docs/PRD_GUARDRAIL_INTEGRATION.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
# PRD: Native Guardion Guardrail Integration for Portkey

## 1. Overview
Integrate Guardion's policy and detector system natively into Portkey's guardrail middleware. This allows users to manage security policies in the Guardion Console and have them automatically applied by the Portkey Gateway using standard Albus-compatible protocols.

## 2. Objective
- Support remote fetching of guardrail definitions (Guardion Policies) via the `/v2/guardrails/{slug}` endpoint.
- Map Guardion Policies/Detectors to Portkey Guardrails/Checks.
- Enable granular visibility of Guardion detections in Portkey metrics and logs.
- Leverage `x-guardion-policy` header to trigger dynamic configuration and guardrail application.

## 3. Technical Requirements

### 3.1. Schema Mapping
The Guard API must return a Portkey-compatible Guardrail response for the `/v2/guardrails/{slug}` endpoint.

**Guardion Policy** → **Portkey Guardrail**
| Guardion Field | Portkey Field | Notes |
| :--- | :--- | :--- |
| `name` (slug) | `slug` | Unique identifier for the policy. |
| `detectors` (list) | `checks` (list) | Each detector maps to a check. |
| `action` | `actions.on_fail` | `block` → `block`, `monitor` → `none`. |

**Guardion Detector** → **Portkey Check**
| Guardion Field | Portkey Field | Notes |
| :--- | :--- | :--- |
| `model` | `id` | Formatted as `guardion.<model>` (e.g., `guardion.pii`). |
| `threshold` | `parameters.threshold` | The sensitivity level. |

### 3.2. Gateway Plugin Integration
The Gateway must have a plugin handler registered for `guardion` that can handle any detector model.
- **Provider**: `guardion`
- **Check Name**: Dynamic (any value following `guardion.`)
- **Action**: Call Guard API `/v1/eval` for the specific detector or policy.

### 3.3. Request Flow
1. **Request**: Client sends `x-guardion-policy: my-security-policy`.
2. **Translation**: `edgeHeaderTranslator` translates this to `x-portkey-config: pc-my-security-policy`.
3. **Auth/Config**: Portkey fetches config `pc-my-security-policy` from Guard API.
4. **Guardrail Fetch**: Config references guardrail `my-security-policy`. Portkey fetches it from `/v2/guardrails/my-security-policy`.
5. **Evaluation**: Portkey iterates through `checks`. For each `guardion.<model>`, it calls the Guardion plugin.
6. **Verdict**: If any check fails and `on_fail` is `block`, the request is intercepted.

## 4. User Experience
- Users can simply provide a policy ID via a header.
- Detections are visible in Portkey's "Agent Trace" and logs with specific detector names.
- No need for complex client-side configuration.
Loading
Loading