Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
156 changes: 53 additions & 103 deletions docs/concepts/scoring.md
Original file line number Diff line number Diff line change
Expand Up @@ -80,57 +80,50 @@ Each dimension has its own detailed breakdown, rating enum, and independent scor

### Scoring Breakdown

The compliance score uses a **60/20/15/5 weighting** across four categories:
The compliance score uses a **penalty-based model** starting at 100 points. Each violation deducts a fixed penalty:

#### Core Fields (60 points)
#### Core Fields

Checks for presence of all 9 required A2A v0.3.0 fields:
Checks for presence of all required A2A fields:

| Field | Points | Purpose |
|-------|--------|---------|
| `protocolVersion` | 6.67 | Declares A2A version (e.g., "0.3.0") |
| `name` | 6.67 | Agent display name |
| `description` | 6.67 | Agent description |
| `url` | 6.67 | Agent base URL |
| `version` | 6.67 | Agent semantic version |
| `capabilities` | 6.67 | Supported capabilities |
| `defaultInputModes` | 6.67 | Accepted MIME types |
| `defaultOutputModes` | 6.67 | Response MIME types |
| `skills` | 6.67 | Available skills array |
| Violation | Penalty | Example |
|-----------|---------|---------|
| Missing critical field | -20 | No `url` or `name` |
| Missing important field | -15 | No `protocolVersion` |
| Missing recommended field | -10 | No `capabilities` |
| Missing optional field | -5 | No `provider` |

**Penalty:** Each missing field costs 6.67 points.

#### Skills Quality (20 points)
#### Skills Quality

Evaluates the quality of the skills array:

- **Skills present** (10 points) - At least one skill defined
- **Required fields** (5 points) - All skills have `id`, `name`, `description`
- **Tags present** (5 points) - All skills have at least one tag

**Penalties:**
- -2 points per skill missing required fields
- -1 point per skill without tags

#### Format Compliance (15 points)
| Violation | Penalty |
|-----------|---------|
| No skills defined | -10 |
| Missing required skill fields (`id`, `name`, `description`) | -2 per skill |
| Skills without tags | -1 per skill |

Validates proper formatting across the agent card:
#### Format Compliance

- **Valid semver** (3 points) - `version` follows semantic versioning (e.g., `1.0.0`)
- **Valid protocol version** (3 points) - `protocolVersion` is `0.3.0`
- **Valid URL** (3 points) - `url` is a proper HTTPS URL
- **Valid transports** (3 points) - Transport protocols are `JSONRPC`, `GRPC`, or `HTTP+JSON`
- **Valid MIME types** (3 points) - Input/output modes are valid MIME types
Validates proper formatting:

**Special penalty:** -10 points for invalid MIME types (security risk)
| Violation | Penalty |
|-----------|---------|
| Invalid MIME types | -10 |
| Invalid URL format | -5 |
| Invalid semver | -5 |
| Invalid protocol version | -5 |
| Invalid transport | -2 |

#### Data Quality (5 points)
#### Data Quality

Checks for data integrity and security:
Checks for data integrity:

- **No duplicate skill IDs** (2 points) - Each skill has a unique identifier
- **Field lengths valid** (2 points) - Names, descriptions within reasonable limits
- **No SSRF risks** (1 point) - URLs don't point to internal/private networks
| Violation | Penalty |
|-----------|---------|
| Duplicate skill IDs | -2 |
| Field length violations | -2 |
| SSRF-risk URLs | -1 |

### Rating Levels

Expand All @@ -152,64 +145,32 @@ Checks for data integrity and security:

---

## 🔐 Dimension 2: Trust (100 points + multiplier)
## 🔐 Dimension 2: Trust (100 points)

**What it measures:** How trustworthy and secure is this agent? Can users verify its authenticity?

**Why it matters:** Trust is critical for adoption. Without signatures, users can't verify your agent's identity. Security claims without proof get penalized.

### The Trust Confidence Multiplier 🔑

**Revolutionary concept:** The presence and validity of cryptographic signatures affects ALL trust claims via a confidence multiplier:

- **✅ Valid JWS signature**: `1.0x` multiplier (full trust)
- **⚪ No signature**: `0.6x` multiplier (unverified claims)
- **❌ Invalid signature**: `0.4x` multiplier (active distrust)

**Why this matters:** An agent claiming strong security without signatures gets reduced trust. This prevents "trust washing" where agents make security claims they can't prove.

### Scoring Breakdown

The trust score uses a **40/25/20/15 weighting** across four categories:

#### Signatures (40 points + confidence key)
**Why it matters:** Trust is critical for adoption. Without signatures, users can't verify your agent's identity.

The foundation of trust:
### Scoring Model

- **Valid signature present** (30 points) - JWS signature verified
- **Multiple signatures** (+3 points) - Redundant verification
- **Comprehensive coverage** (+4 points) - Signature covers all critical fields
- **Recent signature** (+3 points) - Signed within last 90 days
Trust scoring uses a **discrete assignment model** based on the agent card's signature state:

**Penalties:**
- **Invalid signature** (-15 points + 0.4x multiplier) - Worse than no signature!
- **Expired signature** (-10 points + 0.6x multiplier)
| Signature State | Score | Description |
|----------------|-------|-------------|
| Valid signature from trusted issuer | **100** | Fully verified |
| Valid signature, no trusted issuers configured | **80** | Signature valid but no issuer list |
| Valid signature, issuer not in trusted list | **60** | Signature valid but issuer unknown |
| No signature | **20** | Unverified claims |
| Invalid signature | **0** | Active red flag |

#### Provider Information (25 points)
The signature state is the dominant factor. Additional agent card metadata (provider info, security schemes, documentation URLs) contributes to the validation report details but the trust score is primarily driven by signature validity.

Who is behind this agent?
### What Affects Trust

- **Organization specified** (10 points) - `provider.organization` present
- **Provider URL** (10 points) - `provider.url` present and uses HTTPS
- **URL reachable** (+5 bonus) - Provider website responds *(requires live testing)*

#### Security Practices (20 points)

How secure is the implementation?

- **HTTPS-only endpoints** (10 points) - All URLs use HTTPS
- **Security schemes declared** (5 points) - `securitySchemes` defined in agent card
- **Strong authentication** (5 points) - OAuth2, OpenID Connect, or similar

**Penalty:** -10 points for any HTTP URLs (major security risk)

#### Documentation (15 points)

Transparency and user support:

- **Documentation URL** (5 points) - `documentationUrl` provided
- **Terms of Service** (5 points) - `termsOfServiceUrl` provided
- **Privacy Policy** (5 points) - `privacyPolicyUrl` provided
- **Cryptographic signatures** — JWS-signed agent cards receive significantly higher scores
- **Trusted issuers** — Configuring a trusted issuer list and matching against it yields the highest score
- **HTTPS-only endpoints** — HTTP URLs are flagged as security risks
- **Provider information** — Organization and URL presence improves trust indicators

### Rating Levels

Expand All @@ -221,23 +182,12 @@ Transparency and user support:
| 20-39 | **Low Trust** | Minimal security indicators |
| <20 | **Untrusted** | Security red flags present |

### Example: Confidence Multiplier in Action

**Scenario:** Agent card claims strong security (OAuth2, HTTPS-only, security schemes) = 40 raw points

- **With valid signature**: `40 × 1.0 = 40 points` ✅ (Claims verified)
- **Without signature**: `40 × 0.6 = 24 points` ⚠️ (Unverified claims)
- **Invalid signature**: `40 × 0.4 = 16 points` ❌ (Active red flag)

The same security features are worth **167% more** with a valid signature!

### Common Issues

- ❌ No cryptographic signatures (reduces trust by 40%)
- ❌ Missing provider information
- ❌ No documentation or privacy policy URLs
- ❌ Using HTTP instead of HTTPS
- ❌ No security schemes declared
- ❌ No cryptographic signatures (score drops to 20)
- ❌ Invalid signature (score drops to 0)
- ❌ Using HTTP instead of HTTPS (flagged in validation report)
- ❌ Missing provider information (noted in trust details)

---

Expand Down Expand Up @@ -494,7 +444,7 @@ The old `result.score` property still exists but returns `compliance.total` and
Yes. If live testing is not performed, `availability.total` is `None` and `availability.rating` is `AvailabilityRating.NOT_TESTED`.

??? question "Why does my trust score seem low?"
If you have security features but no signature, the 0.6x confidence multiplier reduces your score by 40%. Add JWS signatures to achieve full trust potential.
Without a valid cryptographic signature, the trust score is capped at 20. Adding JWS signatures is the single most impactful change for trust scores. If your signature is valid but you see 60, your issuer may not be in the verifier's trusted issuer list.

??? question "What if I only care about one dimension?"
You can ignore the others, but we recommend considering all three. An agent with perfect compliance but zero trust is still a security risk.
Expand Down
4 changes: 2 additions & 2 deletions docs/getting-started/cicd/2-action.md
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,7 @@ cat > agent-card.json << 'EOF'
"description": "A sample A2A agent",
"url": "https://example.com/agent",
"version": "1.0.0",
"protocolVersion": "0.2.0",
"protocolVersion": "0.3.0",
"provider": {
"organization": "My Company"
},
Expand Down Expand Up @@ -101,7 +101,7 @@ When the action runs, you'll see output like:
```
✅ A2A AGENT VALIDATION PASSED
Score: 75/100
Version: 0.2.0
Version: 0.3.0
Agent passed with warnings

ISSUES FOUND:
Expand Down
20 changes: 10 additions & 10 deletions docs/getting-started/complete-workflow.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,7 @@
# End-to-End Tutorial

<!--
VERIFIED: 2025-12-11
Based on: capiscio-core, capiscio-server, capiscio-sdk-python source code
Source: capiscio-core, capiscio-server, capiscio-sdk-python source code
-->

This tutorial walks through the complete CapiscIO workflow: from creating an A2A agent to deploying it with trust badge enforcement.
Expand Down Expand Up @@ -31,7 +30,7 @@ pip install capiscio
npm install -g capiscio

# Verify installation
capiscio version
capiscio --version
```

---
Expand Down Expand Up @@ -184,7 +183,7 @@ curl -X POST http://localhost:8080/v1/api-keys \
Save the returned API key securely:

```bash
export CAPISCIO_API_KEY="cpsc_live_xxx"
export CAPISCIO_API_KEY="sk_live_xxx"
```

### 3.4 Register Your Agent
Expand Down Expand Up @@ -260,7 +259,7 @@ Badges expire quickly (5 minutes by default). Use the badge keeper daemon:

```bash
capiscio badge keep \
--ca-url "http://localhost:8080" \
--ca "http://localhost:8080" \
--agent-id "$AGENT_ID" \
--api-key "$CAPISCIO_API_KEY" \
--domain "my-agent.example.com" \
Expand Down Expand Up @@ -364,17 +363,18 @@ For simpler deployments, use the SDK middleware instead of the gateway:

```python
from fastapi import FastAPI
from capiscio_sdk import CapiscioMiddleware, SecurityConfig
from capiscio_sdk import SimpleGuard, SecurityConfig
from capiscio_sdk.integrations.fastapi import CapiscioMiddleware

app = FastAPI()

guard = SimpleGuard()

# Add badge verification middleware
app.add_middleware(
CapiscioMiddleware,
config=SecurityConfig.production(
trusted_issuers=["https://registry.capisc.io"],
min_trust_level=2,
),
guard=guard,
config=SecurityConfig.production(),
)

@app.post("/task")
Expand Down
4 changes: 2 additions & 2 deletions docs/getting-started/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -182,8 +182,8 @@ If you need more control, you can still do things step-by-step:
# Generate keys only (no server registration)
capiscio key gen --out-priv private.jwk --out-pub public.jwk

# Derive DID from existing key
capiscio key did --in public.jwk
# Show DID derived from generated key
capiscio key gen --show-did
```

See the [CLI Reference](../reference/cli/index.md) for all options.
Expand Down
6 changes: 3 additions & 3 deletions docs/getting-started/secure/2-sdk.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ python -c "import capiscio_sdk; print(capiscio_sdk.__version__)"

Expected output:
```
0.2.0
2.5.0
```

---
Expand Down Expand Up @@ -56,7 +56,7 @@ async def get_agent_card():
"description": "An A2A agent with CapiscIO security",
"url": "http://localhost:8000",
"version": "1.0.0",
"protocolVersion": "0.2.0",
"protocolVersion": "0.3.0",
"provider": {
"organization": "My Company"
},
Expand Down Expand Up @@ -124,7 +124,7 @@ async def get_agent_card():
"description": "An A2A agent with CapiscIO security",
"url": "http://localhost:8000",
"version": "1.0.0",
"protocolVersion": "0.2.0",
"protocolVersion": "0.3.0",
"provider": {"organization": "My Company"},
"capabilities": {"streaming": False, "pushNotifications": False},
"public_keys": [
Expand Down
2 changes: 1 addition & 1 deletion docs/getting-started/secure/5-production.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ Create a proper `agent-card.json`:
"description": "Production A2A agent for My Company",
"url": "https://agent.mycompany.com",
"version": "1.0.0",
"protocolVersion": "0.2.0",
"protocolVersion": "0.3.0",
"provider": {
"organization": "My Company Inc.",
"url": "https://mycompany.com"
Expand Down
2 changes: 1 addition & 1 deletion docs/getting-started/validate/1-intro.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ Here's a minimal example:
{
"name": "My Agent",
"url": "https://example.com/agent",
"protocolVersion": "0.2.0"
"protocolVersion": "0.3.0"
}
```

Expand Down
6 changes: 3 additions & 3 deletions docs/getting-started/validate/2-install.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ The CapiscIO CLI is available for Node.js and Python. Choose your preferred plat

Expected output:
```
capiscio/2.1.x (darwin-arm64)
capiscio/2.5.0 (darwin-arm64)
```

=== "pip (Python)"
Expand All @@ -42,7 +42,7 @@ The CapiscIO CLI is available for Node.js and Python. Choose your preferred plat

Expected output:
```
capiscio/2.1.x (darwin-arm64)
capiscio/2.5.0 (darwin-arm64)
```

=== "Go (Direct)"
Expand Down Expand Up @@ -142,7 +142,7 @@ cat > agent-card.json << 'EOF'
"description": "A sample A2A agent for learning CapiscIO",
"url": "https://example.com/agent",
"version": "1.0.0",
"protocolVersion": "0.2.0",
"protocolVersion": "0.3.0",
"provider": {
"organization": "My Company"
},
Expand Down
2 changes: 1 addition & 1 deletion docs/getting-started/validate/4-reports.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ Our sample agent card scores 75/100. Let's improve it:
"description": "A sample A2A agent for learning CapiscIO",
"url": "https://example.com/agent",
"version": "1.0.0",
"protocolVersion": "0.2.0",
"protocolVersion": "0.3.0",
"provider": {
"organization": "My Company",
"url": "https://mycompany.com"
Expand Down
3 changes: 0 additions & 3 deletions docs/identity/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -219,9 +219,6 @@ cat .capiscio/did.txt

# Validate that identity is properly configured
capiscio validate agent-card.json

# Resolve any DID
capiscio identity resolve did:key:z6Mk...
```

---
Expand Down
Loading
Loading