Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ node_modules/
dist/
coverage/
.tmp/
.playwright-mcp/
.agent-reliability/
*.log
/progress.md
Expand Down
52 changes: 48 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,9 @@ Verify, harden, and ship AI-agent-assisted codebases in one command.
[![License: MIT](https://img.shields.io/badge/license-MIT-181714.svg)](LICENSE)
[![Node](https://img.shields.io/badge/node-%3E%3D20-315f9f.svg)](package.json)

Agent Reliability Kit scans a repository the way a careful maintainer would before letting AI coding agents work there: agent instructions, verification commands, README quality, secret hygiene, GitHub Actions safety, MCP/tooling risk, and release readiness.
Agent Reliability Kit scans a repository the way a careful maintainer would before letting AI coding agents work there: agent instructions, verification commands, README quality, secret hygiene, GitHub Actions safety, MCP/tooling risk, n8n workflow exports, team policy, and release readiness.

The flagship path is simple: keep `agent-secret-guard` as the sharp security wedge, and use `agent-reliability-kit` as the one command center for agent-era repository reliability.

## Quick Start

Expand All @@ -30,6 +32,16 @@ After npm publication:
npx agent-reliability-kit scan .
```

Optional focused checks:

```bash
ark team-audit . --out .agent-reliability/team
ark mcp-registry . --registry .agent-reliability/mcp-registry.json
ark n8n-scan . --out .agent-reliability/n8n
ark n8n-backup . --backup-dir .agent-reliability/n8n-backup
ark cost-report . --trace .agent-reliability/traces --budget-usd 10
```

The scan writes:

- `.agent-reliability/report.md`
Expand All @@ -52,13 +64,22 @@ AI coding agents fail most often on the unglamorous parts: missing repo rules, u
| Secrets | token-like values, tracked `.env` files, redacted evidence |
| GitHub Actions | validation commands, explicit permissions, risky triggers, pipe-to-shell |
| AI tooling | MCP command configs and prompt-injection-like instruction files |
| MCP registry | private allowlist, trust score, approved commands/URLs, risk owner |
| n8n | public webhooks, command nodes, risky code nodes, workflow secrets, redacted backups |
| Team layer | scan history, policy gates, audit report, dry-run Slack payload |
| Cost guard | local trace token/cost summary and budget alerts |

## CLI

```bash
agent-reliability-kit scan [path]
agent-reliability-kit doctor [path]
agent-reliability-kit init [path]
agent-reliability-kit team-audit [path]
agent-reliability-kit mcp-registry [path]
agent-reliability-kit n8n-scan [path]
agent-reliability-kit n8n-backup [path]
agent-reliability-kit cost-report [path]
```

Examples:
Expand All @@ -68,6 +89,10 @@ ark scan . --min-score 85
ark scan . --format sarif --stdout > agent-reliability.sarif
ark doctor .
ark init .
ark team-audit .
ark mcp-registry .
ark n8n-scan .
ark cost-report . --budget-usd 10
```

Machine-readable stdout stays clean for CI:
Expand All @@ -82,6 +107,22 @@ ark scan . --format sarif --stdout > agent-reliability.sarif

The HTML report is designed for maintainers, contributors, and launch pages. It gives a score, severity counts, repository signals, and next actions for each finding.

## Product Modules

- [Team audit layer](docs/team-layer.md): scan history, policy checks, audit report, and local Slack payload.
- [Private MCP registry](docs/private-mcp-registry.md): team allowlist with trust score, approved commands/URLs, permissions, owner, and reason.
- [n8n safety and backup](docs/n8n-safety-backup.md): risky workflow scanning and redacted Git-friendly backups.
- [AI cost guard](docs/ai-cost-guard.md): local trace cost summaries and budget alerts.
- [Commercial support path](docs/commercial-support.md): open-source boundary and future paid team features.
- [Consolidation roadmap](docs/roadmap-consolidation.md): how small tools roll into the flagship CLI.

![CLI demo](assets/cli-demo.svg)

## Comparisons

- [agent-secret-guard vs gitleaks](docs/comparisons/agent-secret-guard-vs-gitleaks.md)
- [Agent Reliability Kit vs generic linters](docs/comparisons/agent-reliability-kit-vs-generic-linters.md)

## Launch Kit

The repository includes a pre-release launch kit so maintainers can prepare a public launch without inventing copy or sharing private data at the last minute.
Expand All @@ -92,6 +133,8 @@ The repository includes a pre-release launch kit so maintainers can prepare a pu
- [Press kit](docs/launch/press-kit.md)
- [Community responses](docs/launch/community-responses.md)
- [Channel rules](docs/launch/channel-rules.md)
- [Distribution checklist](docs/launch/distribution-checklist.md)
- [Demo GIF script](docs/launch/demo-gif-script.md)
- [Product Hunt draft](docs/launch/product-hunt.md)
- [DEV article draft](docs/launch/devto-article.md)

Expand Down Expand Up @@ -133,9 +176,10 @@ assets/
## Roadmap

- v0.1: CLI scan, doctor, init, Markdown/JSON/HTML/SARIF reports.
- v0.2: richer language detection, monorepo command graph, and config file support.
- v0.3: GitHub Action wrapper and dogfood gallery.
- v0.4: compatibility matrix for Codex, Claude Code, Cursor, Gemini CLI, and OpenCode.
- v0.2: team audit, private MCP registry, n8n safety/backup, and local cost guard.
- v0.3: GitHub Action wrapper, dogfood gallery, and `agent-secret-guard` rule-pack consolidation.
- v0.4: hosted team dashboard prototype, org policy packs, and private MCP approval workflow.
- v0.5: `pr verify`, `trace run`, and compatibility matrix for Codex, Claude Code, Cursor, Gemini CLI, and OpenCode.

## Security

Expand Down
17 changes: 17 additions & 0 deletions assets/cli-demo.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
44 changes: 44 additions & 0 deletions docs/ai-cost-guard.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
# AI Cost Guard

`cost-report` summarizes local JSON or JSONL trace files from coding-agent runs.

```bash
ark cost-report . \
--trace .agent-reliability/traces \
--budget-usd 10 \
--out .agent-reliability/cost
```

It writes:

- `.agent-reliability/cost/cost-report.md`
- `.agent-reliability/cost/cost-report.json`

## Supported Event Shapes

JSONL:

```json
{"provider":"openai","model":"gpt-5.2","inputTokens":1000,"outputTokens":500,"costUsd":0.25}
{"provider":"anthropic","model":"claude-sonnet","usage":{"prompt_tokens":2000,"completion_tokens":1000},"cost_usd":0.75}
```

JSON:

```json
{
"events": [
{
"provider": "openai",
"model": "gpt-5.2",
"inputTokens": 1000,
"outputTokens": 500,
"costUsd": 0.25
}
]
}
```

## Scope

This is a local cost guard, not a provider billing source of truth. It is designed for agent run traces, budget alerts, and "which model burned the most tokens?" debugging.
36 changes: 36 additions & 0 deletions docs/commercial-support.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
# Commercial Support

Agent Reliability Kit is open source and local-first. The natural paid layer is for teams that need shared policy, history, allowlists, and audit proof.

## Open Source

- local CLI
- Markdown/JSON/HTML/SARIF reports
- n8n safety checks
- MCP registry audit
- local scan history
- dry-run Slack payload generation
- redacted n8n backups
- local AI cost reports

## Future Paid Team Layer

- hosted dashboard across repositories
- org-wide policy packs
- private MCP registry and approval workflow
- scan history and trend charts
- Slack/email/webhook alerts
- audit exports for security reviews
- team allowlists for tools, models, commands, and workflow templates

## Support Positioning

Commercial support can start before a hosted SaaS exists:

- paid setup for teams adopting AI coding agents
- repository hardening audit
- MCP registry design
- n8n workflow safety review
- private policy pack maintenance

The CLI should remain useful without a paid account.
22 changes: 22 additions & 0 deletions docs/comparisons/agent-reliability-kit-vs-generic-linters.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
# Agent Reliability Kit vs Generic Linters

Generic linters check code style, syntax, and language-specific rules.

Agent Reliability Kit checks whether a repository is safe and understandable enough for AI-assisted development.

| Surface | Generic linter | Agent Reliability Kit |
| --- | --- | --- |
| Code syntax/style | Strong | Not the focus |
| Agent instructions | No | Yes |
| Verification commands | No | Yes |
| README replayability | No | Yes |
| GitHub Actions risk | Sometimes | Yes |
| Secret-like agent config | Sometimes | Yes |
| MCP allowlist | No | Yes |
| n8n workflow safety | No | Yes |
| Team audit report | No | Yes |
| Local cost report | No | Yes |

## Message For Launch

This is not another JavaScript linter. It is a release-readiness and agent-readiness scanner for teams using Codex, Claude Code, Cursor, Gemini CLI, MCP, and local automation.
28 changes: 28 additions & 0 deletions docs/comparisons/agent-secret-guard-vs-gitleaks.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
# agent-secret-guard vs gitleaks

Use both when possible.

`gitleaks` is a strong general-purpose secret scanner. `agent-secret-guard` and Agent Reliability Kit focus on the agent-era surfaces around those secrets.

| Need | gitleaks | Agent Reliability Kit / agent-secret-guard |
| --- | --- | --- |
| Generic secret detection | Strong | Focused, lightweight |
| AI-agent config context | Limited | Built in |
| MCP config risk | Limited | Built in |
| Local automation notes | Limited | Built in |
| GitHub Actions permission hints | Limited | Built in |
| Redacted reports for agent handoff | Limited | Built in |
| Team policy and audit package | External | Local MVP |

## Positioning

If a team already runs `gitleaks`, Agent Reliability Kit should sit next to it:

```bash
gitleaks detect
ark scan .
ark mcp-registry .
ark team-audit .
```

The product angle is not "replace every scanner." It is "catch the AI-agent and MCP operational risks that generic scanners do not explain well."
30 changes: 30 additions & 0 deletions docs/launch/demo-gif-script.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
# Demo GIF Script

Record this flow after `npm run build`.

Use a clean terminal with no private paths, no real repository names, and no credentials visible.

```bash
npm run build
ark scan tests/fixtures/n8n-risk --format text --stdout --min-score 0
ark mcp-registry tests/fixtures/mcp-registry --out .tmp/demo-mcp
ark cost-report tests/fixtures/cost-trace --budget-usd 0.50 --out .tmp/demo-cost
```

Storyboard:

1. Start on the README hero.
2. Run `ark scan tests/fixtures/n8n-risk --format text --stdout --min-score 0`.
3. Show the critical n8n command execution finding.
4. Run `ark mcp-registry tests/fixtures/mcp-registry --out .tmp/demo-mcp`.
5. Show disabled/unallowlisted MCP server findings.
6. Run `ark cost-report tests/fixtures/cost-trace --budget-usd 0.50 --out .tmp/demo-cost`.
7. Show the budget alert and generated report paths.

Recommended caption:

```text
One local CLI for agent-era repo risk: secrets, MCP allowlists, n8n workflow safety, team audit, and AI cost guard.
```

Never record real configs, browser profiles, Slack webhooks, private repo paths, or token values.
78 changes: 78 additions & 0 deletions docs/launch/distribution-checklist.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
# Distribution Checklist

Use this only after local verification passes.

## Must Be Ready

- `npm run check` passes
- `npm run smoke` passes
- README quick start works from a clean clone
- demo command shows a real finding in under 30 seconds
- no real secrets in screenshots, GIFs, reports, fixtures, or issues
- release notes mention local-only safety boundaries

## Launch Channels

### Show HN

Title:

```text
Show HN: Agent Reliability Kit - local checks for AI-agent-ready repos
```

Post:

```text
I built a local-first CLI for teams using Codex, Claude Code, Cursor, MCP, and n8n.

It scans repo instructions, verification commands, README replayability, secret hygiene, GitHub Actions risk, MCP allowlists, n8n workflow exports, and local AI cost traces.

The goal is not to replace language linters. It is to answer: "Is this repo safe and clear enough to hand to an AI coding agent?"

No account, no telemetry, no cloud required.
```

### Reddit

Target communities:

- `r/ClaudeAI`
- `r/cursor`
- `r/selfhosted`
- `r/devops`
- `r/n8n`
- `r/opensource`

Angle:

```text
I made a local scanner for AI-agent repo risk: MCP allowlists, n8n workflow safety, GitHub Actions permissions, secret-like values, and scan history.
```

### MCP Directories / Awesome Lists

Submit the MCP registry angle:

```text
Agent Reliability Kit helps teams review MCP config files against a private allowlist with trust score, permissions, approved commands/URLs, and risk owner metadata.
```

### Product Hunt

Use after GitHub README, docs homepage, and demo asset are polished.

Tagline:

```text
Local reliability checks for AI-agent-assisted codebases.
```

## Follow-Up Metrics

- GitHub stars
- npm downloads
- CLI runs from issue comments
- opened issues with real repo samples
- discussions asking for team policy/dashboard
- requests for GitHub Action or hosted reporting
Loading