Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 21 additions & 6 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,12 +34,15 @@ Backend config is loaded from environment variables (see `backend/.env.example`)
- `CLERK_WEBHOOK_SECRET` — Svix signature for Clerk subscription webhooks
- `DATABASE_URL` — defaults to `sqlite:///./opensentry.db`
- `FRONTEND_URL` — extra CORS origin (must have scheme, no trailing slash)
- `SEGMENT_CACHE_MAX_PER_CAMERA` — segments cached in memory per camera (default 15, ~30s)
- `SEGMENT_CACHE_MAX_PER_CAMERA` — segments cached in memory per camera (default 60, ~60s)
- `SEGMENT_PUSH_MAX_BYTES` — max bytes per pushed segment (default 2 MB)
- `CLEANUP_INTERVAL` — run cache eviction every N playlist updates (default 20)
- `INACTIVE_CAMERA_CLEANUP_HOURS` — free caches for cameras offline this long (default 24)
- `LOG_RETENTION_DAYS` — stream + MCP + audit + motion + notification log retention (default 90)
- `OFFLINE_SWEEP_INTERVAL_SECONDS` — how often to mark stale rows offline (default 30)
- `REDIS_URL` — Redis connection for rate limiter storage; in-memory fallback when unset (fine for single-instance, but limits don't hold across VMs)
- `SENTRY_DSN` — Sentry error tracking DSN; leave blank to disable (no-ops gracefully)
- `SENTRY_TRACES_SAMPLE_RATE` — Sentry performance trace sample rate (default 0.1)

Frontend config: `VITE_CLERK_PUBLISHABLE_KEY`, `VITE_API_URL`, `VITE_LOCAL_HLS`.

Expand All @@ -64,13 +67,20 @@ backend/
│ │ ├── ws.py # CloudNode WebSocket channel
│ │ └── webhooks.py # Clerk subscription webhook handler
│ ├── mcp/
│ │ └── server.py # FastMCP server + 22 tools + ScopeMiddleware
│ │ ├── server.py # FastMCP server + 22 tools + ScopeMiddleware
│ │ └── activity.py # MCP activity tracking + per-org broadcasting
│ ├── core/
│ │ ├── auth.py # Clerk JWT validation (V1 + V2 permissions), dependencies
│ │ ├── audit.py # Audit log helper (records admin actions)
│ │ ├── codec.py # Video codec string sanitization
│ │ ├── config.py # Environment loading (Config class)
│ │ ├── clerk.py # Clerk SDK init
│ │ ├── database.py # SQLAlchemy engine + session factory + Base
│ │ └── limiter.py # slowapi Limiter instance (tenant-aware key)
│ │ ├── limiter.py # slowapi Limiter instance (tenant-aware key)
│ │ ├── migrations.py # Manual schema migrations (stand-in for Alembic)
│ │ ├── plans.py # Plan limit logic (node quotas per tier)
│ │ ├── sentry.py # Sentry error tracking initialization
│ │ └── versions.py # CloudNode version compatibility checking
│ ├── models/models.py # 13 ORM models (see Data Models below)
│ └── schemas/schemas.py # Pydantic request/response schemas incl. McpKeyCreate
├── scripts/
Expand Down Expand Up @@ -99,6 +109,7 @@ frontend/
│ ├── HlsPlayer.jsx # HLS.js player with Clerk JWT xhrSetup
│ ├── CameraCard.jsx # Live thumbnail + status + actions
│ ├── CameraGridPreview.jsx # Static preview for the landing page
│ ├── DocsDiagrams.jsx # SVG architecture diagrams for the docs page
│ ├── IncidentReportModal.jsx # Markdown + evidence viewer
│ ├── NotificationBell.jsx # Unread badge + inbox popover (SSE-fed)
│ ├── AddNodeModal.jsx # Node creation flow (shows one-time API key)
Expand Down Expand Up @@ -139,7 +150,7 @@ MCP Client ──Bearer osc_…──→ FastMCP → ScopeMiddleware → tools

### Video streaming pipeline

1. CloudNode generates HLS segments via FFmpeg (2-second `.ts` files)
1. CloudNode generates HLS segments via FFmpeg (1-second `.ts` files)
2. CloudNode calls `POST /api/cameras/{id}/push-segment?filename=segment_NNNNN.ts` with the raw `.ts` body and `X-Node-API-Key` header
3. Backend stores the bytes in `_segment_cache[camera_id][filename]`, evicting the oldest once `SEGMENT_CACHE_MAX_PER_CAMERA` is exceeded
4. CloudNode calls `POST /api/cameras/{id}/playlist` with the rolling `stream.m3u8` text
Expand All @@ -151,7 +162,7 @@ MCP Client ──Bearer osc_…──→ FastMCP → ScopeMiddleware → tools
### SPA serving

`main.py` SPA middleware:
- `/api/*`, `/ws/*`, `/install.*`, `/mcp-setup.*` → FastAPI handlers
- `/api/*`, `/ws/*`, `/install.*`, `/mcp-setup.*`, `/downloads/*` → FastAPI handlers
- `POST /mcp` → FastMCP ASGI app (streamable HTTP)
- `GET /mcp` → React `McpPage` (dashboard route)
- `/assets/*` → static files from React build
Expand Down Expand Up @@ -314,6 +325,7 @@ Validation constants (also in `models.py`):
**install.py** (no prefix, no auth):
- `GET /install.sh` / `GET /install.ps1` — CloudNode installer scripts
- `GET /mcp-setup.sh` / `GET /mcp-setup.ps1` — MCP client config helpers
- `GET /downloads/{os_name}/{arch}` — 302 redirect to the latest CloudNode binary on GitHub Releases (os: `linux`/`macos`/`windows`, arch: `x86_64`/`aarch64`/`armv7`)

**ws.py** (no prefix):
- `WS /ws/node` — WebSocket channel for CloudNode realtime (API key in query)
Expand Down Expand Up @@ -390,7 +402,7 @@ cors_origins = [
"https://opensentry-command.fly.dev",
]
```
Plus `FRONTEND_URL` if set (validated: must have scheme, no trailing slash, no embedded whitespace). All methods and headers allowed; credentials allowed.
Plus `FRONTEND_URL` if set (validated: must have scheme, no trailing slash, no embedded whitespace). Allowed methods: `GET`, `POST`, `PUT`, `DELETE`, `OPTIONS`. Allowed headers: `Authorization`, `Content-Type`, `X-Node-API-Key`. Credentials allowed.

## Rate Limiting

Expand Down Expand Up @@ -468,8 +480,11 @@ HLS `GET` paths (`stream.m3u8`, `segment/{file}`) are intentionally unlimited
- `clerk-backend-api` — Clerk authentication
- `pyjwt` — JWT token handling (for V2 permission decoding)
- `slowapi` — Rate limiting
- `redis` — Rate limiter backing store (production; in-memory fallback for dev)
- `httpx` — HTTP client
- `websockets` — WebSocket protocol support (CloudNode channel)
- `svix` — Webhook signature verification
- `sentry-sdk[fastapi]` — Error tracking and performance monitoring
- `python-dotenv` — Environment variable loading

## Development Notes
Expand Down
30 changes: 21 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -92,18 +92,18 @@ The scripts detect which clients you already have and merge an `opensentry` entr
│ USB Camera │ │ FastAPI Backend │ │ React 19 │
│ ↓ │ │ │ │ │
│ FFmpeg (HLS) │──push─────→│ In-memory segment │←─GET───→│ HLS.js │
│ │ segments │ cache (~15 segs/cam) │ proxy │ (video) │
│ │ segments │ cache (~60 segs/cam) │ proxy │ (video) │
│ │──register─→│ SQLite / PostgreSQL │ URLs │ │
│ │──heartbeat→│ Clerk Auth │←─JWT───→│ Clerk Auth │
│ │──WS events │ FastMCP (/mcp) │←──SSE───│ Motion feed │
└──────────────┘ └───────────────────────┘ └──────────────┘
```

**Video pipeline:** CloudNode transcodes USB camera video into HLS segments and pushes each `.ts` file directly to the backend via `POST /api/cameras/{id}/push-segment`. The backend caches segments in memory (15 per camera by default, ~30s buffer) and serves them through the same-origin proxy at `GET /api/cameras/{id}/segment/{file}`. The rewritten playlist contains relative segment URLs, so the browser's Clerk JWT auth header is automatically attached. No S3, no presigned URLs, no third-party storage in the live path.
**Video pipeline:** CloudNode transcodes USB camera video into HLS segments and pushes each `.ts` file directly to the backend via `POST /api/cameras/{id}/push-segment`. The backend caches segments in memory (60 per camera by default, ~60s buffer) and serves them through the same-origin proxy at `GET /api/cameras/{id}/segment/{file}`. The rewritten playlist contains relative segment URLs, so the browser's Clerk JWT auth header is automatically attached. No S3, no presigned URLs, no third-party storage in the live path.

**Authentication:** Clerk handles user sign-up, login, and organization management. The backend validates JWT tokens (V1 and V2 permission formats) and extracts organization-scoped permissions. CloudNodes authenticate with API keys (SHA-256 hashed in the database) passed via `X-Node-API-Key`. MCP clients authenticate with `Authorization: Bearer osc_...` keys (also hashed).

**Storage:** Live segments live in the backend's in-memory cache; they expire automatically once `SEGMENT_CACHE_MAX_PER_CAMERA` is exceeded. Recordings and snapshots live on the CloudNode itself. SQLite is used for development (`opensentry.db`); PostgreSQL for production. Incident snapshots and clips are stored inline on `IncidentEvidence.data` (LargeBinary) — evidence travels with the incident.
**Storage:** Live segments live in the backend's in-memory cache; they expire automatically once `SEGMENT_CACHE_MAX_PER_CAMERA` is exceeded. Recordings and snapshots live on the CloudNode itself in its encrypted SQLite database. SQLite is used for development on the Command Center (`opensentry.db`); PostgreSQL for production. Incident snapshots and clips are stored inline on `IncidentEvidence.data` (LargeBinary) — evidence travels with the incident.

**Real-time:** CloudNodes maintain a WebSocket channel (`/ws/node`) used for commands, status, and motion events. The dashboard subscribes to SSE feeds for motion events (`/api/motion/events/stream`), notifications (`/api/notifications/stream`), and MCP activity (`/api/mcp/activity/stream`).

Expand All @@ -120,12 +120,15 @@ The scripts detect which clients you already have and merge an `opensentry` entr
| `CLERK_WEBHOOK_SECRET` | No | | Svix signature secret for Clerk webhooks |
| `DATABASE_URL` | No | `sqlite:///./opensentry.db` | SQLAlchemy connection string |
| `FRONTEND_URL` | No | `http://localhost:5173` | Extra CORS origin (must include scheme, no trailing slash) |
| `SEGMENT_CACHE_MAX_PER_CAMERA` | No | `15` | Segments cached in memory per camera (~2s each) |
| `SEGMENT_CACHE_MAX_PER_CAMERA` | No | `60` | Segments cached in memory per camera (~1s each) |
| `SEGMENT_PUSH_MAX_BYTES` | No | `2097152` | Max bytes per pushed segment (2 MB) |
| `CLEANUP_INTERVAL` | No | `20` | Run cache eviction every N playlist updates |
| `INACTIVE_CAMERA_CLEANUP_HOURS` | No | `24` | Free caches for cameras offline this long |
| `LOG_RETENTION_DAYS` | No | `90` | Stream, MCP, audit, motion, and notification log retention |
| `OFFLINE_SWEEP_INTERVAL_SECONDS` | No | `30` | How often to flip stale `online` rows to `offline` |
| `REDIS_URL` | No | (empty) | Redis connection for rate limiter storage; in-memory fallback when unset (fine for single-instance, but limits don't hold across VMs) |
| `SENTRY_DSN` | No | (empty) | Sentry error tracking DSN; leave blank to disable (no-ops gracefully) |
| `SENTRY_TRACES_SAMPLE_RATE` | No | `0.1` | Sentry performance trace sample rate (0.0–1.0) |

### Frontend environment variables

Expand Down Expand Up @@ -258,6 +261,7 @@ See [AGENTS.md](AGENTS.md) for the full per-tool list.
|--------|----------|-------------|
| GET | `/install.sh` / `/install.ps1` | CloudNode installer scripts |
| GET | `/mcp-setup.sh` / `/mcp-setup.ps1` | MCP client setup helpers |
| GET | `/downloads/{os}/{arch}` | 302 redirect to the latest CloudNode binary on GitHub Releases (os: `linux`/`macos`/`windows`, arch: `x86_64`/`aarch64`/`armv7`) |

### System

Expand Down Expand Up @@ -327,13 +331,20 @@ backend/
│ │ ├── ws.py # WebSocket channel (heartbeat, commands, motion events)
│ │ └── webhooks.py # Clerk subscription webhooks
│ ├── mcp/
│ │ └── server.py # FastMCP server + 22 tools + ScopeMiddleware
│ │ ├── server.py # FastMCP server + 22 tools + ScopeMiddleware
│ │ └── activity.py # MCP activity tracking + per-org broadcasting
│ ├── core/
│ │ ├── auth.py # Clerk JWT validation (V1 + V2), dependencies
│ │ ├── audit.py # Audit log helper (records admin actions)
│ │ ├── codec.py # Video codec string sanitization
│ │ ├── config.py # Environment loading (Config class)
│ │ ├── clerk.py # Clerk SDK initialization
│ │ ├── database.py # SQLAlchemy engine + session factory
│ │ └── limiter.py # slowapi Limiter instance
│ │ ├── limiter.py # slowapi Limiter instance (tenant-aware key)
│ │ ├── migrations.py # Manual schema migrations (stand-in for Alembic)
│ │ ├── plans.py # Plan limit logic (node quotas per tier)
│ │ ├── sentry.py # Sentry error tracking initialization
│ │ └── versions.py # CloudNode version compatibility checking
│ ├── models/models.py # 13 ORM models (see table above)
│ └── schemas/schemas.py # Pydantic request/response schemas
├── scripts/
Expand Down Expand Up @@ -364,8 +375,9 @@ frontend/
│ # HeartbeatBanner (first-heartbeat polling after
│ # node creation, localStorage-backed),
│ # WelcomeHero (Admin / Member empty-state heroes),
│ # CameraGridPreview, EmptyState, PublicLayout,
│ # LandingNav, LandingFooter, LoadingSpinner
│ # CameraGridPreview, DocsDiagrams, EmptyState,
│ # PublicLayout, LandingNav, LandingFooter,
│ # LoadingSpinner
├── hooks/ # useNotifications, useMotionAlerts, usePlanInfo,
│ # useSharedToken, useToasts
└── services/api.js # Typed client for every backend endpoint
Expand Down Expand Up @@ -408,7 +420,7 @@ Deployed on [Fly.io](https://fly.io) via GitHub Actions:
3. Live video segments are cached in the backend's process memory — no external storage
4. Clerk handles authentication (no user database needed)

Memory sizing: each camera uses ~3.75 MB of cache (`SEGMENT_CACHE_MAX_PER_CAMERA × ~250 KB per segment`). The default 1 GB Fly instance comfortably handles ~150 cameras with headroom. Bump `[[vm]] memory_mb` if you need more.
Memory sizing: each camera uses ~7.5 MB of cache (`SEGMENT_CACHE_MAX_PER_CAMERA × ~125 KB per 1-second segment`). The default 1 GB Fly instance comfortably handles ~130 cameras with headroom. Bump `[[vm]] memory_mb` if you need more.

Production URL: [opensentry-command.fly.dev](https://opensentry-command.fly.dev)

Expand Down
1 change: 1 addition & 0 deletions SECURITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ Always run the latest version. The current release is tracked in `backend/pyproj
| **Audit Logging** | Stream access tracked with user ID, IP, and user agent |
| **Encrypted Storage** | CloudNode encrypts API key at rest with AES-256-GCM |
| **Webhook Verification** | Clerk webhooks verified via Svix signature |
| **Security Headers** | All responses include `X-Content-Type-Options: nosniff`, `X-Frame-Options: DENY`, `Referrer-Policy: strict-origin-when-cross-origin`, `Permissions-Policy: camera=(), microphone=(), geolocation=()`, and `Strict-Transport-Security` (HTTPS) |

## Reporting a Vulnerability

Expand Down
10 changes: 10 additions & 0 deletions backend/.env.example
Original file line number Diff line number Diff line change
Expand Up @@ -25,3 +25,13 @@ INACTIVE_CAMERA_CLEANUP_HOURS=24

# Log retention (stream, MCP, and audit logs — days)
LOG_RETENTION_DAYS=90

# Redis for rate limiter storage (production).
# In dev the empty default is fine — slowapi falls back to in-memory storage.
# In production set this to a managed instance (Upstash on Fly, etc.) so
# limits hold across VMs.
# REDIS_URL=redis://localhost:6379

# Sentry error tracking (leave blank for local dev — no-ops gracefully)
# SENTRY_DSN=https://...@sentry.io/...
# SENTRY_TRACES_SAMPLE_RATE=0.1
Loading