Skip to content

Latest commit

 

History

History
415 lines (346 loc) · 22.7 KB

File metadata and controls

415 lines (346 loc) · 22.7 KB

Architecture

System architecture, module overview, and data flow for RetroLoop.


System Diagram

┌─────────────────────────────────────────────────────────────────────────┐
│                              Client                                     │
│                                                                         │
│   Next.js 16 (App Router)         Socket.IO Client                      │
│   ┌──────────────────────┐        ┌──────────────────────┐              │
│   │  React 19 + Tailwind │        │  WebSocketProvider   │              │
│   │  AuthProvider        │        │  (real-time events)  │              │
│   │  AnalyticsProvider   │        │                      │              │
│   └────────┬─────────────┘        └──────────┬───────────┘              │
│            │ REST (fetch)                     │ Socket.IO                │
└────────────┼──────────────────────────────────┼─────────────────────────┘
             │ :3000 → :3001/api                │ :3001 (ws)
             ▼                                  ▼
┌─────────────────────────────────────────────────────────────────────────┐
│                           NestJS 11 Backend                             │
│                                                                         │
│   ┌────────────────────┐          ┌──────────────────────┐              │
│   │  REST Controllers  │          │  RetroGateway         │              │
│   │  /api/*            │          │  (Socket.IO server)   │              │
│   │                    │          │                       │              │
│   │  Guards:           │          │  Guards:              │              │
│   │   JwtAuthGuard     │          │   WsJwtGuard          │              │
│   │   ThrottlerGuard   │          │   WsRateLimitGuard    │              │
│   │   MaintenanceGuard │          │                       │              │
│   └────────┬───────────┘          │  Handlers:            │              │
│            │                      │   SessionHandler      │              │
│            │                      │   CardHandler         │              │
│   ┌────────▼───────────┐          │   VoteHandler         │              │
│   │  Services          │◄────────►│   ReactionHandler     │              │
│   │  (business logic)  │          │   TimerHandler        │              │
│   └────────┬───────────┘          │   ColumnHandler       │              │
│            │                      │   ActionItemHandler   │              │
│   ┌────────▼───────────┐          └──────────────────────┘              │
│   │  Repositories      │                                                │
│   │  (data access)     │                                                │
│   └────────┬───────────┘                                                │
│            │ Drizzle ORM                                                │
└────────────┼────────────────────────────────────────────────────────────┘
             │
             ▼
┌─────────────────────┐     ┌─────────────────────┐
│  PostgreSQL 16      │     │  Internal App :9090  │
│                     │     │  /metrics (Prometheus)│
│  15 tables          │     └─────────────────────┘
│  UUID primary keys  │
│  JSONB for boards   │
└─────────────────────┘

Monorepo Layout

retroloop/
├── apps/
│   ├── back/                 # NestJS backend
│   │   ├── src/
│   │   │   ├── main.ts                 # Bootstrap, CORS, pipes, Swagger
│   │   │   ├── app.module.ts           # Root module (Joi config validation)
│   │   │   ├── internal.module.ts      # Metrics-only app on INTERNAL_PORT
│   │   │   ├── modules/               # Feature modules
│   │   │   ├── gateways/              # WebSocket gateway + handlers
│   │   │   ├── repositories/          # Drizzle data-access layer
│   │   │   ├── database/              # Schema, migrations config, DB module
│   │   │   └── common/               # Shared guards, filters, DTOs
│   │   ├── drizzle/                   # SQL migration files
│   │   └── test/                      # E2E test setup
│   └── front/                # Next.js frontend
│       ├── app/                       # App Router (route groups)
│       ├── components/                # UI components by feature
│       ├── contexts/                  # React context providers
│       ├── hooks/                     # Custom hooks (API + UI)
│       └── lib/                       # API client, utilities
└── packages/
    └── shared-types/         # TypeScript types shared between apps
        └── src/
            ├── types/                 # Domain interfaces
            ├── enums/                 # UserRole, CardType, ActionItemStatus
            └── events/                # WebSocket event type contracts

Backend Modules

The backend is organized into NestJS feature modules. Two modules — DatabaseModule and RepositoriesModule — are registered globally so every other module can inject database access without importing them.

Core Domain

Module Controller Purpose
Auth /api/auth/* Register, login, Google OAuth, anonymous login, password reset, email verification
Users /api/users/* Profile management, password change, account deletion, GDPR data export
Workspaces /api/workspaces/* Workspace CRUD, member management, invite tokens, visibility settings
Retrospectives /api/retros/* Retro CRUD, configuration (vote limits, timer, blind voting), completion, markdown export
Boards /api/boards/* Board columns (stored as JSONB), column add/update/delete/reorder
Cards /api/cards/* Card CRUD, move between columns, optimistic locking via version field
Voting /api/votes/* Vote add/remove with atomic transactions, per-user and per-card limits
Reactions Emoji reactions on cards (WebSocket only, no REST controller)
Action Items /api/action-items/* Action item CRUD, status tracking, assignee, due dates
Templates via Workspaces Custom board templates per workspace

Platform

Module Controller Purpose
Health /api/health Terminus health checks (database connectivity)
Admin /api/admin/* System admin: user/workspace management, maintenance mode, stats aggregation
Growth /api/admin/growth/* Signup, retention, funnel, churn, feature adoption analytics
Insights /api/workspaces/:id/insights Per-workspace analytics: word cloud, sentiment, top topics, activity
Metrics /metrics (internal :9090) Prometheus metrics: HTTP latency, DB pool, system stats
Audit — (global) Audit log recording (no controller — written by other modules)
Event Tracking /api/events/* Page view tracking, user event recording
Email — (global) Mailgun integration for password reset and email verification

WebSocket Gateway

The RetroGateway handles all real-time communication. Domain logic is split into focused handler classes:

Handler Events Auth
SessionHandler retro:join, retro:leave, presence Retro access
CardHandler card:create, card:update, card:delete, card:move, card:group, card:ungroup Retro access
VoteHandler vote:add, vote:remove Retro access
ReactionHandler reaction:add, reaction:remove Retro access
TimerHandler timer:start, timer:pause, timer:resume, timer:stop Facilitator (configurable)
ColumnHandler column:update, column:reorder Facilitator only
ActionItemHandler action-item:create, action-item:update, action-item:delete Retro access

See docs/WEBSOCKET.md for the full protocol reference.


Database Schema

All tables use UUID primary keys and timestamptz columns. Managed by Drizzle ORM with forward-only SQL migrations in apps/back/drizzle/.

Entity Relationship Diagram

users
 ├──< workspaces            (ownerId)
 ├──< workspace_members     (userId)
 ├──< retrospectives        (facilitatorId)
 ├──< cards                 (authorId)
 ├──< votes                 (userId)
 ├──< reactions             (userId)
 ├──< action_items          (assigneeId, nullable)
 ├──< sessions              (userId)
 ├──< user_events           (userId, nullable)
 ├──< audit_logs            (actorId, nullable)
 └──< custom_templates      (createdBy)

workspaces
 ├──< workspace_members     (workspaceId)
 ├──< retrospectives        (workspaceId)
 ├──< action_items          (workspaceId)
 └──< custom_templates      (workspaceId)

retrospectives
 ├──1 boards                (retrospectiveId)
 ├──< cards                 (retrospectiveId)
 ├──< votes                 (retrospectiveId)
 ├──< reactions             (retrospectiveId)
 └──< action_items          (retrospectiveId)

cards
 ├──< votes                 (cardId)
 ├──< reactions             (cardId)
 └──< action_items          (cardId, nullable)

Tables

Table Key columns Notes
users email (unique), passwordHash, googleId, isAnonymous, isSystemAdmin, isSuspended Supports email/password, Google OAuth, and anonymous auth. Reset/verification tokens hashed (SHA-256).
workspaces name, ownerId → users, isPublic, inviteToken Invite tokens hashed before storage.
workspace_members (userId, workspaceId) composite PK, role Roles: participant (default), facilitator.
retrospectives workspaceId, facilitatorId, maxCardsPerUser, maxVotesPerUser, maxVotesPerCard, timerEndAt, timerPausedRemaining, isPublic, allowParticipantTimers, isBlindVoting, isCompleted 0 = unlimited for card/vote limits. Timer state persisted for server restart recovery.
boards retrospectiveId (1:1), columns (JSONB) Columns stored as BoardColumnJson[] — id, title, description, color, icon, order.
cards authorId, retrospectiveId, columnId, groupId, version groupId links cards in the same group. version enables optimistic locking.
votes userId, cardId, retrospectiveId Atomic add with row-level locking to prevent race conditions.
reactions userId, cardId, retrospectiveId, emoji 10 valid emojis enforced server-side.
action_items retrospectiveId, workspaceId, assigneeId, status, dueDate Status: openin_progressdone / cancelled.
sessions userId, isActive, durationSeconds, ipAddress, userAgent Login session tracking for analytics.
user_events userId, eventType, metadata (JSONB) Event sourcing for analytics (page views, feature usage).
audit_logs actorId, action, targetType, targetId, metadata (JSONB), ipAddress Full audit trail for admin actions.
daily_stats date (unique), counters for users/retros/cards/votes/etc. Pre-aggregated daily metrics.
weekly_stats weekStart (unique), counters Pre-aggregated weekly metrics.
custom_templates workspaceId, createdBy, columns (JSONB) Reusable board column configurations.

Frontend Architecture

Route Groups

The Next.js App Router uses route groups to apply different layouts and auth requirements:

Group Path examples Auth Layout features
(auth) /login, /signup, /forgot-password, /reset-password Redirects to /dashboard if authenticated Split-panel: branding left, form right
(app) /dashboard Required Top navigation, user menu, onboarding tour
(admin) /admin/* Required + isSystemAdmin Sidebar navigation
(public-app) /retro/[id], /workspace/[id] None WebSocketProvider, ConnectionStatus
(root) /, /docs, /privacy, /terms, /join/* None Landing page / standalone

Provider Hierarchy

<ErrorBoundary>
  <AuthProvider>                    ← checks /auth/me on mount
    <AnalyticsProvider>             ← GDPR-aware page view tracking
      <OnboardingProvider>          ← (app routes only) 3-step welcome tour
        <WebSocketProvider>         ← (retro routes only) Socket.IO connection
          {children}
        </WebSocketProvider>
      </OnboardingProvider>
    </AnalyticsProvider>
  </AuthProvider>
</ErrorBoundary>

Component Organization

components/
├── ui/            Base primitives (Button, Input, Modal, DropdownMenu, Card)
├── auth/          Login/signup forms, social login, feature spotlight
├── brand/         Logo, SVG illustrations
├── retro/         Retro board: columns, cards, settings, action items panel, template selector
├── workspace/     Workspace cards, settings modal, invite management
├── admin/         Admin sidebar, data tables, confirm dialog, maintenance banner
├── growth/        Charts (line, bar, donut, funnel, heatmap), period selectors
├── insights/      Word cloud, sentiment trends, activity charts, top topics
├── marketing/     Landing page: hero, features, about, footer
├── onboarding/    Tour tooltip, onboarding steps
├── forms/         Password input with show/hide
└── user/          Edit profile modal

API Client

lib/api-client.ts wraps all REST calls via fetch() with credentials: 'include' (httpOnly cookies). Endpoints are organized by domain — auth, users, workspaces, retrospectives, boards, cards, votes, action items, admin, growth, events. Custom hooks in hooks/api/ provide data-fetching with loading and error states.


Data Flow

Authentication

1. User submits credentials (POST /api/auth/login)
2. Backend validates, generates JWT (sub: userId, email)
3. JWT set as httpOnly cookie (auth_token), secure in production
4. Frontend AuthProvider detects auth via GET /api/auth/me
5. WebSocketProvider connects (cookie sent via withCredentials: true)
6. WsJwtGuard extracts JWT from cookie on every WS message

Retrospective Session

1. Facilitator creates retro via REST (POST /api/retros)
   → creates retrospective + board (in transaction)
2. Participants open retro page → WebSocket connects
3. Client emits retro:join(retroId)
   → server verifies access (membership or public)
   → joins Socket.IO room
   → responds with retro:joined { retro, participants }
4. All subsequent operations go through WebSocket:
   card:create → CardHandler → CardsService → Repository → DB
                               → server.to(retroId).emit('card:created', card)
5. Facilitator completes retro via REST (POST /api/retros/:id/complete)

Card Operations (WebSocket)

Client                    Server                         Database
  │                         │                               │
  │── card:create ─────────►│                               │
  │                         │── verifyRetroAccess() ───────►│
  │                         │── createWithLimitCheck() ────►│
  │                         │   (atomic transaction         │
  │                         │    + row-level lock)          │
  │                         │◄── Card ─────────────────────│
  │◄── card:created ───────│                               │
  │   (broadcast to room)   │                               │

Voting (Atomic Transaction)

Client                    Server                         Database
  │                         │                               │
  │── vote:add ────────────►│                               │
  │                         │── verifyRetroAccess() ───────►│
  │                         │── addVoteAtomic() ──────────►│
  │                         │   BEGIN                       │
  │                         │   SELECT ... FOR UPDATE       │ ← row lock
  │                         │   check maxVotesPerUser       │
  │                         │   check maxVotesPerCard       │
  │                         │   INSERT vote                 │
  │                         │   COMMIT                      │
  │                         │◄─────────────────────────────│
  │◄── vote:added ─────────│                               │

Timer Lifecycle

1. Client emits timer:start { duration }
2. Server persists timerEndAt = now + duration to DB
3. Server schedules setTimeout for expiration
4. Server broadcasts timer:started { duration, endAt }
5. Clients count down locally from endAt
6. On pause: server clears timeout, stores timerPausedRemaining
7. On resume: server computes new timerEndAt, reschedules timeout
8. On expiration or stop: server broadcasts timer:stopped, clears DB fields
9. On server restart: reconstructTimersFromDatabase() queries active timers
   and reschedules timeouts from persisted timerEndAt values

Access Control

REST requests:
  JwtAuthGuard → extract JWT from cookie → attach user to request
  ThrottlerGuard → 100 req/min global rate limit
  MaintenanceModeGuard → 503 during maintenance (exempts /health, /admin, /auth/login)
  RetroAccessGuard → check workspace membership or public access (per-endpoint)
  EmailVerifiedGuard → enforce email verification (per-endpoint)

WebSocket messages:
  WsJwtGuard → extract JWT from cookie or handshake.auth → attach user to socket
  WsRateLimitGuard → per-event sliding-window rate limit
  WsAuthorizationService.verifyRetroAccess() → workspace membership or public
  WsAuthorizationService.verifyFacilitator() → facilitator-only actions

Shared Types

The @retro/shared-types package provides TypeScript interfaces consumed by both apps. It must be built (yarn workspace @retro/shared-types build-lib) before either app can compile.

Domain Model

Type Key fields
User id, email, name, avatarUrl, isAnonymous, isSystemAdmin, isSuspended
Workspace id, name, ownerId, isPublic, inviteToken
WorkspaceMember userId, workspaceId, role (facilitator / participant)
Retrospective id, title, workspaceId, facilitatorId, vote/card limits, timer fields, feature flags
Board id, retrospectiveId, columns: BoardColumn[]
BoardColumn id, title, description, color, icon (ColumnIcon union), order
Card id, content, authorId, columnId, groupId, version, published
Vote id, userId, cardId, retrospectiveId
Reaction id, userId, cardId, emoji, retrospectiveId
ActionItem id, content, assigneeId, status (ActionItemStatus), dueDate, workspaceId

Enums

Enum Values
UserRole facilitator, participant, observer
CardType went_well, to_improve, action_item
ActionItemStatus open, in_progress, done, cancelled

WebSocket Contracts

ClientToServerEvents and ServerToClientEvents interfaces define every event name and payload type. Both apps import these to get compile-time type safety on socket emit/on calls.


Observability

Concern Implementation
Health checks GET /api/health — Terminus, checks DB connectivity
Prometheus metrics Internal app on :9090/metrics — HTTP latency, DB pool, memory, event loop lag
Audit logs audit_logs table — actor, action, target, metadata, IP
Event tracking user_events table — page views, feature usage, signup sources
Error tracking Sentry (optional, via SENTRY_DSN)
Logging Pino with correlation IDs, request/response serialization
WebSocket metrics wsMetricsStore — active connections, rooms, participants

Security

Layer Mechanism
Authentication JWT in httpOnly / sameSite=strict / secure cookies
Password storage bcrypt (10 rounds)
Token storage SHA-256 hashed before persisting reset/verification/invite tokens
CORS Origin restricted to FRONTEND_URL, credentials enabled
Rate limiting REST: 100 req/min global (ThrottlerGuard) + per-endpoint throttles. WebSocket: per-event sliding window (WsRateLimitGuard)
Input validation Global ValidationPipe (whitelist, forbidNonWhitelisted, transform)
Headers Helmet.js security headers
Error sanitization Production errors stripped of stack traces; correlation ID returned
Maintenance mode Global guard returns 503, exempts health/admin/login
Config validation Joi schema enforces required vars and formats at startup