Skip to content

feat(multiplayer): add P2P online matches with live camera verification#99

Draft
BenedictHomuth wants to merge 24 commits intomainfrom
bh/ai-dart-detection
Draft

feat(multiplayer): add P2P online matches with live camera verification#99
BenedictHomuth wants to merge 24 commits intomainfrom
bh/ai-dart-detection

Conversation

@BenedictHomuth
Copy link
Copy Markdown
Collaborator

@BenedictHomuth BenedictHomuth commented Jan 7, 2026

This PR transforms the application into a networked multiplayer platform while strictly maintaining the privacy and offline integrity of the original local mode. It introduces Peer-to-Peer (WebRTC) connectivity for remote darts matches and enhances profile customization with real-time video features.

✨ Key Features

🏠 Online Lobby System

  • Host & Join: Users can generate a unique Room ID or join an existing session via a short-code entry.
  • Live Presence: A real-time "Connected Guests" list provides visual confirmation (green status badges) to both the Host and all Guests before the match begins.
  • Auto-Start Handshake: When the Host initiates the match, an INIT_GAME signal broadcasts the configuration, automatically redirecting all guests from the Lobby to the active Game Board with synchronized settings.

📹 Camera Verification (Full Mesh)

  • Integrated Video: WebRTC video feeds are embedded directly into the gameplay view via a slide-out "Live Feeds" drawer, which sits alongside the main controls for simultaneous tracking.
  • Mesh Topology: Every participant establishes a direct connection with every other participant. This ensures that Player A can verify Player B's throws even if the Host experiences connectivity issues.
  • Dynamic Calling: Media streams are initiated as soon as cameras are activated, ensuring video handshakes happen automatically upon entering the game screen.

📸 Webcam Avatar Capture

  • Profile Integration: Added a new webcam capture flow in the profile creation/edit steps.
  • Toggleable Interface: To respect user privacy, the camera does not start automatically. A dedicated "Webcam" toggle opens a modal to preview and capture a snapshot.
  • Automatic Processing: Captured frames are drawn to a canvas, converted to WebP, and automatically resized using the project's existing avatar utility before being saved to the profile.

🔄 Real-Time Synchronization

  • Deterministic State: Every action (THROW_DART, UNDO_THROW, NEXT_TURN, TOGGLE_MULTIPLIER) is broadcast as a lightweight JSON payload. The local reducer applies these actions deterministically, ensuring scores and statistics are identical on all screens.
  • Remote Abort: If a match is aborted remotely, guests are automatically redirected to the match history view to prevent UI lock-ups.

🔒 Privacy & Offline Isolation

We have implemented a strict Lazy Initialization pattern to guarantee that "Local Mode" remains 100% air-gapped from any networking logic:

  • Null-by-Default: The Peer object (responsible for signaling and server connection) is initialized as null in the global context.
  • Explicit Activation: The network handshake (new Peer()) occurs only inside the hostGame() and joinGame() triggers. The webcam for avatars only activates upon explicit user request within the modal.
  • Mode Guarding: All gameplay broadcasting and camera logic is wrapped in a strict check: if (state.matchMode === 'online'). In a local match, the network stack is completely bypassed.

🛡️ Technical Implementation

  • Component Modularization: Extracted and generalized the VideoStream logic into a reusable component (renderer/components/media/VideoStream.tsx).
  • Ref-based Media Handling: Implemented forwardRef in the VideoStream component to allow parent components (like the Profile Creator) to access the underlying <video> element for frame capture and canvas processing.
  • Global Context: A central MultiplayerContext manages the complex lifecycle of WebRTC connections, media streams, and data channels.
  • Regression Testing: Added a specialized Vitest suite (renderer/tests/multiplayer/gameSync.test.ts) that simulates network traffic to verify state machine synchronization.
  • Internationalization (i18n): Full English (en) and German (de) localization for all new components (30+ new translation keys).

⚠️ Security Audit

  • Public Signaling: Initial handshakes use the default PeerJS cloud server; however, game data and video flow directly between peers once connected.
  • IP Exposure: Due to the P2P nature of WebRTC, participants' IP addresses are visible to each other during an active session.
  • Unauthenticated Access: Rooms are currently secured by the Room ID; there is no secondary password or "Host-Kick" mechanism in this version.

@BenedictHomuth BenedictHomuth self-assigned this Jan 7, 2026
@BenedictHomuth BenedictHomuth marked this pull request as draft January 7, 2026 23:22
@timderes
Copy link
Copy Markdown
Owner

timderes commented Jan 7, 2026

I was just thinking: maybe we should split online and local play into two separate routes @BenedictHomuth

Comment thread renderer/context/MultiplayerContext.tsx Outdated
Comment thread renderer/pages/[locale]/match/playing.tsx Outdated
BenedictHomuth and others added 6 commits January 8, 2026 22:53
Eliminated the redundant <Title> element from MatchSettingsForm and retained it in the lobby page to avoid duplicate section headers. This improves UI clarity by ensuring the match settings title appears only once.
Displays a notification when no webcam is detected during profile creation. Added localized error messages for both English and German.
@timderes

This comment was marked as outdated.

Replaced hardcoded English text in the multiplayer lobby page with localized strings. Added new translation keys for descriptions, placeholders, and button labels in both English and German locale files.
@timderes
Copy link
Copy Markdown
Owner

timderes commented Jan 9, 2026

@BenedictHomuth Its possible to connect to a random string (eg "ddd"). The UI will return a lobby with "Connect to Lobby"

timderes and others added 6 commits January 9, 2026 13:45
Replaced hardcoded English strings in the multiplayer lobby page with translation keys. Added missing translation entries for both English and German locales to support new UI labels and actions.
@BenedictHomuth
Copy link
Copy Markdown
Collaborator Author

Addressed, both the invalid lobby id and the infinite loop of the webcam

@timderes
Copy link
Copy Markdown
Owner

timderes commented Jan 9, 2026

So the invalid room id input is fix 👍

But it still crashes when I use two instances on my PC

@BenedictHomuth
Copy link
Copy Markdown
Collaborator Author

BenedictHomuth commented Jan 9, 2026

It might be that the webcam creates a race condition, when both instances "start" the multiplayer game. Here both try to access the camera at the exact same time. As Windows is to my knowledge very restrictive on this (e.g., only one process can use the camera at a time), this might be the issue here.

If you want to test this, I recommend you to explicitly deny one of the instances access to the camera/mic and check.

@timderes
Copy link
Copy Markdown
Owner

I actually tried both times without a webcam connected. Maybe thats the problem

BenedictHomuth and others added 3 commits January 11, 2026 22:27
Changed the screenshot button in StepThree to use the new 'profile:takeScreenshot' translation key. Added 'takeScreenshot' entries to both English and German locale files for profile.
@timderes timderes added enhancement New feature or request renderer Issues related to the renderer process multiplayer labels Feb 3, 2026
@BenedictHomuth
Copy link
Copy Markdown
Collaborator Author

🔄 Proposal: Migrate Multiplayer from PeerJS to Colyseus

As we consider scaling our multiplayer features, we should evaluate moving from our current Peer-to-Peer (PeerJS) architecture to an Authoritative Client-Server (Colyseus) architecture.

Here is a breakdown of the pros, cons, and major architectural shifts this migration would entail.

🌟 Pros of Switching to Colyseus

  • Authoritative Game State: Colyseus is an authoritative server framework. It prevents cheating because the server maintains the single "source of truth" rather than trusting individual peer clients.
  • Built-in State Synchronization: Automatically handles state diffs and deltas. Clients only receive exactly what changed, making network traffic highly efficient.
  • Better Reliability: P2P architectures (like PeerJS) can struggle with NAT traversal, strict firewalls, and dropped connections if the host leaves. Colyseus provides a centralized, stable server connection via WebSockets.
  • Matchmaking & Scaling: Colyseus comes with built-in matchmaking out of the box and is designed to scale horizontally across multiple node processes or servers.

⚠️ Cons & Challenges of Switching

  • No Native Media Streams (Video/Audio): PeerJS currently handles both our data synchronization AND our WebRTC media streams (webcams/mics). Colyseus only handles data (via WebSockets). We would need to either:
    1. Rebuild WebRTC signaling on top of Colyseus for media streams.
    2. Adopt a dedicated media server (like LiveKit or Mediasoup).
    3. Keep PeerJS exclusively for the video/audio feature.
  • Infrastructure Overhead: We currently rely on the free public PeerJS cloud server (0.peerjs.com) and don't host anything ourselves. Moving to Colyseus requires deploying and maintaining a dedicated backend Node.js server.
  • Paradigm Shift: Moving from a decentralized mesh network to a centralized server means rewriting the entire MultiplayerContext, game loops, and how we handle player lobbies.

🎯 Summary

Switching to Colyseus is a major architectural change. It will significantly improve data reliability, anti-cheat, and matchmaking, but it requires us to take on server hosting costs and implement a completely separate solution for our existing Video/Audio streams.

@timderes for your consideration.

@timderes
Copy link
Copy Markdown
Owner

What's your opinion @BenedictHomuth? Worth a try or should we stay with the P2P solution. I'm not really sure 😅

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request multiplayer renderer Issues related to the renderer process

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants