▍ B L U E L I N E

A self-driving car that runs a real autonomy stack in your browser —
and a neural driver you train, evolve, and race against it. Live. No backend.

Perception → Planning → Control · a classical stack   •   Trained live · imitation + neuroevolution   •   0 collisions · 9 scenarios

How it works  ·  Watch it drive  ·  The neural drivers  ·  Proof  ·  Run it

Two schools of autonomy, side by side

Blueline is a working implementation of the modular autonomy pipeline from real self-driving
research — visualised like a Tesla — plus a second, learned stack you train yourself.
Two entire approaches, directly comparable, at 60 FPS in a browser tab.
_{~6,400 lines of TypeScript. No game engine. No ML library. No server.}

🧭 Classical stack Every named algorithm, wired end to end — Frenet lattice planning, IDM/MOBIL traffic, Kalman tracking, Stanley control. Verified headless in Node.

🧠 Learned stack A pure-TS MLP (backprop + Adam) trained by behavioural cloning + DAgger, and a policy found by neuroevolution — no teacher, no framework.

📊 Scorecard + race Live safety / comfort / efficiency scoring, and a head-to-head benchmark of all three drivers on one identical seeded course.

🎬 Watch it drive

Chase cam — cruising the block, lane markings, live telemetry.	Neural net at the wheel — trained live (watch the loss fall), then driving.
Top-down — a rounded 90° junction, cross-streets, radar, occupancy grid.	Rain — weather physically degrades sensor range (95 m → 68 m).

The modular autonomy pipeline

Every arrow is a real module. The ego never reads ground truth — it plans off noisy
sensor detections fused by a Kalman tracker, exactly like the real thing.

%%{init: {'theme':'base','themeVariables':{'primaryColor':'#1b1d22','primaryTextColor':'#e8eaed','primaryBorderColor':'#1fd18b','lineColor':'#5fe6ad','fontSize':'14px'}}}%%
flowchart LR
  subgraph WORLD["🌍 World"]
    ENV["Traffic · IDM + MOBIL<br/>Pedestrians · Lights · Weather"]
  end
  subgraph PERCEPTION["👁 Perception"]
    SENSE["Sensor<br/>noise · occlusion · dropout"] --> TRACK["Kalman Tracker<br/>const-velocity + association"]
    TRACK --> GRID["Occupancy Grid"]
  end
  subgraph DECIDE["🧠 Decision"]
    PRED["Prediction"] --> FSM["Behaviour FSM<br/>cruise · follow · overtake<br/>yield · stop · emergency"]
    FSM --> PLAN["Frenet Lattice Planner<br/>quintic + quartic · cost-ranked"]
  end
  subgraph ACT["🕹 Control"]
    STANLEY["Stanley + PID"] --> BICYCLE["Kinematic Bicycle"]
  end
  ENV --> SENSE
  TRACK --> PRED
  GRID --> PLAN
  PLAN --> STANLEY
  BICYCLE -->|"ego pose"| ENV

  LEARN["🎓 Learned driver<br/>MLP · imitation / evolved"] -.->|"can replace<br/>Decision + Control"| STANLEY
  PLAN -.->|"demonstrations"| LEARN

  classDef s fill:#16181d,stroke:#2b2e34,color:#e8eaed;
  classDef learn fill:#1a2b22,stroke:#1fd18b,color:#5fe6ad;
  class ENV,SENSE,TRACK,GRID,PRED,FSM,PLAN,STANLEY,BICYCLE s;
  class LEARN learn;

Loading

The stack, module by module — with source links

Layer	Technique	Source
World model	Arc-length Frenet frame (station s, lateral d) + curvature over a closed centreline	world/ReferencePath.ts
Perception — sensing	Range + line-of-sight sensor with Gaussian noise, occlusion & random dropout	perception/Sensor.ts
Perception — tracking	Kalman-filter multi-object tracker (const-velocity), NN data association, track lifecycle, object classes	perception/Tracker.ts
Occupancy	Ego-centred occupancy grid	perception/OccupancyGrid.ts
Prediction	Constant-velocity roll of each track's estimated velocity (lateral term for pedestrians)	planner/FrenetPlanner.ts
Behaviour	Finite-state machine: cruise / follow / overtake / emergency / yield / stop	behavior/BehaviorPlanner.ts
Planning	Frenet lattice: quintic + quartic polynomial trajectories, cost-based selection, collision checking	planner/FrenetPlanner.ts · core/poly.ts
Control	Stanley lateral tracker + PID longitudinal	control/
Vehicle	Kinematic bicycle model	vehicle/Vehicle.ts
Traffic	IDM car-following + MOBIL lane changes	traffic/
Urban	Traffic lights with stop-line planning	world/TrafficLight.ts

The entire simulation core imports zero Three.js and runs headless in Node — which is exactly why every claim below is a passing test, not a screenshot.

📐 The mathematics behind the blue line

Every box in that diagram is a real equation the car solves ~60 times a second.
Here is the whole stack, straight from the source — nothing hand-waved.

🛣 Geometry — the Frenet frame

The world is re-expressed along the road: station $s$ (distance along) and lateral offset $d$ (distance off centre). A world point, and the road's curvature, are

$$\mathbf{p}(s,d) = \mathbf{c}(s) + d\mathbf{n}(s) \qquad \mathbf{n}(s) = \big(-\sin\theta(s),\ \cos\theta(s)\big)$$

$$\kappa(s) = \left|\frac{d\theta}{ds}\right| \approx \frac{\big|\theta(s+\Delta s) - \theta(s-\Delta s)\big|}{2\Delta s}$$

so the car knows precisely how hard to slow for a bend, capping speed at the lateral-acceleration limit $a^{\text{lat}}_{\max}=2.2$ m/s²:

$$v_{\max}(s) = \sqrt{\frac{a^{\text{lat}}_{\max}}{\kappa(s)}}$$

_{→ world/ReferencePath.ts · sim/Simulation.ts}

🧭 Planning — the Werling Frenet lattice

Each frame samples a lattice of manoeuvres. Lateral motion is a quintic (six boundary conditions — position, velocity and acceleration pinned at both ends); longitudinal is a quartic (terminal position left free):

$$d(t) = \sum_{i=0}^{5} a_i t^i \qquad\qquad s(t) = \sum_{i=0}^{4} b_i t^i$$

Every candidate is scored, and the car draws the arg-min:

$$J = \underbrace{k_j\int_0^T\big(\dddot{d}^2 + \dddot{s}^2\big)dt + k_t T}_{\text{comfort}} + \underbrace{k_v(v_d - v_T)^2}_{\text{speed}} + \underbrace{k_{lc}|\ell - \ell_0| + k_b|\ell - \ell^{\star}|}_{\text{lane}} + \underbrace{k_p\sum_{\text{obstacles}}\frac{1}{\text{gap}}}_{\text{proximity}}$$

_{colliding candidates take a +10⁶ penalty, dynamically-infeasible ones +10⁵ → planner/FrenetPlanner.ts · core/poly.ts}

🚗 Traffic — IDM + MOBIL

Ambient cars follow the Intelligent Driver Model — a desired gap, and an acceleration that closes it:

$$s^{\star}(v,\Delta v) = s_0 + \max\left(0,\ vT + \frac{v\Delta v}{2\sqrt{ab}}\right) \qquad a_{\text{IDM}} = a\left[1 - \left(\frac{v}{v_0}\right)^{\delta} - \left(\frac{s^{\star}}{s}\right)^{2}\right]$$

and change lanes by MOBIL only when the net incentive beats a threshold — never braking the new follower past $b_{\text{safe}}$:

$$\Delta a_{\text{self}} + p\big(\Delta a_{\text{new}} + \Delta a_{\text{old}}\big) + a_{\text{bias}} > \Delta a_{\text{th}}$$

_{→ traffic/IDM.ts · traffic/MOBIL.ts}

🕹 Control — Stanley + PID + bicycle

Steering drives both the heading error $\psi_e$ and the cross-track error $e$ to zero:

$$\delta = \psi_e + \arctan\left(\frac{ke}{v + k_{\text{soft}}}\right)$$

Speed is a PID on the target-speed error; the vehicle itself is a kinematic bicycle — nonholonomic, "can't slide sideways":

$$u = k_p e + k_i\int e\ dt + k_d\dot e \qquad\qquad \dot x = v\cos\psi \quad \dot y = v\sin\psi \quad \dot\psi = \frac{v}{L}\tan\delta \quad \dot v = a$$

_{→ control/Stanley.ts · control/PID.ts · vehicle/Vehicle.ts}

👁 Perception — the Kalman filter

Each tracked object is a constant-velocity state $\mathbf{x} = [p_x, p_y, v_x, v_y]^{\top}$, smoothed by a Kalman filter that predicts, then corrects against the noisy detection $\mathbf{z}$:

$$\hat{\mathbf{x}}^{-} = F\hat{\mathbf{x}} \qquad P^{-} = F P F^{\top} + Q$$

$$\mathbf{y} = \mathbf{z} - H\hat{\mathbf{x}}^{-} \qquad S = H P^{-} H^{\top} + R \qquad K = P^{-} H^{\top} S^{-1}$$

$$\hat{\mathbf{x}} = \hat{\mathbf{x}}^{-} + K\mathbf{y} \qquad P = (I - K H)P^{-}$$

_{the ego plans off the estimate, never ground truth → perception/Tracker.ts}

🧠 Learning — MLP, backprop, Adam

The neural driver is a tanh MLP with a linear head, trained to minimise MSE against the expert's action:

$$a^{(l)} = \tanh\big(W^{(l)}a^{(l-1)} + b^{(l)}\big) \qquad \mathcal{L} = \frac{1}{2N}\sum_{n} \big\lVert a^{(L)}_n - y_n \big\rVert^2$$

Gradients flow back through the tanh, and Adam takes the step:

$$\delta^{(L)} = a^{(L)} - y \qquad \delta^{(l)} = \big(W^{(l+1)\top}\delta^{(l+1)}\big)\odot\big(1 - a^{(l)}\odot a^{(l)}\big)$$

$$m \leftarrow \beta_1 m + (1-\beta_1)g \quad v \leftarrow \beta_2 v + (1-\beta_2)g^2 \quad \theta \leftarrow \theta - \frac{\eta\hat{m}}{\sqrt{\hat{v}} + \epsilon}$$

_{β₁=0.9, β₂=0.999, ε=10⁻⁸, Xavier init, tanh′ = 1−a² → learn/NN.ts}

🧬 Evolution — survival of the fittest driver

With no teacher, genomes are scored by the worst of two rollouts (so the winner must generalise, not memorise), then bred by tournament selection + uniform crossover + annealed Gaussian mutation:

$$\mathcal{F}(\theta) = \min_{r\in{1,2}} \sum_{t}\Big(\underbrace{v_t\Delta t}_{\text{progress}} - 0.6\max(0,\ |d_t|-2)\Delta t\Big) - \text{penalty}_{\text{crash / off-road}}$$

$$\sigma_g = 0.25\left(1 - \frac{g}{G}\right) + 0.03 \qquad \text{(mutation scale, annealed over generations)}$$

_{→ learn/Evolution.ts}

📊 The scorecard

And all three drivers are ranked on one honest scale:

$$\text{safety} = 100 - 30 n_{\text{col}} - 120 f_{\text{off-road}} - 5\max(0,\ 6 - g_{\min})$$

$$\text{comfort} = 100 - 14\overline{|a|} - 5\overline{|j|} \qquad\quad \text{efficiency} = 100\cdot\frac{\bar{v}}{v_d}$$

$$\text{overall} = 0.5\cdot\text{safety} + 0.25\cdot\text{comfort} + 0.25\cdot\text{efficiency}$$

_{→ sim/Metrics.ts}

_{Geometry to gradients — that's the whole car, ~6,400 lines of it.}

🧠 The learned drivers — no ML library

A pure-TypeScript MLP with backprop + Adam (learn/NN.ts) learns to drive from a
16-feature view of the road. Two ways to teach it — watch the loss curve fall in real time.

🎓 Imitation + DAgger The classical stack drives; the network records (state → action) demonstrations and clones them. Then DAgger flips it — the learner drives while the expert labels the states it actually visits, curing the covariate-shift drift that sinks naive cloning. Result: holds its lane at max 0.99 m off centre (half-width 5.55 m) over 1.05 km, 0 collisions. → ImitationAgent.ts · Trainer.ts

🧬 Neuroevolution — no teacher A genetic algorithm evolves the weights, scored purely by driving-rollout fitness (distance minus penalties for leaving the road or hitting traffic). Selection + crossover + mutation discover a driver from random weights. Result: fitness climbs 53 → 378 over 16 generations; the champion then drives 1.52 km, 0 collisions. → Evolution.ts

A safety shield (AEB + lane-keeping + virtual guardrail) guards the learned drivers in play —
but switches off during evolution, so fitness judges the raw policy, not the shield.

📊 Proof it actually works

Everything is verified headless and deterministically in Node (seeded RNG) — no browser,
no hand-waving. Reproduce every number yourself (see Run it).

_{In-browser head-to-head — all three drivers over one identical seeded course. Classical wins on balance;
the evolved policy is fastest but rougher. All three: zero collisions.}

smoke.ts — the classical stack across 9 scenarios

[highway]     0.74 km   CRUISE                                  crashes 0   ped-hits 0
[highway@fast]1.09 km   CRUISE                                  crashes 0   ped-hits 0
[stalled]     0.51 km   CRUISE · OVERTAKE                       crashes 0   ped-hits 0
[cutin]       0.47 km   CRUISE · OVERTAKE                       crashes 0   ped-hits 0
[trucks]      0.58 km   CRUISE · OVERTAKE · FOLLOW              crashes 0   ped-hits 0
[crossing]    0.53 km   CRUISE · YIELD · OVERTAKE · STOP        crashes 0   ped-hits 0
[occluded]    0.58 km   CRUISE · OVERTAKE · YIELD · STOP        crashes 0   ped-hits 0
[jaywalker]   0.53 km   CRUISE · YIELD · STOP                   crashes 0   ped-hits 0
[rush]        0.62 km   CRUISE · YIELD · STOP                   crashes 0   ped-hits 0

SMOKE PASS — all scenarios: on-road, zero collisions, pedestrians safe.

learn-test.ts — the net learns DAgger 4: samples 60000, loss 0.0230 network now driving: [highway] 1.05 km | max|d| 0.99 (half 5.55) | crash 0 [trucks] 0.61 km | max|d| 1.14 (half 5.55) | crash 0 LEARN-TEST PASS

evo-test.ts — evolution from scratch gen 16/16: best 378.2, avg 151.4 fitness: 53.1 -> 378.2 champion drives: 1.52 km | crash 0 EVO-TEST PASS

Plus lights-test.ts (stops on red, goes on green) and weather-test.ts (rain/fog degrade the sensor). All green.

🎬 One-click scenarios

Highway · Trucks (overtake a convoy) · Stalled car · Cut-in · Dense traffic ·
Crossing pedestrian · Occluded ped (steps out from behind a stalled car, seen late) ·
Jaywalker (emergency stop) · 🚦 Lights · 🌆 Rush hour

Deep-link any of them: ?scenario=occluded&cam=top&weather=rain

Run it

npm install
npm run dev        # → http://localhost:5173
npm run build      # typecheck + production bundle → dist/  (drop on any static host)

Headless verification — reproduce every number above, no browser needed

npx tsx scripts/smoke.ts        # 9 scenarios: on-road, zero vehicle & pedestrian collisions
npx tsx scripts/learn-test.ts   # the neural net learns to drive (imitation + DAgger)
npx tsx scripts/evo-test.ts     # neuroevolution discovers a driver from random weights
npx tsx scripts/lights-test.ts  # ego obeys the traffic light
npx tsx scripts/weather-test.ts # rain/fog degrade perception

Each asserts real, measurable behaviour — stays on road, zero collisions, loss falls, fitness climbs, red light obeyed.

Keyboard  ·  1–9 scenarios  ·  T train  ·  E evolve  ·  C/N/V switch driver  ·  Space pause  ·  R reset  ·  B benchmark

Under the hood

TypeScript · Three.js · Vite. Fixed-timestep deterministic physics & planning, decoupled from the render loop. The neural networks — MLP with backprop + Adam, and the genetic algorithm — are written by hand; there is no TensorFlow, no PyTorch, no ONNX. No backend, no build-time secrets. It's a static site.

The render layer (Three.js + UnrealBloom, the green path ribbon, candidate-trajectory viz, the Tesla-style DOM HUD) is a thin skin over a headless core — which is why the same code that renders in the browser is the code the tests drive in Node.

Roadmap

• Classical AV stack — Frenet planner, IDM/MOBIL, Stanley + PID, Kalman tracking
• Pedestrians + hard cases — crossing / occluded / jaywalker
• City map — rounded 90° junctions, cross-streets, traffic lights
• Imitation learning driver — behavioural cloning + DAgger
• Neuroevolution driver — learn from scratch, no teacher
• Analytics — live scorecard + head-to-head benchmark
• Weather — rain/fog that degrades perception
• Intersections with cross-traffic & turns
• Reinforcement learning (policy-gradient) driver
• Recording & replay

_{Built by @codewithfourtix — perception, planning, and a little bit of evolution.}