Eldarin is a GPS-denied autonomous drone navigation stack that replaces GPU-heavy visual navigation with neuromorphic processing (event cameras + spiking neural networks) and Hyperdimensional Computing (HDC).
The system is partitioned into three compute domains with distinct latency/criticality profiles, as specified in the System Architecture Outline:
| Domain | Hardware | Responsibilities | Loop Character |
|---|---|---|---|
| Inner Loop | 1+ high-performance MCUs | EKF sensor fusion, LQR motor control, motor ESC interface | Hard real-time, kHz-rate, flight-critical |
| Neuromorphic Perception | FPGA implementing SNN runtime | Event-stream processing, SNN feature tracking | Event-driven, asynchronous, sparse |
| Mission / State Layer | Cortex-A flight computer (RPi-class), ROS2, NEON SIMD | HDC-EVIO, HDC-SLAM, path planning, payload integration | Soft real-time, 10s–100s of Hz |
Design principle: wherever a conventional pipeline would use a GPU-intensive algorithm (dense optical flow, feature-based VIO, dense SLAM, learned perception), substitute a sparse, event-driven, or hyperdimensional equivalent.
┌───────────────────────────────────────────────────────────────────┐
│ SENSOR SUITE │
│ Stereo Event Cameras │ 6-DoF IMU │ Optical Flow │ IR Lasers │
│ Barometer │ Payload Sensors │
└───────────────┬───────────────────────────────────┬───────────────┘
│ │
┌───────────────▼──────────┐ ┌─────────────────────▼───────────────┐
│ FPGA — Neuromorphic │ │ MCU — Inner Loop (hard real-time) │
│ • SNN Feature Tracking │ │ • Extended Kalman Filter │
│ • Event-stream encode │ │ • LQR Motor Controller │
│ • (opt) HV encoding │ │ • Motor ESC interface │
│ Event-driven, sparse │ │ kHz-rate, flight-critical │
└───────────────┬──────────┘ └─────────────────────┬───────────────┘
│ │
│ SensorPackets │ StateEstimate
▼ ▼ corrections
┌───────────────────────────────────────────────────────────────────┐
│ Cortex-A — Mission / State Layer (soft real-time, NEON SIMD) │
│ │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────────────┐ │
│ │ HDC-EVIO │ │ HDC-SLAM │ │ Path Planning │ │
│ │ Ego-motion │ │ Landmark │ │ • Waypoint following│ │
│ │ estimation │ │ map + loop │ │ • Coverage patterns │ │
│ │ from HD │ │ closure │ │ • Payload-adaptive │ │
│ │ vectors │ │ │ │ • Obstacle avoidance│ │
│ └──────┬───────┘ └──────┬───────┘ └──────────┬───────────┘ │
│ │ │ │ │
│ └─────────────────┴──────────────────────┘ │
│ Control targets → MCU LQR │
└───────────────────────────────────────────────────────────────────┘
The MCU inner loop can keep the vehicle stable using only IMU + optical flow + distance sensors, even if the FPGA or flight computer degrades or restarts. Higher layers improve the state estimate and provide goals; they are not required for basic stabilization.
# Run the full navigation system simulation
python main_navigation.py --config config/navigation.yaml --simulate --duration 60
# Run component self-tests
python main_navigation.py --test_all
# Benchmark HDC-EVIO pipeline
python main_navigation.py --benchmark_evio
# Benchmark HDC-SLAM pipeline
python main_navigation.py --benchmark_slam# Training
python main.py --config config/train_visdrone.yaml --data_root /path/to/VisDrone
# Inference
python inference.py --checkpoint checkpoints/best_model.pth --input video.mp4
# Visual Odometry (training-free)
python inference.py --config config/inference.yaml --mode vo --input events.npygit clone https://github.com/Enotrium/Eldarin.git
cd Eldarin
# Create conda environment
conda create -n eldarin python=3.10
conda activate eldarin
# Install PyTorch
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118
# Install core dependencies
pip install -r requirements.txt
# Optional: SNN framework for FPGA deployment
pip install snntorch lava-numpyPurpose: extract usable feature information from the asynchronous event stream without frame reconstruction or GPU inference. The FPGA implements a neuromorphic SNN runtime — neuron/synapse update logic, spike routing, and an interface for loading trained network weights. Output is sparse activations, keeping downstream bandwidth and power low.
Purpose: replace computationally expensive sensor fusion (especially event-based VIO) and dense SLAM with operations on high-dimensional binary/bipolar vectors (bind, bundle, permute, similarity search).
Two pipelines:
- HDC-EVIO — ego-motion estimation from encoded spatial-inertial hypervectors (based on Renner et al. 2024, Nature Machine Intelligence)
- HDC-SLAM — compact associative memory of landmark hypervectors for place recognition and mapping
Implementation notes:
- Torchhd is mature for prototyping, but a custom bit-packed implementation is preferred for deployment
- NEON SIMD on Cortex-A is essential: 128-bit XOR, popcount, and related primitives are the inner-loop operations of binary HDC
- Flight computer should be at least quad-core so HDC-EVIO, HDC-SLAM, and path planning run concurrently on dedicated cores
| Sensor | Measures | Primary Consumers |
|---|---|---|
| Stereo event cameras | Per-pixel brightness changes (sparse, µs-latency) | SNN feature tracking on FPGA |
| 6-DoF IMU | Linear acceleration, angular velocity | MCU EKF; spatial-inertial hypervector encoding |
| Downward optical flow camera | Frame-to-frame shifts (ground-relative velocity) | MCU EKF; spatial-inertial hypervector encoding |
| Infrared laser rangefinders | Directional depth / distance | MCU EKF; both hypervector encoders |
| Barometer | Absolute altitude | Spatial-inertial and feature hypervector encoding |
| Payload sensors | Mission-specific | Path planning (mission goal adjustment) |
- Event camera stream → SNN Feature Tracking on the FPGA
- SNN output fans out to two encoders:
- Spatial-Inertial Hypervector Encoding — fuses SNN features with IMU, optical flow, IR depth, barometer
- Feature Hypervector Encoding — encodes landmark/appearance information for mapping
- Spatial-inertial hypervectors → HDC-EVIO (event visual-inertial odometry)
- Feature hypervectors → HDC-SLAM (compact associative map of landmarks)
- HDC-EVIO sends accurate state corrections back down to the MCU's EKF
- Path Planning consumes position/motion (HDC-EVIO), the environment map (HDC-SLAM), and mission goals derived from payload sensor data
- Path planning emits control targets (waypoints) to the MCU
- Extended Kalman Filter fuses IMU, optical flow, and IR distance at high rate, corrected periodically by HDC-EVIO
- LQR Controller takes the state estimate and control targets, computes actuator commands
- Motor ESC executes commands
| Milestone | Description |
|---|---|
| M1 — Stable flight | Embedded team's EKF + LQR flying with IMU/optical-flow/distance only |
| M2 — Perception bring-up | SNN runs on FPGA against recorded event data; HDC pipelines run offline |
| M3 — CPU benchmark gate | HDC latency/power numbers on Cortex-A drive FPGA-vs-CPU encoding decision |
| M4 — Closed-loop integration | HDC-EVIO corrections feeding the live EKF; HDC-SLAM map feeding path planning |
| M5 — Mission demo | GPS-denied flight with payload-driven waypoint adjustment |
- Where does hypervector encoding live? Benchmark HDC encoding latency/throughput on the Cortex-A (with NEON) first. Move encoding (and possibly more) to FPGA only if CPU numbers don't meet budget.
- MCU count and partitioning — one MCU running EKF + LQR, or split estimation and control across two.
- Torchhd vs. custom bit-packed HDC — prototype in Torchhd, define a migration path to a custom implementation.
- SNN architecture and training method — network topology, training approach, and weight deployment to FPGA runtime.
- Hardware selection — specific MCUs, FPGA part, Cortex-A board, and sensors; confirm domestic supply chain availability.
| Dataset | Modalities | Task |
|---|---|---|
| VisDrone | RGB | Detection + Tracking |
| UAVDT | RGB | Vehicle Detection/Tracking |
| UAV3D | RGB + 3D Boxes | 3D Detection/Tracking |
| FRED | RGB + Event | Drone Detection |
| MVSEC | Stereo + Event | Multi-vehicle |
| Event Camera Dataset | Events + IMU + Ground Truth | Visual Odometry |
| Eq. | Name | Formula |
|---|---|---|
| 1–2 | FPE Encoding | s = Σ_{(x,y)∈E} h₀ˣ ⊗ v₀ʸ |
| 3 | Codebook | s = Φ · I |
| 4 | Hierarchical Resonator | ĥ(t+1) = (1-γ)·ĥ(t) + γ·f(H H† (ŝ(t) ⊙ v̂* ⊙ m̂*)) |
| 5–7 | Population Vector | h_out = Σ i·h_sim(i) / Σ h_sim(i) |
| 8 | Camera→Map | m(t) = Λ(s(t) ⊗ h^{h_out} ⊗ v^{v_out}) ⊗ r^{r_out} |
| 9 | Anchored Map Update | m̂(t+1) = μ₁·m̂(t) + μ₂·m̂(0) + (1-μ₁-μ₂)·m(t) |
| 10 | IMU Fusion | r̂(t) = r̂(t-1) ⊗ r_seed^{IMU(t)} |
MIT License. See LICENSE file.
Areas of particular interest:
- EKF/LQR tuning for real hardware
- HDC-EVIO accuracy benchmarking against conventional VIO
- SNN-on-FPGA throughput characterization
- Multi-UAV swarm consensus testing
- Payload-adaptive mission planning extensions
- NEON SIMD optimisation of HDC operations
Links: System Outline | Renner et al. VO | FPGA Event Encode | arthedain-1 VSA/HDC | Yan et al. 2026