feat: add lane detection deployment pipeline#787
Conversation
|
✅ STM32 CI: firmware build succeeded
|
🔍 TSF Validation Results
📋 Lint Output📊 Traceability Graph✅ Graph generated successfully Download artifacts to view: TSF Validation Results
🧪 Unit Test ResultsOverall Status: ✅ PASSED
📊 CoverageCoverage report available in artifacts. 🔍 Coverage Change Validation
Full coverage reports available in workflow artifacts 🔒 CodeQL (filtered SARIF summary)
🧪 Unit Test ResultsOverall Status: ✅ PASSED
📊 CoverageCoverage report available in artifacts. 🔍 Coverage Change Validation
Full coverage reports available in workflow artifacts
|
melaniereis
left a comment
melaniereis
left a comment
There was a problem hiding this comment.
Overall, this is a strong contribution. The runtime is well structured, and the README explains the deployment flow clearly.
I like that the pipeline separates Hailo inference from CPU post-processing, and that the main output is exposed as lane_lines, which is the right abstraction for a future steering controller.
I would request a few changes before merging, mainly to make the runtime safer and the documentation more aligned with the code.
Main code points:
preprocess()modifies the original frame in-place, which can affect debug or recorded output.decode()should validate all expected Hailo output groups before indexing them.- In
compute_steering(), I have one question: is it intentional thatnp.interp()still returns a value when the line does not reach the lookahead row? If yes, the docstring should mention it. If not, we should skip those lines. --recordshould probably require--debug, otherwise it silently does nothing.- Camera subprocess cleanup could be more robust by waiting for the process to terminate.
For the README:
- “smoothing” should probably be described as temporal persistence;
- the code can fit one or more line segments per lane class, not always one line per lane;
- debug overlays are only useful when recorded, since the script does not display them;
steeringshould be described as an experimental starter signal, not final control logic;crosswalkis detected by the model but excluded fromlane_lines.
The perception pipeline is solid and close to merge. I would just adjust these points first to avoid confusion and keep a clear separation between lane geometry extraction and future vehicle control.
|
|
||
| def decode(outputs): | ||
| cv2_list, cv3_list, cv4_list, proto = sort_outputs(outputs) | ||
| if proto is None or len(cv2_list) != 3: |
There was a problem hiding this comment.
I think we should validate all the expected output groups before indexing them.
Right now we only check proto and cv2_list, but later the code also accesses cv3_list[i] and cv4_list[i] inside the loop. If one of those lists is incomplete, the runtime could fail with an IndexError.
Maybe this check should include all three output groups:
if (
proto is None
or len(cv2_list) != 3
or len(cv3_list) != 3
or len(cv4_list) != 3
):
return [], protoThis would make the decoder more robust if the HEF output structure changes or if an inference result is incomplete.
| for tensor in outputs.values(): | ||
| t = np.squeeze(tensor).transpose(2, 0, 1) | ||
| c, h, w = t.shape | ||
| if h == MODEL_SIZE // 4 and c == 32: |
There was a problem hiding this comment.
This output sorting works, but it depends heavily on tensor shapes.
That is practical for this deployment, but it could be fragile if the model or HEF output structure changes later. In that case, a tensor could be classified into the wrong group without an obvious error.
Could we add optional debug logging of the output names and shapes, at least on the first inference when --debug is enabled?
For example:
if debug:
for name, tensor in raw.items():
print(name, tensor.shape)It does not need to print every frame, but having this available would make future model updates easier to debug.
| order = np.argsort(ys) | ||
| # interpolate x at the lookahead row; if the line doesn't reach it, | ||
| # np.interp clamps to the nearest endpoint (so we still get a value) | ||
| xs.append(float(np.interp(y_look, ys[order], pts[order, 0]))) |
There was a problem hiding this comment.
Question/doubt: is it intentional that we still compute a steering point when the line does not actually reach the lookahead row?
The docstring says this function returns None if no line reaches the lookahead row. However, np.interp() clamps to the nearest endpoint when y_look is outside the detected line range. So we may still get a value from a line that does not really cover the lookahead area.
If the intended behavior is to only use lines that reach the lookahead row, maybe we should skip lines outside that range:
if y_look < ys.min() or y_look > ys.max():
continue
xs.append(float(np.interp(y_look, ys[order], pts[order, 0])))If the clamping behavior is intentional, then I think the docstring should be updated to make that explicit.
| class_masks = build_class_masks(detections, proto) | ||
| class_masks = memory.update(class_masks) # temporal | ||
| lane_lines = extract_lane_lines(class_masks, w, h) # geometry | ||
| steering = compute_steering(lane_lines, w, h) |
There was a problem hiding this comment.
At the moment, compute_steering() averages all detected lane line positions together. That is useful to expose a basic lateral error, but the actual control strategy still needs to decide which lane classes define the drivable corridor, how to behave when only one lane line is visible, how to handle stale temporal detections, and how the final controller will consume this signal. So I would keep this as experimental/helper output for now and make sure the README also describes steering as a starter signal, not as final control logic.
| source .venv/bin/activate | ||
| python main.py --config training_config.yaml | ||
| python3 deploy_lanes_headless.py # driving (headless, ~30 FPS) | ||
| python3 deploy_lanes_headless.py --debug # also draw masks/lines + steering overlay |
There was a problem hiding this comment.
I think the debug command may be slightly misleading as written.
The code creates the debug overlay, but it does not display a window with cv2.imshow(). The overlay only becomes visible/useful if we save it with --record.
Maybe the README should show:
python3 deploy_lanes_headless.py --debug --record dbg.avi
# save annotated debug video with masks, fitted lines, and steering overlayThis would be clearer for a headless Pi workflow.
| python3 deploy_lanes_headless.py # driving (headless, ~30 FPS) | ||
| python3 deploy_lanes_headless.py --debug # also draw masks/lines + steering overlay | ||
| python3 deploy_lanes_headless.py --source clip.mp4 # test on a video | ||
| ``` |
There was a problem hiding this comment.
Since the script supports --record, I think we should document it in the run examples.
Suggested addition:
python3 deploy_lanes_headless.py --debug --record dbg.avi
# save annotated debug videoThis is probably the most useful way to inspect the debug output when running headless on the Pi.
| Each frame the loop produces: | ||
| - `lane_lines` — `{class_name: [ (N,2) float arrays in frame pixels ]}` — the | ||
| fitted lane geometry (this is what a steering controller consumes). | ||
| - `steering` — `(target_x, err)` starter signal (`err ∈ [-1 left, +1 right]`). |
There was a problem hiding this comment.
I would describe steering more carefully here so it is not confused with final control logic.
The current runtime exposes a starter/debug signal, not a complete steering controller. Also, the code does not explicitly clamp err to [-1, +1].
Maybe this could be reworded as:
- `steering` — `(target_x, err)` experimental starter signal. `err` is the
normalized lateral offset of the detected target from the image center:
negative = target left of center, positive = target right of center.
This is not a final steering controller.This keeps the distinction clear between perception output and future control logic.
| `LOOKAHEAD_FRAC`. | ||
|
|
||
| ## Model facts | ||
| - YOLOv8n-seg, 5 classes: `center_continuous_lane, center_dashed_lane, |
There was a problem hiding this comment.
Could we mention that crosswalk is detected by the model but excluded from lane_lines? The model has a crosswalk class, but the runtime only extracts lane geometry from the lane classes. That distinction may be useful for future contributors.
Suggested addition:
`crosswalk` is detected as a segmentation class for scene awareness/debug, but it is excluded from the lane-line geometry consumed by the future steering controller.| compile). The training weights and dataset are kept in the team's model/data | ||
| store, not in this repo. | ||
|
|
||
| > **Why v8 and not v11?** The module was originally planned around YOLO11n-seg. |
There was a problem hiding this comment.
This explanation is useful and I think it should stay, because it explains an important model decision.
Small suggestion: the README could keep a shorter version, while the full DFC error and troubleshooting details live in compilation.md.
For example:
YOLO11n-seg was tested but did not produce a deployable Hailo-8 `.hef` with the
standard DFC flow because of the `C2PSA` attention block. YOLOv8n-seg is
pure-convolutional, compiles cleanly, and met the runtime target.This keeps the README readable while preserving the technical details elsewhere.
2 participants
Pull Request
Related issue(s): #680
Type of change
Summary
Adds the lane-detection deployment path for the Pi/Hailo pipeline, including the headless runtime, compiled model assets, and supporting documentation for install, compilation, and usage.
How to test / Validation
Run
python3 ADAS/lane-detection/deploy_lanes_headless.py --source clip.mp4on a Pi/Hailo setup, or use--debugto inspect masks and fitted lane lines.Checklist
Risks and backward compatibility
None
Related / dependent PRs
None
Approval: Requires a minimum of 2 approvals.
Action: The feature branch MUST be deleted upon successful merge.