Skip to content

fix(dronecan): count stall-rail trips in esc.Status error_count#54

Draft
AlexKlimaj wants to merge 1 commit into
ark-releasefrom
claude/error-counters
Draft

fix(dronecan): count stall-rail trips in esc.Status error_count#54
AlexKlimaj wants to merge 1 commit into
ark-releasefrom
claude/error-counters

Conversation

@AlexKlimaj

Copy link
Copy Markdown
Member

Problem

uavcan.equipment.esc.Status.error_count was desync_happened alone, and that counter is incremented in exactly one place — the jump-desync check in runtimeProcessDesyncCheck (Src/runtime_loop.c:125). Every other way an established run dies left it untouched:

  • the INTERVAL_TIMER stall rail (faultHandleBemfIntervalStall) bumps bemf_timeout_happened and restarts the loop
  • the blind-grind rail kicks INTERVAL_TIMER to 46000 so that stall rail is guaranteed to trip on the next main-loop pass
  • the blind-step / miss-bucket limit in bemf_zc.c:84 hands off the same way

So the continuous partial desync documented in the block comment above the grind rail — 423 blind steps/s at a 19.5% miss rate, duty cut engaging and releasing in a limit cycle to 102 A — reported error_count = 0 to the flight controller for the entire event.

Change

Add fault_stall_trips, a monotonic since-boot counter incremented at the stall rail under the existing zero_crosses > 100 gate, and sum it with desync_happened in a new faultErrorCount().

uint32_t faultErrorCount(void)
{
	return desync_happened + fault_stall_trips;
}

Src/DroneCAN/DroneCAN.c now calls that instead of reading desync_happened directly.

Why the gate matters

A start attempt that never got going is the stuck-rotor rail's job, with its throttle-scaled tolerance (bemf_timeout 100 below input 150). Heavy props legitimately kick many times at low throttle — counting those would make error_count climb on every normal start. Reusing the same gate the episode charge already uses keeps that out.

Why only two addends

The stall rail is the single aggregation point for the grind and blind/miss-limit paths, so counting it once is what keeps one physical failure from being reported as two or three events. Deliberately not separate addends:

candidate why not
blind-grind rail always escalates into the stall rail by design
blind-step / miss-bucket limit same handoff
episode-rail latch a consequence of accumulating the above, and a state (ESC_FAULT_STUCK), not an event

Attribution between those sub-causes belongs in the AM32-reserved FlexDebug payload — see follow-up below.

Concurrency

fault_stall_trips is volatile: it is read from the main loop for telemetry while the grind rail that forces the trip runs in faultDesyncEpisodeTick1kHztenKhzRoutine, i.e. ISR context (Mcu/f051/Src/stm32f0xx_it.c:210). The stall rail itself is main-loop (main.c:515).

Note desync_happened remains a plain uint32_t — pre-existing, and its only writer and reader are both main-loop, so it is left alone here.

Cost

Measured on ARK_4IN1_F051 with HWCI_PERF=1 (the binding target — make size-check-ark):

baseline this PR delta
FLASH 26156 B 26164 B +8 B (limit 26265)
RAM 6872 B 6880 B +8 B (limit 7200)

faultErrorCount() is garbage-collected on non-CAN targets since nothing calls it; the 4 B counter itself still lands in .bss there. That is deliberate — gating it would mean #ifdef DRONECAN_SUPPORT inside the fault path for 4 bytes.

Verification

  • make AM32_SITL_CAN and make ARK_4IN1_F051 build clean under -Wall -Wextra -Werror.
  • make size-check-ark PASS (94.63% flash / 86.00% RAM, both under gate).
  • Full SITL suite: 29 passed.
  • make check_format passes with clang-format 22.1.5, the version CI pins. (Heads up for anyone reproducing locally: with clang-format 18 the check reports 4 pre-existing failures in Mcu/f415, Mcu/g431, and Mcu/v203 — that is version skew, not drift, and those files are untouched here.)
  • make cppcheck not run — cppcheck isn't installed in this environment.

Test added

The existing test_dronecan_throttle_and_esc_status now asserts an honest spool-up to a steady 4-6k rpm reports error_count == 0 — the regression guard for the zero_crosses gate. Run 3× independently to confirm it isn't flaky.

There is no positive test that the counter increments: inducing a genuine stall-rail trip in SITL is not something the current harness does, so that side is unverified in CI and wants a bench check.

Follow-up, deliberately not here

Per-cause attribution in FlexDebug. Note PR #37 already brings in an upstream FlexDebug v2 (duty/input/ADC/flags), so the breakdown should land on top of #37 as v3 rather than independently claiming v2 — and it needs the matching hwci host-side decoder change.

🤖 Generated with Claude Code

https://claude.ai/code/session_01HoD65bRT7unXhETnZ13WEq


Generated by Claude Code

error_count was desync_happened alone, which is incremented in exactly
one place - the jump-desync check in runtimeProcessDesyncCheck. Every
other way an established run dies left it untouched:

- the INTERVAL_TIMER stall rail (faultHandleBemfIntervalStall) bumps
  bemf_timeout_happened and restarts the loop
- the blind-grind rail kicks INTERVAL_TIMER to 46000 so that stall rail
  is guaranteed to trip on the next main-loop pass
- the blind-step / miss-bucket limit in bemf_zc.c hands off the same way

So the continuous partial desync documented at the top of the grind rail
(423 blind steps/s at a 19.5% miss rate, duty cut engaging and releasing
in a limit cycle to 102 A) reported error_count 0 to the flight
controller for the entire event.

Add fault_stall_trips, a monotonic since-boot counter incremented at the
stall rail under the existing zero_crosses > 100 gate, and sum it with
desync_happened in the new faultErrorCount(). The gate matters: a start
attempt that never got going is the stuck-rotor rail's job, with its
throttle-scaled tolerance, and heavy props legitimately kick many times
at low throttle - counting those would make error_count climb on every
normal start.

The stall rail is the single aggregation point for the grind and
blind/miss-limit paths, so counting it once is what keeps one physical
failure from being reported as two or three events. The episode-rail
latch is likewise not a separate addend: it is a consequence of
accumulating the above and is a state (ESC_FAULT_STUCK), not an event.
Attribution between the sub-causes belongs in the AM32-reserved
FlexDebug payload and is left to a follow-up.

The counter is volatile: it is read from the main loop for telemetry
while the grind rail that forces the trip runs in tenKhzRoutine, i.e.
ISR context.

Extend the existing SITL esc.Status test to assert a healthy spool-up to
a steady 4-6k rpm reports error_count 0 - the regression guard for the
zero_crosses gate.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HoD65bRT7unXhETnZ13WEq
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants