Add a way to have MQTT uplinks uplink multiple copies of packets#10531
Add a way to have MQTT uplinks uplink multiple copies of packets#10531ianmcorvidae wants to merge 7 commits into
Conversation
There was a problem hiding this comment.
Pull request overview
This PR adds an opt-in (compile-time USERPREFS_MQTT_UPLINK_ALL_SEEN) mechanism to publish multiple heard copies of the same mesh packet to MQTT, primarily to support analytics (e.g., seeing SNR from later rebroadcasts / more complete traceroute paths). It refactors receive-side MQTT publish logic into a dedicated Router::publishReceivedToMqtt() helper and invokes it from duplicate-filtering paths so packets that would normally be dropped as dupes can still be uplinked.
Changes:
- Refactor receive-side MQTT uplink logic into
Router::publishReceivedToMqtt()and call it fromRouter::handleReceived(). - When
USERPREFS_MQTT_UPLINK_ALL_SEENis enabled, publish packets to MQTT even in dupe/upgrade filter paths (FloodingRouter/NextHopRouter). - Add
USERPREFS_MQTT_UPLINK_ALL_SEENdefault + template entry inuserPrefs.jsonc.
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| userPrefs.jsonc | Adds a commented userPrefs knob for enabling “uplink all seen” behavior. |
| src/configuration.h | Defines USERPREFS_MQTT_UPLINK_ALL_SEEN with default 0. |
| src/mesh/Router.h | Adds publishReceivedToMqtt() declaration and adjusts DecodeState for forward-declare. |
| src/mesh/Router.cpp | Implements Router::publishReceivedToMqtt() and routes receive-side MQTT publish through it. |
| src/mesh/NextHopRouter.cpp | Calls publishReceivedToMqtt() from dupe/upgrade filter paths when enabled. |
| src/mesh/FloodingRouter.cpp | Calls publishReceivedToMqtt() from dupe/upgrade filter paths when enabled. |
Comments suppressed due to low confidence (2)
src/mesh/NextHopRouter.cpp:88
- Same const-correctness issue as above:
const_castis used to callpublishReceivedToMqtt()from aconst meshtastic_MeshPacket*context, but the helper can mutate the packet viaperhapsDecode(). Prefer changing the helper API to takeconstand work on copies, or pass a copy here.
#if USERPREFS_MQTT_UPLINK_ALL_SEEN
publishReceivedToMqtt(const_cast<meshtastic_MeshPacket *>(p), DecodeState::DECODE_FAILURE);
#endif
src/mesh/FloodingRouter.cpp:66
- Same const-correctness concern: casting away const to call
publishReceivedToMqtt()allows packet mutation inside a code path that otherwise treatspas immutable. Consider using a copy or adjusting the helper signature to acceptconstinput and work on copies.
#if USERPREFS_MQTT_UPLINK_ALL_SEEN
publishReceivedToMqtt(const_cast<meshtastic_MeshPacket *>(p), DecodeState::DECODE_FAILURE);
#endif
|
Code looks fine, but is this feature appropriate for the general firmware, as opposed to a private fork? It seems to be a specific use case, since to do something useful with it additional software on the other end would be required too. Maybe could be re-written as a module, so it gets packets out of the pipeline, rather than changes to the core router code? |
|
I'm not sure, I guess. Most tools that ingest MQTT data for analytics that I've seen already do fine with this (e.g. MeshView), not least of all since we had an accidental version of this in the hop-upgrade code for a few releases. I made it gated behind compile flags/userPrefs since it's inappropriate for the public MQTT probably, though only in a "bit more traffic" way, not a "cause problems for nodes" sort of way. Given it requires compiling custom anyway, I'm not terribly opposed to it being kept separate if that's more of the firmware dev consensus (despite of course preferring not to need to maintain a fork!), but it would make the fork maintenance easier if we could still bring in the refactor splitting off the reusable uplink logic. I have it on my todo list to go through the copilot review comments soon as well, so I'll try to keep an eye on the possibility of doing it in a module, though my feeling is the MQTT stuff is rather deeply embedded right now and it might need hook points we don't currently have. |
|
I should also mention that testing this the past week or so I've spotted a gap in my logic, which is that it still only ever uplinks one copy of any packet sent by itself (so no first-rebroadcasts heard by the MQTT uplink of its own packets). This might not be too bad to leave but if it's not more intrusive than what's already here I figured I'd try to patch that part. |
…d multiple times (compile flag + userPrefs)
ed2c56d to
3c7efde
Compare
... when they're heard multiple times (hidden behind compile flag + userPrefs)
This is entirely for MQTT-based analytics. Uplinking packets multiple times means analytics tools and users can see SNRs for rebroadcasts after the first heard by each uplink, and it's especially useful for traceroutes, where entire paths that might otherwise not appear can come by way of later-heard copies of the packet.
My experience level with C++ is not amazing so I'm suspecting there are things here that aren't needed, despite my best efforts to rein in the generated code. The primary parts of this PR are just moving MQTT uplink logic into a dedicated
Router::publishReceivedToMqttand then calling it from a variety of places to catch packets that otherwise wouldn't make it to the regular uplink location.Some specific question areas are the necessity of lines 870-874 in Router.cpp, serving as a dummy implementation I guess, and whether the gate on MQTT being enabled at all should be moved to live outside the newly created function. I'll try to test it on a few more devices, but it's also not a huge change and it works on at least one esp32 and nrf52, so I'm not too worried it'll turn up issues.
🤝 Attestations