feat: Dynamic light sleep support for ESP32 platform#7121
Conversation
9526383 to
d8f2fcd
Compare
|
This is a ton of work, thank you very much |
|
If you have time .... I "maintain" the GPS code - just wondering what the impact might be there or whether we might need further extensions? |
|
AFAIK dynamic light-sleep in ESP32 platform makes the chip light-sleep when FreeRTOS waits for an event or there is an explicit vTaskDelay call (=~ sleep in Arduino). So most of the functionality shouldn't be impacted as Arduino runs all the code in one loop, which waits different intervals depending on the type of thread (using vTaskDelay under the hood). However, for critical modules or core functionality I added an explicit call to PowerFSM.trigger to exit LS state and allow for at least 300ms of uninterrupted processing time; another approach is to use "preflightSleepObserver" which prevents entering LS state if there is some work pending (like TX queue is not empty). I took a look at GPS code and I think we should exit LS state when we change state to GPS_ACTIVE and prevent entering LS as long as the state doesn't change. I'll take a look at the code and propose some changes soon. |
|
If there is a chance to have this feature merged, I'd appreciate getting some feedback on the additional repos to be hosted under |
|
@fifieldt changes committed; device should not go into light-sleep when GPS position is requested (=module is in GPS_ACTIVE state). I'm unable to test as I'm on vacation right now. |
151def2 to
a42c8f6
Compare
a42c8f6 to
892f77b
Compare
892f77b to
4223701
Compare
4223701 to
f58f5e5
Compare
|
hi,
|
058e904 to
50151fc
Compare
| #if defined(ARCH_ESP32) && HAS_WIFI && !HAS_TFT | ||
|
|
||
| #if defined(ARCH_ESP32) && !HAS_TFT | ||
| #ifdef HAS_WIFI |
There was a problem hiding this comment.
Would it also make sense to have the HAS_ESP32_DYNAMIC_LIGHT_SLEEP (and HAS_ESP32_PM_SUPPORT, HAS_32768HZ) as a bool as well, instead of just checking #ifdef ?
It's more broad discussion about convention. This has to be decided on project level I guess.
There was a problem hiding this comment.
sure, I'll check these definitions and I can update them to bool. I see there is inconsistency about how we use HAS_XYZ flags in the project. I will create a separate PR for this.
|
General critique about having a lot of While I've already make a boot loop with those asserts because of a wrong setting, the more concerning part is that Surely some of the calls can fail in runtime or due to a wrong setting. Should we then crash the device? There is a known issue here that some ESP32 devices corrupt their device configuration when running on solar power. This not only increases the risk, but also cause the device to become irrecoverable, because of a wrong setting. Example: I'd recommend an approach that would fail in a recoverable way. |
The linker fails on ESP32 (not S3). |
This fails because the ESP32 platform has very limited IRAM available, and enabling PM capabilities pushes it beyond the limit. I’ll explore making the ESP-IDF components lighter to reduce memory usage without affecting Meshtastic functionality, but that may not be feasible. In the end, we may have to accept that dynamic light-sleep isn’t compatible with the ESP32. In newer ESP-IDF versions (5.x compatible with Arduino 3.2.x), there’s a new sdkconfig option CONFIG_ESP_SYSTEM_ESP32_SRAM1_REGION_AS_IRAM, which makes more RAM available as IRAM, allowing the build to succeed. However, adopting this would be a breaking change for the project, since it requires moving from Arduino 2.x to 3.x. I’ve already created and tested such a build with the current Meshtastic version, but this shift is too significant to include in this PR. |
Understood, I'll check if there is any assertion, which relies on user settings and make them fail safe. However I'd recommend to keep assertions for all calls, which use predefined values (board configuration, etc.). IMO this is the only way to validate the changes with all devices during alpha testing phase. Let me know what you think. |
2815ae5 to
89b296d
Compare
89b296d to
4b22d1d
Compare
4b22d1d to
2e860a8
Compare
|
I'm testing this on ESP32C3 and so far the testing is looking very good. My only nitpick is that I think that there should be a comment in e.g. |
I can't provide any specific reference for that particular value, but my tests show that anything less than 2048 breaks Bluetooth when dynamic light-sleep is enabled; I couldn't find any Espressif materials explaining how to optimize this value. This is my initial implementation of keeping NodeDB in PSRAM, which significantly reduces heap usage: |
Hey :) I think it does - when dynamic light sleep is enabled, all scheduled tasks will take place as requested. ESP-IDF will wake-up the device when next OSThread is to be run, so routines in Power.c should run as expected. |
I believe it may be missing from several variants as defined in repo (i.e. T-Lora Pager, no schematic is available, but it can be enabled successfully). |
|
@thebentern is there a chance you could help with the item below? I can set up the CI pipeline and migrate the repository contents. After that, I’ll update the PR so all links to the custom Arduino framework point to releases under the Meshtastic GitHub organization. |
|
I can confirm that both 32k crystal switchover (as well as failure to switch over) and DFS are active and working with this PR: I still need to test whether the INA219 will work EARLY in the boot (it is detected very early) to drop into deep sleep if the battery voltage is below the threshold, but this is excellent work! |
|
I do not see light sleep getting enabled, only periodic AGC reset attempts and messages about gpio_hold_en being attempted on output-capable IOs: I do have light sleep enabled in the configuration: I also do not see the "Using INA on I2C addr 0x%x for charging detection" message on bootup, although I don't see why I shouldn't: I have an INA219, I have telemetry enabled (I see those messages in the log and being transmitted over LoRa), I'm not excluding environmental sensing and |
|
yep I just realized that after adding some explicit debugging. I've got trivial patch to fix that since it's cluttering up the debug logs. Any idea why the INA219 is correctly detected and the configuration does reference it correctly ( |
|
Does INA219 get properly detected with stock firmware ? It may be out of scope of my changes but I can take a look later during the day. |
yep it's absolutely detected by your branch (and stock) without any changes whatsoever. The power telemetry is also working great: |
|
it looks like the AGC reset seems to never re-enter light sleep -- every minute it attempts an AGC reset, but I never see I'm digging into this a little more but wanted to mention it in case you might already know what's going on |
Please see my comment above (the one about PM lock and FreeRTOS, AGC resets do not matter here). |
|
nevermind, I think I just got the point of what you said. 🤦 If Now the logs make more sense: What's interesting is that receiving a packet also wakes up with the same wakeCause -- I would have expected it to wake from GPIO. |
|
I've also figured out the INA219 not being used for battery level -- it's the same issue as I had when I was doing this last year: the battery level stuff is VERY strongly tied to AnalogBatteryLevel, which isn't at all what I'm doing on my own hardware. I'll rely on the brownout tripping deep sleep instead for the time being. I'm excited to see your PR merged, this is very excellent work. Thank you! |
|
I’ve been testing the Meshtastic develop branch firmware for several days, compiled with the only modification of using the following in the heltec v3 platformio.ini: platform_packages = With this build on the Heltec V3, the LiPo battery current consumption drops from about 100 mA to around 50 mA, and I haven’t noticed any side effects with Bluetooth or WiFi enabled. In my case, with a 1200 mAh LiPo battery, I can finally exceed 20 hours of operation while keeping Bluetooth continuously connected to the phone app. @m1nl, you’ve done a great job reducing power consumption by enabling some PM features for the ESP32-S3. By porting the additional dynamic light sleep patches, it’s probably to achieve even better results. |
|
I developed a tiny node (Heltec HT-CT62 based) and with my earlier changes it would basically stay at ~11mA which is roughly the limit (SX1262 with LNA on and in RX mode is ~10mA) -- with this PR I'm measuring ~16-18mA but have to verify that it's not because of the environmental sensor, which can draw a few mA due to the heater. This patch does enable both DFS and DLS -- I tried briefly just |
|
I am noticing WAY more frequent wakeups from light sleep than I expected: I thought it might be related to being connected via bluetooth but after disconnecting and rebooting, I am still seeing the same. Why is power status waking us up every 2-3 seconds? |
|
I also noticed if I disable bluetooth in the settings the node tends to hang after about a minute or two of booting. I have not been able to debug this further yet. Re-enabling bluetooth seems to correct the behaviour (and also seems to have reset the waking up every two seconds which makes no sense at all) |
Interesting, thanks for feedback! These are the sdkconfig options I've enabled (pretty standard): |
It does not happen with my devices. Are you using any of these interrupts below? Power thread interval should 20 seconds at minimum. |
Actually it makes sense - without Bluetooth enabled, sleep intervals are much longer and interrupts can be missed unless GPIOs are set to wake up device from sleep. I didn't notice any hang when Bluetooth is disabled - when does it happen? Are you able to provide fragment of logs from device around the time when it hangs? |
It will hang in the middle of emitting a log message to the serial port -- I haven't made any changes around enabling GPIO for wake from interrupt, but perhaps dynamic light sleep is skipping this part (I see log messages saying otherwise, but every wakeup I've documented has been from a timer, not a GPIO, even when receiving a packet). I'll re-test of course but was surprised at this. I was relying on VERY long sleeps with my hacked up version of power saving and that was working reliably since I think the sleep interval was something on the order of 10 or even 30 minutes for housekeeping/transmitting telemetry (with GPIO interrupts waking it earlier of course). |
|
@m1nl Hi! Heltec make revision of their v4 schematic, and released v.4.3 board version There was huge fix improving RX sensitivity (merge #9906) so on today 2.7.21 (this one) is the only one firmware that works fine on 4.3 boards and allow them TX\RX normally If it possible to ask m1nl version with powersaving for 2.7.21 ? 🙏 Thank you |
|
I have been using this PR branches revision on my daily carry for around a month now without issue (and extremely decent battery consumption). Wondering what is remaining to have this merged, as believe this to be extremely beneficial. |
|
This needs to be reworked now that the Arduino 3.x work has been merge (to |
|
@dblaber Totally agree, seems what @vidplace7 mentioned moves this PR even further from merging 😢 |
|
Hey, I will try to migrate the changes. Actually I did test them with pioarduino before. But timing is bad for me right now, need at least 2 weeks. |
|
In an attempt to make a 0.5w solar node for esp32 (I'm glutten for punishment) I've tweaked a few existing features to save power. The expectation is that this will only work if wifi AND bluetooth is disabled, meaning the only way to manage the node is through remote admin. Even serial will require a reboot of the node. Using a Debug Mate with custom firmware to count mAh and mWh, I can get a Xiao Wio ESP32 down to a 1 watt hour per day with very busy mesh traffic. This might be good for very specific solar routers. I have this running on a Heltec V4.3 with a estimated 2Wh/day. The changes are #10570 #10571 #10572 and #10582 .. These are the settings I'm working with on with the above changes. At the very least, #10582 should be interesting since light sleep does not advance ticks, causing incorrect (inflated) chutil. |
fixes #6660
I've finally implemented the Feature Request I created a few months ago. Since then, I've been testing the dynamic light-sleep functionality and haven't encountered any issues.
In the initial version, legacy light-sleep support with the default Arduino framework could lead to problems after my changes. To address this, I had to rewrite and refactor parts of the code to ensure compatibility with builds compiled using the default PlatformIO Arduino framework.
I tested the updated implementation on a Heltec V3 board in the following scenarios:
There are a few topics that need discussion:
meshtasticorganization.platformio.inifiles for the hardware variants I was able to test. I shared customized builds with others, who confirmed they were working as expected.variant.hfiles for the hardware variants which do have 32kHz oscillator, according to Espressif docs, it helps with Bluetooth stability when dynamic light-sleep is enabledLinks to repositiories mentioned in 1) :
🤝 Attestations