Feature: Automated Solar Clipping Mitigation (using native clear-sky models)

[rholligan]

Introduction:

This PR introduces a native clipping buffer to Predbat. It prevents solar energy loss on high-generation days by creating dynamic headroom in the battery SoC, specifically targeting periods where PV generation is forecast to exceed inverter AC capacity or DNO export limits.

This implementation integrates with the existing cloud model. It extends SolarAPI to compute and store new pv_clearsky arrays (supporting multiple native upstream sources like ha_solcast_clearsky, solcast_api, and openmeteo). Clear-sky arrays are used alongside a lightweight auto-tuner to safely estimate peak generation and align it with hardware limits.

Resolves issue #1206, supersedes PR #4036.

Key Features:

• Dynamic Buffer Calculation: Calculates exactly how much spare capacity the battery needs to safely absorb any solar generation that exceeds the inverter limits.
• Cost-Optimized Headroom Creation: Integrates directly into Predbat's financial solver by adding a cost penalty whenever the battery doesn't have enough headroom. This naturally forces Predbat to prioritize exporting or deferring charge just before a solar peak hits, guaranteeing the necessary spare capacity.
• Clear-Sky Cloud Modelling: Introduces ha_solcast_clearsky integration alongside a robust clipping_auto_tune mechanism. This explicitly accounts for those short "break in the clouds" spikes, ensuring the buffer is safely sized even when the standard hourly forecast is smoothed out.
• Defensive HA Polling: Hardened solcast.py with safe float(val or 0.0) casting to gracefully handle missing HA clear-sky sensor history without crashing.

Configuration Parameters:

• clipping_buffer_enable: Master toggle for the clipping buffer feature.
• clipping_clearsky_source: Selection of clear-sky forecast source (recommends ha_solcast_clearsky). This setting is fully exposed in the Home Assistant configuration panel.
• clipping_use_clearsky_peaks: Toggle to use Clear-Sky generation arrays rather than standard PV forecast arrays to size the buffer.
• clipping_auto_tune: Automatically learns the scale difference between standard forecasts and hardware limits based on past clipping behavior.
• clipping_amplification: Manual multiplier for the standard solar forecast (used if Auto-Tune is disabled).
• clipping_cost_weight: Multiplier to add a harsher financial penalty for clipping, forcing the optimizer to prioritize creating headroom.
• clipping_limit_override: Manual configuration of the inverter AC ceiling in Watts.
• clipping_buffer_max_kwh / start_time / end_time: Manual settings to force a fixed static buffer size and time window instead of using the dynamic calculation.

Developed with Gemini ✦

[jkpe]

@rholligan

Apart from installing https://github.com/autoSteve/ha-solcast-clearsky do I need to make any changes to my Solcast configuration or integration? I’ve got a 5kW inverter so I’ve got that set as my AC limit on Solcast’s website, or should I remove this so that we’re getting unclipped data? I think Solcast clips it for you if you set an AC limit.

[rholligan]

@rholligan

Apart from installing https://github.com/autoSteve/ha-solcast-clearsky do I need to make any changes to my Solcast configuration or integration? I’ve got a 5kW inverter so I’ve got that set as my AC limit on Solcast’s website, or should I remove this so that we’re getting unclipped data? I think Solcast clips it for you if you set an AC limit.

As you say, set Solcast to your actual array capacity, and let Predbat handle the clipping calculation (so in my case I set both Capacity DC (modules) in kW and Capacity AC in kW in Solcast to 6.4 kW).

[jkpe]

I had a play with this. Initially I couldn't get it to predict any clipping, the clipping chart was showing a ClearSky peak of 4.83, on my 5kW inverter there would be a little bit of clipping with my average 400-500w load. Does this consider load?

This led me to try predbat_clipping_limit_override and noticed a bug when using predbat_clipping_limit_override set to 4000w it would result in Clipping status: 41.22 kWh clipping buffer forecast between 05:25 and 21:25 tomorrow (4.0kW limit override active). The fix is in this commit jkpe@70dd020

After this fix I now see Clipping status: 0.76 kWh clipping buffer forecast between 09:55 and 13:55 tomorrow (4.5kW limit override active). which seems very accurate, however Predbat is still charging me to 100% the night before.

This is possibly due to my config below.

predbat_metric_min_improvement_export 16p (My export rate is 15p and I don't want predbat to export randomly)
predbat_metric_pv_calibration_enable is false
predbat_calculate_export_on_pv is false
predbat_combine_charge_slots is true

[rholligan]

I had a play with this. Initially I couldn't get it to predict any clipping, the clipping chart was showing a ClearSky peak of 4.83, on my 5kW inverter there would be a little bit of clipping with my average 400-500w load. Does this consider load?

This led me to try predbat_clipping_limit_override and noticed a bug when using predbat_clipping_limit_override set to 4000w it would result in Clipping status: 41.22 kWh clipping buffer forecast between 05:25 and 21:25 tomorrow (4.0kW limit override active). The fix is in this commit jkpe@70dd020

After this fix I now see Clipping status: 0.76 kWh clipping bpredbat_metric_min_improvement_exportuffer forecast between 09:55 and 13:55 tomorrow (4.5kW limit override active). which seems very accurate, however Predbat is still charging me to 100% the night before.

This is possibly due to my config below.

predbat_metric_min_improvement_export 16p (My export rate is 15p and I don't want predbat to export randomly) predbat_metric_pv_calibration_enable is false predbat_calculate_export_on_pv is false predbat_combine_charge_slots is true

Thanks for the feedback and the commit! Your unit fix was spot on and is now merged.

You were absolutely right to suspect your config for the 100% charging issue. Because your predbat_metric_min_improvement_export (16p) was higher than your export rate (15p), the optimizer was rejecting the anti-clipping export window. I've updated the logic to bypass this cost check specifically for anti-clipping windows, so your headroom target should now be respected.

I also found and fixed a bug where leaving clipping_clearsky_source on auto failed to load any ClearSky data, which likely explains why you saw no clipping initially. Out of interest, are you using the ha-solcast-clearsky integration?

Please pull the latest changes and give it a try. If you can, post your output clipping chart so I can see how it's performing. Sharing mine below for reference:

If it's helpful, I also use these cards to help monitor my config:

yaml
type: entities
title: Clipping Configuration
entities:
  - switch.predbat_clipping_buffer_enable
  - switch.predbat_clipping_auto_tune
  - input_number.predbat_clipping_amplification
  - input_number.predbat_clipping_cost_weight
  - input_number.predbat_clipping_buffer_start_offset
  - input_number.predbat_clipping_buffer_end_offset
  - input_number.predbat_clipping_buffer_max_kwh
  - input_number.predbat_clipping_limit_override
  - predbat.clipping_status
  - predbat.clipping_tomorrow
  - predbat.clipping_target_soc

yaml
type: history-graph
entities:
  - entity: input_number.predbat_clipping_buffer_end_offset
  - entity: input_number.predbat_clipping_buffer_start_offset
  - entity: input_number.predbat_clipping_amplification
  - entity: sensor.solcast_clear_sky_today
  - entity: sensor.solcast_clear_sky_tomorrow
  - entity: sensor.solcast_clear_sky_d3
hours_to_show: 168

[giddie]

Let me know if there’s anything we can do to move this into main.

Just an idea, since I'm only an interested observer: I think this PR is quite big, and it's introducing two significant features:

• Using "clear skies" instead of Solcast 90% for the irradiance ceiling in the cloud model used to predict clipping
• Introducing a buffer to ensure clipping cannot occur

My thought is: I've noticed PredBat does OK preventing clipping if the solar prediction is accurate. The issue is when the 90% forecast is too low. So a good 80% of the problem should be fixed by just the first feature: using clear skies instead of the 90% forecast.

The clipping buffer is a nice optimisation, but it's the last 20%.

So I wonder if it's worth splitting the PR in two. That way @springfall2008 will have a much easier time reviewing the changes, and we'll be in with a greater shot of getting it merged?