entso-e quarterly prices

[Habbie]

On October 1st, the energy market switched from hourly prices to quarterly prices. The entso-e fetcher in this app just assumes the datapoints in the entso-e API XML are per hour, leading to funny results:

Most users won't notice this, as the entso API key in the code is no longer valid, and thus they fall back to EasyEnergy data, which is still hourly. If you put in your own entso API key (I know I did, but I don't remember why; perhaps EasyEnergy was broken at some point. I noticed today that their API does not always respond) at some point, you likely also see the broken result.

I'm working on some code changes to plot the quarterly data, but I have no idea when I will have anything to PR. For now this issue serves as documentation to say: if you have this problem, you could consider "breaking" your entso key again, to get hourly data from the EasyEnergy API instead.

[IgorYbema]

Correct. I was planning to fix this in a few days also. Also wanted to add a way to set a custom API key and a way to choose between 15m and 60m timing

[Habbie]

Also wanted to add a way to set a custom API key and a way to choose between 15m and 60m timing

nice!

[Habbie]

looks like the easyenergy API we use stopped working yesterday..

[IgorYbema]

Yes I noticed it also. Didn't finish the work yet though.
But I managed to get the entsoe prices and average them per hour back to one hour prices.

If you replace the entsoe function by this you temporarily have it back working again. Put your own API token in there.
I didn't test it good yet but let me know your feedback.

        function getCurrentTariffsEntsoe() {
                var now = new Date();
                currentHour = now.getHours();
                startHour = currentHour - settings.lookbackHours; // start the graph at the start point set
                now.setHours(startHour,0,0,0);
                var endDate = new Date(now.getTime() + ((settings.lookforwardHours + settings.lookbackHours) * 3600 * 1000)); // end the graph at the end piont set

                var xmlhttp = new XMLHttpRequest();
                xmlhttp.onreadystatechange=function() {
                        if (xmlhttp.readyState == 4) {
                                if (xmlhttp.status == 200) {
                                        var res = xmlhttp.responseText
                                        var tariffsTemp = []
                                        var i = res.indexOf("<Period>")
                                        var j = res.indexOf("</Period>")
                                        while ( i > 0 ) {
                                                var period = res.slice(i+8,j)
                                                res = res.slice(j+9)
                                                i = period.indexOf("<start>")
                                                j = period.indexOf("</start>")
                                                var start = period.slice(i+7,j)
                                                var quoteTime = Date.parse(start) - 900000 //because of in the while loop it gets added first
                                                i = period.indexOf("<price.amount>")
                                                while ( i > 0 ) {
                                                        var p1 = period.indexOf("<position>")
                                                        var positionString = period.slice(p1+10);
                                                        var p2 = positionString.indexOf("</position>");
                                                        var position = positionString.slice(0,p2) / 1;
                                                        period = period.slice(i+14)
                                                        j = period.indexOf("</price.amount>")
                                                        var quotePrice = period.slice(0,j) / 1000
                                                        var quoteTime = quoteTime + 900000 // for now this is good, every next index is one hour later
                                                        var quoteTarrif = {timestamp: quoteTime, tariff: quotePrice}
                                                        if (quoteTime >= now.getTime() && quoteTime <= endDate.getTime() ) {
                                                                tariffsTemp.push(quoteTarrif)
                                                        }
                                                        //check if next position is one up, if not fill up the gaps
                                                        var nextp1 = period.indexOf("<position>")
                                                        if (nextp1 > 0) {
                                                                var nextpositionString = period.slice(nextp1+10);
                                                                var nextp2 = nextpositionString.indexOf("</position>");
                                                                var nextposition = nextpositionString.slice(0,nextp2) / 1;
                                                                if (nextposition > (position + 1)) {
                                                                        while (nextposition > (position + 1)) {
                                                                                position = position + 1;
                                                                                quoteTime = quoteTime + 900000;
                                                                                quoteTarrif = {timestamp: quoteTime, tariff: quotePrice}
                                                                                if (quoteTime >= now.getTime() && quoteTime <= endDate.getTime() ) {
                                                                                        tariffsTemp.push(quoteTarrif)
                                                                                }
                                                                        }
                                                                }
                                                        }

                                                        i = period.indexOf("<price.amount>")
                                                }
                                                // next period
                                                i = res.indexOf("<Period>")
                                                j = res.indexOf("</Period>")
                                        }
                                        tariffsTemp.sort(function(a, b){return a.timestamp - b.timestamp});
                                        datapoints = tariffsTemp.length;
                                        if ( ((datapoints - settings.lookbackHours) < 6) && (settings.lookforwardHours > 6 ) ) {
                                        //if (datapoints < 24) {
                                                console.log("SpotEnergy: ENTSOE URL fetch returned not enough datapoints!");
                                                getCurrentTariffsEasyEnergy();
                                                return;
                                        }

                                        minTariffValue = 1000;
                                        maxTariffValue = -1000;

                                        var tariffs = [];
                                        for (var i = 0; i < (tariffsTemp.length / 4); i++) {
                                                tariffs[i] = (tariffsTemp[i*4].tariff + tariffsTemp[(i*4)+1].tariff + tariffsTemp[(i*4)+2].tariff + tariffsTemp[(i*4)+3].tariff) / 4;
                                                if (minTariffValue > tariffs[i]) {
                                                        minTariffValue = tariffs[i];
                                                }
                                                if (maxTariffValue < tariffs[i]) {
                                                        maxTariffValue = tariffs[i];
                                                }
                                        }

                                        datapoints = tariffs.length;
                                        tariffValues = tariffs.slice();

                                        // calculate the quartiles for the low and high tariff
                                        var quartiles;
                                        if (settings.algoMedian) {
                                                quartiles = SpotenergyJS.getQuartilesMedian(tariffs);
                                        } else {
                                                quartiles = SpotenergyJS.getQuartilesAverage(tariffs);
                                        }
                                        tariffQ1 = quartiles[0];
                                        tariffMedian = quartiles[1];
                                        tariffQ3 = quartiles[2];

                                        // set the current tariff and normalize
                                        currentTariffUsage = tariffs[settings.lookbackHours];
                                        if (settings.domoticzEnable) { updateDomoticz(); }

                                }
                                else {
                                        console.log("SpotEnergy: ENTSOE URL fetch failed!");
                                        getCurrentTariffsEasyEnergy();
                                }
                        }
                }
                var urlAppend = "TimeInterval=" + encodeURIComponent(now.toISOString() + "/" + endDate.toISOString());
                var urlEntsoe = "https://web-api.tp.entsoe.eu/api?securityToken=your-own-token-here&documentType=A44&Out_Domain=10YNL----------L&In_Domain=10YNL----------L&" + urlAppend;
                console.log("SpotEnergy entsoe url: " + urlEntsoe);
                xmlhttp.open("GET", urlEntsoe, true);
                xmlhttp.send();
        }

[Habbie]

But I managed to get the entsoe prices and average them per hour back to one hour prices.

Well, -I- don't want that (but others might) - the per 15 minute changes within hours are sometimes bigger than the differences from hour to hour. But I can probably try it soon and will report back anyway.

[Habbie]

I didn't test it good yet but let me know your feedback.

(please ignore the colours, I screenshot with cat /dev/fb0 > screenshot.data and import with GIMP and it does not get the RGB order right).

Looks good to me! The tile looks fine as well.

[Habbie]

I've taken your patch in production on my living room Toon until I get the display of quarterly prices right, or you beat me to it :) so if it breaks, I'll find out soon enough

[Habbie]

I've taken your patch in production on my living room Toon until I get the display of quarterly prices right, or you beat me to it :) so if it breaks, I'll find out soon enough

I saw an empty graph one time last week, but it's been working fine otherwise.

[Habbie]

Except the entso-e API is currently broken, and giving old data, and this code does not detect that, so it's plotting the wrong thing. This is not a new problem with the quarterly prices, of course.

[Habbie]

The one with your patch somehow started doing this again. I did not debug anything yet.

However,

looks like the easyenergy API we use stopped working yesterday..

looks like this works again, but enabling it (by breaking my entso key) yields a nonsense graph.

Ah: it is "working" but 26 october is just.. missing. And as we know, the qml code we have here does not really look at all the individual Timestamps that come out an API response (both entso and easyenergy) to check that they are what we expect.

[Habbie]

Again just noting this here so people know it's not them: the graph broke on both my Toons. One running Igor's patch, one running an unfinished patch of my own, so I guess easyenergy broke again.

[Habbie]

Looks like easyenergy's data stopped at "2026-01-01T22:00:00+00:00" . Let's hope it's just a glitch like 26 october.

[Habbie]

(Just making notes so people know they're not alone)

January 2 was just another glitch, and then another one when we switched to Daylight Saving Time, but right now easyenergy's API appears to be completely gone, and entso-e meanwhile changed even more.

[Habbie]

easyenergy's API appears to be completely gone

confirmed here: home-assistant/core#154286 (comment)

[Habbie]

I looked around (just a bit) at other data sources, and Nordpool is looking interesting.