feat(market_data): add zone-addressed market data (ENTSOE + UK power)#80
Merged
Conversation
Adds a new client.market_data namespace exposing observational European
power-market data (renewable generation, load, day-ahead forecasts, and
prices) addressed purely by market_zone plus a curated, backend-agnostic
variable vocabulary.
The ENTSOE and UK-power Query Engine endpoints are hidden as internal
backends. A capability matrix in _mapping.py routes each (zone, variable)
pair: EU zones -> /v1/entsoe (DE -> DE_LU), GB renewables/load -> /v1/uk-power,
GB prices/load_forecast -> ENTSOE GB. ENTSOE wind is the onshore+offshore
total, summed client-side. get_data returns one tidy pandas DataFrame
[time, market_zone, variable, value, unit].
Time handling is DST-safe: response timestamps are normalized through UTC
then converted to the requested IANA time_zone, avoiding mixed-offset
object columns and dropped/duplicated transition hours.
Verification: - QE contract: live/services/query-engine/src/query_engine/{entsoe,uk_power}/router.py
- just check-commit passes (lint + tests)
Co-authored-by: Cursor <cursoragent@cursor.com>
…stitching Parse power-forecast response timestamps through UTC before converting to the requested time zone, so a range spanning a daylight-saving transition no longer produces an object-dtype `time` column. This was raising `TypeError: cannot subtract DatetimeArray from ndarray` inside `get_day_ahead_timeseries` (e.g. GB init_hour=9 over a window crossing the spring-forward). Add regression tests covering mixed-offset parsing plus end-to-end stitching across spring-forward, fall-back, leap day, and year-boundary ranges. Co-authored-by: Cursor <cursoragent@cursor.com>
Query realised solar, wind, and load for Germany and Great Britain through the zone-addressed market_data API, showing the same unified call works across the ENTSO-E and UK-power backends. Co-authored-by: Cursor <cursoragent@cursor.com>
Guard against silently comparing a forecast with the wrong data: assert that `_to_dataset` only relabels instants across time zones (never changes values) and that stitched day-ahead values come from the correct init run at the correct lead. Co-authored-by: Cursor <cursoragent@cursor.com>
Verify on the real API that power-forecast values are time-zone invariant, stitched day-ahead values match a direct query of their source run, market forecast and actual share the same time grid, and solar actuals are physically plausible (non-negative, ~0 overnight). Co-authored-by: Cursor <cursoragent@cursor.com>
Reformat the zone-addressed market_data modules and tests so `just lint` (ruff-format --all-files) passes; whitespace/line-wrapping only, no behaviour change. Co-authored-by: Cursor <cursoragent@cursor.com>
…andling ENTSO-E publishes imbalance prices per direction (Long/Short). The single `imbalance_prices` variable summed both rows, which doubled single-price markets (DE/BE, where Long == Short) and produced meaningless values for dual-pricing markets (FR/NL, where Long != Short). Expose `imbalance_price_long` / `imbalance_price_short` and filter to one direction instead of summing. Non-PSR variables (prices, load) now collapse ENTSO-E revision duplicates rather than summing them, while PSR generation still sums its components (wind = onshore + offshore). Route DE imbalance prices to the "DE" control area instead of the DE_LU bidding zone used for the rest of Germany's data; that is where the Transparency Platform publishes them (DE_LU returns nothing). Stop advertising GB day-ahead/imbalance prices and load forecast: no backend serves them, so requesting them now raises a clear "not supported" error instead of silently returning an empty frame. Co-authored-by: Cursor <cursoragent@cursor.com>
niasie
approved these changes
Jun 5, 2026
jua-mark
approved these changes
Jun 5, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a new
client.market_datanamespace for observational European power-market data, addressed purely bymarket_zoneplus a curated, backend-agnostic variable vocabulary. ENTSOE and UK-power are hidden as internal backends.Vocabulary
solar,wind,load,solar_forecast,wind_forecast,load_forecast,day_ahead_prices,imbalance_prices. Wind is the onshore+offshore total.Routing (capability matrix in
_mapping.py)/v1/entsoe(DE->DE_LU);solar/windfromgeneration_actual(PSR-summed), forecasts fromwind_solar_forecast_da./v1/uk-power.load_forecast/ prices -> ENTSOEGBzone (the uk-power endpoint serves no prices).(zone, variable)raises a clearValueError.Timezone / DST
Request datetimes serialized via
.isoformat()(naive treated as UTC by the server, documented). Responses parsed DST-safe: normalized through UTC, thentz_convertto the requested IANAtime_zone-- no mixed-offset object columns, no dropped/duplicated transition hours.Test plan
just check-commitpasses locally (lint + unit tests; 182 passed)Made with Cursor