Forecasting load, wind and solar with machine learning, and judging it honestly

Beat ‘same hour yesterday’ on a held-out month of SPP’s hourly demand, score point and band forecasts honestly, and find the leak in a model that looks too good

Modeler lesson M10 of Energy Systems, Step by Step: split a year of real hourly SPP demand by time, score persistence and seasonal-naive baselines, fit a linear model on features known at 00:00, find look-ahead leaks, test once on a held-out month, and judge bands with coverage and pinball loss; a Python notebook adds gradient boosting and a small neural network.
  • Modeler · L3
  • 40 min guided · 60 min lab · about 90 min practice
  • After lesson 4 and M7

SPP hourly demand, 2025: baselines, a learned model and a band, scored by time

  1. Split by time: train on 6,383 hours (8 January to 30 September 2025), choose on October (744 hours), test once on November (721).
  2. On October, same hour yesterday misses by 1,126 MW on average (MAPE 3.6%); same hour last week by 2,232 MW; the latest hour held flat by 3,186 MW.
  3. Persistence error, issued every hour from the reading h hours earlier, in % of each series' 2025 maximum, at 1 and 12 hours ahead: demand 1.4 and 10.0; wind 2.8 and 17.5; solar 6.3 and 48.7. Solar is zero in 377 of 744 October hours.
  4. The best of 16 feature mixes on October: same hour yesterday + same hour last week + the latest actual + Saturday and Sunday, 1,083 MW, 3.8% better than same hour yesterday.
  5. Leaks: adding the hour before scores 670 MW on October, adding the day's actual average 822 MW; at 00:00 neither is known for every hour of the day.
  6. November, opened once: 981 MW against 1,070 MW for same hour yesterday (8.3% better).
  7. An 80% band from October's misses, pooled over all hours (−1,801 MW to +1,861 MW), holds 83.1% of November's hours; its pinball loss is 398.1 MW.
  8. The three worst November days (Sunday 30 November, Monday 10 November, Thursday 27 November) hold 25% of the month's absolute error.

The summaries on this page

Summary MAE or share
Same hour yesterday, October and November 1,126 MW and 1,070 MW
Best mix by October, October and November 1,083 MW and 981 MW
Skill against same hour yesterday, November 8.3%
80% band holds, November 83.1%

With JavaScript on, you build and score these forecasts yourself, step by step.

  • Data: real hourly demand, wind and solar of the Southwest Power Pool (SPP) in 2025, as reported to EIA-930 (accessed 2 Nov 2026).
  • Cleaning: done afterwards, with the whole year in hand: 699 of 2025's 8,760 demand hours were repaired from the day-ahead forecast scaled to actuals on both sides, and frozen wind and solar readings filled.
  • So: a demonstration on cleaned data, not an operational backtest. Inputs are not exactly what was known at 00:00, and repaired hours are scored with the rest; the data do not mark them.
  • Scale: each series is stored in thousandths of its 2025 maximum; wind and solar stay so. Demand is that share of SPP's 2025 peak hour, 54,411 MW, so values are rounded to about 54 MW.
  • Clock: Central time with daylight saving: 9 March has 23 hours and 2 November 25. A forecast is issued at 00:00 for every hour of that day.
  • Same hour: the same clock hour on the day (or week) before. A clock hour that occurs twice (2 November) is read as the later one; a missing one (9 March), as the hour before it.

About 40 guided minutes in nine steps. Every forecast is fitted in your browser; nothing is sent anywhere. Need a refresher on MAE and bands? Lesson 4.

Build it in Python · notebook lab, about 60 minutes, then 90 minutes of practice

The same split and features in pandas, with assert cells against the page.

Then gradient boosting and a small neural network on the linear mix's inputs and on a wider set, quantile forecasts, and a rolling-origin check that chooses on earlier months only.

  1. Download the notebook, the lesson settings and the course's hourly data into one folder (in Colab, upload the two data files with the folder icon).
  2. Open it in Google Colab (File → Upload notebook) or locally with Python 3, pandas, NumPy and scikit-learn 1.1 or later.
  3. Run all cells (under a minute: about 20 seconds on a 2025 laptop), then try the five practice tasks at the end.

Notebook (.ipynb) Lesson settings Hourly data

What you just did, as diagrams

Baseline first
A simple rule, such as same hour yesterday, that a model must beat on the same hours.
Split by time
Train on the past, choose on the next month, test once on the month after; never shuffle hours.
Known at 00:00
Every input must exist when the forecast is issued; a later actual is a leak.
Skill
One minus the model's MAE over the baseline's, on the same hours.
Band and pinball loss
Coverage says how often reality lands inside; pinball loss also charges for width.
Bad days
A month's average hides the few days that hold much of the error.
Go deeper · ModelerThe formulas: rules, the regression, skill and pinball loss
  • Symbols. \(t\) an hour; \(y_t\) its actual demand and \(\hat y_t\) a forecast of it; \(x_t\) the list of inputs for hour \(t\), starting with a 1 for the intercept; \(w\) one weight per input; \(X\) every training hour’s \(x_t\), one row each; \(^\top\) the transpose (K6); \(\tau\) a quantile level and \(\hat q\) the forecast’s quantile at that level.
  • Horizon. A forecast issued at 00:00, the end of the day before’s last hour \(t_0\), for hour \(t\) has horizon \(h = t - t_0\), 1 to 24 (25 on 2 November). An input is allowed only if it is known at \(t_0\).
  • Baselines. Persistence \(\hat y_t = y_{t_0}\); same hour yesterday: \(y\) at the same clock hour the day before (24 hours earlier on most days, 23 or 25 next to a clock change); same hour last week: the same clock hour seven days before (the seasonal naive rule).
  • A weighted sum. With \(w = (2,\ 0.6,\ 0.3)\) and \(x_t = (1,\ 40,\ 38)\), the intercept’s 1 and then same hour yesterday and last week in GW, \(\hat y_t = 2 + 0.6 \times 40 + 0.3 \times 38 = 37.4\) GW (illustrative numbers).
  • Regression. \(\hat y_t = w^\top x_t\), with \(w\) minimizing \(\sum_t (y_t - w^\top x_t)^2\) over the training hours: the normal equations \(X^\top X\, w = X^\top y\), solved by Cholesky as in M9.
  • Skill. \(1 - \text{MAE}_\text{model} / \text{MAE}_\text{baseline}\) on the same hours; 0 is no better than the baseline.
  • Band. Quantiles \(q_\tau\) of October’s misses \(y_t - \hat y_t\), pooled over all hours, give the band \(\hat y_t + q_\tau\); the 80% band runs from \(q_{0.1}\) to \(q_{0.9}\).
  • Pinball loss at level \(\tau\): \(\tau (y - \hat q)\) if \(y \ge \hat q\), else \((1 - \tau)(\hat q - y)\). At the 90th percentile, a 100 MW miss above the forecast quantile costs 90 MW; the same miss below it costs 10 MW. The page averages it over the 19 levels 5% to 95%. It is a proper score: in expectation, no band beats the true quantiles.
Go deeper · ModelerWhat the lab adds, and what real forecasters use
  • Model ladder. Baseline, then linear regression, then gradient-boosted trees, then neural networks; each must still beat the baseline on held-out hours. The lab fits gradient boosting and a small neural network on this page’s features and split.
  • Weather. Real day-ahead load forecasts use temperature forecasts; actual temperatures as inputs would be a leak, an oracle bound at best.
  • Rolling origin. One test month is a small sample: refit and test month after month, and report the spread.
  • Further reading. Hyndman and Athanasopoulos’s free textbook covers baselines, accuracy and intervals in depth.

Check yourself

Myth or fact? “A learned model easily beats a simple rule.”

  • Myth
  • Fact
Answer

Myth. The best of 16 linear mixes beat same hour yesterday by 3.8% on October and 8.3% on November. Report skill against the baseline, not the model alone.

Myth or fact? “A model that scores 670 MW where the best fair one scores 1,083 MW is simply better.”

  • Myth
  • Fact
Answer

Myth. It used the hour before each forecast hour: known at 00:00 for the day's first hour, unavailable for every later one. An unexpectedly good score is a reason to check for leakage.

Myth or fact? “The band that holds the most hours is the best band.”

  • Myth
  • Fact
Answer

Myth. At twice October's width the 80% band held 98.6% of November's hours, but its pinball loss was the worst of the widths tried. Aim for the stated share with the narrowest band.

A model's MAE is 950 MW; same hour yesterday's is 1,000 MW on the same hours. Its skill score?

  • 95%
  • 5%
  • 50 MW
  • −5%
Answer

5%: 1 − 950 ÷ 1,000 = 0.05.

A forecast issued at 00:00 on 14 October for the hour ending 18:00 on 14 October. Which input is allowed?

  • Demand at 17:00 on 14 October
  • The actual temperature on 14 October
  • Demand at 18:00 on 13 October
  • The average demand of 14 October
Answer

Demand at 18:00 on 13 October: 24 hours before the target, and already known at 00:00 on 14 October. The others lie after the forecast is issued.

Numbers, sources and what’s next

Numbers to remember

Number What it means Source
1,126 MW SPP's demand, October 2025: the average miss of same hour yesterday, issued at 00:00 (3.6% MAPE). Our calculation from EIA-930 data · Oct 2025
8.3% Skill of the mix chosen on October against same hour yesterday, on the held-out November. This lesson's model · EIA-930 data · Nov 2025
83.1% Share of November's hours inside the 80% band built from October's misses, pooled over all hours. This lesson's model · EIA-930 data · Nov 2025
377 of 744 October 2025 hours with zero solar output in the course's SPP data: MAPE cannot score them. Our calculation from EIA-930 data · Oct 2025

Keep going

Back to the course map

Related lab: a pandas and scikit-learn notebook that rebuilds the page's split and checks it, then gradient boosting and a small neural network on the linear mix's inputs and on a wider set, about 60 minutes, then 90 minutes of practice.

Where this came up in research

From the course author's research: a wind turbine's real power curve is a band, not a line. Adding the turbine's theoretical curve to about two hundred measurements can narrow that band, but whether it helps depends on how the fit is scored. (Li et al., ASME IDETC 2023)

In the reported nine-turbine test, nonlinear multi-fidelity regression achieved lower RMSE for most turbines; performance depended on the evaluation metric.

Testing both point and probabilistic metrics showed that the gain depended on which error matters.

The same lesson as this page: say which score, on which held-out data. The method is in M9.

Case file CF8: learning a wind turbine's power curve from scarce data.

Sources and further reading

Spotted an error, or stuck? Write to hlli@ksu.edu. Corrections are made in place and dated here.

Text licensed CC BY 4.0, code MIT: teachers and students may reuse and adapt it, with credit to Honglin Li.