Forecasts and surprises: how do you plan for weather you can’t see yet?

Score a real forecast, then plan for its misses

Lesson 4 of Energy Systems, Step by Step: score a real grid operator’s day-ahead demand forecast against what happened, beat or lose to a simple baseline, see when percentage errors mislead, draw a band instead of a line, and find the scheduling cushion that balances the cost of being short against the cost of spare capacity, under illustrative prices.
  • Foundations to Analyst · L1–L2
  • Lesson 4 of 14
  • 15 min
  • After lesson 2 and lesson 3

A real forecast against what happened

Southwest Power Pool demand on Monday 10 February 2025, hour by hour, against the day-ahead forecast: the forecast missed by 1.6% on average; the largest miss was 1.3 GW at 16:00. 323436384000:0006:0012:0018:0024:00GW

With JavaScript on, you score this forecast, compare it with a simple baseline, draw a band of possibilities and choose how much spare capacity to schedule.

Real data: hourly demand, the day-ahead demand forecast, and solar and wind output of the Southwest Power Pool (SPP) in 2025, as reported to EIA-930 (accessed 2 Nov 2026). Costs in step 6 are illustrative.

Forecasts are from January–April only. From May the forecast SPP reports runs 8–15% above its reported demand in every month; the cause is unverified (see “Why only January to April?” below).

Need a refresher? Toolbox K5 (mean, spread and percentiles of forecast errors).

What you just did, as diagrams

Day-ahead forecast
The operator's estimate of every hour of tomorrow's demand, used to schedule plants.
Forecast error
Actual minus forecast; positive means demand came in higher than expected.
Persistence
The simplest forecast, “tomorrow looks like today”: the baseline any real forecast must beat.
MAE and MAPE
The average miss in MW, and in percent of actual; percentages break down near zero.
Band
A range around the forecast that held the actual value in a stated share of past hours; its future coverage still has to be tested.
Scheduling cushion
When being short costs more than spare capacity, the cheapest schedule sits above the forecast (illustrative prices).

Myth or fact? “Forecasting tomorrow's demand is guesswork: ‘same as today’ does about as well.”

  • Myth
  • Fact
Answer

Myth. In January–April 2025, SPP's average miss was 55% smaller than that of “same hour yesterday”.

Myth or fact? “MAPE is a fair score for a solar forecast.”

  • Myth
  • Fact
Answer

Myth. It cannot score night hours, and a 1 MW miss at dawn can count as much as a large miss at noon; use MAE or a skill score.

Myth or fact? “A good 80% band holds the actual value in about 80% of the hours of every single day.”

  • Myth
  • Fact
Answer

Myth. The 80% counts all the January–April hours used to build the band, not each day. Misses bunch together: on 4 February 2025, 19 of 24 hours fell outside the 80% band.

Go deeper · AnalystThe scores in symbols

With \(A_t\) the actual and \(F_t\) the forecast in hour \(t\), over \(n\) hours:

\[\text{MAE} = \frac{1}{n}\sum_t |A_t - F_t|, \qquad \text{MAPE} = \frac{1}{n}\sum_t \frac{|A_t - F_t|}{A_t}, \qquad \text{bias} = \frac{1}{n}\sum_t (A_t - F_t).\]

  • Two denominators. MAPE divides each miss by the actual value \(A_t\). The signed error in the chart and the band divide by the forecast: \(e_t = (A_t - F_t)/F_t\).
  • The 80% band runs from \(F_t(1+q_{0.1})\) to \(F_t(1+q_{0.9})\), the 10th and 90th percentiles of past \(e_t\). For SPP in January–April 2025, \(q_{0.1} = -2.1\%\) and \(q_{0.9} = +3.8\%\).
  • In-sample. The band is built and checked on the same months; a fair test uses months it has not seen.
  • Skill score against a baseline \(B\): \(\;1 - \text{MAE}_F / \text{MAE}_B\). Zero means no better than the baseline; negative means worse.
  • For SPP, January–April 2025, over the hours both could be scored: \(1 - 657/1{,}444 \approx 0.55\) against “same hour yesterday”.
Go deeper · ModelerLopsided penalties and the cheapest cushion
  • The idea. A missing MWh costs more than an unused one, so the cheapest cushion covers demand in most hours, not all.
  • How many hours. The share to cover is the shortage price over the sum of both prices, with \(c_u\) the price of a missing MWh and \(c_o\) of an unused one:

\[\frac{c_u}{c_u + c_o} = \frac{300}{300 + 10} \approx 0.97\]

  • On this page. Applied to the January–April misses, the rule gives a cushion of +5.9%; the cheapest available slider setting is +6.0%.
  • Next. Lesson M10 builds forecasts and scores them on months they have not seen.
Go deeperWhy only January to April?
Forecast SPP reported ÷ demand it reported, monthly average, 2025
Month Ratio Month Ratio
January 0.988 July 1.097
February 0.989 August 1.080
March 0.998 September 1.095
April 0.996 October 1.113
May 1.085 November 1.108
June 1.089 December 1.147
  • EIA asks each operator for the day-ahead forecast it makes in its normal course of business, which need not be defined exactly like the demand it reports.
  • From May the forecast ran above demand in 93% of hours. A reporting or definition mismatch is possible, but the cause is unverified.
  • Leaving May to December out is an editorial choice, pending validation.
Go deeperWhat the lesson leaves out
  • Operators update forecasts through the day and re-schedule plants; only the day-ahead forecast is scored here.
  • Reserves also cover sudden plant trips (lesson 3), not only forecast misses.
  • A single town is harder to forecast, in percent, than a whole region, where many customers' surprises partly cancel.
  • The cushion game uses one average forecast level and illustrative prices. It leaves out reserve availability, response speed, plant outages and transmission, so its answer is not a reserve requirement.

Check yourself

Actual demand in three hours: 30, 32 and 34 GW. The forecast said 31 GW each hour. What is the MAE?

  • 0 GW
  • 1.67 GW
  • 5 GW
  • 1 GW
Answer

1.67 GW: the misses are 1, 1 and 3 GW, so (1 + 1 + 3) ÷ 3 ≈ 1.67. The signed misses (−1, +1, +3) would average +1 GW: that is the bias, not the MAE.

A missing MWh costs $300 and an unused one $10. In roughly what share of hours does the cheapest scheduling cushion cover demand?

  • 50%
  • 80%
  • 97%
  • 100%
Answer

About 97%: 300 ÷ (300 + 10) ≈ 0.97. Covering every hour would mean scheduling for the worst miss ever seen, at a high price the rest of the time.

Numbers, sources and what’s next

Numbers to remember

Number What it means Source
1.9% SPP's average day-ahead demand forecast miss (MAPE), January–April 2025, over 2,716 hours: about 0.65 GW. Our calculation from EIA-930 data · Jan–Apr 2025
55% How much smaller SPP's average miss was than that of “same hour yesterday”. Our calculation from EIA-930 data · Jan–Apr 2025
−2.1% to +3.8% The middle 80% of SPP's misses, as demand relative to the forecast. Our calculation from EIA-930 data · Jan–Apr 2025
3% → 19% Average miss of “wind stays as it is now”, one hour and one day ahead, as a share of SPP's largest wind hour of 2025 (24.0 GW). Our calculation from EIA-930 data · 2025

Next on the starter path

Lesson 5 · Capacity factor: why is 1 MW not always 1 MW?

In the real world

U.S. balancing authorities that serve customers report a day-ahead demand forecast and their actual demand to EIA, so you can score any of them in the Hourly Electric Grid Monitor.

Check what each one reports before you compare them: SPP's later 2025 months show why.

From the course author's research: a wind turbine's real power curve is a band, not a line. Adding the turbine's theoretical curve to about two hundred measurements can narrow that band, but whether it helps depends on how the fit is scored. (Li et al., ASME IDETC 2023)

Case file CF8: learning a wind turbine's power curve from scarce data.

Sources and further reading

Spotted an error, or stuck? Write to hlli@ksu.edu. Corrections are made in place and dated here.

Text licensed CC BY 4.0, code MIT: teachers and students may reuse and adapt it, with credit to Honglin Li.