Python and data for energy: how do you load, plot and check a year of hourly data?

Find the bad hours in a real year of grid data

Bridge lesson B2 of Energy Systems, Step by Step: load a real year of hourly Southwest Power Pool data, fix its time stamps, find a spike and frozen sensors, run a physics check, resample, read a duration curve and capacity factors, and write assert checks; a pandas notebook repeats every step.
  • Analyst to Modeler · L2–L3
  • 20 min guided · 60 min lab
  • After lesson 5 and K7

What the checks find in SPP's 2025 data

Step What you find In pandas
Load 8,760 rows: one per hour, in MW, stamped in UTC at the end of each hour. df.shape
Inspect Demand reads 1,505 MW for one hour on 21 June, under 5% of the median: almost certainly a reporting error. df.describe()
Time On local time, labelled by each hour's start: 363 days of 24 hours, 9 March with 23 and 2 November with 25. tz_convert("America/Chicago")
Repeats Runs of 3 or more identical non-zero readings: 671 wind hours and 1,377 solar hours, the counts the data build flagged. (s != s.shift()).cumsum()
Plot From 15 to 21 December every column stands still for 155 hours; solar holds 617 MW through the nights. df.loc[...].plot()
Physics Solar above 5% of its peak between midnight and 4 a.m.: 214 hours in the raw file, none in the cleaned file. df.index.hour < 4
Resample Prairie Town's 150 MW wind farm makes the most energy in March (70.7 GWh) and the least in September (45.1 GWh). resample("MS").sum()
Duration Demand is above 90% of its peak for 180 hours; the farms' capacity factors are 52.8% (wind) and 23.4% (solar), illustrative. sort_values()
Assert The raw file fails 3 of 5 asserts; the cleaned file passes all 5. assert ...

With JavaScript on, you run each step yourself on the real data, with the pandas code beside it.

Data: hourly demand, wind and solar of the Southwest Power Pool, 2025, from EIA's Hourly Electric Grid Monitor (Form EIA-930, public domain); accessed 2 November 2026.
Raw file: EIA's adjusted values before this course's cleaning; each fuel's columns summed, negatives set to zero.
Cleaned file: the shapes Prairie Town uses.
Illustrative: 150 MW wind, 200 MW solar, 450 MW demand, as in lesson 0.

Do it in pandas · notebook lab, about 60 minutes

The notebook repeats every step in pandas, and an assert cell checks each result.

Run it in Google Colab (upload it with the two data files), or locally with Python 3, pandas and matplotlib.

Notebook (.ipynb) Raw SPP data (JSON) Cleaned shapes (JSON)

Need a refresher? Toolbox K8 (Git and reproducible notebooks).

What you just did, as a flowchart and a checklist

Time stamps
EIA-930 stamps each hour in UTC at its end; label hours by their start and convert to local time before grouping by day.
Range, repeats, physics
Three checks flag three kinds of suspect data: a spike, a possibly stuck feed, and sun at night.
Mean or sum
Average power uses the mean; energy uses the sum, since each hour's MW × 1 h is its MWh.
Duration curve
The year's hours sorted from highest to lowest; for a plant, the area under it over the full box is the capacity factor.
Assert
A check that stops the notebook with a message when the data break a rule.
Go deeper · ModelerWhy one hour of offset matters to a model
  • An hourly model reads value t as the average power over hour t, the hour its time stamp labels; times one hour, that is its energy.
  • Shift the stamps by one hour and the solar peak moves an hour, which changes when a battery should charge and what it earns.
  • Days of 23 and 25 hours break code that assumes 24 rows a day: keep a time-zone-aware index and let pandas place each hour.

Modeler's note: the lab's notebook is the template for every later lab; M10 (forecasting) and M14 (learning from data) start from a frame checked like this one.

Go deeper · AnalystA capacity factor needs a capacity

A capacity factor is energy divided by (nameplate capacity × hours). EIA-930 reports generation, not capacity, so a real fleet's capacity factor needs a second source, such as EIA's generator inventory (Form EIA-860).

Observed peak output is not a capacity measurement. If it is below nameplate capacity, using it as the denominator overstates the capacity factor.

Prairie Town's farms are defined to reach full output in SPP's best hour, which is why their 52.8% and 23.4% are labelled illustrative. Lesson 5 builds capacity factors from scratch.

Go deeperHow the data build filled the gaps, and its night rule
Problem What the data build did
Repeated wind or solar readings Replaced by the mean of the same hour on up to seven valid days before and after.
Implausible demand hours Replaced by the day-ahead forecast, rescaled to the actual demand around the gap.
Solar at night, 20 February to 14 March Raw solar barely changed between day and night but never repeated exactly, so the repeat rule kept it. A night rule sets solar to zero.
  • The night rule is a documented heuristic for this SPP, Central-time example: hours whose monthly median solar is zero, 22:00 to 05:00 all year, and hours when the sun is below the horizon at SPP's centre.
  • What it proves. Hours set to zero pass a night check by construction; the check does not test the daytime values, so look at those too.
  • The notebook applies the median and clock parts of the rule to the raw file.

R4 (data you can defend) turns these choices into a written data note.

Check yourself

Myth or fact? “Every local day has 24 hours.”

  • Myth
  • Fact
Answer

Myth. Where clocks change for daylight saving, one spring day has 23 hours and one autumn day has 25; in 2025 in U.S. Central time, 9 March and 2 November.

Myth or fact? “If the summary statistics look normal, every row is fine.”

  • Myth
  • Fact
Answer

Myth. 617 MW is a normal solar value, but not at 2 a.m.; repeat and physics checks look at rows and their times, not just at summaries.

Myth or fact? “Average wind output divided by the year's best hour is the wind fleet's capacity factor.”

  • Myth
  • Fact
Answer

Myth. A capacity factor divides by nameplate capacity. The best hour is not a capacity measurement; if it is below nameplate, that ratio overstates the capacity factor.

A wind farm's hourly output is 100 MW in every hour of one 24-hour day. How much energy did it make that day?

  • 100 MWh
  • 2,400 MWh
  • 4.2 MW
  • 2,400 MW
Answer

2,400 MWh: energy = power × time, 100 MW × 24 h. On a 23- or 25-hour day it would be 2,300 or 2,500 MWh.

You resample hourly MW values to months. Which gives each month's energy?

  • The mean
  • The sum
  • The maximum
  • The median
Answer

The sum, in MWh (divide by 1,000 for GWh). The mean gives the month's average power in MW.

Numbers, sources and what’s next

Numbers to remember

Number What it means Source
8,760 Rows in a year of hourly data (8,784 in a leap year). arithmetic
23 h, 25 h 9 March and 2 November 2025 in U.S. Central time, when the clocks change. EIA-930 SPP data · 2025
54.4 GW SPP's highest hourly demand in 2025, on 29 July. EIA-930 · data for 2025, accessed November 2026
1,377 h SPP solar hours in 2025 flagged as repeated readings (3 or more identical non-zero values in a row). EIA-930 · this lesson's check
180 h Hours a year that demand spends above 90% of its peak (SPP's 2025 shape). EIA-930 · this lesson's check

Next

B3 · How do you read a formulation, and check that a model is right?

Related lab: a notebook (about 60 minutes) that repeats every step in pandas, with auto-check cells.

Where this leads

The data are the same Southwest Power Pool year behind Prairie Town in lesson 0; lessons 2 (the shape of demand) and 5 (capacity factor) read it without code.

R4 (data you can defend) builds on it, and so do M10 (forecasting) and M14 (learning from data).

Sources and further reading

Spotted an error, or stuck? Write to hlli@ksu.edu. Corrections are made in place and dated here.

Text licensed CC BY 4.0, code MIT: teachers and students may reuse and adapt it, with credit to Honglin Li.