Learning from few runs: surrogates, Gaussian processes and multi-fidelity data

Fit a wind turbine’s power curve from 16 records with a Gaussian process, check its band on 400 new records, and add the maker’s curve as a cheap second source

Modeler lesson M9 of Energy Systems, Step by Step: fit a Gaussian process to a few records of an illustrative wind turbine, let the records choose its settings by maximum likelihood, score its band on held-out records for calibration, add the maker’s power curve as a cheap second source, and see when that helps and when it misleads; a Python notebook rebuilds it.
  • Modeler · L3
  • 40 min guided · 45 min lab · about 60 min practice
  • After M7 and K6

One turbine, 16 records, two fits (illustrative)

The first 16 records of the toy turbine, all below 12 m/s, with two fits and their 90% bands: one source (the records alone) and two sources (the maker's curve plus the records). The one-source band widens above 12 m/s; the two-source band follows the maker's curve there. −50005001,0001,5002,0002,50036912151820wind speed, m/skW
  1. The records sit furthest below the maker's curve at 11–12 m/s, 239 kW on average: gusts smooth the curve's corner at rated speed.
  2. With fixed settings, the 90% band is widest where there are no records; records up to 15.5 m/s cut its mean width from 1,220 kW (16 records) to 576 kW (32).
  3. The 16 records choose a length-scale of 4.6 m/s, noise 80 kW and signal size 780 kW (maximum likelihood on a grid of 3,808 settings).
  4. RMSE on its own records against 400 new ones: 75 kW and 126 kW for the records' choice; 2 kW and 506 kW for a very short length-scale.
  5. On 400 new records the 90% band holds 95.5% (sampling alone: about ± 3 points): 99% at 3–8 m/s, 88% at 8–13 m/s, 100% at 13–20 m/s.
  6. Adding the maker's curve as a second source cuts the RMSE from 126 kW to 85 kW and the band's mean width from 391 kW to 287 kW; the band holds 92.0% overall but 82% at 8–13 m/s (one source: 88%).
  7. Two sources are 41 kW closer with 16 records and 5 kW with 64.
  8. With a load-reducing mode above 12.5 m/s that the maker's curve leaves out, the two-source band holds 58% of new records at 13–20 m/s; the one-source band holds 100%.
  9. In that case pinball loss rates the two fits about equal (27.9 and 27.7 kW), while RMSE is 92 kW and 107 kW.

The summaries on this page

Summary Value
Records to fit, to score 16 and 400 (illustrative, seeded)
Records' choice (one source) 4.6 m/s, 780 kW, 80 kW
RMSE, one and two sources 126 kW and 85 kW
90% band holds, one and two sources 95.5% and 92.0%
Load-reducing mode, holds at 13–20 m/s 100% and 58%

With JavaScript on, you fit and score these curves yourself, step by step.

  • Toy: one invented 2,000 kW turbine, not a real one. Its field curve is the maker's curve averaged over gusts (12% turbulence) and 3% lower.
  • Scatter: power scatter combines 15 kW of background noise with an approximate wind-error term, 0.3 m/s times the curve's slope, so it grows where the curve is steep.
  • Records: ten-minute averages, wind speeds Weibull-distributed (k = 2, c = 8 m/s) between 3 and 20 m/s; 64 to fit (seed 9, used in order) and 400 others to score (seed 2023), generator mulberry32.
  • Fits: a squared-exponential kernel and one noise level; settings by maximum likelihood on a fixed grid. Settings for 24 to 64 records are stored with the page and checked by its test.

About 40 guided minutes in nine steps. Every fit runs in your browser; nothing is sent anywhere.

Build it in Python · notebook lab, about 45 minutes, then 60 minutes of practice

The same records and the same fits in NumPy, with assert cells against the page; then scikit-learn's Gaussian process with its own optimizer, a noise level that changes with wind speed, and a surrogate of M7's slow model.

  1. Download the notebook and the lesson data into one folder (in Colab, upload the data with the folder icon).
  2. Open it in Google Colab (File → Upload notebook) or locally with Python 3, NumPy, SciPy and scikit-learn.
  3. Run all cells (under a minute: about 5 seconds on a 2025 laptop), then try the five practice tasks at the end.

Notebook (.ipynb) Lesson data

What you just did, as diagrams

Surrogate
A cheap stand-in for something costly to run or to measure, fitted to a few samples.
Gaussian process
A smooth best guess with a band: narrow near the records, wide where there are none.
Length-scale and noise
How far one record reaches, and how much scatter the fit expects around the curve.
Held-out records
Records the fit never saw; only they show how good it is.
Calibration
A 90% band should hold about 90% of new records, overall and within each range of wind speed.
Two sources
A cheap curve that covers every speed, plus a few costly records; it helps when its shape is right.
Go deeper · ModelerThe formulas: the fit, its band and the records' choice
  • Symbols. \(v\) wind speed (m/s); \(y\) the records’ power, one entry per record (kW); \(n\) the number of records; \(I\) the \(n \times n\) identity matrix; \(^\top\) the transpose, rows written as columns (K6); \(|K|\) the determinant of \(K\); \(\sigma\) the band’s standard deviation.
  • Kernel. \(k(v, v') = s_f^2 \exp\!\left(-\frac{(v - v')^2}{2\ell^2}\right)\), with \(\ell\) the length-scale (m/s) and \(s_f\) the signal size (kW); the records carry noise \(s_n\) (kW): \(K = [k(v_i, v_j)] + s_n^2 I\).
  • Mean. A basis \(h(v)\) sets the shape far from the records: \([1]\) for one source, \([1,\ f(v)/1000]\) with the maker’s curve \(f\) for two. \(H\) stacks \(h\) at every record, one row each. Its weights come from generalized least squares, \(\hat\beta = (H^\top K^{-1} H)^{-1} H^\top K^{-1} y\).
  • Prediction. \(m(v) = h(v)^\top \hat\beta + k_v^\top K^{-1}(y - H\hat\beta)\), where \(k_v\) lists \(k(v, v_i)\) for every record.
  • Band. The curve’s variance is \(s_f^2 - k_v^\top K^{-1} k_v + u^\top (H^\top K^{-1} H)^{-1} u\) with \(u = h(v) - H^\top K^{-1} k_v\); a new record adds \(s_n^2\). The 90% band is \(m \pm 1.645\,\sigma\).
  • Solving. One Cholesky factor \(K = LL^\top\) gives every solve; it costs about \(n^3/3\) operations, which is why GP training sets stay small (vectors and matrices: K6).
  • The records’ choice. Maximize \(\log p(y) = -\tfrac12 r^\top K^{-1} r - \tfrac12 \log|K| - \tfrac n2 \log 2\pi\), \(r = y - H\hat\beta\), over a grid of \(\ell \times s_f \times s_n\); the page’s grid has 3,808 settings.
  • Pinball loss. For quantile level \(\tau\) and forecast quantile \(q\): \(\tau (y - q)\) if \(y \ge q\), else \((1-\tau)(q - y)\); the page averages it over the 19 levels 5% to 95% of \(N(m, \sigma^2)\).
Go deeper · ModelerTwo sources in the literature, and surrogates of slow models
  • Kennedy and O’Hagan’s form. The costly source is \(\rho\) times the cheap one plus an independent GP for the gap. Here the cheap curve is known exactly at every speed, so it enters as a basis function; when the cheap source is itself a slow model, both levels get a GP.
  • Nonlinear fusion. When the costly source is not a scaled copy of the cheap one, nonlinear multi-fidelity models put the cheap output in as an extra input; the lab’s open code compares both.
  • Slow models. The same fit replaces a model too slow to run a thousand times: run it at a few inputs, fit the GP, then sample the GP, as M7 did with its fast toy. The lab’s practice task 4 does this for M7’s model.
  • Scores across sites. When a fit must work at places with no records, hold out whole places, not single records: M10 splits by time for the same reason.

Check yourself, then try it

Myth or fact? “A Gaussian process's 90% band holds 90% of new records at every wind speed.”

  • Myth
  • Fact
Answer

Myth. In step 5 it held 99% at 3–8 m/s but 88% at 8–13 m/s. A band is only as good as its kernel and noise assumptions: check it on held-out records, by range.

Myth or fact? “Adding a cheap second source can only make a fit better.”

  • Myth
  • Fact
Answer

Myth. With the load-reducing mode, the two-source band held 58% of new records at 13–20 m/s against the one-source band's 100%: cheap data mislead where they are wrong and no record corrects them.

Myth or fact? “A fit that passes close to all its records is a good fit.”

  • Myth
  • Fact
Answer

Myth. The very short length-scale in step 4 missed its own records by 2 kW and new ones by 506 kW.

A 90% band holds 86% of 400 new records. Two standard errors of a share near 90% are √(0.9 × 0.1 ÷ 400) × 2 = 3 points. What do you conclude?

  • Nothing: 86% is close to 90%
  • The band is likely too narrow: 4 points below is outside the sampling range
  • The band is too wide
  • The records are wrong
Answer

Likely too narrow: 86% is 4 points below 90%, outside the approximate 95% sampling range of ± 3 points for independent records, so sampling alone is unlikely to explain it. Consecutive real records are not independent, so their range is wider.

You fit a one-source GP to 16 records, all below 12 m/s. Where will its band be widest?

  • At 6 m/s, where records are dense
  • At rated speed, where the curve bends
  • Above 12 m/s, beyond the last record
  • The same everywhere
Answer

Above 12 m/s: with no records there, the fit falls back on its mean and its band opens up.

Try it yourself with open code

Optional, about 30 minutes each; needs Python 3.10 or later (lesson B2).

Try it yourself: the framework fits the three models to an invented turbine type and wind farm in a few seconds. Change the training size and the kernel-density bandwidth, or compare linear and nonlinear fusion on a test function. Its numbers say nothing about any real turbine.

  • Clone and install it (its README's quick start), then run python examples/sensitivity.py: compare the three models' RMSE at 50, 100, 200 and 400 training points.
  • Run python examples/probabilistic.py and read the coverage at each nominal level, as in step 5.

mfgpr-wind-power-curve: public example repository · MIT licence · synthetic data.

Try it yourself: the framework runs end to end on invented plants in about half a minute; another example compares target choices against a negative control. Its numbers say nothing about any real plant, data product or forecast system.

  • Run python examples/minimal.py: in its printed scores, does each model beat the constant baseline?

wind-uq-surrogate: public example repository · MIT licence · synthetic data.

This work was developed based upon funding from the Alliance for Sustainable Energy, LLC, Managing and Operating Contractor for the National Renewable Energy Laboratory for the U.S. Department of Energy.

Numbers, sources and what’s next

Numbers to remember

Number What it means Source
126 → 85 kW RMSE on 400 new records of the toy's one-source and two-source fits, both from the same 16 records. This lesson's model (illustrative)
± 3 points Approximate 95% sampling range of the share of 400 independent new records inside a 90% band (two standard errors). This lesson's model; K5
58% against 100% Share of new records at 13–20 m/s inside the 90% band, two sources against one, when the maker's curve is wrong there. This lesson's model (illustrative)
n³ ÷ 3 Floating-point operations to factor a GP's matrix for n records (Cholesky): about 1,400 for 16 records, about 330 million for 1,000. Arithmetic, as in Go deeper

Keep going

Back to the course map

Related lab: a NumPy notebook that refits the page's records and checks them, then scikit-learn's Gaussian process, a noise level that changes with wind speed and a surrogate of M7's model, about 45 minutes, then 60 minutes of practice.

Where this came up in research

From the course author's research: a wind turbine's real power curve is a band, not a line. Adding the turbine's theoretical curve to about two hundred measurements can narrow that band, but whether it helps depends on how the fit is scored. (Li et al., ASME IDETC 2023)

Linear (autoregressive) and nonlinear multi-fidelity Gaussian process regression fuse the turbine type's theoretical power curve (low fidelity) with a few hundred kernel-density-sampled SCADA points from the open SDWPF dataset (high fidelity), and are compared with single-fidelity regression on point and probabilistic metrics.

The kernel density sample keeps the shape of the scatter without manual cleaning; a compact training set matters because a Gaussian process solve grows with the cube of the training points.

In a research project (2022–2023), the course author worked on this question: can uncertainty metrics of a wind power model's inputs (weather data products, forecast ensembles and turbine-level data) predict calibrated plant-level power uncertainty at wind plants that have no power measurements?

The course author developed the surrogate-modeling workflow: calibrated uncertainty targets, surrogate models and validation on held-out plants.

Inputs: uncertainty metrics of four kinds: how a series varies over time (moving windows), how data sources disagree, how widely a forecast ensemble spreads, and how values vary across a plant's turbines (a variogram).

Targets: at plants with power measurements, the scale of a predictive distribution around the point forecast, chosen to minimize pinball loss.

Surrogates: Gaussian process regression and other regressors learn a mapping from the metrics to that scale, then predict probabilistic power at plants without measurements.

Validation holds out whole plants, scored with pinball loss, CRPS and reliability, with recalibration of the predictive distributions.

A surrogate is useful only if it beats a constant baseline on held-out plants.

The project has no paper. The public framework is a fresh implementation tested on synthetic data; project data and results are not public.

This work was developed based upon funding from the Alliance for Sustainable Energy, LLC, Managing and Operating Contractor for the National Renewable Energy Laboratory for the U.S. Department of Energy.

This page's turbine is a toy: invented round numbers, not the studies' data, equations or results.

Case file CF8: learning a wind turbine's power curve from scarce data.

Sources and further reading

Spotted an error, or stuck? Write to hlli@ksu.edu. Corrections are made in place and dated here.

Text licensed CC BY 4.0, code MIT: teachers and students may reuse and adapt it, with credit to Honglin Li.