Which system, which boundary, which method?
Place a question on the scale ladder, draw its boundary, and defend a method against two alternatives
The task
- Produce: a design card for one brief: its scale, its boundary (decided or computed, fixed inputs, left out), a time step and horizon, a method family, and a defence of that method against two alternatives.
- From: three illustrative briefs at three scales, each a different type of question.
- Done when: the card states the scale, the three boundary lists, a time step and horizon, and a method defended against two alternatives, one from another family; your R1 card covers the rest of Framing.
About 30 minutes guided here, then about an hour on your own: take your own question card from R1 through the same moves, and write the defence in the same six parts.
The moves, in order
- Place the question on the scale ladder: device and plant, microgrid and community, regional grid, city and region, nation.
- Draw its boundary: what the study decides or computes, the fixed inputs it uses but does not change, and what it leaves out and says so.
- Pick a time step that shows what the metric needs, and a horizon that covers the decision.
- Choose a method family, then defend it against a simpler version and against another family, and name its main limit.
Three briefs, with a researcher's answers
Keep the hospital running (prescriptive, S1)
What sizes of solar, battery and backup generator keep Prairie Campus's hospital supplied through a three-day winter outage at the lowest 20-year cost, compared with today's diesel generator alone?
Assumed: the switch to running alone (islanding) works. The study checks whether there is enough energy, not the switch itself.
- Scale: S1: one campus that can run on its own, sized by its owner under the utility's rules. The tariff and the feeder's limits come from the regional grid above.
- Decided or computed (in the study: it decides or computes these):
- Sizes of solar, battery and generator (These are the decisions the study makes)
- Fixed inputs (in the study: used, but not changed by it):
- The hospital's critical load, hour by hour (A measured or estimated input; the study does not change it)
- The utility's tariff and the feeder's export limit (Set at the scale above; fixed inputs here)
- Hourly winter weather for solar output (An input: a design winter, or several past ones)
- Diesel fuel stored on site when the outage starts (A fixed limit in the outage case)
- Left out (outside the study: stated as a limit):
- Wholesale prices across the Plains Grid (The campus pays a tariff, not wholesale prices: say so as a limit)
- Road access for fuel trucks during the storm (Left out and stated: the design must last three days on stored fuel)
- Who owns the system and who pays for it (A separate question about ownership, noted as a limit of the cost result)
- Time step and horizon: 1 hour, hourly steps follow solar's daily cycle and the battery's charge; a typical year plus the outage.
- Fits well: Optimization, Simulation, Mixed methods.
- Helps with a part: Statistics and econometrics, Qualitative methods.
- Poor fit for this question: Machine learning, System dynamics, Agent-based modelling, Equilibrium modelling.
- A researcher's defence (Optimization):
- Family: Optimization
- Method: A sizing mixed-integer program (MILP) that picks the cheapest sizes keeping the critical load served.
- Time step and horizon: Hourly steps over a typical year, plus a three-day winter outage.
- Alternative 1: a linear program (LP) with continuous sizes
- Why not alternative 1: The generator has a minimum load and comes in whole units, which only whole-number decisions capture.
- Alternative 2: Simulation
- Why not alternative 2: The question asks for the cheapest mix, not a ranking of mixes I already have; a simulation still checks the chosen design.
- Main limit: Perfect foresight during the outage: the model knows the coming weather.
- Reduced or checked by: Test the chosen design in a rule-based simulation that does not know the future.
Did the rebate reach low-income homes? (evaluative, S3)
Did Prairie City's 2023 heat-pump rebate raise installations in its low-income neighbourhoods, compared with similar neighbourhoods in cities without a rebate?
- Scale: S3: a city programme, set by the council with local money and rules. Federal incentives come from the nation above; the homes below act on them.
- Decided or computed (in the study: it decides or computes these):
- Heat-pump installations per 1,000 homes, by neighbourhood and month (The outcome the study measures)
- The choice of comparison neighbourhoods (A design decision of the study, and the one that matters most)
- Fixed inputs (in the study: used, but not changed by it):
- The rebate's rules, amount and start date (Fixed facts of the programme)
- Neighbourhood income and housing data from the census (Input data, used to match similar neighbourhoods)
- Federal tax credits available everywhere (Assumed to affect both groups alike; check eligibility and other policies that could shift their trends differently)
- Left out (outside the study: stated as a limit):
- Whether the heat pumps lowered bills (A different question, for a later study)
- Installers' waiting lists (Left out and stated: they may have limited uptake)
- Grid upgrades for the new heating load (Outside a question about uptake)
- Time step and horizon: 1 month, monthly counts show the change around the rebate's start; 2019 to 2025, before and after.
- Fits well: Statistics and econometrics, Mixed methods.
- Helps with a part: Machine learning, Agent-based modelling, Qualitative methods.
- Poor fit for this question: Optimization, Simulation, System dynamics, Equilibrium modelling.
- A researcher's defence (Statistics and econometrics):
- Family: Statistics and econometrics
- Method: A difference-in-differences comparison of installations in low-income neighbourhoods.
- Time step and horizon: Monthly installations by neighbourhood, 2019 to 2025.
- Alternative 1: a before-and-after count in Prairie City alone
- Why not alternative 1: Prices and federal credits changed everywhere at once, so a before-and-after count mixes them with the rebate.
- Alternative 2: Agent-based modelling
- Why not alternative 2: It would assume how households respond, which is what this study must measure.
- Main limit: It assumes both groups would have moved in parallel without the rebate.
- Reduced or checked by: Compare their trends before 2023, and repeat with a second comparison group.
A large load that can pause (predictive, S2)
How would a 300 MW load that can pause for up to four hours during scarcity change the Plains Grid's expected unserved energy in a heat-wave year, compared with the same load running flat?
Pausing rule (illustrative): up to four hours in a row per event, at most ten events a year, and paused work is not made up later. Paused energy is counted apart from unserved energy.
- Scale: S2: adequacy is planned for a region, such as a market operator's or a utility's planning area, with its transfer limits and assumptions about its neighbours. The load only decides when it pauses.
- Decided or computed (in the study: it decides or computes these):
- When the load pauses, hour by hour (Set inside the study by a pausing rule)
- Random generator outages (Drawn many times by the study, to give a range)
- Fixed inputs (in the study: used, but not changed by it):
- Hourly demand and weather for a heat-wave year (Input data: a past heat-wave year, or a scaled one)
- Today's plants and the transfer limits between zones (Fixed: the question asks about the system as it is)
- Imports from neighbouring grids (Fixed limits, stated)
- Left out (outside the study: stated as a limit):
- The load's contract: who pays for pausing (A separate question about contracts and tariffs)
- Voltage and stability in the seconds after a trip (A different time scale, and a different study)
- New plants built because the load arrived (A planning question; this one keeps the fleet fixed)
- Time step and horizon: 1 hour, adequacy studies count unserved energy hour by hour; one year, run many times.
- Fits well: Simulation.
- Helps with a part: Optimization, Statistics and econometrics, Agent-based modelling, Qualitative methods, Mixed methods.
- Poor fit for this question: Machine learning, System dynamics, Equilibrium modelling.
- A researcher's defence (Simulation):
- Family: Simulation
- Method: A Monte Carlo adequacy simulation that runs the heat-wave year many times with random outages.
- Time step and horizon: Hourly steps; one year, run many times.
- Alternative 1: a check of the single worst hour
- Why not alternative 1: The question asks how often and how much, not only whether.
- Alternative 2: Optimization
- Why not alternative 2: The question keeps today's plants and asks for a risk, not a plan.
- Main limit: Outages are drawn independently, while a heat wave can cause several at once.
- Reduced or checked by: Add a run with linked outages as a sensitivity.
With JavaScript on, you make each choice yourself, with a note on how well it fits, and your design stays in this browser.
The briefs, Prairie Campus, Prairie City, the Plains Grid and the 300 MW load are illustrative. The fit notes are teaching judgements, not rules: several method families can answer one question well.
Two answers to the same task
The task
Defend a method for: “How sensitive is a national storage build-out to a one-year disruption of one mineral, and which design choices reduce the risk?”
Weak answer · Developing
Method: capacity-expansion optimization [1] Why: it is the standard tool for national planning [2] Result: it gives the optimal answer [3] Covers: the whole country [4]
- Falls short: A family is named, but nothing says why it fits a question about sensitivity and risk.
- Falls short: Chosen by habit: the rubric's Developing anchor for method fit.
- Falls short: Optimal only inside the model's assumptions, and one run gives no range.
- Falls short: A scale, but no boundary, time step or horizon.
Strong answer · Strong on method fit
Reading: a conditional planning study: how do model-optimal storage plans change across disruption cases? [1] Scale and boundary: S4; the national power system's builds and hourly operation; mineral supply enters as given disruption cases; recycling left out [2] Method: a capacity-expansion model (optimization), run under many disruption cases drawn at random [3] Time step and horizon: yearly steps to 2040, representative hours inside each year [4] Alternative 1: a single run with expected prices [5]; it hides the risk the question asks about Alternative 2: a system-dynamics model of mineral supply [6]; it captures market feedback, not the grid's hourly needs Main limit: perfect foresight inside each case; the results are model-optimal plans, not predicted deployment [7] Reduced by: fixing early builds before the disruption is known [8]
- Works: Says what an optimization can answer here: the best plan under each case, not a forecast of what will be built.
- Works: The three boundary lists: what is decided or computed, what is fixed and what is left out.
- Works: Matches the question: “how sensitive” needs a range, not one plan.
- Works: Time step and horizon stated, matched to a build-out over years.
- Works: A simpler version of the same family, rejected for a reason.
- Works: Another family, rejected for a reason: what the Proficient anchor asks for.
- Works: The limit stated, with what the results do and do not mean.
- Works: The limit partly mitigated: the Strong anchor for method fit.
| Weak answer | Strong answer | |
|---|---|---|
| Why this family | Habit | Tied to what the question asks |
| Boundary | “The whole country” | Decided, fixed and left out |
| Time step and horizon | Not stated | Yearly to 2040, representative hours |
| Alternatives | None | One in the family, one from another |
| Limit | None | Stated, what results mean, partly reduced |
Nine method families, side by side
| Family | Answers | Needs | Main limit |
|---|---|---|---|
| Optimization | What is the best choice under stated rules and goals? | Decisions, a goal, rules (constraints) and their data. | Best only inside the model, for its stated goal; a full-horizon run also assumes perfect foresight, unless built to decide step by step (a rolling horizon). |
| Simulation | What happens when the system runs by given rules, under given conditions? | Operating rules, inputs over time and the designs to test. | It tests the designs you give it; it does not search for the best one. |
| Statistics and econometrics | How much, how often, and did a change cause an effect? | Observed data with variation; for a causal claim, a comparison group. | It sees only the observed past; causal claims rest on assumptions about the comparison. |
| Machine learning | What will the next value be, or which pattern is this? | Many examples, honest baselines and data held back for testing. | Least reliable outside the range of its training data, and it explains little about why. |
| System dynamics | How do stocks, flows and feedback shape a trend over years? | A causal-loop structure and parameters for each flow. | It works with aggregates, and many parameters rest on judgement. |
| Agent-based modelling | How do many deciding actors produce a system outcome? | Rules for each kind of actor, and data on how they behave. | Results depend on behaviour rules that are hard to check against data. |
| Equilibrium modelling | Where do prices and quantities settle when many markets or players interact? | Supply and demand, market rules and how players respond to prices. | Static versions describe a settled state, not the path to it; most simplify how players behave. |
| Qualitative methods | How and why do people and institutions decide as they do? | Interviews, documents or cases, with consent and a plan for the analysis. | It gives no numbers that generalise; this course teaches it at awareness level only. |
| Mixed methods | Questions that need both numbers and people's reasons. | A model or data analysis, a qualitative part, and a plan for how each informs the other. | More time, and both parts must answer the same question. |
Check your own work against the rubric
| Competence | Developing | Proficient (the bar) | Strong |
|---|---|---|---|
| Framing Look for: the scale and the three boundary lists are stated; the rest of Framing (audience, metric with units, counterfactual, falsification) comes from your R1 card. |
Question vague or untyped; no audience or counterfactual. | Typed question with audience or decision-maker, metric with units, counterfactual, scale and boundary; one falsification statement. | Also explains why the question matters to its audience and what answer would change a decision. |
| Method fit Look for: two alternatives named, one from another family, each rejected for a reason tied to the question. |
Method chosen by habit. | Method defended against two alternatives, one of them a different family. | Limits of the chosen method stated and partly mitigated. |
Five scales: who decides, and typical steps and horizons
- S0 Device and plant: decides what to install at one site, and how to run it; the owner, within tariffs and codes. Time step: minutes to an hour; horizon: a day to the equipment's life.
- S1 Microgrid and community: decides sizes, islanding, load priorities, ownership; a facility owner or co-op, under the utility's rules. Time step: minutes to an hour; horizon: an outage to about 20 years.
- S2 Regional grid: decides commitment, dispatch, reserves, transmission, new plants; the grid operator or utility, and regulators. Time step: seconds to an hour; horizon: the next hour to several decades.
- S3 City and region: decides buildings, heat, transport, land use, local supply; city and county, utilities and owners. Time step: an hour to a year; horizon: to 2040 or 2050.
- S4 Nation: decides targets, policy instruments, infrastructure, security stocks, trade; national governments and regulators. Time step: representative hours to a year; horizon: decades.
Myth or fact? “A more detailed model is always more accurate.”
- Myth
- Fact
Answer
Myth. Detail adds data needs and uncertainty of its own; match the detail to the question, and say what you left out.
Myth or fact? “An optimal plan predicts what will happen.”
- Myth
- Fact
Answer
Myth. It is the best plan under the model's assumptions; real decisions and real paths depart from it, which is why a prescriptive answer comes with its limits.
Myth or fact? “Saying what a study leaves out makes its answer more useful, not less.”
- Myth
- Fact
Answer
Fact. A reader can only use an answer if they know where it holds; an unstated boundary invites them to use it where it does not.
Go deeper · ResearcherWhere the defence goes next
Go deeperWhat this studio leaves out
- Spatial resolution: zones, buses, neighbourhoods or grid cells matter as much as the time step, and are chosen the same way.
- Coupling models: many studies link two families, for example an optimization inside a Monte Carlo loop. The “mixed” label here covers only models with a qualitative part.
- Qualitative methods are taught at awareness level only; a fully qualitative study needs a mentor trained in them.
Sources and what’s next
Tools named on this page
| Tool | What it is for | Link |
|---|---|---|
| PyPSA | An open energy-system modelling framework in Python (optimization). | docs.pypsa.org |
| JuMP | An open modelling language for optimization in Julia. | jump.dev |
| Pyomo | An open modelling language for optimization in Python. | pyomo.org |
The other tools in the comparison table are named as examples, without links.
Next
Sources and further reading
- DeCarolis, J. et al. (2017). Formalizing best practice for energy system optimization modelling. Applied Energy 194, 184–198.
- Pfenninger, S., Hawkes, A. and Keirstead, J. (2014). Energy systems modeling for twenty-first century energy challenges. Renewable and Sustainable Energy Reviews 33, 74–86.
- Trutnevyte, E. (2016). Does cost optimization approximate the real-world energy transition? Energy 106, 182–193.
- Sovacool, B. K., Axsen, J. and Sorrell, S. (2018). Promoting novelty, rigor, and style in energy social science: Towards codes of practice for appropriate methods and research design. Energy Research & Social Science 45, 12–42.
Spotted an error, or stuck? Write to hlli@ksu.edu. Corrections are made in place and dated here.
Text licensed CC BY 4.0, code MIT: teachers and students may reuse and adapt it, with credit to Honglin Li.