Joseph F. De Carolis, Administrator of the U.S. Energy Information Administration (EIA), has some general advice for anyone studying long-term energy forecasts:
“Whatever you do, don’t start believing the numbers,” De Carolis said at the MIT Energy Initiative (MITEI) fall meeting. “When you sit in front of a computer and watch models spit out numbers, you tend to actually start believing those numbers with a high degree of precision. Don’t be fooled. Always be skeptical.”
The event is part of MITEI’s new lecture series, “MITEI Presents: Advancing the Energy Transition,” which connects the MIT community with energy experts and leaders working on the science, technology, and policy solutions urgently needed to accelerate the energy transition.
The main point of De Carolis’ talk, titled “Be Humble and Prepare for the Unexpected: Lessons from the Energy Transition,” was not that energy models are unimportant. In contrast, De Carolis said, energy models provide stakeholders with a framework that allows them to consider current decisions in the context of potential future scenarios. But he repeatedly stressed the importance of taking uncertainty into account and not treating these projections like a “crystal ball.”
“We can use models to help shape our decision-making strategies,” De Carolis said. “We know there’s a lot of uncertainty about the future. We don’t know what’s going to happen, but by incorporating that uncertainty into our models, we can provide a path forward.”
Dialogue, not prediction
EIA is the statistical and analytical agency of the U.S. Department of Energy, with a mission to collect, analyze, and disseminate independent and objective energy information to help stakeholders make more informed decisions. While EIA analyzes the impacts of energy policies, the agency itself does not develop policy or provide policy advice. De Carolis, a former professor and university faculty researcher in the Department of Civil, Construction, and Environmental Engineering at North Carolina State University, noted that EIA does not need to get permission from anyone else in the federal government before releasing its data or reports. “That independence is important to us because it means we can focus on doing our job and providing the best information possible,” he said.
Among the many reports produced by the EIA is the agency’s Annual Energy Outlook (AEO), which forecasts energy production, consumption, and prices in the U.S. Every two years, the agency also produces an AEO retrospective report that shows the relationship between past forecasts and actual energy indicators.
“The first question people ask is, ‘Should we use these models to make predictions?'” De Carolis says. “My answer to that question is, ‘No, you shouldn’t.’ When you use models to make predictions, the results are often very pessimistic.”
De Carolis points to past projections of the expansion of nuclear power in the United States being wildly inaccurate as an example of the problems inherent in forecasting. Still, he notes, energy models “still have a lot of really valuable uses.” Rather than using models to predict future energy consumption or prices, De Carolis said stakeholders should use the models to inform their own thinking.
“[Models] are just tools that help us think about the energy future and make assumptions,” De Carolis said. “They help generate a dialogue between different stakeholders about a complex issue. If we want to think about something like the energy transition and start a dialogue, that dialogue needs some basis. If we have a systematic representation of the energy system that we can carry into the future, we can start discussing the model and its implications. We can also identify the main sources of uncertainty and knowledge gaps.”
Uncertainty Modeling
The key to working with energy models is to account for uncertainty, not eliminate it, De Carolis says. He points out that one way to better understand uncertainty is to look at past projections and see how they differed from actual outcomes. De Carolis points to two “surprises” that occurred over the past few decades: the rapid growth of shale oil and natural gas production (which had the effect of limiting coal’s share of the energy market and reducing carbon emissions), and the rapid growth of wind and solar power. In both cases, market conditions changed much faster than energy modelers predicted, leading to inaccurate forecasts.
“All these reasons led to CO2 emissions being significantly higher than what actually happened ,” De Carolis said. “We’re a statistical agency, so we look really carefully at the data, but it can take a while to identify the signal among the noise.”
While EIA does not publish forecasts in the AEO, reference cases in the agency’s reports are sometimes interpreted as projections. To illustrate the unpredictability of future outcomes in its 2023 AEO publication, the agency added “cones of uncertainty” to its projections of energy-related carbon dioxide emissions, with a range of outcomes based on the difference between past projections and actual outcomes. One cone captures 50 percent of the historical forecast error, and the other represents 95 percent of the historical error.
“They capture any bias that exists in our projections,” De Carolis said of the uncertainty cones. “We record this number because we are comparing actual emissions with the projections. But this has drawbacks: who is to say that past projection errors will apply in the future? We don’t know, but I think we might still learn something useful from this exercise.”
The Future of Energy Modeling
“There are so many challenges that keep me up at night as a modeler” about the future, De Carolis said. These include the impacts of climate change; how those impacts affect demand for renewable energy; how industry and governments overcome obstacles in building clean energy infrastructure and supply chains; technological innovation; and increased energy demand from data centers running compute-intensive workloads.
“What about geothermal power? Thermonuclear? Solar power in space? De Carolis asked. “Should those be included in the model? What technological innovations are we missing? And of course there are the unknowns. I can’t imagine including them on this list, but I think they probably will.”
In addition to obtaining the widest range of results, De Carolis said, EIA wants to produce reports that are flexible, agile, transparent and accessible, and can easily incorporate new model capabilities to provide timely analysis. To that end, the agency has launched two new initiatives. First, AEO 2025 will use an improved version of the National Energy Modeling System, which includes modules for hydrogen production and pricing, carbon management and hydrocarbon supply. The second, an effort called “Project Blue Sky,” aims to develop the agency’s next-generation energy system model, which De Carolis said will be modular and open source.
De Carolis noted that energy systems are complex and rapidly evolving, and warned that a fear of “mental shortcuts” and mistakes can lead modelers to overlook possible future developments. “We have to be humble and intellectually honest about what we know,” he said. “That way we can honestly communicate to decision makers our thoughts about what will happen in the future.”
