Parth Vaishnav- School for Environment and Sustainability, University of Michigan, Ann Arbor, MI, United States
Quantitative analyses may aim to provide actionable answers to policy questions and to generate tools or insights for decision-making. Given the deep uncertainties involved in any realistic reckoning of policy questions, this study argues that only the second of these goals is achievable. Here, this argument is illustrated by considering analyses of how the electrification of an activity changes the damage from the air pollution emissions that occur because of that activity. The sources of uncertainty in such an analysis include the long life of the technologies being studied. Consequently, the structure and operation of the electricity grid might change because of the new technology and independent of it. Analysts must make subjective choices about what to include in their analysis and what to exclude. For example, policies modeled in isolation may, in reality, be bundled with other policies; interactions between technologies may be missed if the analysis focuses on only one technology; and certain benefits or costs may be neglected because they lie outside the scope of the analysis and the expertise of the analyst. Quantitative policy analysis must aim to be part of the broader discussions in society that ultimately determine what policies get implemented.
Introduction
Morgan et al. (1992) identify a variety of motivations for policy analysis: from the desire to inform a policy decision to the development and demonstration of new methods and tools. Morgan et al. (1992) call the former of these motivations “substance-focused.” Within this category, motivations range from “answering” policy questions in a form that leads to direct implementation to “illuminat[ing] and provid[ing] insight on a general area of policy concern for a variety of interested parties.” From this perspective, it is extraordinarily difficult for analysts to answer complex policy questions based solely on quantitative analysis. Even very careful analyses must satisfy themselves with providing broad insights and developing tools that might inform broader policy discussions.
This argument is illustrated by considering analyses of how the electrification of an activity changes the damage from the air pollution emissions that occur because of that activity. Electrification is defined as a switch to using electricity to power activities that currently require the distributed combustion of fossil fuels. Examples of such activities include light transportation (Michalek et al., 2011; Holland et al., 2016; Yuksel et al., 2016) and space heating in homes (Hanova and Dowlatabadi, 2007; Vaishnav and Fatimah, 2020; Deetjen et al., 2021). This fuel switch is seen by analysts as an essential decarbonization strategy (Davis et al., 2018).
Attributional and consequential analyses of electrification
Whether or not electrification actually reduces greenhouse gas emissions—and Harms from emissions of short-lived pollutants—depends on how much pollution is produced in generating electricity for the application in question.
A recent National Academies of Science Engineering and Medicine (NASEM, 2022) report argues that, when assessing policies that induce a change—such as a shift from a fossil fuel to electricity—analysts must account for the change in harms induced by the change in fuel.
The NASEM report argues that, in practice, this means that it is incorrect to simply use the average emissions intensity, expressed in the mass of pollutants per unit of electricity produced, based on the current operation of the electricity system. Using average emissions is an attributional approach: it should be used only to apportion the harms from the current operation of the power system among current uses. Almost by definition, this makes attributional approaches unsuitable for studying the effects of electrification.
Instead, for small changes in demand, analysts must model how the operation of the current electricity system will change if more electricity is demanded. This amounts to asking which of the existing generators will produce more to meet the new demand. The NASEM (2022, p. 190–194) report outlines four key approaches to performing this analysis: regression based on past operations of the power system, modeling of the current or future operation of the power system, the use of proxies (e.g., non-baseload generation), and inferences based on data about the real-time operation of the power system. Each approach has significant limitations.
For large changes in demand, analysts must model how new demand will change the composition, structure, and operation of the power system. This consequential analysis is not straightforward. Analysts must make many assumptions—about policies, about the relative costs of different technologies, and about the often volatile prices of commodities—to produce estimates of emissions from future systems.
Average emissions from the existing power system are a physical quantity that can (and is) directly measured. The notion of changes in marginal emissions from current and future systems is a conceptual construct that is not directly related to any physical quantity. Using a consequential approach to answer questions that are of interest to policymakers presents three challenges.
Challenge 1: uncertainty induced by long-lived technologies
Modern personal vehicles in the United States are projected to last nearly 20 years (Zhu et al., 2021). Electric appliances such as heat pumps may have similar lifetimes (Staffell et al., 2012). An analysis that assumes unchanging marginal emission factors from the electricity grid ignores the possibility of better performance over the lifetime of a new technology than in its first year of operation. Some studies approximate this improvement in performance by assuming that electricity grid marginal emission factors fall in line with average emission factors (Vaishnav and Fatimah, 2020; Deetjen et al., 2021). However, studies of historical regression-based emissions factors have shown that this improvement has not occurred in the United States (Holland et al., 2022b). A response to this study questioned the appropriateness of using marginal emissions factors designed to reflect small, short-term changes to the grid to study the effect of changes that are neither small nor short term (Gagnon et al., 2022).
An alternative approach is exemplified by the Cambium data set assembled by the National Renewable Energy Laboratory (Gagnon et al., 2023). Cambium calculates long-run marginal emission factors by comparing two alternative runs: one with a baseline level of demand and another in which demand in each hour is perturbed by a substantial amount. In each of these runs, a capacity expansion model (CEM) and an economic dispatch model are both run. The CEM captures the fact that new generators and transmission capacity may need to be built in response to large and persistent changes in demand. The Cambium modeling approach also adjusts the generation mix to ensure that existing state and national renewable portfolio standards are met.
This approach differs from current approaches in two ways. First, it accounts for structural rather than operational changes. Second, it offers a way of modeling the effects of changes that occur alongside a large change in demand but not entirely because of it. This could be especially relevant to policymakers, who may want to account not only for the fact that the operation and structure of the grid will change because of the new demand a policy induces but that also, in the long term, the power system will undergo changes unrelated to the new demand.
There is, nonetheless, “an inescapable degree of subjectivity” (Holland et al., 2022a) in how short- or long-term consequential emissions are modeled. While it is theoretically possible to put bounds on the consequences of those subjective choices, the computational requirements and barriers to entry in terms of the depth of expertise needed for such analyses are substantial.
Challenge 2: uncertainty induced by choices related to the system boundary
An important source of uncertainty is the choice of system boundary, wherein some aspects of the consequences of a policy may be left out of the decision. Here, three examples are discussed.
First, an analysis may ignore that policies may not be implemented in isolation from each other but as bundles with other unrelated policies. For example, utilities may require that owners of electric vehicles switch to time-of-use rates (DTE Electric Company, 2023). Therefore, a policy to encourage the adoption of electric vehicles may have the unintended (and unmodeled) effect of switching users to dynamic rates, which may affect how they use other electricity-consuming appliances. An analyst must grapple with the diversity in utility responses to electrical vehicle adoption and the diversity of user responses.
Second, there might be current or future synergies between different technologies, which may not be accurately modeled. For example, a heating ventilation and air-conditioning contractor might advise a client that installing an electrical heat pump is financially more attractive if the client also installs rooftop solar panels and improves the insulation of their home. Vehicle-to-grid and vehicle-to-home technologies might allow users or service providers to manipulate household electricity load profiles in ways that meaningfully change the impact of electric vehicles on the power system.
Third, deploying a technology might produce benefits that are either unrelated or indirectly related to energy or the environment. For example, Michalek et al. (2011) quantify the ways in which electric vehicles might reduce geopolitical risk, military spending, and volatility in fuel costs. Analyses of weatherization often focus on energy, cost, or air pollution benefits (or harms); (Fowlie et al., 2018) but often ignore the significant health benefits associated with better-insulated homes (Howden-Chapman et al., 2007; Tonn et al., 2021). In advocating for ambitious technical targets for batteries for aviation, Viswanathan et al. (2022) note that the effort to achieve these targets will have spillover benefits for electric road vehicles. Deploying technologies might produce learning effects, which might shift the balance of benefits and costs in ways that are seldom captured in models; for example, significant learning, defined as the reduction in cost for every doubling of deployed capacity, has been observed for electric vehicles (Taylor et al., 2005; Rubin et al., 2015; Malhotra and Schmidt, 2020). The deployment of technologies can catalyze the construction of supporting infrastructure, which, in turn, can make the technology more attractive. Li et al. (2017) demonstrate this positive feedback loop in the case of electric vehicles and charging infrastructure, arguing that investing in charging infrastructure is more cost-effective than subsidizing EVs (electric vehicles) directly.
Challenge 3: reconciling present and future perspectives
The 2022 NASEM report notes that attributional LCA (ALCA) “estimates emissions as they are or could be in some projected future state (emphasis added)” (20). In a future where the electricity grid is substantially—if not fully—decarbonized, an ALCA would show that widespread electrification is unambiguously better than the continued use of fossil fuels. Nonetheless, a consequential analysis performed from today's perspective might suggest that many changes made in the direction of that future increase environmental harms.
The first solution to this conundrum is to identify those strategies that reduce harms even in the short term and prioritize them, while continuing to deploy fossil fuels in applications where they do less harms given the current and near-future electricity grid (Williams et al., 2012). A criticism of this approach is that any continued reliance on fossil fuels risks creating lock-ins and stranded assets (Bertram et al., 2015). A second criticism is that a managed, sequential deployment of technology presumes more control over how the energy transition unfolds than is realistic. A third criticism is that any detailed recommendations about the correct sequencing could suffer from false precision. All the sources of uncertainty described earlier mean that detailed recommendations based on small differences between alternatives run the risk of being an artifact of what the analyst chose to include (or not) in the analysis (Lave, 1996).
A second solution is to take a heuristic approach. In this view, what matters in most contexts1 is that a combination of electrification and grid decarbonization offers a pathway to net zero emissions, whereas the distributed combustion of fossil fuels does not. While eschewing detailed recommendations based on differences that are smaller than the surrounding uncertainty, analysts may restrict themselves to hot-spot analyses that identify great potential harms (or benefits) that might be ignored in policies that are focused on energy or greenhouse gas emissions. For example, an early study of electric vehicles with lead–acid batteries found that the harms from excess emissions of lead from smelters would far exceed any benefits from reduced greenhouse gas emissions (Lave et al., 1995). A criticism of the heuristic approach stems from the fact that the extent of warming is a function of cumulative greenhouse gas emissions (IPCC, 2021). A trade-off exists between the indirect decarbonization benefits of policies that increase greenhouse gas emissions in the near term (e.g., through learning to reduce costs and accelerate full adoption) and their contribution to the cumulative stock of atmospheric greenhouse gases. The analysis must grapple with this trade-off. A second criticism of a heuristic approach is that resources—including money, attention, and political will—are finite. Failing to allocate them optimally can carry potentially large opportunity costs (Tengs et al., 1995).
What can policy analysts say about electrification?
The net-zero emissions energy systems study by Davis et al. (2018) identified sectors, including load-following electricity, as difficult to decarbonize. Sectors such as light transportation and the residential sector were, however, flagged as straightforward to decarbonize. What makes assessing the effects of electrification complicated is that the straightforward-to-decarbonize sectors are coupled with load-following electricity, which is hard to decarbonize. Arguably, the overall goal of studies of the environmental consequences of electrification is to elucidate the evolving nature of that coupling.
In doing so, analysts studying electrification must recognize that different approaches and assumptions might be legitimate, given subtle differences in the specifics of the decision that the analysis is seeking to inform. For example, if only the near-term implications of an electrification policy are of interest, it may be appropriate to ignore structural changes to the grid that result from that electrification or that occur alongside it.
For the analysis to have broader relevance, it must be repeated using different approaches (e.g., short- or long-range marginal emissions or average emissions from a future grid), and differences in the results must be discussed. Analysts must identify what assumptions are the most consequential and give users of the analysis the means to easily substitute their own assumptions instead.
Finally, consumers of analysis must ensure that there is a match between the question they are trying to answer and the question that a study has answered. They should pay attention to differences in time scale (e.g., short vs. long term), goals (e.g., reducing short-term harms vs. long-term transformation), and scope (e.g., a standalone intervention vs. numerous intertwined changes). Where these differences are large, they should be cautious about basing policy on the conclusions of the study.
What can policy analysts say about policy choices?
Quantitative policy analyses in service of the energy transition should comply with guidance on how to conduct good policy analysis in general. Morgan et al. (1992) identify “Ten Commandments” for good policy analysis: (1) do your homework with literature, experts, and users; (2) let the problem drive the analysis; (3) make the analysis as simple as possible but not simpler; (4) identify all significant assumptions; (5) be explicit about decision criteria and policy strategies; (6) be explicit about uncertainties; (7) perform systematic sensitivity and uncertainty analysis; (8) iteratively refine the problem statement and the analysis; (9) document clearly and completely; and (10) expose the work to peer review.
While it is difficult to meet all these strictures fully, analysts and decision-makers should grow warier of analyses as they veer further away from these commandments. Policy analysis can provide clear answers to scientific questions, provided there is “unambiguous data or well-founded theoretical insight” (Morgan, 1978, p. 971). If it becomes too difficult to track all assumptions or adequately characterize sensitivities and uncertainties, one must question the reliability of any conclusions. Consequential analyses of the effects of electrification must yoke together multiple models from domains as diverse as epidemiology and power system analysis. Arguably, they make it extraordinarily hard for analysts to obey Morgan et al. (1992) commandments. Conclusions from these analyses must be presented with a corresponding degree of humility and even skepticism.
In his critique of benefit analysis, Lave (1996) described the method as foremost a means of structuring complex problems, arguing that “the option identified as having the largest net benefit does not have a strong claim to being the best social choice” (129). In the same vein, given the depth of uncertainty associated with decisions pertaining to electrification, quantitative analysis ought to be identified as one (but not the only) tool to aid decision-making rather than a means of generating optimal policy prescriptions. This approach has been described as “modeling for insights” (Huntington et al., 1982).
John Stuart Mill defined representative democracy as government by discussion. Quantitative policy analysis must accept that it forms part of a discussion (Mill, 1861; Harris, 1956) and must—if at all possible—seek to make that discussion more productive.
Data availability statement
The original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding author.
Author contributions
PV: Writing—original draft, Writing—review and editing.
Funding
The author(s) declare that no financial support was received for the research, authorship, and/or publication of this article.
Conflict of interest
The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher's note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
Footnotes
1. ^There are some applications (e.g., aviation) where it is not clear that full electrification is feasible.
References
Bertram, C., Johnson, N., Luderer, G., Riahi, K., Isaac, M., Eom, J., et al. (2015). Carbon lock-in through capital stock inertia associated with weak near-term climate policies. Technol. Forecast. Soc. Change 90, 62–72. doi: 10.1016/j.techfore.2013.10.001
Davis, S. J., Lewis, N. S., Shaner, M., Aggarwal, S., Arent, D., Azevedo, I. L., et al. (2018). Net-zero emissions energy systems. Science 360, eaas9793. doi: 10.1126/science.aas9793
Deetjen, T. A., Walsh, L., and Vaishnav, P. (2021). US residential heat pumps: the private economic potential and its emissions, health, and grid impacts. Environ. Res. Lett. 16, 084024. doi: 10.1088/1748-9326/ac10dc
DTE Electric Company (2023). “Rate Schedule no. D1.9: Electric Vehicle Rate,” in Rate Book for Electric Service (Detroit, MI), D-14.03. Available online at: https://www.michigan.gov/-/media/Project/Websites/mpsc/consumer/rate-books/electric/dte/dtee1cur.pdf?rev=d72535d2c1964a5d88dc2cef986b1e7f (accessed August 1, 2023).
Fowlie, M., Greenstone, M., and Wolfram, C. (2018). Do energy efficiency investments deliver? Evidence from the weatherization assistance program. The Q. J. Econ. 133, 1597–1644. doi: 10.1093/qje/qjy005
Gagnon, P., Cowiestoll, B., and Schwarz, M. (2023). Cambium 2022 Scenario Descriptions and Documentation. Golden, CO: National Renewable Energy Lab.
Gagnon, P. J., Bistline, J. E. T., Alexander, M. H., and Cole, W. J. (2022). Short-run marginal emission rates omit important impacts of electric-sector interventions. Proc. Nat. Acad. Sci. 119, e2211624119. doi: 10.1073/pnas.2211624119
Hanova, J., and Dowlatabadi, H. (2007). Strategic GHG reduction through the use of ground source heat pump technology. Environ. Res. Lett. 2, 044001. doi: 10.1088/1748-9326/2/4/044001
Harris, A. L. (1956). John Stuart Mill's theory of progress. Ethics 66, 157–175. doi: 10.1086/291054
Holland, S. P., Kotchen, M. J., Mansur, E. T., and Yates, A. J. (2022a). Reply to Gagnon et al.: short-run estimates vs. long-run conjectures. Proc. Nat. Acad. Sci. 119, e2214219119. doi: 10.1073/pnas.2214219119
Holland, S. P., Kotchen, M. J., Mansur, E. T., and Yates, A. J. (2022b). Why marginal CO2 emissions are not decreasing for US electricity: Estimates and implications for climate policy. Proc. Nat. Acad. Sci. 119, e2116632119. doi: 10.1073/pnas.2116632119
Holland, S. P., Mansur, E. T., Muller, N. Z., and Yates, A. J. (2016). Are there environmental benefits from driving electric vehicles? The importance of local factors. Am. Econ. Rev. 106, 3700–3729. doi: 10.1257/aer.20150897
Howden-Chapman, P., Matheson, A., Crane, J., Viggers, H., Cunningham, M., Blakely, T., et al. (2007). Effect of insulating existing houses on health inequality: cluster randomised study in the community. BMJ 334, 460. doi: 10.1136/bmj.39070.573032.80
Huntington, H. G., Weyant, J. P., and Sweeney, J. L. (1982). Modeling for insights, not numbers: the experiences of the energy modeling forum. Omega 10, 449–462. doi: 10.1016/0305-0483(82)90002-0
IPCC (2021). “Summary for policymakers,” in Climate Change 2021: The Physical Science Basis. Contribution of Working Group I to the Sixth Assessment Report of the Intergovernmental Panel on Climate Change, eds. V. Masson-Delmotte, P. Zhai, A. Pirani, S. L. Connors, C. Péan, S. Berger, et al. (Cambridge: Cambridge University Press), 3–32.
Lave, L. B. (1996). “Benefit-cost analysis: do the benefits exceed the costs?,” in Risks, Costs, and Lives Saved: Getting Better Results from Regulation, ed. R. W. Hahn (Oxford: Oxford University Press).
Lave, L. B., Hendrickson, C. T., and McMichael, F. C. (1995). Environmental implications of electric cars. Science 268, 993–995. doi: 10.1126/science.268.5213.993
Li, S., Tong, L., Xing, J., and Zhou, Y. (2017). The market for electric vehicles: indirect network effects and policy design. J. Assoc. Environ. Res. Econ. 4, 89–133. doi: 10.1086/689702
Malhotra, A., and Schmidt, T. S. (2020). Accelerating low-carbon innovation. Joule 4, 2259–2267. doi: 10.1016/j.joule.2020.09.004
Michalek, J. J., Chester, M., Jaramillo, P., Samaras, C., Shiau, C. S. N., Lave, L. B., et al. (2011). Valuation of plug-in vehicle life-cycle air emissions and oil displacement benefits. PNAS 108, 16554–16558. doi: 10.1073/pnas.1104473108
Morgan, M. G. (1978). Bad science and good policy analysis. Science 201, 971–971. doi: 10.1126/science.201.4360.971
Morgan, M. G., Henrion, M., and Small, M. (1992). Uncertainty: A Guide to Dealing with Uncertainty in Quantitative Risk and Policy Analysis. Cambridge: Cambridge University Press.
NASEM (2022). Current Methods for Life Cycle Analyses of Low-Carbon Transportation Fuels in the United States. Washington, DC: National Academies Press.
Rubin, E. S., Azevedo, I. M. L., Jaramillo, P., and Yeh, S. (2015). A review of learning rates for electricity supply technologies. Energ. Policy 86, 198–218. doi: 10.1016/j.enpol.2015.06.011
Staffell, I., Brett, D., Brandon, N., and Hawkes, A. (2012). A review of domestic heat pumps. Energ. Environ. Sci. 5, 9291–9306. doi: 10.1039/c2ee22653g
Taylor, M. R., Rubin, E. S., and Hounshell, D. A. (2005). Regulation as the mother of innovation: the case of SO2 control. Law Policy 27, 348–378. doi: 10.1111/j.1467-9930.2005.00203.x
Tengs, T. O., Adams, M. E., Pliskin, J. S., Safran, D. G., Siegel, J. E., Weinstein, M. C., et al. (1995). Five-hundred life-saving interventions and their cost-effectiveness. Risk Anal. 15, 369–390. doi: 10.1111/j.1539-6924.1995.tb00330.x
Tonn, B., Hawkins, B., Rose, E., and Marincic, M. (2021). Income, housing and health: Poverty in the United States through the prism of residential energy efficiency programs. Energ. Res. Soc. Sci. 73, 101945. doi: 10.1016/j.erss.2021.101945
Vaishnav, P., and Fatimah, A. M. (2020). The environmental consequences of electrifying space heating. Environ. Sci. Technol. 54, 9814–9823. doi: 10.1021/acs.est.0c02705
Viswanathan, V., Epstein, A. H., Chiang, Y. M., Takeuchi, E., Bradley, M., Langford, J., et al. (2022). The challenges and opportunities of battery-powered flight. Nature 601, 519–525. doi: 10.1038/s41586-021-04139-1
Williams, J. H., DeBenedictis, A., Ghanadan, R., Mahone, A., Moore, J., Morrow, W. R., et al. (2012). The technology path to deep greenhouse gas emissions cuts by 2050: the pivotal role of electricity. Science 335, 53–59. doi: 10.1126/science.1208365
Yuksel, T., Tamayao, M. A. M., Hendrickson, C., Azevedo, I. M. L., and Michalek, J. J. (2016). Effect of regional grid mix, driving patterns and climate on the comparative carbon footprint of gasoline and plug-in electric vehicles in the United States. Environ. Res. Lett. 11, 044007. doi: 10.1088/1748-9326/11/4/044007
Keywords: decarbonization, electrification, policy analysis, energy transition, lifecycle analysis
Citation: Vaishnav P (2023) How can quantitative policy analysis inform the energy transition? The case of electrification. Front. Sustain. Energy Policy 2:1271301. doi: 10.3389/fsuep.2023.1271301
Received: 02 August 2023; Accepted: 07 September 2023;
Published: 09 October 2023.
Edited by:
Saba Siddiki, Syracuse University, United StatesReviewed by:
Noah Kittner, University of North Carolina at Chapel Hill, United StatesEric Hittinger, Rochester Institute of Technology (RIT), United States
Copyright © 2023 Vaishnav. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Parth Vaishnav, parthtv@umich.edu