Why we care about the scientific credibility of emissions data
We’re at a pivotal moment in the quest to reduce methane emissions while guaranteeing the energy supply that modern life demands. Policymakers, industry, academics, and the environmental community are striving to implement new policies that curb emissions and balance economic impacts. It’s a tightrope walk that affords only one simple, yet challenging, way forward: follow the credible scientific evidence.
What makes for scientifically credible work?
Scientifically credible work is peer reviewed, based on transparent data, and authored by independent researchers. Let’s take a deeper dive into how these three components help establish scientific credibility:
Is the evidence presented peer reviewed?
Peer review is a process where experts in a scientific field assess and provide feedback on a new study. Peer reviewers will consider things like data quality and processing, study assumptions, novelty, methods, and conclusions. They can identify flaws and limitations, as well as conflicts of interest. A study will only be published once these expert concerns are satisfactorily addressed. The peer review process is imperfect, but it’s our best way to put sunshine on the data and assumptions being made for the evidence we’re considering. Not all peer-reviewed science is perfect, and not all evidence that hasn’t gone through peer review is untrustworthy, but any study lacking peer review should be regarded with care.
Are the data transparent?
This is a tricky one, particularly in the context of emissions because there’s a constant balance between data privacy and the need for transparency. Insight M (formerly Kairos Aerospace) has a track record of balancing data privacy (through complete data anonymization) and transparency in our work, because we believe that some level of transparency is required in order to assess the quality of our data. Transparency can include information like where, when, and how you sampled, how many measurements you collected, data quality concerns and limitations, as well as the distribution of your measurements. Studies that lack this information cannot be assessed fully, making it difficult to rely on their conclusions.
Are the lead authors of the study independent?
When data collection is challenging or expensive, such as basin-scale emissions measurements, scientific partnerships between data providers and academic researchers can generate very high quality studies. In these cases, it is critical that the academic partners lead the analysis and generate study conclusions. This produces a more credible study than a paper written solely by employees of a company with direct financial incentives behind their perspective. Insight M has always sought out independent study authors where possible to both minimize bias and capitalize on the significant expertise of academic experts to produce sound results.
Why does scientific credibility matter?
Developing policy– be it national policies, corporate operating principles, or other guiding frameworks– requires making complex decisions about how to balance goals and targets that may have significant, interconnected trade-offs. Every decision has potential downstream and upstream effects. It’s therefore critical that we use the highest quality information possible as the basis for our decisions. We cannot afford anything less.
Therefore, valid and credible scientific understanding can and must be the bedrock to our policy making processes. It leads to regulations that are as sound, defensible, and effective as the quality of data allows. It points us toward better outcomes while avoiding prejudices and self-interest. And it provides a framework to avoid bias and ensure a level playing field for industry to innovate, compete, and grow.
We’re seeing this play out in the Environmental Protection Agency’s (EPA) evolution of methane policy from its 2015 methane rule, or “OOOOa” rule, to today’s proposed methane rules, “OOOOb” and “OOOOc”. Back in 2015, the handful of available studies were limited to those that examined a relatively small number of oil and gas sites, concluding that most methane emitted from numerous small sources. EPA relied on this credible, but sparse, understanding to craft the OOOOa rule optimized for curbing small sources.
In the years since, our ability to measure and quantify the field dynamics of methane emissions has grown substantially, leading to an updated scientific understanding that large, rare super emitters are the source of most methane emitted in the field. EPA’s recently proposed rules incorporate new, peer-reviewed scientific evidence into its proposal. Where super emitters are completely absent from OOOOa, in OOOOb the EPA incorporates strategies for tackling large emissions into its framework, and built the case for doing so upon the most current scientific studies.
As this example demonstrates, good policy follows from credible science, and as scientific understanding evolves so too must any recommended action.
The community should commit to upholding quality science
In many cases, the people most deeply involved in the science of understanding emissions have made it their life’s work. Policymakers do not have the luxury of focusing on one single issue like methane emissions to understand it as deeply as possible. Similarly, industry experts have many other issues to consider and weigh.
Therefore, members of the community must commit to uphold scientific integrity, and rely on credible science to develop and promote new policy. It’s the only way to get things right on the critical issue of methane emissions.