Different Interpretations of Same Data Is Routine | The Institute for Creation Research
Different Interpretations of Same Data Is Routine
Are people growing weary of scientific experts giving them contradictory instructions for dealing with the COVID-19 pandemic? It seems that there may be an increasing willingness by politicians and ordinary citizens alike to question the reliability of scientific pronouncements by experts—at least by those addressing this pandemic. Some state governors have taken to interpreting the data (as well as all of the different expert opinions) for themselves. This has led to different state responses. One positive result of the COVID-19 pandemic may be that people may get a more realistic assessment of the limits of science when they are forced to experience firsthand the actual uncertainty of scientific conclusions by individual experts based on differing—or downright conflicting—interpretations of the same set of data.

Getting scientific agreement on relatively straightforward interpretations of data is simple enough. If a panel of experts was asked the question: Is the spread of COVID-19 an outbreak, an epidemic, or a pandemic? The data clearly support the interpretation that the world is facing a pandemic. However, agreement on interpretations of data breaks down quickly when more complex questions are applied to it. One of the best studies illustrating this problem was recently published as it relates to neuroimages.

By way of background, a CT scan, magnetic resonance image (MRI), or X-ray taken of the brain or spinal cord is called a neuroimage. Radiologists and others with specialized training interpret what they see to make a diagnosis or give recommendations for treatment. Like all other areas of science, there is plenty of room for interpretation. This study of neuroimaging experts, published in Nature,1 acknowledged that “data analysis workflows in many scientific domains have become increasingly complex and flexible.” But how flexible are the analyses of—and conclusions about—neuroimages? To find out, the researchers initiated a large collaborative project consisting of 70 teams of neuroimaging experts from around the world called the Neuroimaging Analysis Replication and Prediction Study (NARPS).

The teams were given the raw functional MRI data along with the protocols for prior experiments. Each team used one of three different image analysis software packages. But they were free to decide how to analyze the data and to apply their own interpretative methodology in order to test nine hypotheses about specific brain activity. The teams disagreed on interpretations for five out of nine hypotheses. The researchers comment on how complete flexibility by different individuals to interpret the same data can result in diverse conclusions:

Here we assess the effect of this flexibility on the results of functional magnetic resonance imaging by asking 70 independent teams to analyse the same dataset, testing the same 9 ex-ante hypotheses. The flexibility of analytical approaches is exemplified by the fact that no two teams chose identical workflows to analyse the data. This flexibility resulted in sizeable variation in the results of hypothesis tests…[this] variation in reported results was related to several aspects of analysis methodology…[and thus] our findings show that analytical flexibility can have substantial effects on scientific conclusions….

The results do not necessarily imply that any of the different interpretations are wrong. But it does clearly illustrate how highly trained experts can come to conflicting interpretations of the same data, depending on the interpretive framework they use to approach the data.

This fact is very relevant to all areas of science. Let’s briefly consider it as it applies to the controversies surrounding the theory of evolution, the veneration of science in society, and scientific consensus.

Previously, ICR reported on a contentious meeting of the British Academy and the Royal Society held on November 7-9, 2016, focused on reconciling evolutionary theory with contradictory observations and mechanisms of adaptation.2 Interestingly, divisions at the Royal Society along with the study on neuroimaging experts illustrate an important point that creationists have been saying. Evolutionists often claim that they have a mountain of data to support their position while creationists have none. Creationists contend that they have the same data, but interpret it very differently. Similarly, one evolutionist, Kevin Laland of the University of St Andrews, was coming from the minority position at the Royal Society meeting and he observed,

This tension was manifest in the discussions where different interpretations of the same findings were voiced…The conference brought home a key point—these debates are not about data but rather about how findings are interpreted and understood.3

So, it seems that the debate generally isn’t over which side has data. Rather, the debate is about which interpretation better explains the data.

Since different interpretations of data is quite normal in science, then part of the frustration that people feel with the conflicting guidance from COVID-19 experts may stem from inaccurate beliefs about how certain pronouncements by scientists actually are. Many people may be unaware of how often conflicting interpretations of the same data arises between scientists. So, how might they absorb beliefs extolling the pre-eminence of scientific thinking compared to other ways of gaining knowledge? A recent podcast interview by Nature with Mario Livio about his new book, Galileo and the Science Deniers, may give a clue.4 When asked about important lessons that could be gleaned from his book Livio stated,

Well, the real lesson is ‘believe in science.’ It’s not that science is always right, but science has this ability to self-correct. So we have to believe in science, and we have to put the science first, and before any kind of political considerations, conservatism, religious beliefs and things like that. This is a big lesson.

Yet, self-correction is not exclusive to science. Also, scientists could exploit their supposed impartial scientific authority to advance their personal political or religious agendas.

Finally, one may wonder how a true “scientific consensus” can emerge, given that the same data is often interpreted very differently. One might legitimately ask if the consensus is a real unity of interpretation among scientists, or if there might be other factors in play. Perhaps the experiences of people during the COVID-19 pandemic has exposed that claims of scientific consensus are somewhat disingenuous and that the “consensus” was more like “follow the leader.” People got to observe how guidance pronounced by one scientific expert (who is perceived by his or her colleagues as the best in the field) was continually parroted by others possessing expert qualifications. This repetition—or consensus—gave a sense of scientific authority for obeying the guidance. Unfortunately, people were soon informed that the guidance that they were once told to obey was the exact opposite of what they should now be doing. How can they know this? Because the scientific consensus said so.

1. Botvinik-Nezer, R. et al. Variability in the analysis of a single neuroimaging dataset by many teams. Published on nature.com May 20, 2020 accessed May 24, 2020.
2. Guliuzza, R. J. Schism in Evolutionary Theory Opens Creationist Opportunity. Creation Science Update. Posted on ICR.org May 18, 2017, accessed May 24, 2020.
3. Laland, K. N. 2017. Schism and Synthesis at the Royal Society. Trends in Ecology & Evolution. 32(5): 316–317
4. Anonymous. Podcast: Galileo and the science deniers, and physicists probe the mysterious pion (Time Marker 16:06). Nature. Posted on nature.com May 6, 2020 accessed May 24, 2020.

*Randy Guliuzza is ICR’s National Representative. He earned his Doctor of Medicine from the University of Minnesota, his Master of Public Health from Harvard University, and served in the U.S. Air Force as 28th Bomb Wing Flight Surgeon and Chief of Aerospace Medicine. Dr. Guliuzza is also a registered Professional Engineer.
The Latest
Are Creationists Conspiracy Theorists?
Paul Braterman, emeritus professor of chemistry at the University of Glasgow, recently claimed online that creationism “meets all the criteria”...

How Algae Do Fine When Tossed at Sea
How would you do if someone spun you around every few seconds all day long? Marine algae repeatedly get tossed about in coastal surf, and they cope quite...

Butterfly Wing Design Repudiates Evolution
The takeoff and flight of butterflies has long been derided by evolutionists as being an unstable and inefficient product of evolution. However, a new...

Reconciling Two Different Calculations of the Hubble Constant
An interesting article1 on physics.org caught my attention. Its title is “Solved: The Mystery of the Expansion of the Universe.”...

Institute for Creation Research Awarded Highest Rating from Charity...
DALLAS, February 11, 2021–The Institute for Creation Research (ICR) recently earned a 4-star rating from Charity Navigator—America’s...

Reminder: Saturn’s Moon Titan Really Looks Young
Scientists led by Valerio Poggiali of Cornell University’s Center for Astrophysics and Planetary Science have used Cassini spacecraft data to learn...

Is Newfound Dinosaur Fossil the Biggest Ever?
Whether dealing with athletes, sports teams, skyscrapers, or animals, we always want to know who or what takes the top spot. Researchers just published...

Inside February 2021 Acts & Facts
What is the mission of the Institute for Creation Research? What legacy did Dr. Henry M. Morris III leave behind? How will our Ice Age model impact...

Creation Kids: Jupiter
You’re never too young to be a creation scientist! Kids, discover fun facts about God’s creation with ICR’s special Creation Kids learning...

Planting, Watering, and God's Increase
During construction of the ICR Discovery Center for Science & Earth History, I spent a lot of time working with various crews of artists and craftsmen....