Different Interpretations of Same Data Is Routine | The Institute for Creation Research

Different Interpretations of Same Data Is Routine
Are people growing weary of scientific experts giving them contradictory instructions for dealing with the COVID-19 pandemic? It seems that there may be an increasing willingness by politicians and ordinary citizens alike to question the reliability of scientific pronouncements by experts—at least by those addressing this pandemic. Some state governors have taken to interpreting the data (as well as all of the different expert opinions) for themselves. This has led to different state responses. One positive result of the COVID-19 pandemic may be that people may get a more realistic assessment of the limits of science when they are forced to experience firsthand the actual uncertainty of scientific conclusions by individual experts based on differing—or downright conflicting—interpretations of the same set of data.

Getting scientific agreement on relatively straightforward interpretations of data is simple enough. If a panel of experts was asked the question: Is the spread of COVID-19 an outbreak, an epidemic, or a pandemic? The data clearly support the interpretation that the world is facing a pandemic. However, agreement on interpretations of data breaks down quickly when more complex questions are applied to it. One of the best studies illustrating this problem was recently published as it relates to neuroimages.

By way of background, a CT scan, magnetic resonance image (MRI), or X-ray taken of the brain or spinal cord is called a neuroimage. Radiologists and others with specialized training interpret what they see to make a diagnosis or give recommendations for treatment. Like all other areas of science, there is plenty of room for interpretation. This study of neuroimaging experts, published in Nature,1 acknowledged that “data analysis workflows in many scientific domains have become increasingly complex and flexible.” But how flexible are the analyses of—and conclusions about—neuroimages? To find out, the researchers initiated a large collaborative project consisting of 70 teams of neuroimaging experts from around the world called the Neuroimaging Analysis Replication and Prediction Study (NARPS).

The teams were given the raw functional MRI data along with the protocols for prior experiments. Each team used one of three different image analysis software packages. But they were free to decide how to analyze the data and to apply their own interpretative methodology in order to test nine hypotheses about specific brain activity. The teams disagreed on interpretations for five out of nine hypotheses. The researchers comment on how complete flexibility by different individuals to interpret the same data can result in diverse conclusions:

Here we assess the effect of this flexibility on the results of functional magnetic resonance imaging by asking 70 independent teams to analyse the same dataset, testing the same 9 ex-ante hypotheses. The flexibility of analytical approaches is exemplified by the fact that no two teams chose identical workflows to analyse the data. This flexibility resulted in sizeable variation in the results of hypothesis tests…[this] variation in reported results was related to several aspects of analysis methodology…[and thus] our findings show that analytical flexibility can have substantial effects on scientific conclusions….

The results do not necessarily imply that any of the different interpretations are wrong. But it does clearly illustrate how highly trained experts can come to conflicting interpretations of the same data, depending on the interpretive framework they use to approach the data.

This fact is very relevant to all areas of science. Let’s briefly consider it as it applies to the controversies surrounding the theory of evolution, the veneration of science in society, and scientific consensus.

Previously, ICR reported on a contentious meeting of the British Academy and the Royal Society held on November 7-9, 2016, focused on reconciling evolutionary theory with contradictory observations and mechanisms of adaptation.2 Interestingly, divisions at the Royal Society along with the study on neuroimaging experts illustrate an important point that creationists have been saying. Evolutionists often claim that they have a mountain of data to support their position while creationists have none. Creationists contend that they have the same data, but interpret it very differently. Similarly, one evolutionist, Kevin Laland of the University of St Andrews, was coming from the minority position at the Royal Society meeting and he observed,

This tension was manifest in the discussions where different interpretations of the same findings were voiced…The conference brought home a key point—these debates are not about data but rather about how findings are interpreted and understood.3

So, it seems that the debate generally isn’t over which side has data. Rather, the debate is about which interpretation better explains the data.

Since different interpretations of data is quite normal in science, then part of the frustration that people feel with the conflicting guidance from COVID-19 experts may stem from inaccurate beliefs about how certain pronouncements by scientists actually are. Many people may be unaware of how often conflicting interpretations of the same data arises between scientists. So, how might they absorb beliefs extolling the pre-eminence of scientific thinking compared to other ways of gaining knowledge? A recent podcast interview by Nature with Mario Livio about his new book, Galileo and the Science Deniers, may give a clue.4 When asked about important lessons that could be gleaned from his book Livio stated,

Well, the real lesson is ‘believe in science.’ It’s not that science is always right, but science has this ability to self-correct. So we have to believe in science, and we have to put the science first, and before any kind of political considerations, conservatism, religious beliefs and things like that. This is a big lesson.

Yet, self-correction is not exclusive to science. Also, scientists could exploit their supposed impartial scientific authority to advance their personal political or religious agendas.

Finally, one may wonder how a true “scientific consensus” can emerge, given that the same data is often interpreted very differently. One might legitimately ask if the consensus is a real unity of interpretation among scientists, or if there might be other factors in play. Perhaps the experiences of people during the COVID-19 pandemic has exposed that claims of scientific consensus are somewhat disingenuous and that the “consensus” was more like “follow the leader.” People got to observe how guidance pronounced by one scientific expert (who is perceived by his or her colleagues as the best in the field) was continually parroted by others possessing expert qualifications. This repetition—or consensus—gave a sense of scientific authority for obeying the guidance. Unfortunately, people were soon informed that the guidance that they were once told to obey was the exact opposite of what they should now be doing. How can they know this? Because the scientific consensus said so.

References
1. Botvinik-Nezer, R. et al. Variability in the analysis of a single neuroimaging dataset by many teams. Published on nature.com May 20, 2020 accessed May 24, 2020.
2. Guliuzza, R. J. Schism in Evolutionary Theory Opens Creationist Opportunity. Creation Science Update. Posted on ICR.org May 18, 2017, accessed May 24, 2020.
3. Laland, K. N. 2017. Schism and Synthesis at the Royal Society. Trends in Ecology & Evolution. 32(5): 316–317
4. Anonymous. Podcast: Galileo and the science deniers, and physicists probe the mysterious pion (Time Marker 16:06). Nature. Posted on nature.com May 6, 2020 accessed May 24, 2020.

*Randy Guliuzza is ICR’s National Representative. He earned his Doctor of Medicine from the University of Minnesota, his Master of Public Health from Harvard University, and served in the U.S. Air Force as 28th Bomb Wing Flight Surgeon and Chief of Aerospace Medicine. Dr. Guliuzza is also a registered Professional Engineer.
The Latest
NEWS
''Surprisingly Recent'' Lunar Volcanism?
Tiny volcanic glass beads suggest “surprisingly recent” lava flows on the moon that are “difficult to reconcile with the accepted...

CREATION PODCAST
On the Origin of Racism | The Creation Podcast: Episode 83
Racism and its foul fruits have plagued humanity for thousands of years and in the past couple of centuries it seems to have only reared...

NEWS
Methuselah-Like Longevity in Pre-Flood Mammals
Genesis claims that people in the pre-Flood world routinely attained 900-year lifespans. The best-known example is Methuselah, who had the longest recorded...

NEWS
Was an Insect Ancestor Discovered?
There is nothing simple about an animal group called the euarthropods (phylum Euarthropoda), which includes insects, crustaceans, and extinct trilobites. Evolutionists...

NEWS
October 2024 ICR Wallpaper
"The people who walked in darkness Have seen a great light; Those who dwelt in the land of the shadow of death, Upon them a light...

NEWS
Collapsed Utah Arch Prompts Questions about Arch Formation
We lost a natural wonder to gravity and erosion on Thursday, August 8, 2024.1 Those who visited Double Arch, also called “Hole in the...

ACTS & FACTS
ICR 2024 Resource Catalog
At the Institute for Creation Research, our mission is not only to conduct research demonstrating how science confirms Scripture but also to share this...

CREATION.LIVE PODCAST
Beetle Blasts and Biomimetics | Creation.Live Podcast: Episode...
Though tiny, the bombardier beetle is a fascinating masterclass in design. Evolutionists claim that this explosive insect came about by chance,...

NEWS
Another Arch Collapse at a National Park
Erosion and other natural forces upon sedimentary formations such as exposed cliffs and arches belie the millions of years during which they allegedly...

CREATION PODCAST
Living in Light of Genesis | The Creation Podcast: Episode 82
The world tells us that the book of Genesis is, if not entirely, at least partially a myth. We are told that history, archaeology, and science...