TFN Survey's Flawed Methodology Invalidates It | The Institute for Creation Research

TFN Survey's Flawed Methodology Invalidates It

In an effort to influence the Texas State Board of Education to ban the teaching of the weaknesses apparent in evolutionary theory, the Texas Freedom Network (TFN) Education Fund sponsored and released a survey in 2008 titled “Evolution, Creationism & Public Schools: Surveying What Texas Scientists Think about Educating Our Kids in the 21st Century.”1 ICR News previously published a review of some of the serious logical fallacies of this survey.2

Led by sociologist and evolution proponent Raymond A. Eve of the University of Texas at Arlington, the 59-question survey was sent to 1,019 biology and biological anthropology faculty members of public universities and large private institutions in Texas. There are four flaws in the methodology of the survey, any one of which would suffice to invalidate it: 1) Improper generalization, 2) lack of anonymity, 3) inadequate expertise, and 4) vague, meaningless questions.

Improper Generalization

TFN’s report repeatedly states that its data accurately represent “Texas scientists.” However, the opinions of university biology professors cannot be generalized to all Texas scientists. TFN errs by claiming that its survey represents the opinions of scientists in government, private industry, or non-biological fields.

Also, studies have shown that most American university faculty members are far more atheistic than their non-university counterparts and the public.3 Good survey construction incorporates features to control for known selection biases such as these, but the TFN made no such effort.

Lack of Anonymity

Standard survey procedures must ensure the complete anonymity of respondents, otherwise minority opinions can be underrepresented. The TFN effectively invalidated its survey by failing to use third party contractors, send out anonymous ID numbers, or employ other respondent-protective practices. In fact, on page 6 of the survey report, the authors describe how respondents could be “profiled” based on their education, teaching experience, etc. Fear of this profile being linked to their institution is exactly what discourages minority opinions and why good surveys are designed to prevent it.

The TFN is candid in its hostility towards views that dissent from Darwinism, yet this fact was not controlled for in the survey design. Without confidence that minority opinions—possibly a significant number—were not underreported, the survey findings and conclusions are not valid.

Inadequate Expertise

Over 78 percent of those surveyed selected the response that there was “no difference between ‘creationism’ and ‘intelligent design.’” Whereas the survey did contain responses to identify training or experience teaching evolution, the survey contained no verification that respondents had any training in intelligent design or creation science. There was no test showing that respondents could even accurately identify views unique to intelligent design, creation science, or views held by both groups. Such questioning elicits invalid caricatures, not legitimate conclusions.

Vague, Meaningless Questions

All five agree-disagree statements reported in the survey failed the standard of specificity. The statements’ lack of specificity force a non-scientific “mind-reader” approach as to what respondents mean by “yes” answers or in gauging their thoughts on social impact.

For example, one agree-disagree statement on the survey was rendered, “Teaching ‘Weaknesses’ of Evolution Impairs Ability to Compete for 21st Century Jobs.” But what jobs? And how many jobs? Does this include jobs as computer scientists, physicians, engineers, physicists, lawyers, or only evolutionary biologists? The statement’s lack of specificity makes it impossible to interpret what a “yes” response means. This question cannot justify TFN’s conclusion “that Emphasizing ‘Weaknesses’ of Evolution Would Substantially Harm Students’ College Readiness and Prospects for 21st Century Jobs.” This “finding” only bolsters the false claim that belief in evolution is vital for properly performing “jobs.”

Another statement read, “It is possible for someone who accepts evolutionary biology to have religious faith.” Faith in what? Faith in evolution? The Flying Spaghetti Monster? The Force? Both a nominal religious faith and a deeply held religious faith generate a “yes” response. In regard to significance, what is the difference between the 91 percent of evolutionary biologists who answered “yes” to this statement and intelligent design advocates, who also agree that evolution is compatible with religious faith, or creationists, who say evolution is a religious faith? Thus, the question yields no meaningful data.

The TFN presented its survey as a serious scientific instrument with legitimate data. But it would not survive any standard peer-review process for publication because its findings and conclusions are invalidated by at least four flawed methodologies. This survey would benefit from critical analysis in the same way that the theory of evolution would benefit from having its weaknesses discussed.

References

  1. Eve, R.A, and C. A. Belhadi. 2008. Evolution, Creationism & Public Education: Surveying What Texas Scientists Think about Educating Our Kids in the 21st Century. A Report from the Texas Freedom Network Education Fund, accessed online January 19, 2009.
  2. Thomas, B.Texas 'Freedom' Network, UTA Professor Oppose Academic Freedom in Public Schools. ICR News. Posted on icr.org November 26, 2008, accessed January 19, 2009.
  3. Tobin, G. A. and A. K. Weinberg. 2007. Profiles of the American University, Volume II: Religious Beliefs and Behavior of College Faculty. San Francisco, CA: Institute for Jewish & Community Research.

* Dr. Guliuzza is ICR’s National Representative.

Article posted on January 22, 2009.

The Latest
NEWS
Creation's Easter Message
While many Christians still consider the creation doctrine a fringe issue, a proper understanding of the Christian message finds creation at its core...

NEWS
ICR Veteran Don Barber Retires
Don Barber   After 34 years with the Institute for Creation Research, Director of Enterprise Technology Don Barber will retire...

CREATION.LIVE PODCAST
The Sanctity of Life | Creation.Live Podcast: Episode 23
Abortion is a big issue culturally and in the church. How can believers love our neighbors and act as the hands and feet of Christ when it comes...

NEWS
Plant Receptors Are Designed to Control Immunity and Development
God has designed plants to continuously track their environment.1 They do so with specially designed detectors (also called receptors) on...

CREATION PODCAST
Homo Naledi: The Lies Behind Evolution's Rising Star | The Creation...
Homo naledi, once evolution's 'rising star,' was considered to be a prime example of human evolution. But is it really proof? Or...

NEWS
A Subsurface Ocean on Mimas?
Scientists have analyzed data obtained from the Cassini spacecraft and concluded that irregularities in the orbit of Saturn’s moon Mimas indicate...

NEWS
In Theaters March 20 & 21: The Ark and the Darkness
Is Genesis true? What about Noah’s Flood? How did Noah fit the animals on the ark? Wasn’t it a local flood? Have you asked these questions? Though...

NEWS
Enigmatic Fossil Plants
The pre-Flood world thousands of years ago was unlike the world of today.1 Unfamiliar animals and plants were common, and there were a lot...

NEWS
Surprisingly Colorful Fossil Snail Shells
Finding organic compounds such as flexible dinosaur collagen and complete bone cells1,2 is becoming common, much to the shock and consternation...

NEWS
March 2024 ICR Wallpaper
"He is not here; for He is risen, as He said." (Matthew 28:6 NKJV) ICR March 2024 wallpaper is now available for mobile, tablet,...