TFN Survey's Flawed Methodology Invalidates It | The Institute for Creation Research
TFN Survey's Flawed Methodology Invalidates It

In an effort to influence the Texas State Board of Education to ban the teaching of the weaknesses apparent in evolutionary theory, the Texas Freedom Network (TFN) Education Fund sponsored and released a survey in 2008 titled “Evolution, Creationism & Public Schools: Surveying What Texas Scientists Think about Educating Our Kids in the 21st Century.”1 ICR News previously published a review of some of the serious logical fallacies of this survey.2

Led by sociologist and evolution proponent Raymond A. Eve of the University of Texas at Arlington, the 59-question survey was sent to 1,019 biology and biological anthropology faculty members of public universities and large private institutions in Texas. There are four flaws in the methodology of the survey, any one of which would suffice to invalidate it: 1) Improper generalization, 2) lack of anonymity, 3) inadequate expertise, and 4) vague, meaningless questions.

Improper Generalization

TFN’s report repeatedly states that its data accurately represent “Texas scientists.” However, the opinions of university biology professors cannot be generalized to all Texas scientists. TFN errs by claiming that its survey represents the opinions of scientists in government, private industry, or non-biological fields.

Also, studies have shown that most American university faculty members are far more atheistic than their non-university counterparts and the public.3 Good survey construction incorporates features to control for known selection biases such as these, but the TFN made no such effort.

Lack of Anonymity

Standard survey procedures must ensure the complete anonymity of respondents, otherwise minority opinions can be underrepresented. The TFN effectively invalidated its survey by failing to use third party contractors, send out anonymous ID numbers, or employ other respondent-protective practices. In fact, on page 6 of the survey report, the authors describe how respondents could be “profiled” based on their education, teaching experience, etc. Fear of this profile being linked to their institution is exactly what discourages minority opinions and why good surveys are designed to prevent it.

The TFN is candid in its hostility towards views that dissent from Darwinism, yet this fact was not controlled for in the survey design. Without confidence that minority opinions—possibly a significant number—were not underreported, the survey findings and conclusions are not valid.

Inadequate Expertise

Over 78 percent of those surveyed selected the response that there was “no difference between ‘creationism’ and ‘intelligent design.’” Whereas the survey did contain responses to identify training or experience teaching evolution, the survey contained no verification that respondents had any training in intelligent design or creation science. There was no test showing that respondents could even accurately identify views unique to intelligent design, creation science, or views held by both groups. Such questioning elicits invalid caricatures, not legitimate conclusions.

Vague, Meaningless Questions

All five agree-disagree statements reported in the survey failed the standard of specificity. The statements’ lack of specificity force a non-scientific “mind-reader” approach as to what respondents mean by “yes” answers or in gauging their thoughts on social impact.

For example, one agree-disagree statement on the survey was rendered, “Teaching ‘Weaknesses’ of Evolution Impairs Ability to Compete for 21st Century Jobs.” But what jobs? And how many jobs? Does this include jobs as computer scientists, physicians, engineers, physicists, lawyers, or only evolutionary biologists? The statement’s lack of specificity makes it impossible to interpret what a “yes” response means. This question cannot justify TFN’s conclusion “that Emphasizing ‘Weaknesses’ of Evolution Would Substantially Harm Students’ College Readiness and Prospects for 21st Century Jobs.” This “finding” only bolsters the false claim that belief in evolution is vital for properly performing “jobs.”

Another statement read, “It is possible for someone who accepts evolutionary biology to have religious faith.” Faith in what? Faith in evolution? The Flying Spaghetti Monster? The Force? Both a nominal religious faith and a deeply held religious faith generate a “yes” response. In regard to significance, what is the difference between the 91 percent of evolutionary biologists who answered “yes” to this statement and intelligent design advocates, who also agree that evolution is compatible with religious faith, or creationists, who say evolution is a religious faith? Thus, the question yields no meaningful data.

The TFN presented its survey as a serious scientific instrument with legitimate data. But it would not survive any standard peer-review process for publication because its findings and conclusions are invalidated by at least four flawed methodologies. This survey would benefit from critical analysis in the same way that the theory of evolution would benefit from having its weaknesses discussed.

References

  1. Eve, R.A, and C. A. Belhadi. 2008. Evolution, Creationism & Public Education: Surveying What Texas Scientists Think about Educating Our Kids in the 21st Century. A Report from the Texas Freedom Network Education Fund, accessed online January 19, 2009.
  2. Thomas, B.Texas 'Freedom' Network, UTA Professor Oppose Academic Freedom in Public Schools. ICR News. Posted on icr.org November 26, 2008, accessed January 19, 2009.
  3. Tobin, G. A. and A. K. Weinberg. 2007. Profiles of the American University, Volume II: Religious Beliefs and Behavior of College Faculty. San Francisco, CA: Institute for Jewish & Community Research.

* Dr. Guliuzza is ICR’s National Representative.

Article posted on January 22, 2009.

The Latest
NEWS
"100-Million-Year-Old" Bacteria?
Researchers from the Japan Agency for Marine-Earth Science and Technology have recovered what they claim are 100-million-year-old microbes. Drilling 74.5...

NEWS
Origin of Tectonic Plates Best Explained by the Flood
A new study published in Nature Communications claims to have figured out how the tectonic plates may have originated.1 It’s been over...

NEWS
Give Today and Support Biblical Creation Ministry
North Texas Giving Day is here once again! Will you help us reach the next generation? Your generosity will empower ICR and our Dallas Museum—the...

NEWS
Mammalian Brains Prove Evolutionary Disconnect
The evolutionary model of brain development predicted that the complexity of neural connectivity should have increased as brains became larger and the...

NEWS
North Texas Giving Day Next Week!
North Texas Giving Day is almost here! Will you help us reach the next generation? Your generosity will empower ICR and the ICR Discovery Center for Science...

NEWS
Celebrating One Year at the ICR Discovery Center
Thank you to all who joined us for the ICR Discovery Center for Science & Earth History’s First Anniversary Celebration! ICR staff enjoyed meeting...

NEWS
New Evidence Hurts Feathered Dinosaur Theory
Is a dinosaur still a dinosaur if it has flight feathers? A new study points to some fascinating evidence that contradicts the idea of feathered dinosaurs...

NEWS
New Planetarium Show: Exploring Deep Space
The ICR Discovery Center for Science & Earth History is celebrating its first anniversary since the Grand Opening on September 2, 2019. To commemorate...

NEWS
Inside September 2020 Acts & Facts
What's the story behind ICR's new President? What milestone are we celebrating this month at the ICR Discovery Center? How do comets affirm...

NEWS
Sunflower Heliotropism: August Sunlight for Making Tons of Seeds
August is an important month for sunflowers—those gigantic, bright-yellow flowers with brownish, round seed heads bordered by radiating yellow ligules...