March 4, 2021

Two sides of the same misconceived coin

"Science is a continuous process gradually improving our understanding of how the natural world works," writes NCSE Executive Director Ann Reid, countering the misconception that science is just a Big Box of Facts or a Big Cloud of Hot Air.

"Science is neither a big box of facts nor a big cloud of hot air."

Misconception of the Month emblemHow would you feel if you visited your favorite news website tomorrow morning and saw the headline: “Study finds surfing internet at least 8 hours per day adds years to life, eating nachos extends benefit”? C’mon, admit it. You’d definitely click. And then what? How would you decide whether the results of the study really lived up to that headline?

In an example that extreme, I think most of us would have our healthy skepticism goggles firmly in place. After all, the finding is in such opposition to everything you’ve heard about diet and exercise that, even if you really, really wanted it to be true, you probably wouldn’t get your hopes up. Indeed, what you’d probably find, upon reading the article, was that the results were based on interviews with the widows of three elderly men, one of whom lived three years longer than the others after having spent his golden years eating nachos on the couch. Not a very rigorous study Especially when you reach the end of the article and the author notes that the study was paid for by Big Chip.

But questionable science results in the news are not always so easy to spot. Indeed, it isn’t even possible to sort new scientific findings into just two bins: laudable and laughable, right or wrong, sheep or goats. If only it were that simple! Instead, there are any number of other categories we might propose including, but would hardly be limited to: preliminary, incomplete, promising, predictable, over-hyped, misleading, misrepresented, outright fraudulent, and — very occasionally — genuinely revolutionary.

And that brings us to one side of this month’s misconception coin: Science is a Big Box of Facts. Science isn’t static, or black and white; it is a continuous process gradually improving our understanding of how the natural world works. I sometimes like to say that science is a verb, not a noun. Science is not the accreted collection of human knowledge of the natural world, a big pile of pebbles representing true things that scientists add to throughout their careers. But it’s easy to see how this misconception takes root. I think many of us leave high school — and maybe even college — having wandered through some random collection of facts and equations that we had to memorize and regurgitate on command. You could be forgiven for coming away with the impression that 1) science is hella boring and 2) new scientific findings are either true or false, worthy of eventually making it into textbooks or not.

(Incidentally, one of the truly tragic consequences of that kind of science education is that it almost certainly drives away a lot of people who would be marvelous scientists if they realized how dynamic and creative science can be and instead retains people who either like or are at least willing to tolerate memorizing things. Not that the latter can’t be good scientists [she says, self-consciously] — but we’re potentially leaving a lot of talent on the table.)

It’s really important that students come to understand the provisional, contingent, and sometimes halting nature of science. As part of the second nature of science lesson in NCSE’s lesson plans, the one designed to address the misconception that science is a rote process of collecting facts, students work through either the history of germ theory or plate tectonics to learn how scientific understanding of complex phenomena changes as new evidence emerges.

This is an instance where students could really benefit from learning science as science is practiced — in this particular case, learning how scientists evaluate evidence to decide what’s credible and what isn’t.

But reader, there’s a catch. Yes, it’s crucially important to learn that science is open-minded, fluid, subject to self-correction, and never the final word, but only the best understanding we currently have. Because that’s all true! Unfortunately, that critical and powerful characteristic of science can be weaponized against science itself. If you believe that changes in scientific conclusions are always just one study away, why not conclude that sitting on the couch eating nachos is a good life strategy? Maybe that’s where the scientific research is headed! Similarly, if scientists are human like anyone else, subject to their own biases and cultural blind spots, prone to error and loath to admit it, is it wrong to discount findings that are inconvenient or distasteful? The scientist may be wrong, or deluded, or lying! And what about all that uncertainty? If scientists freely admit that they cannot say beyond the shadow of a doubt that we will never discover that sitting on couches noshing nachos is bad for you, what the heck are you supposed to think anyway? You’re telling me to trust science, but you’re also telling me it’s as precarious as a stack of eggs! Here it is, the other side of this month’s misconception coin: Science is a Big Cloud of Hot Air.

Clearly what we have here is a false dilemma. Science is neither a Big Box of Facts nor a Big Cloud of Hot Air: scientific knowledge aspires, and often manages, to be “simultaneously reliable and subject to change,” in the words of the National Science Teaching Association’s recent position statement on teaching the nature of science.

So what to do? This is an instance where students could really benefit from learning science as science is practiced — in this particular case, learning how scientists evaluate evidence to decide what’s credible and what isn’t. This is definitely a more important component of scientific literacy than any particular collection of facts. In fact, I’d be willing to trade understanding of the Krebs cycle and Boyle’s Law, with a side order of the parallelogram of forces, for having all students leave high school with a firm grasp of this skill and the wherewithal to ask questions such as: Who conducted the study? Do the authors have expertise in that field and a track record of research results that have stood the test of time? How well designed was the study? Did it include enough subjects and were they followed for an adequate amount of time? How well supported are the study’s conclusions? Are the authors suggesting that one thing caused another when it’s just as possible that the two things simply happened at the same time? Did the authors consider, and test for, other possible explanations for what they found? Do they dismiss all contradictory evidence as fraudulent? Who paid for the study? Does that present any conflict-of-interest concerns? How was this research reported — in a peer-reviewed journal, on a preprint server, on an interest group’s website, or as a YouTube video?

In the culmination of NCSE’s nature of science lessons (the fifth lesson is coming soon), students actually undertake this very process, and they do it by digging into the science of masks as defenses against infectious diseases (like, oh, I don’t know, COVID-19?). They go out and look for as much information about masks as they can find. They actually design and carry out their own experiments to evaluate how well masks work. Then they work through the process of evaluating the information they found. Which websites are credible? Which studies were well-designed? (Having been through the experience of designing their own experiments makes them considerably more discerning about study design.) What are scientists pretty sure of and what’s still uncertain? In the end, students have a much deeper understanding of what we know about masks and how we know it.

This is a critically important process. But I feel duty-bound to add one caveat. None of these criteria is determinative all by itself. After all, sometimes scientists make important discoveries well outside of their own area of expertise. Some studies published without peer review turn out to be seminal. Just because a study is paid for by private industry, its findings aren’t automatically wrong. And sometimes, once in a great while, a study that reaches a conclusion seemingly at odds with all current understanding initiates a scientific paradigm shift. But these are all warning signs that something is up, so it’s important to check for any of these red flags.

In his series Cosmos, Carl Sagan, the pre-eminent science communicator of the television age, popularized a phrase that perfectly captures this balancing act: “Extraordinary claims require extraordinary evidence.” Science is not a collection of facts. It is always provisional. Our understanding of the world around us will not be the same a year or a decade from now. That’s science’s great strength, even if it does mean that we all have to be careful consumers of scientific information.

Read all our Misconception of the Month articles.

NCSE Executive Director Ann Reid

Ann Reid is a former Executive Director of NCSE.

reid@ncse.ngo