Evaluating NCSE's curriculum study field test

NCSE’s curriculum study, launched in July 2021 to test the efficacy of new and revised curricula for nature of science, climate change, and evolution, is well under way. The two-year study includes both evaluation and research components. The evaluation component will help us answer questions about the effectiveness of the lessons such as: What changes in student knowledge and attitude do the lessons support? How easy is it for teachers to implement the lessons in their classrooms? To what degree do the curricula adequately address science education standards? The research component will examine how teachers’ participation in a professional community — which is an integral part of the curriculum study —  influences their classroom practices and other factors that support teacher implementation of new curriculum.

During the fall 2021 semester, teacher participants implemented the nature of science lessons and provided NCSE staff with feedback. In order to evaluate student understanding and attitudes toward the big ideas in the curriculum, teacher participants used surveys administered to students prior to starting and then again following the conclusion of the lessons. The Student Understanding of Science and Scientific Inquiry (SUSSI) survey, developed by Ling L. Liang and her colleagues, focuses on six essential principles emphasized in K–12 science education standards on the nature of science.

As adapted for our use, these are:

1. Observations and Inferences: Science is based on both observation and inferences guided by the current perspectives of scientists.
2. Tentativeness of Science: Scientific knowledge is both tenable and durable. We should have confidence in scientific knowledge while realizing that such knowledge may change with new evidence or reconceptualizing previous evidence.
3. Scientific Theories and Laws: Scientific theories and laws explain and describe scientific phenomena. They are subject to change in the light of new or reorganized evidence.
4. Social and Cultural Influences on Science: People from all cultures contribute to science. As a human endeavor, science is influenced by and reflects the values of the society and culture in which the science is conducted, interpreted, and accepted.
5. Imagination and Creativity in Scientific Investigations: Science is based on observations and inferences of the natural world that originate from human imagination and logical reasoning.
6. Process of Scientific Investigation: There is no single universal step-by-step scientific method that all scientists follow. Scientific knowledge is constructed in a variety of ways including observation and experimentation.

To assess their understanding of the nature of science, students are presented with 24 statements that represent a mix of informed views and naive interpretations of the nature of science as found in scientific literature. They rank each statement on a five- point Likert scale ranging from strongly agree to strongly disagree. Responses are then analyzed for each of the concepts. Results are classified as naive, transitional, or informed views.

The curriculum field testers have shared a summary of student results prior to using the nature of science lessons. Initial pre-survey results show consistency across schools involved in the NCSE curriculum study and with science education research on the public understanding of the nature of science. The results are, on average, as follows:

Observations and Inferences: Informed

Tentativeness of Science: Informed

Scientific Theories and Laws: Naive

Social and Cultural Influences on Science: Transitional

Imagination and Creativity in Science: Naive

Process of Scientific Investigation: Transitional

The curriculum field testers have shared the different ways in which they are already using the SUSSI results to influence their teaching. Ericca Thornhill, environmental science teacher at Southern Boone High School in Ashland, Missouri, is using the survey results to address areas of lower proficiency for her students and to be more careful with the language she uses when describing the process of science and scientific investigation. Jennifer Broo, Advanced Placement biology teacher, convinced her entire science department at Mariemont High School in Cincinnati, Ohio, to survey their students. They intend to use the schoolwide results to fuel data-driven conversations about student progress through their science courses so that all students have an informed understanding of science upon graduation. At the McAllie School in Chattanooga, Tennessee, biology teacher Michael Lowry is conducting his own curriculum evaluation. He administered SUSSI to all his students and will be teaching the nature of science lessons to half his classes; he’ll teach the other half with materials he used previously.

NCSE’s nature of science curriculum, developed by our teacher ambassadors along with NCSE staff, includes classroom activities to help students develop informed views of science while addressing misconceptions that lead to or reinforce naive views. For example, both Lesson Set 2: Science is a Never-Ending Process and Lesson Set 4: Science is about the Evidence include activities in which students examine how our understanding of germs and evolution have changed over time as the result of the development of new technologies, which in turn altered theories about these phenomena. Lesson Set 3: Science is an Inquiry-Based Process provides students with an understanding of the ways in which innovation drives our under- standing of how cells function, which then leads to new questions sparking new innovations to aid investigation.

We will continue to gather, analyze, and evaluate the data resulting from the implementation of our lessons. And we will continue to conduct research as a means to add to the body of knowledge about science education. Stay tuned for more news from the evaluation and research front over the course of the next year and a half.