Summative Evaluation of the Impact of Secrets of the Universe Giant Screen Film and Related Small Screen Videos on Middle School Students

With funding from the National Science Foundation, Secrets of the Universe (SOTU) is a multimedia project that has at its core a 40-minute 3D giant screen film directed by a collaboration of K2 Communications Inc., The Stephen Low Company, and University of California Davis Department of Physics. To support and extend the impact of the film for these audiences, the project features an educational website with a variety of resources, including online videos of science content and characters from the film.

The independent evaluation team from Knight Williams Inc. conducted a summative evaluation of the SOTU project in two phases. Phase 1 addressed the impact on 5th to 8th graders of viewing online seven minutes of small screen videos, focused either on science content or science characters, prior to viewing on the next day the 40-minute SOTU giant screen film. The evaluation assessed the perceived impact of the pre-film types on the film experience, the appeal of the film itself, and the impact of both media on student learning and interest regarding science and scientists, with additional analyses by gender and by younger and older grade levels. Through conversation groups with middle school girls, Phase 2 further explored girls’ experience with the science characters in the pre-film videos and film, focusing on the impact of the science characters on the girls’ thoughts of scientists and their opinions of the effectiveness of the science characters as role models in encouraging their interest in science.

 This report presents the evaluation findings and concludes with recommendations relating to the use of the SOTU videos and supplemental materials that the project team may want to  address while the film is still showing in theaters.