Ice Stories Summative Evaluation Compilation Report

Knight Williams Research Communications (Knight Williams, Inc), an independent evaluation firm specializing in the development and evaluation of science education media, conducted the summative evaluation for Ice Stories. The evaluation focused on the extent to which the project achieved the goals described in the Exploratorium's grant to the National Science Foundation (NSF) Arctic Research and Education, Antarctic Coordination and Information program within the Division of Research on Learning in Formal and Informal Settings (DRL). The NSF DRL program provided funding for both the project and independent evaluation beginning in 2007, as described on the project award page http://www.nsf.gov/awardsearch/showAward.do?AwardNumber=0733048: Ice Stories proposed by the Exploratorium strives to create public awareness of the International Polar Year (IPY) and the multi-disciplinary range of IPY research, increase public understanding of the process of scientific research and stimulate an enhanced relationship between IPY research and public outreach activities Following from these goals, the evaluation focused on two primary audiences. The first audience was a professional audience of 29 scientists who received media production training to enhance their ability to produce dispatches about their research for posting on the Ice Stories website while they conducted fieldwork in Antarctica or the Arctic. The scientists' dispatches targeted the project's second audience, a public audience comprised of lay adults, educators, students, and others interested in learning about the International Polar Year (IPY), scientific research conducted at the poles, and the scientists pursuing this work. As the Ice Stories project served these two audiences in diverse ways over a 3-year project period, the evaluation team prioritized evaluating the appeal and effectiveness of the project's primary training and web-based deliverables as experienced by these respective audiences. The evaluation strategy used a mixed-methods approach aimed at generating both a well-rounded and in-depth look at the project's contributions over time, drawing on the following methods: i) observations of the scientist media training workshops, ii) paper surveys administered to the scientists at the conclusion of the training, iii) follow-up telephone interviews with the scientists following their field work, iv) online surveys to visitors of the Ice Stories website available through the homepage and email invitation, v) web metrics captured for the site over the three year project period, and vi) a two group posttest only randomized experiment to examine recruited visitors' learning from the site. The mixed methods approach resulted in 7 separate reports, the findings from which have been merged for use in this final project report for convenience.