NOVA “Making Stuff Season 2″” Summative Evaluation Report

Concord Evaluation Group (CEG), led by Dr. Christine Paulsen, was hired by WGBH to perform an evaluation of NOVA’s Making Stuff Season 2 series (MS2), website, and outreach activities. CEG also conducted a separate set of formative evaluation activities during the early stages of resource development. The findings from the formative evaluation have been delivered separately. The goal of the summative evaluation was to explore the extent to which MS2 activities were successful at achieving NOVA’s intended impacts: 1. To increase public understanding that basic research leads to technological innovation; 2. To increase and sustain public awareness and excitement about innovation and its impact on society; and 3. To establish a community of practice that enhances the frequency and quality of collaboration among STEM researchers and informal educators. To empirically assess whether MS2 achieved these impacts, the following four studies were undertaken by Dr. Paulsen at CEG: • Study 1 – TV Viewing Experiment: This study explored the impact of the MS2 television series on a national audience compared to a control group sample of non-viewers. • Study 2 – Website User Experiment: This study explored the impact of the MS2 website and its web-based resources on a national audience compared to a control group sample of non-users. • Study 3 – Longitudinal Study: This study explored changes over time in individuals who were exposed to MS2 TV and/or web-based resources. • Study 4 – Community of Practice Evaluation: This study included interviews with a sample of MS2 outreach community members,1 surveys of all MS2 outreach community members, and an exploration of the online relationships of MS2 outreach community members to one another, in particular, whether the Community of Practice networks grew or changed over time. This report details methodologies, empirical findings, and assesses the evidence for MS2 impacts.