Summative Evaluation of the Skyline Exhibition

Chicago Children's Museum (CCM) contracted with Randi Korn & Associates, Inc. (RK&A) to evaluate Skyline a National Science Foundation-funded exhibition designed to facilitate family learning of Science, Technology, Engineering, and Math (STEM) concepts relevant to building stable structures. RK&A conducted all three phases of evaluation for Skyline front-end, formative, and summative; select findings from the summative evaluation are presented here. Skyline's target audience is families with children between the ages of 5 and 10 years of age. RK&A conducted 100 observations of children in the target age range, 100 in-depth interviews 50 with children in the target age range and 50 with caregivers visiting with a child in the target age range, and analyzed 43 recordings of families who completed the exhibition's photo-narrative experience (a story-telling station that allowed structured visitor reflection). All data were collected in June and July 2008 in the Skyline exhibition. CCM developed audience impacts within NSF's Framework for Evaluating Impacts of Informal Science Education Projects. This framework requires museums to define impacts within six categories awareness, knowledge, or understanding, engagement or interest, attitude, behavior, skills, and other as well as indicators as evidence of these impacts. Notably, Skyline achieved nearly all of its intended impacts. Results from one impact (out of three) are presented here as an example.Impact: Families will gain a new understanding of how buildings stand up by increasing their understanding of relevant STEM concepts (awareness, knowledge, or understanding). CCM developed five indicators as evidence of achieving this impact and achieved all five save one small exception (see number three below):(1) The percent of children who engaged in building (71 percent) increased, and no statistically significant difference was found between male and female children whereas in the front-end evaluation, data showed that males were more likely to build than females;(2) The percent of observed children who successfully created a stable, free-standing structure (40 percent) increased compared to the front-end evaluation (11 percent); (3) The percent of children who attempted to build a structure increased (58 percent) compared to the front-end evaluation (36 percent); while CCM's original goal was 65 percent, these data still show a significant improvement over the front-end evaluation in children's attempts to and ultimate success in creating stable, free-standing structures.(4) Three-quarters of adult interviewees and slightly more than one-half of child interviewees used STEM-based concepts when discussing their building experience; and,(5) Slightly more than one-half of visitor groups (who gave permission for RK&A to analyze their photo-narrative experience) used STEM-based concepts when discussing their building experience.