Spyhounds Pilot: Evaluation Report

Concord Evaluation Group (CEG) conducted an evaluation of the Spyhounds pilot test in October-December 2011. The goal of the evaluation was to assess the online resources for appeal and interest, as well as to provide WGBH with data on how a full-scale year-long project could be structured. CEG recruited a national sample of 5-8 year old children to participate in the pilot test. We conducted a pre-test survey to measure science-related knowledge (kids only), attitudes and interest (parents and kids). We then invited families to use the online resources during the pilot test and surveyed the kids and parents at the end of the pilot test to assess changes in knowledge, attitudes, and interest as well to gather subjective data on their experiences with the Spyhounds resources that can be used as formative data for development of future resources. The final sample contained 115 children across 21 states. Kids ranged in age from 5-8, with an average age of 7. The sample was evenly split between boys (n = 57) and girls (n = 58). The racial and ethnic background of the families in our sample mirrored national proportions: White (n = 85, 73.9%), Hispanic (n = 16, 13.0%), Black (n = 15, 13.0%), Asian (n = 10, 8.7%), Native American or American Indian (n = 2, 1.8%). Most parents reported that their family income was average, but the sample did include a range of socioeconomic levels: Better than the average family (n = 25, 21.7%), Same as the average family (n = 78, 67.8%), Worse than the average family (n = 12, 10.4%). About half of the kids had watched FETCH! with Ruff Ruffman prior to the study. The survey used in this study is included in the appendix of the report.