Evaluation of Design Squad, Season 3: Final Report

Design Squad, produced by WGBH-Boston (http://www.wgbh.org), premiered on the Public Broadcasting Service in October 2009. Design Squad is a reality television series that encourages kids ages 9-12 years to “show off their smarts as they design and build working solutions for real-world clients—people who are hungry for clever ideas from a new generation of innovators.” Each season, the series culminates in a final episode when the top two scorers compete for a $10,000 college scholarship from the Intel Foundation. The underlying educational goals of Design Squad are to: (1) Increase students' knowledge of engineering and the design process; (2) Improve the public image of engineering; and (3) Encourage further exploration in the fields of engineering, math, and science. Concord Evaluation Group performed an evaluation of the third season of Design Squad in 2009-2010. We found: (1) Design Squad, through its website, has succeeded in reaching out to a widespread and extensive audience, including traditionally underserved audiences such as non-white, ethnic minorities; girls; and low income audiences. (2) Design Squad, through its outreach partnerships, has succeeded in reaching out to a widespread and extensive audience, including traditionally underserved audiences such as non-white, ethnic minorities; girls; and low income audiences. (3) The Design Squad website and outreach partnerships are serving the needs of educators and students in both formal and informal educational settings. (4) Outreach partners and teachers perceive Design Squad as an effective means for getting kids excited about STEM and supplementing their existing science curricula. (5) The Design Squad resources had a statistically significant impact on student learning of key science concepts. (6) The Design Squad resources facilitated student development of skills that are essential to performing engineering challenges. (7) Design Squad helped students develop a deeper understanding of engineering. (8) Students’ attitudes about engineering and about engineering stereotypes improved significantly after using Design Squad. (9) The vast majority of students reported that they wanted to watch more Design Squad on TV or the Web. (10) Teachers used Design Squad to supplement their curricula and to support their classroom goals. (11) Design Squad enabled teachers to encourage problem solving and teamwork among students. (12) All teachers agreed that Design Squad made them feel comfortable leading design challenges and helped them to use a student-centered problem solving process with their students. (13) Teachers were enthusiastic about using Design Squad again. (14) Teachers were able to integrate the Design Squad videos with the design challenges to reinforce the existing curricula, but the video clips may need to be longer to be more effective. This report includes the survey protocols used in the study.