Summative Evaluation of FETCH Outreach – Season II

FETCH, a children's television series produced by WGBH since 2005, is a competition-based game/reality show for 6-10 year-old children that includes both animation and live action footage. Goodman Research Group, Inc. (GRG), a research firm specializing in the evaluation of educational programs, materials, and services, served as the external evaluator for the FETCH series in Season Two. GRG's evaluation focused on the science and engineering challenges presented in the show, and assessed the influence of the series on children's understanding of science and engineering concepts and processes. GRG used survey and interview methods to gather pre- and post-viewing data from students; 168 fourth grade students in Massachusetts (n=97) and southern California (n=74) answered science and engineering process and content survey questions before and after watching five FETCH episodes. A five-minute interview was also conducted individually with a sub-sample of 117 students (51 in MA and 66 in CA) to further explore their understanding of the science and engineering concepts and processes featured in FETCH. KEY FINDINGS The FETCH series continues to have high appeal for the target audience in its second season. The majority of students (ranging from 67% to 91%) either "completely loved" or "liked a lot" each episode that they watched. The most common response about what they liked best about the show was that they liked learning science in a fun way. Students showed overall gains in their knowledge about science concepts and processes after watching FETCH. Overall, the students showed a significant increase in their scores on the science questions from the pre- to the post-tests. This difference persisted even when the sample was separated out by gender, age, FETCH viewing habits, and state of residence. FETCH was effective at increasing children's understanding of science concepts. The students scored significantly higher on all eight science content questions on the post-test than they did on the pre-test. Also, when asked explicitly to list the science ideas that they learned from FETCH, students mentioned, on average, at least one idea related to the questions on the survey. FETCH was effective at increasing children's application of science concepts. The students showed significant increase in the scores from pre- to post- test on the science application questions, as well as on the explanations they gave for their answers. These open-ended explanations clearly indicated an increase in students' application of science concepts related to characteristics of mammals and gravity and friction. FETCH was moderately effective at increasing children's understanding and application of science processes. Questions were asked about four science and engineering processes: (1) designing an experiment, (2) controlling variables, (3) brainstorming ideas, and (4) testing and redesigning products. Students improved significantly from pre- to post-tests in their learning about the design of an experiment; they were more likely on the post- tests to describe scientific steps such as making observations, using graphs to measure growth, and using control variables in the experiment. For the other three concepts, there was an upward trend in pre- to post-scores, although the differences were not statistically significant. Students more readily discussed FETCH science concepts than they did FETCH science processes. While students were articulate about the science ideas and concepts they learned from FETCH, they had more difficulty discussing science processes. RECOMMENDATIONS Focus on science processes during FETCH's next season. GRG recommends that WGBH continue to focus on science processes as one of the primary themes of the show in the upcoming year. This would include highlighting as explicitly as possible the science processes in each episode, through the use of graphics, diagrams and other visual aids. Make specific alterations to the show's format, while keeping the basic model intact. Because students enjoy the format and appear to benefit from the fun way in which science information is presented, GRG recommends that WGBH maintain the basic format of the show. However, the show can be further enhanced by adding a separate section at the end of each episode that reinforces both the science processes (and science concepts) that are highlighted in that episode. Introduce new science processes to the viewers while also helping them generalize these ideas. GRG recommends that the content and characters on the show emphasize and demonstrate how science processes can be generalized. For example, if the episode is dealing with the concept of experimentation and various strategies to enhance an experiment, the cast members could be shown engaging in multiple experiments. They could then be shown drawing parallels among the different experiments, which would reinforce the notion of generalizing the concept. This report includes pre- and post- surveys, interview questions, and coding scheme for interviews.