Responsive Virtual Human Museum Guides: Summative Evaluation

April 1st, 2012 | EVALUATION

The University of Southern California's Institute for Creative Technologies (ICT) and the Museum of Science, Boston (MoS) were awarded an Informal Science Education grant from the National Science Foundation (#0813541) for the project, Responsive Virtual Human Museum Guide. The goal of the project was to use computer-generated character animation, artificial intelligence, and natural language processing to create interactive characters, or virtual humans, that could engage in face-to-face communication with museum visitors. During the three year project, the MOS and ICT project teams created three exhibits that included or highlighted virtual humans: The Twins, Science Behind, and Coach Mike. The Institute for Learning Innovation (ILI), a non-profit organization specializing in research and evaluation in informal learning settings, served as the independent, external summative evaluator for the Responsive Virtual Human Museum Guides project. The summative evaluation of the project was conducted from May to October 2011. For the summative evaluation, each exhibit was assigned two conditions, allowing for a quasi-experimental design. The evaluation used observations of interactions at the exhibits, interviews with visitors directly after their interaction, and follow-up questionnaires distributed online six week after the museum visit. The summative evaluation of the Responsive Virtual Humans Museum Guides project demonstrated mixed results in incorporating virtual humans into the museum environment. Generally, the Twins and Science Behind (taken together) were successful at achieving the impacts the team identified for the exhibits. The study of Coach Mike, on the other hand, demonstrated that the exhibit was only somewhat successful in achieving the identified impacts. The appendix of this report includes observation protocols, interview protocols, and surveys used in the study.



Team Members

Susan Foutz, Evaluator, Institute for Learning Innovation
Jeanine Ancelet, Evaluator, Institute for Learning Innovation
Kara Hershorin, Evaluator, Institute for Learning Innovation
Liz Danter, Evaluator, Institute for Learning Innovation
University of Southern California, Contributor
Museum of Science, Contributor


Funding Source: NSF
Funding Program: AISL
Award Number: 0813541
Funding Amount: 2062116

Related URLs

Responsive Virtual Human Museum Guides


Audience: Evaluators | General Public | Museum | ISE Professionals
Discipline: Computing and information science | Education and learning science | Technology
Resource Type: Evaluation Reports | Interview Protocol | Observation Protocol | Research and Evaluation Instruments | Summative | Survey
Environment Type: Exhibitions | Games | Simulations | Interactives | Media and Technology | Museum and Science Center Exhibits