Responsive Virtual Human Museum Guides: Summative Evaluation

The University of Southern California's Institute for Creative Technologies (ICT) and the Museum of Science, Boston (MoS) were awarded an Informal Science Education grant from the National Science Foundation (#0813541) for the project, Responsive Virtual Human Museum Guide. The goal of the project was to use computer-generated character animation, artificial intelligence, and natural language processing to create interactive characters, or virtual humans, that could engage in face-to-face communication with museum visitors. During the three year project, the MOS and ICT project teams created three exhibits that included or highlighted virtual humans: The Twins, Science Behind, and Coach Mike. The Institute for Learning Innovation (ILI), a non-profit organization specializing in research and evaluation in informal learning settings, served as the independent, external summative evaluator for the Responsive Virtual Human Museum Guides project. The summative evaluation of the project was conducted from May to October 2011. For the summative evaluation, each exhibit was assigned two conditions, allowing for a quasi-experimental design. The evaluation used observations of interactions at the exhibits, interviews with visitors directly after their interaction, and follow-up questionnaires distributed online six week after the museum visit. The summative evaluation of the Responsive Virtual Humans Museum Guides project demonstrated mixed results in incorporating virtual humans into the museum environment. Generally, the Twins and Science Behind (taken together) were successful at achieving the impacts the team identified for the exhibits. The study of Coach Mike, on the other hand, demonstrated that the exhibit was only somewhat successful in achieving the identified impacts. The appendix of this report includes observation protocols, interview protocols, and surveys used in the study.