Early prediction of visitor engagement in science museums with multimodal learning analytics

October 22nd, 2020 | RESEARCH

Modeling visitor engagement is a key challenge in informal learning environments, such as museums and science centers. Devising predictive models of visitor engagement that accurately forecast salient features of visitor behavior, such as dwell time, holds significant potential for enabling adaptive learning environments and visitor analytics for museums and science centers. In this paper, we introduce a multimodal early prediction approach to modeling visitor engagement with interactive science museum exhibits. We utilize multimodal sensor data including eye gaze, facial expression, posture, and interaction log data captured during visitor interactions with an interactive museum exhibit for environmental science education, to induce predictive models of visitor dwell time. We investigate machine learning techniques (random forest, support vector machine, Lasso regression, gradient boosting trees, and multi-layer perceptron) to induce multimodal predictive models of visitor engagement with data from 85 museum visitors. Results from a series of ablation experiments suggest that incorporating additional modalities into predictive models of visitor engagement improves model accuracy. In addition, the models show improved predictive performance over time, demonstrating that increasingly accurate predictions of visitor dwell time can be achieved as more evidence becomes available from visitor interactions with interactive science museum exhibits. These findings highlight the efficacy of multimodal data for modeling museum exhibit visitor engagement.

Document

icmi1322-emersonA.pdf

Team Members

Andrew Emerson, Author, North Carolina State University
Nathan Henderson, Author, North Carolina State University
Jonathan Rowe, Co-Principal Investigator, North Carolina State University
Wookhee Min, Author, North Carolina State University
Seung Lee, Author, North Carolina State University
James Minogue, Co-Principal Investigator, North Carolina State University
James Lester, Principal Investigator, North Carolina State University

Citation

Identifier Type: doi
Identifier: 10.1145/3382507.3418890

Publication: Proceedings of the 2020 International Conference on Multimodal Interaction
Page(s): 107-116

Funders

Funding Source: NSF
Funding Program: Advancing Informal STEM Learning (AISL)
Award Number: DRL-1713545
Funding Amount: $1,951,956.00

Related URLs

Multimodal Visitor Analytics: Investigating Naturalistic Engagement with Interactive Tabletop Science Exhibits

Tags

Audience: Learning Researchers | Museum | ISE Professionals
Discipline: Climate | Ecology | forestry | agriculture
Resource Type: Conference Proceedings | Research
Environment Type: Games | Simulations | Interactives | Media and Technology | Museum and Science Center Exhibits

Linkedin   Youtube   Facebook   Instagram
Search: repository | repository and website pages | website pages
NSF logo

This material is supported by National Science Foundation award DRL-2229061, with previous support under DRL-1612739, DRL-1842633, DRL-1212803, and DRL-0638981. Any opinions, findings, conclusions, or recommendations contained within InformalScience.org are those of the authors and do not necessarily reflect the views of NSF.

NSF AISL Project Meetings

Privacy Policy | Terms of Use | Contact Us