Augmented Reality: Accessible Dynamic Informal STEM Learning for Deaf and Hard-of-Hearing

This project explores augmented reality (AR) technology as a way to make STEM content accessible to Deaf and Hard-of-Hearing (DHH) learners in "live" presentation settings found within science centers and museums. The current ways of providing communication and information access to DHH museum visitors require DHH learners to split their attention between the visual focus of the experience and the interpreter or captioning display. As a result, DHH learners are forced to decide which information to miss during live presentations, thus presentations may not be fully comprehended by the DHH learner. Furthermore, this issue perpetuates a lack of inclusivity and accessibility in informal STEM learning environments. The work focuses on investigating approaches to develop an AR platform with a near-eye display that will allow DHH visitors to receive signed or captioned instruction while still looking at and interacting with the intended exhibits. The new platform will allow for transmission of live and spontaneous instruction. Researchers will evaluate and recommend efficient ways to make information at science centers and museums accessible to the DHH population based on data collected from DHH learners. This project is funded by the Advancing Informal STEM Learning program's Pilot and Feasibility Studies which seek new approaches to design and development of STEM learning to be accessible to all learners including underrepresented groups in informal environments.

The researchers will investigate the following research question: how can the use of Augmented Reality (AR) as a method of support service delivery improve the engagement of Deaf and Hard-of-Hearing (DHH) adolescent learners in semi-structured learning environments? Both formative and summative evaluations will be conducted from two different angles: (1) an engineering-centric evaluation focused on technical development and (2) an educational-research centric evaluation focused on pilot studies on user experiences, attitudes, and learning outcomes. Both qualitative and quantitative data on the use of this technology will be collected. The evaluation methods include attitudinal surveys (pre/post), ethnographic observations, pre/post tests of content knowledge, and semi-structured group and individual interviews. This project will be conducted by researchers in the Center on Access Technology at National Technical Institute for the Deaf, one of the colleges of Rochester Institute of Technology, in partnership with the Rochester Museum & Science Center, which includes the Rochester Challenger Learning Center. This project represents a first step in setting future directions for research & development and to make educational materials more accessible to the DHH community.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.