Engagement | Josh Gutwill
"We use the term 'active engagement'. We want people to author the experience for themselves in some way, to self-direct the experience. That means doing things according to their own interests and desires, not just following a set of instructions that has been laid out by the museum."
2018 Interview Highlights:
How do you define "engagement"?
For us, engagement is really about the amount of time people are spending on a given learning activity. So for us the definition of engagement became spending time in an exhibit with attention focused on the learning materials provided.
How does your concept of engagement potentially differ from that of other people?
First of all, there’s a really big distinction between trying to assess engagement in a formal environment and doing it in an informal environment. Maybe the Exploratorium is unusual in this respect, but I don’t think so. From everything I know about informal environments, people there really vote with their feet, so just looking at whether people are spending time and focusing their attention on something in informals is a pretty strong indicator of engagement. In informals I think there is a pretty well-accepted definition of what engagement is—it has to do with time and attention.
We add the word “active,” so we say “active engagement.” We want people to author the experience for themselves in some way, to self-direct the experience. That means doing things according to their own interests and desires, not just following a set of instructions that has been laid out by the museum.
How are you measuring or assessing engagement in your work, and what are the tradeoffs, if any, in your approach?
We assess engagement of the learner, and we also assess aspects of the design itself. We measure holding time with fixed gaze or touching the exhibit, manipulating the exhibit in some way. That’s observational. Then we do audio and video as a method and we code the audio and video in lots of different ways, depending on the project. Sometimes we do self-reports, so we can just ask people. We’ve done think-aloud protocols with people, retrospective or video, and exit interviews. We’ve done surveys asking them how engaging something was or how interesting it was.
Regarding tradeoffs, it’s hard to validate self-reports. It’s hard to make sure that people are being accurate. It’s sometimes hard for coders to be sure of gaze. With holding time, you don’t always know what’s happening in learners’ minds, and we think it’s a good proxy for engagement, but for some people that might not be accurate. Finally, the video analysis work is just very, very time-intensive. It takes a really long time to code. We try to use special video coding software. We used to use Studiocode, but now we use DataView, for which you don’t have to transcribe the video—you can just code it directly on the video.
What led you to study engagement in your work?
At the Exploratorium, we have a lot of phenomenon-based interactive exhibits that offer rich learning opportunities, but we noticed that people weren’t spending a lot of time or going very deeply into the phenomena, at least from our perspective. So way back around 2002 or so, we started a project to develop or redesign exhibits that would really encourage people to go much more deeply into the phenomena. That project was called Active Prolonged Engagement (APE).
How did you use engagement or the APE idea in various projects?
In the Tinkering Studio, people spend a lot more time, an hour or an hour and a half, during which they’re engaged in some learning activity. We wanted to think about what learning really looked like in the Tinkering Studio, and we created a learning dimensions framework. One of the dimensions in that framework was engagement. We retained the time aspect of it, so it was a behavioral assessment, essentially the amount of time an activity would hold attention. We added the emotional dimension of engagement, the sense of being emotionally invested, and the way we looked for that was whether people were displaying emotions that looked like to us pride, joy, disappointment, or frustration. Another big one was that a lot of times people in an exhibit or in the Tinkering Studio will finish whatever they had been doing, and they’ll look up and come out of whatever state they were in, maybe a flow state, and look around. For us, it showed major engagement if they then went back into the activity and did something else. That would be a wonderful indicator of a high level of engagement.
How and why do you think engagement matters for science learning or science communication?
I think engagement is a foundational aspect of learning. Can you even have learning without engagement?