Engagement | Christian Schunn
"We consistently find that affective, cognitive, and behavioral engagement don’t travel together. That is, some experiences work really well on one of those dimensions but very relatively poorly on the others, and those have long-term consequences."
2018 Interview Highlights:
What is your working definition of “engagement” and how does your concept of engagement differ from that of others?
Chris: The word “engagement,” like many theoretical words, has a thousand definitions. We focus on what is happening in the moment a learner participant is experiencing a particular exhibit or classroom situation. It has to do with the characteristics of the experience. I think a number of others think about it in a similar way, and it can be described as the ABC model of engagement. The Affective engagement elements: Are you enjoying this moment or feeling bored in the moment? The Behavioral component: Are you actually doing the activity or experience, or are you off task? Then the Cognitive elements: You might have your hands on, but your mind off, so are you really thinking about making connections, cognitively participating? That’s a consensus view among people who are studying engagement that it’s the thing that’s happening in the moment.
How do you measure or assess engagement in your work?
Chris: We have explored a number of ways of doing it, and short surveys is the way we do it most regularly. It’s efficient in terms of providing timely feedback. You can do it with large sets of learners. You can get really into the “for whom,” who’s actually benefiting, who is really engaging, who is not. And it doesn’t consume a lot of time, which is important because kids doing informal learning really don’t want to take a lot of time away from that learning. Short surveys seem to be the right compromise. They often give you insights. As you’re watching a class, you might have some beliefs about what’s happening inside the learners’ minds, but often you’re surprised and find out that someone who looks like they’re thoughtfully engaged is actually totally off task. And somebody who looks like they’re in la-la land is actually just deeply thinking about the experience, and without asking you have no way of knowing.
And what are some tradeoffs to doing a survey versus some other kind of measure?
Chris: Our approach takes a bit of learner time, which is a cost, but it’s directly focused on the things we’re trying to learn about. There were a lot of questions that we thought in advance, “Boy, these are awesome questions,” and they turned out to be horrible. So it isn’t easy to come up with a survey of engagement that really works.
What advice would you give practitioners who want to integrate your findings about engagement into their work?
Chris: We consistently find that affective, cognitive, and behavioral engagement don’t travel together. That is, some experiences work really well on one of those dimensions but very relatively poorly on the others, and those have long-term consequences. If you’re not measuring each of the elements, you might not even notice that your designed experience isn’t working for some other aspect, or at least not for a lot of learners. You’re thinking, “Boy, they look so engaged, and they’re totally doing the activity—”
How do you measure the three different aspects of engagement?
Chris: The survey has separate questions that really try to focus on one aspect separately from the others. We also do a lot of careful measurement studies to make sure that we include all the aspects in the survey and through the interviews with participants, make sure they’re interpreting the questions about each aspect individually and not just overall. That actually is one of the ways in which you can come up with bad questions, questions that you think are about one thing and they interpret it in another way. That’s especially the case for questions related to boredom being about affect versus being about cognition.
What are the big questions in informal science education, science communication, or even formal education for the next five or 10 years regarding engagement?
Chris: Well, I think there’s a “designing for subgroups” question we need to explore—how to help people navigate through things that are likely to be good experiences for them. We also want to take on personalized learning in informal contexts that use engagement as a way of framing what personalized learning should look like. But there’s a challenge there, which is supporting multiple aspects of engagement, not just the affective side. Sometimes free-choice learning leans heavily toward the affective side of things, and to have all the long-term outcomes that we want, I think we want to support choices toward experiences that are deeply engaging across the board. To do that, we need to figure out how to predict who will engage in what way and how to direct them in ways that they feel like they got a good deal, in both the short and long term.