Frustration and Confusion are Important Learning Elements

“Artificial Intelligence has made it possible to develop intelligent tutors or intelligent educational systems capable of adapting to each student to personalize education.”

This is a popular article made in collaboration with Tecnológico de Monterrey’s Writing Lab.

Far from what we might think, negative emotions such as boredom, frustration, and confusion are essential elements of effective learning. The evidence does confirm it. The key is to know how to handle these emotions. Technological advances in Artificial Intelligence (AI) such as intelligent tutors and machine learning help us understand the role of negative emotions in learning.

The relevance of emotions in several crucial aspects of life has been reconsidered since Descartes’ Error‘s publication by the neuroscientist Damasio (1994). In the educational field, however, the lack of empirical evidence supporting the transcendence of emotions opens up a fascinating field due to the possibility of computationally modeling the cognitive state and students’ affective state. Modeling these states could be understood as mathematically representing the degree of learning or affectivity at a particular moment during learning. These models use AI to update in real-time, depending on their activities with the learning tool. In this example video starting at minute 5, a method for modeling the affective state in real-time is illustrated.

Photo: Prostock-Studio downloaded from https://www.istockphoto.com/

”Meta-affectivity describes the ability 1) to be aware of the emotional states experienced during learning and 2) to use emotional regulation strategies to convert negative states to positive ones for better learning.”

Artificial Intelligence has also made it possible to develop intelligent tutors or intelligent educational systems capable of adapting to each student to personalize education. Although the intelligent tutors’ success as mechanisms to support learning has been demonstrated over the years, there are questions regarding the role of emotions: How to model the affective states that influence learning? How to use the facial points that are described in this video with learning? How do emotions relate to learning? In this scenario, researchers such as Ryan Baker at the University of Pennsylvania and Sidney D’Mello at the University of Colorado, Boulder, have conducted scientific studies to determine the role of emotions, particularly negative emotions such as frustration or confusion (Sutter-Brandenberger et al. 2018). These studies’ results demonstrate the adverse effects of boredom and reveal that some emotions such as frustration are not as damaging as might be assumed (Baker et al. 2010; D’Mello and Graesser 2012; Andres et al. 2019).

Thanks to these advances, it has been established that far from being unnecessary or harmful, frustration and confusion are vital and determinant affective states for learning. One of the most recent studies (Spann et al., 2019) presents that learning can be predicted from students’ emotions. In their study, the authors experimented with students learning physics in a serious game environment (see Fig1). When students became frustrated or bored when resolving physics problems, they promptly used cognitive reappraisal as an emotional regulation technique.

Fig. 1 A problem in the serious game, "Physics Playground." — Fig. 1 A problem in the serious game, “Physics Playground.”

In its simplest form, cognitive reappraisal is the student’s ability to reconsider a situation from a different perspective to minimize negative emotions. In the game “Physics Playground,” students who gained meaningful learning and reported negative emotional states such as frustration and boredom used cognitive reappraisal to regulate these emotions. These students simply imagined that they were playing an easier video game to feel differently and, thus, learn.

A less-studied concept related to emotions during learning is meta-affectivity (Debellis and Goldin, 2006). The term was coined by making an analogy with the concept of meta-cognition, which is defined as the level of awareness that the student has about his own knowledge and the techniques that allow him to learn better. Meta-affectivity, therefore, describes the ability 1) to be aware of the emotional states experienced during learning and 2) to use emotional regulation strategies to convert negative states to positive ones for better learning. This hypothesis has been proposed, but there has been very little research to obtain empirical evidence that validates the theory. However, meta-affectivity could describe the students’ willingness to regulate their emotions themselves and learn. It could also explain why only some students in the “Physics Playground” study used cognitive reappraisal successfully.

“The results are fascinating. Meta-affective students (18.54% of the sample) had significantly higher learning than the non-meta-affective students (81.46% in our study).”

In a study conducted in Mexico to understand the role of meta-affectivity in learning mathematics, it was found that it is possible to characterize meta-affectivity and that, also, students who possess this characteristic learn better, although they are not the ones who have the most positive emotions.

Students were asked to interact with the intelligent tutor, Scooter (Walker et al., 2011). For 80 minutes, the students resolved the intelligent tutor’s problems and were interrupted every 10 minutes to report how they felt. Currently, at Tecnologico de Monterrey, a more efficient way of recognizing emotions is being implemented, which is to use machine learning to recognize the emotions automatically in the context of technology-mediated education (similar to the video). There is no need to interrupt the students in this scenario, and there is a more effective identification of emotions than when the students are self-evaluating these. In the study, the options provided to students included four emotions: boredom, frustration, confusion, and concentration, in addition to the neutral state. Data related to learning, affectivity, and meta-connectivity were analyzed using computer learning and statistical techniques with two objectives: 1) understanding whether meta-affectivity was observed in the data collected during the study, and 2) describing the affective dynamics observed and associating them with the learning achievement.

The results are fascinating. Meta-affective students (18.54% of the sample) had significantly higher learning than the non-meta-affective students (81.46% in our study). These students also achieved better scores on the standardized national exam in Mexico, the ENLACE test. It is noteworthy that less than a fifth of the sample students were meta-affective, suggesting that this skill should be taught.

The second objective was to analyze the affective dynamics of the students. In this sense, it was found that meta-affective students reported significantly more moments of frustration than non-meta-affective students. On the other hand, students who were not meta-affective reported higher states of concentration. This result catches the eye because it can be thought that non-meta-affective students learning math report being focused, but, in reality, they do not learn as well as the meta-affective students.

Finally, it was found that almost a quarter of students (23.43%) had affective behavior consisting of sequences of boredom followed by frustration. This behavior was associated with poor learning, and it might be because the students, aware of their boredom, decided not to do anything about it, leading them to feel frustrated. Because these students do not know how to regulate boredom or frustration, they get caught up in a negative cycle that leads them not to learn. An open question is whether these students could benefit from utilizing cognitive reappraisal.

These results are significant because they demonstrate that self-regulation of boredom and frustration leads to better learning as long as students are aware. The results also show that the lack of meta-affectivity causes negative behaviors among the students and suggests that using cognitive reappraisal strategies could be beneficial for this group. Finally, these results suggest that boredom and frustration are vital to learning, even for the most successful students.

The results of this meta-affectivity study will appear in an article in the Journal of Artificial Intelligence in Education. I invite the Edu Bits subscribers to read the article extensively. This field of research is fascinating, and much remains to investigate. Nowadays, due to COVID-19, most of our students learn online, so strategies for automatic recognition of emotions can be implemented to identify students who lack meta-affectivity. It would be possible to offer them a set of strategies that allow them to regulate negative emotions.

About the author

Genaro Rebolledo is a research professor on the Monterrey Campus of Tecnologico de Monterrey. He earned his Ph.D. from the University of Sussex and completed his postdoctoral stay at University College London (UCL) in the United Kingdom. He has done joint research work with Ryan Baker, focusing on Artificial Intelligence in Education. A more extended version of this article will be published in the International Journal of Artificial Intelligence in Education (Rebolledo-Mendez et al., 2021).

This article is adapted from the special publication in the International Journal of Artificial Intelligence in Education (2021). DOI: https://doi.org/10.1007/s40593-021-00247-1 Link to publication https://link.springer.com/article/10.1007/s40593-021-00247-1

References

Andres, J. M. A. L., Paquette, L., Ocumpaugh, J., Jiang, Y., Baker, R. S., Karumbaiah, S., et al. (2019). Affect sequences and learning in Betty’s brain. In ACM International Conference Proceeding Series (pp. 383–390). New York, NY, USA: Association for Computing Machinery. https://doi.org/10.
1145/3303772.3303807

Baker, R.S., Corbett, A.T., Koedinger, K.R. (2006) Responding to Problem Behaviors in Cognitive Tutors: Towards Educational Systems Which Support All Students. National Association for the Dually Diagnosed (NADD) Bulletin, 9 (4), 70-75.

Baker, R. S., D’Mello, S. K., Rodrigo, M. M. T., & Graesser, A. C. (2010). Better to be frustrated than bored: The incidence, persistence, and impact of learners’ cognitive-affective states during interactions with three different computer-based learning environments. International Journal of Human-Computer Studies, 68(4), 223–241. https://doi.org/10.1016/j.ijhcs.2009.12.003

D’Mello, S., & Graesser, A. (2012). Dynamics of affective states during complex learning. Learning and Instruction, 22(2), 145–157.

Debellis, V. A., & Goldin, G. A. (2006). Affect and meta-affect in mathematical problem solving: A representational perspective. Educational Studies in Mathematics, 63(2), 131–147. https://doi.org/10.1007/s10649-006-9026-4

Rebolledo-Mendez, G., Huerta-Pacheco, N. S., Baker, R. S., & Boulay, B. du. (2021). Meta-affective behavior within an intelligent tutoring system for mathematics. International Journal of Artificial Intelligence in Education, To appear

Spann, C. A., Shute, V. J., Rahimi, S., & D’Mello, S. K. (2019). The productive role of cognitive reappraisal in regulating affect during game-based learning. Computers in Human Behavior, 100, 358–369. https://doi.org/https://doi.org/10.1016/j.chb.2019.03.002

Sutter-Brandenberger, C. C., Hagenauer, G., & Hascher, T. (2018). Students’ self-determined motivation and negative emotions in mathematics in lower secondary education—Investigating reciprocal relations. Contemporary Educational Psychology, 55, 166–175. https://doi.org/https://doi.org/10.1016/j.cedpsych.2018.10.002

Walker, E., Ogan, A., Baker, R. S. J. D., De Carvalho, A., Laurentino, T., Rebolledo-Mendez, G., & Castro, M. J. (2011). Observations of collaboration in cognitive tutor use in Latin America. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (Vol. 6738 LNAI, pp. 575–577).

Edited by Rubí Román (rubi.roman@tec.mx) – Observatory of Educational Innovation.

Translation by Daniel Wetta.

About Writing Lab

The Writing Lab is a TecLabs initiative dedicated to the development of the culture of research in educational innovation and the improvement of the academic production of the teaching community at Tecnológico de Monterrey.

Financing and technical support are provided only for Tec de Monterrey’s professors to attend conferences or publish articles in journals.

For more information visit: https://writinglab-tec.com/

Facebook: http://facebook.com/WritingLabTEC

Twitter: @TecWriting

Instagram: instagram.com/writinglabtec/