Research Article

The effect of an agent tutor’s integration of cognitive and emotional gestures on cognitive load, motivation, and achievement

Soonri Choi 1 2 * , Soomin Kang 2 , Kyungmin Lee 1 2 , Hongjoo Ju 1 , Jihoon Song 1 *
More Detail
1 Hanyang University, Seoul, SOUTH KOREA2 Kwangwoon University, Seoul, SOUTH KOREA* Corresponding Author
Contemporary Educational Technology, 16(1), January 2024, ep491, https://doi.org/10.30935/cedtech/14101
Published: 05 January 2024
OPEN ACCESS   1237 Views   883 Downloads
Download Full Text (PDF)

ABSTRACT

This study proposes that the gestures of an agent tutor in a multimedia learning environment can generate positive and negative emotions in learners and influence their cognitive processes. To achieve this, we developed and integrated positive and negative agent tutor gestures in a multimedia learning environment directed by cognitive gestures. The effects of emotion type on cognition were examined in terms of cognitive load, learning motivation, and achievement. The subjects were 46 university students in Gyeonggi Province, South Korea. The students were divided into three learner groups: cognition, cognition + negative emotion, and cognition + positive emotion. The learners watched a tutorial lecture on the Notion note-taking app by an agent tutor. Data analysis was conducted using one-way ANOVA to determine the cognitive load, learning motivation, and achievement. The results showed that the positive emotion design was more effective in terms of intrinsic cognitive load, learning motivation, and achievement but had a higher extrinsic cognitive load. However, even the negative + passive group showed more positive learning than the cognition group. Although this study focused on gestures by an agent tutor, it implies that such gestures in multimedia learning contexts must be informed by emotional as well as cognitive design to provide a more meaningful learning experience.
Keywords: agent tutor, gesture effect, cognitive theory model, cognitive-affective model

CITATION (APA)

Choi, S., Kang, S., Lee, K., Ju, H., & Song, J. (2024). The effect of an agent tutor’s integration of cognitive and emotional gestures on cognitive load, motivation, and achievement. Contemporary Educational Technology, 16(1), ep491. https://doi.org/10.30935/cedtech/14101

REFERENCES

  1. Atkinson, R. K. (2002). Optimizing learning from examples using animated pedagogical agents. Journal of Educational Psychology, 94(2), 416-427. https://doi.org/10.1037/0022-0663.94.2.416
  2. Ba, S., Stein, D., Liu, Q., Long, T., Xie, K., & Wu, L. (2021). Examining the effects of a pedagogical agent with dual-channel emotional cues on learner emotions, cognitive load, and knowledge transfer performance. Journal of Educational Computing Research, 59(6), 1114-1134. https://doi.org/10.1177/0735633121992421
  3. Baylor, A. L., & Kim, S. (2009). Designing nonverbal communication for pedagogical agents: When less is more. Computers in Human Behavior, 25(2), 450-457. https://doi.org/10.1016/j.chb.2008.10.008
  4. Baylor, A. L., & Kim, Y. (2005). Simulating instructional roles through pedagogical agents. International Journal of Artificial Intelligence in Education, 15(2), 95-115. https://doi.org/10.1007/s40593-015-0055-y
  5. Bless, H., Clore, G., Schwarz, N., Golisano, V., Rabe, C., & Wolk, M. (1996). Mood and the use of scripts: Does a happy mood really lead to mindlessness? Journal of Personality and Social Psychology, 71(4), 665-679. https://doi.org/10.1037/0022-3514.71.4.665
  6. Chen, O., Castro-Alonso, J. C., Paas, F., & Sweller, J. (2018). Extending cognitive load theory to incorporate working memory resource depletion: Evidence from the spacing effect. Educational Psychology Review, 30(2), 483-501. https://doi.org/10.1007/s10648-017-9426-2
  7. Chen, Z. H. (2012). We care about you: Incorporating pet characteristics with educational agents through reciprocal caring approach. Computers & Education, 59(4), 1081-1088. https://doi.org/10.1016/j.compedu.2012.05.015
  8. Cowan, N. (2014). Working memory underpins cognitive development, learning, and education. Educational Psychology Review, 26(2), 197-223. https://doi.org/10.1007/s10648-013-9246-y
  9. Dinçer, S., & Doğanay, A. (2017). The effects of multiple-pedagogical agents on learners’ academic success, motivation, and cognitive load. Computers & Education, 111, 74-100. https://doi.org/10.1016/j.compedu.2017.04.005
  10. Fredrickson, B. L. (2001). The role of positive emotions in positive psychology: The broaden-and-build theory of positive emotions. American Psychologist, 56(3), 218-226. https://doi.org/10.1037/0003-066X.56.3.218
  11. Fredrickson, B. L. (2003). The value of positive emotions: The emerging science of positive psychology is coming to understand why it’s good to feel good. American Scientist, 91(4), 330-335. https://doi.org/10.1511/2003.26.330
  12. Heidig, S., & Clarebout, G. (2011). Do pedagogical agents make a difference to student motivation and learning? Educational Research Review, 6(1), 27-54. https://doi.org/10.1016/j.edurev.2010.07.004
  13. Heller, B., & Procter, M. (2010). Animated pedagogical agents and immersive worlds: Two worlds colliding. In G. Veletsianos (Ed.), Emerging technologies in distance education (pp. 301-316). Edmonton.
  14. Horovitz, T., & Mayer, R. E. (2021). Learning with human and virtual instructors who display happy or bored emotions in video lectures. Computers in Human Behavior, 119, 106724. https://doi.org/10.1016/j.chb.2021.106724
  15. Kim, Y., & Baylor, A. L. (2016). based design of pedagogical agent roles: A review, progress, & recommendations. International Journal of Artificial Intelligence in Education, 26(1), 160-169. https://doi.org/10.1007/s40593-015-0055-y
  16. Lawson, A. P., & Mayer, R. E. (2022). Does the emotional stance of human and virtual instructors in instructional videos affect learning processes and outcomes? Contemporary Educational Psychology, 70, 102080. https://doi.org/10.1016/j.cedpsych.2022.102080
  17. Lawson, A. P., Mayer, R. E., Adamo-Villani, N., Benes, B., Lei, X., & Cheng, J. (2021). Do learners recognize and relate to the emotions displayed by virtual instructors? International Journal of Artificial Intelligence in Education, 31(1), 134-153. https://doi.org/10.1007/s40593-021-00238-2
  18. LeDoux, J. E., & Brown, R. (2017). A higher-order theory of emotional consciousness. Proceedings of the National Academy of Sciences, 114(10), E2016-E2025. https://doi.org/10.1073/pnas.1619316114
  19. Leppink, J., Paas, F., Van der Vleuten, C. P., Van Gog, T., & Van Merriënboer, J. J. (2013). Development of an instrument for measuring different types of cognitive load. Behavior Research Methods, 45(4), 1058-1072. https://doi.org/10.3758/s13428-013-0334-1
  20. Li, W., Wang, F., Mayer, R. E., & Liu, H. (2019). Getting the point: Which kinds of gestures by pedagogical agents improve multimedia learning? Journal of Educational Psychology, 111(8), 1382-1395. https://doi.org/10.1037/edu0000352
  21. Li, W., Wang, F., Mayer, R. E., & Liu, T. (2022). Animated pedagogical agents enhance learning outcomes and brain activity during learning. Journal of Computer Assisted Learning, 38(3), 621-637. https://doi.org/10.1111/jcal.12634
  22. Lin, L., Atkinson, R. K., Christopherson, R. M., Joseph, S. S., & Harrison, C. J. (2013). Animated agents and learning: Does the type of verbal feedback they provide matter? Computers & Education, 67, 239-249. https://doi.org/10.1016/j.compedu.2013.04.017
  23. Lozano, S. C., & Tversky, B. (2006). Communicative gestures facilitate problem solving for both communicators and recipients. Journal of Memory and Language, 55(1), 47-63. https://doi.org/10.1016/j.jml.2005.09.002
  24. Martha, A. S. D., & Santoso, H. B. (2019). The design and impact of the pedagogical agent: A systematic literature review. Journal of Educators Online, 16(1). https://doi.org/10.9743/jeo.2019.16.1.8
  25. Mayer, R. E. (2020a). Multimedia learning. Cambridge University Press.
  26. Mayer, R. E. (2020b). Searching for the role of emotions in e-learning. Learning and Instruction, 70, 101213. https://doi.org/10.1016/j.learninstruc.2019.05.010
  27. Mayer, R. E. (2021). Evidence-based principles for how to design effective instructional videos. Journal of Applied Research in Memory and Cognition, 10(2), 229-240. https://doi.org/10.1016/j.jarmac.2021.03.007
  28. Moreno, R., Reislein, M., & Ozogul, G. (2010). Using virtual peers to guide visual attention during learning: A test of the persona hypothesis. Journal of Media Psychology, 22(2), 52-60. https://doi.org/10.1027/1864-1105/a000008
  29. Paas, F., & van Merriënboer, J. J. G. (2020). Cognitive-load theory: Methods to manage working memory load in the learning of complex tasks. Current Directions in Psychological Science, 29(4), 394-398. https://doi.org/10.1177/0963721420922183
  30. Park, S. (2016). Virtual avatar as an emotional scaffolding strategy to promote interest in online learning environment. In S. Y. Tettegah, & M. Gartmeier (Eds.), Emotions, technology, design, and learning (pp. 201-224). Academic Press. https://doi.org/10.1016/B978-0-12-801856-9.00010-4
  31. Pessoa, L. (2008). On the relationship between emotion and cognition. Nature Reviews Neuroscience, 9(2), 148-158. https://doi.org/10.1038/nrn2317
  32. Petri, H. L., & Govern, J. M. (2012). Motivation: Theory, research, and application. Cengage Learning.
  33. Phelps, E. A. (2004). Human emotion and memory: Interactions of the amygdala and hippocampal complex. Current Opinion in Neurobiology, 14(2), 198-202. https://doi.org/10.1016/j.conb.2004.03.015
  34. Pi, Z., Liu, R., Ling, H., Zhang, X., Wang, S., & Li, X. (2022). The emotional design of an instructor: Body gestures do not boost the effects of facial expressions in video lectures. Interactive Learning Environments. https://doi.org/10.1080/10494820.2022.2105898
  35. Plass, J. L., & Kalyuga, S. (2019). Four ways of considering emotion in cognitive load theory. Educational Psychology Review, 31(2), 339-359. https://doi.org/10.1007/s10648-019-09473-5
  36. Plass, J. L., & Kaplan, U. (2016). Emotional design in digital media for learning. In S. Tettegah, & M. Gartmeier (Eds.), Emotions, technology, design, and learning (pp. 131-162). Academic Press. https://doi.org/10.1016/B978-0-12-801856-9.00007-4
  37. Plass, J. L., Homer, B. D., MacNamara, A., Ober, T., Rose, M. C., Pawar, S., Hovey, C, M., & Olsen, A. (2020). Emotional design for digital games for learning: The effect of expression, color, shape, and dimensionality on the affective quality of game characters. Learning and Instruction, 70, 101194. https://doi.org/10.1016/j.learninstruc.2019.01.005
  38. Reeves, B., & Nass, C. (1996). The media equation. Cambridge University Press.
  39. Reyes, M. R., Brackett, M. A., Rivers, S. E., White, M., & Salovey, P. (2012). Classroom emotional climate, student engagement, and academic achievement. Journal of Educational Psychology, 104(3), 700-712. https://doi.org/10.1037/a0027268
  40. Rolland, R. G. (2012). Synthesizing the evidence on classroom goal structures in middle and secondary schools: A meta-analysis and narrative review. Review of Educational Research, 84(4), 396-435. https://doi.org/10.3102/0034654312464909
  41. Russell, J. A. (2003). Core affect and the psychological construction of emotion. Psychological Review, 110(1), 145-172. https://doi.org/10.1037/0033-295X.110.1.145
  42. Ryan, R. M., Connell, J. P., & Plant, R. W. (1990). Emotions in nondirected text learning. Learning and Individual Differences, 2(1), 1-17. https://doi.org/10.1016/1041-6080(90)90014-8
  43. Schunk, D. H. (2011). Learning theories: An educational perspective. Pearson.
  44. Sharot, T., & Phelps, E. A. (2004). How arousal modulates memory: Disentangling the effects of attention and retention. Cognitive, Affective, & Behavioral Neuroscience, 4(3), 294-306. https://doi.org/10.3758/CABN.4.3.294
  45. Sweller, J. (2020). Cognitive load theory and educational technology. Educational Technology Research and Development, 68(1). https://doi.org/10.4135/9781483346397.n55
  46. Sweller, J., Ayres, P., & Kalyuga, S. (2011). Cognitive load theory. Springer. https://doi.org/10.1007/978-1-4419-8126-4
  47. Sweller, J., van Merriënboer, J. J. G., & Paas, F. (2019). Cognitive architecture and instructional design: 20 years later. Educational Psychology Review, 31(2), 261-292. https://doi.org/10.1007/s10648-019-09465-5
  48. Tsai, J., Bowring, E., Marsella, S., Wood, W., & Tambe, M. (2012). A study of emotional contagion with virtual characters [Paper presentation]. The International Conference on Intelligent Virtual Agents. https://doi.org/10.1007/978-3-642-33197-8_8
  49. Um, E., Plass, J. L., Hayward, E. O., & Homer, B. D. (2012). Emotional design in multimedia learning. Journal of Educational Psychology, 104(2), 485-498. https://doi.org/10.1037/a0026609
  50. van der Meij, H., van der Meij, J., & Harmsen, R. (2015). Animated pedagogical agents effects on enhancing student motivation and learning in a science inquiry learning environment. Educational Technology Research and Development, 63(3), 381-403. https://doi.org/10.1007/s11423-015-9378-5
  51. Veletsianos, G. (2012). How do learners respond to pedagogical agents that deliver social-oriented non-task messages? Impact on student learning, perceptions, and experiences. Computers in Human Behavior, 28(1), 275-283. https://doi.org/10.1016/j.chb.2011.09.010
  52. Veletsianos, G., & Russell, G. S. (2014). Pedagogical agents. In J. M. Spector, M. D. Merrill, Jan Elen, & M. J. Bishop (Eds.), Handbook of research on educational communications and technology (pp. 759-769). Springer. https://doi.org/10.1007/978-1-4614-3185-5_61
  53. Wang, F. X., Li, W. J., Mayer, R. E., & Liu, H. S. (2018). Animated pedagogical agents as aids in multimedia learning: Effects on eye-fixations during learning and learning outcomes. Journal of Educational Psychology, 110, 250-268. https://doi.org/10.1037/edu0000221
  54. Xie, H., Mayer, R. E., Wang, F., & Zhou, Z. (2019). Coordinating visual and auditory cueing in multimedia learning. Journal of Educational Psychology, 111(2), 235-255. https://doi.org/10.1037/edu0000285
  55. Yung, H. I., & Paas, F. (2015). Effects of cueing by a pedagogical agent in an instructional animation: A cognitive load approach. Journal of Educational Technology & Society, 18(3), 153-160. https://doi.org/10.1080/10494820.2020.1857784