•  
• 

Introduction

Affective adaptation holds great promise for promoting productive affective states that improve the learning experience. With a decade of research on affective computing that has yielded foundational results on affect and learning, we are now well positioned to address a central question in the field: How can we design learning environments that adaptively respond to students’ affect to create the most effective, engaging learning experiences while simultaneously promoting improved attitudes toward learning? Answering this question will significantly advance the field, contribute directly to improved learning experiences for all students, and introduce the opportunity to achieve transformative improvements for underserved students, who stand to see particular benefit from these adaptive technologies.

Project Description

The project has three major thrusts: 1) capture rich multimodal data of students’ affective experiences while interacting with a fully instrumented learning environment with spoken dialogue; 2) design, develop, and iteratively refine an affect understanding model that integrates students’ natural language, nonverbal behavior, physiological response, and task action phenomena; and 3) design, develop, and iteratively refine an integrated affect and dialogue management model that adaptively responds to students’ affective states in the course of their learning interactions. To achieve these goals, the project will undertake iterative refinement and evaluation that begins with fully instrumented laboratory studies and culminates in month-long classroom studies at middle schools.

publications

2019
[17]	Predicting Dialogue Breakdown in Conversational Pedagogical Agents with Multimodal LSTMs. Wookhee Min, Kyungjin Park, Joseph Wiggins, Bradford Mott, Eric Wiebe, Kristy Elizabeth Boyer, James Lester. Proceedings of the International Conference on Artificial Intelligence in Education, 2019, pp. 195-200. [bib]
[16]	Take the Initiative: Mixed Initiative Dialogue Policies for Pedagogical Agents in Game-Based Learning Environments. Joseph B. Wiggins, Mayank Kulkarni, Wookhee Min, Kristy Elizabeth Boyer, Bradford Mott, Eric Wiebe, James Lester. Proceedings of the International Conference on Artificial Intelligence in Education, 2019, pp. 314-318. [bib]
2018
[15]	Affect-based Early Prediction of Player Mental Demand and Engagement for Educational Games. Joseph B. Wiggins, Mayank Kulkarni, Wookhee Min, Kristy Elizabeth Boyer, Bradford Mott, Eric Wiebe, James Lester. The 14th AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment (AIIDE'18), 2018, pp. 243-249. [bib]
[14]	User Affect and No-Match Dialogue Scenarios: An Analysis of Facial Expression. Joseph B. Wiggins, Mayank Kulkarni, Wookhee Min, Kristy Elizabeth Boyer, Bradford Mott, Eric Wiebe, James Lester. Proceedings of the 4th International Workshop on Multimodal Analyses enabling Artificial Agents in Human-Machine Interaction (MA3HMI), 2018, pp. 6-14. [bib]
2017
[13]	"Thanks Alisha, Keep in Touch": Gender Effects and Engagement with Virtual Learning Companions. Lydia G. Pezzullo, Joseph B. Wiggins, Megan H. Frankosky, Wookhee Min, Kristy Elizabeth Boyer, Bradford W. Mott, Eric N. Wiebe, James C. Lester. Proceedings of the International Conference on Artificial Intelligence in Education, Wuhan, China, 2017, pp. 299—310. [bib]
2016
[12]	Gender Differences in Facial Expressions of Affect During Learning. Alexandria K. Vail, Joseph F. Grafsgaard, Kristy Elizabeth Boyer, Eric N. Wiebe, James C.. Lester. Proceedings of the 24th International Conference on User Modelling, Adaptation, and Personalization, Halifax, Canada, 2016, pp. 65-74. [bib]
[11]	Predicting Learning from Student Affective Response to Tutor Questions. Alexandria K. Vail, Joseph F. Grafsgaard, Kristy Elizabeth Boyer, Eric N. Wiebe, James C. Lester. Proceedings of the 13th International Conference on Intelligent Tutoring Systems, Zagreb, Croatia, 2016, pp. 154-164. [bib]
[10]	The Affective Impact of Tutor Questions: Predicting Frustration and Engagement. Alexandria K. Vail, Joseph B. Wiggins, Joseph F. Grafsgaard, Kristy Elizabeth Boyer, Eric N. Wiebe, James C.. Lester. Proceedings of the 9th International Conference on Educational Data Mining, Raleigh, North Carolina, 2016, pp. 247-254. [bib]
[9]	Do You Think You Can? The Influence of Student Self-Efficacy on the Effectiveness of Tutorial Dialogue for Computer Science. Joseph B. Wiggins, Joseph F. Grafsgaard, Kristy Elizabeth Boyer, Eric N. Wiebe, James C. Lester. International Journal of Artificial Intelligence in Education, 2016, pp. 1-24. [bib]
2015
[8]	The Mars and Venus Effect: The Influence of User Gender on the Effectiveness of Adaptive Task Support. Alexandria K. Vail, Kristy Elizabeth Boyer, Eric N. Wiebe, James C. Lester. Proceedings of the 23nd International Conference on User Modelling, Adaptation, and Personalization, Dublin, Ireland, 2015, pp. 265-276. [bib]
[7]	Mind the Gap: Improving Gender Equity in Game-based Learning Environments with Learning Companions. Philip Sheridan Buffum, Kristy Elizabeth Boyer, Eric N. Wiebe, Bradford W. Mott, James C. Lester. Proceedings of the International Conference on Artificial Intelligence in Education, Madrid, Spain, 2015, pp. 64-73. [bib]
[6]	Choosing to Interact: Exploring the Relationship Between Learner Personality, Attitudes, and Tutorial Dialogue Participation. Aysu Ezen-Can, Kristy Elizabeth Boyer. Proceedings of the International Conference on Educational Data Mining, Madrid, Spain, 2015, pp. 125-129. [bib]
[5]	Discovering Individual and Collaborative Problem-Solving Modes with Hidden Markov Models. Fernando J. Rodríguez, Kristy Elizabeth Boyer. Proceedings of the 17th International Conference on Artificial Intelligence in Education (AIED), Madrid, Spain, 2015, pp. 408-418. [bib]
[4]	Modeling Self-Efficacy Across Age Groups with Automatically Tracked Facial Expression. Joseph F. Grafsgaard, Seung Y. Lee, Bradford W. Mott, Kristy Elizabeth Boyer, James C. Lester. Proceedings of the International Conference on Artificial Intelligence in Education, Madrid, Spain, 2015, pp. 582-585. [bib]
[3]	Semantic Grounding in Dialogue for Complex Problem Solving. Xiaolong Li, Kristy E. Boyer. Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics and Human Language Technology (NAACL HLT), Denver, Colorado, 2015, pp. 841-850. [bib]
[2]	Classifying Student Dialogue Acts with Multimodal Learning Analytics. Aysu Ezen-Can, Joseph F. Grafsgaard, James C. Lester, Kristy Elizabeth Boyer. Proceedings of the International Conference on Learning Analytics and Knowledge (LAK), Poughkeepsie, NY, 2015, pp. 280-289. [bib]
2014
[1]	Predicting Learning and Engagement in Tutorial Dialogue: A Personality-Based Model. Alexandria Katarina Vail, Joseph F. Grafsgaard, Joseph B. Wiggins, James C. Lester, Kristy Elizabeth Boyer. Proceedings of the 16th International Conference on Multimodal Interaction (ICMI), Istanbul, Turkey, 2014, pp. 255-262. [bib]