The Dialogue HMM Project

Building computational models of the structure of human dialogue.

Introduction

Human dialogue is the realization of complex cognitive, emotional, and social phenomena. If we can model this dialogue well, we can not only learn fundamental truths about how humans communicate with each other and learn through dialogue, we can also build intelligent systems capable of interacting in rich natural language. However, building computational models of dialogue is a challenging problem. The rich phenomena that characterize the dialogue process introduce significant uncertainty in many forms, so extracting “rules” that hold in every circumstance is not possible. To cope with this uncertainty, we utilize statistical models, which use probabilities to express relationships.

Hidden Markov Models

All statistical models attempt to capture the relationships within some set of observations. In dialogue, we can observe many things, for example, the actual words that were used, the tone of voice, or facial expressions and gestures. We can visualize dialogue as a linear sequence of these observations.

However, there are some aspects of dialogue that generally cannot be directly observed. For example, the two dialogue participants may have goals that are never explicitly stated. Representing this unobservable structure within a statistical model can lead to a better model fit. This is the motivation behind utilizing hidden Markov models (HMMs), which allow for both a hidden and an observable layer of probabilistic structure. These models can also be visualized in a time-slice view; at each time step, there is a hidden state and a corresponding observation that we say was emitted by the hidden state.


Learning the Structure of Dialogue

Each hidden state is characterized by a probability distribution over the observations it could emit. Therefore, we can visualize an HMM as a set of hidden states with their associated emission probabilities. When we learn HMMs of annotated dialogue, we can discover ways in which the hidden state structure is associated with outcomes of interest, such as student knowledge gain.

Future Work

Numerous findings from the Dialogue HMM project have demonstrated that these models hold great promise for modeling human dialogue (see related publications below for details). Particularly rich areas for future work include the following:

publications

2015
[18]Discovering Individual and Collaborative Problem-Solving Modes with Hidden Markov Models. Fernando J. Rodríguez, Kristy Elizabeth Boyer. Proceedings of the 17th International Conference on Artificial Intelligence in Education (AIED), Madrid, Spain, 2015, pp. 408-418. [bib]
2011
[17]An Affect-Enriched Dialogue Act Classification Model for Task-Oriented Dialogue. Kristy Elizabeth Boyer, Joseph Grafsgaard, Eun Young Ha, Robert Phillips, James C. Lester. Proceedings of the International Conference of the Association for Computational Linguistics (ACL), Portland, Oregon, 2011, pp. 1190-1199. [bib]
[16]The Impact of Task-Oriented Feature Sets on HMMs for Dialogue Modeling. Kristy Elizabeth Boyer, Eun Young Ha, Robert Phillips, James C. Lester. Proceedings of the 12th Annual SIGDIAL Meeting on Discourse and Dialogue, Portland, Oregon, 2011, pp. 49-58. [bib]
[15]Investigating the Relationship Between Dialogue Structure and Tutoring Effectiveness: A Hidden Markov Modeling Approach. Kristy Elizabeth Boyer, Robert Phillips, Amy Ingram, Eun Young Ha, Michael D. Wallis, Mladen A. Vouk, James C. Lester. International Journal of Artificial Intelligence in Education (IJAIED), vol. 21 no. 1, 2011, pp. 65-81. [bib]
2010
[14]Dialogue Act Modeling in a Complex Task-Oriented Domain. Kristy Elizabeth Boyer, Eun Young Ha, Robert Phillips, Michael D. Wallis, Mladen A. Vouk, James C. Lester. Proceedings of the 11th Annual SIGDIAL Meeting on Discourse and Dialogue, Tokyo, Japan, 2010, pp. 297-305. [bib]
[13]Leveraging Hidden Dialogue State to Select Tutorial Moves. Kristy Elizabeth Boyer, Robert Philips, Eun Young Ha, Michael D. Wallis, Mladen A. Vouk, James C. Lester. Proceedings of the 5th Workshop on Innovative Use of NLP for Building Educational Applications (BEA), Los Angeles, California, 2010, pp. 66-73. [bib]
[12]Characterizing the Effectiveness of Tutorial Dialogue with Hidden Markov Models. Kristy Elizabeth Boyer, R. Phillips, A. Ingram, Eun Young Ha, M. Wallis, M. Vouk, J. Lester. Proceedings of the 10th International Conference on Intelligent Tutoring Systems (ITS), Pittsburgh, Pennsylvania, 2010, pp. 55-64. [bib]
[11]Principles of Asking Effective Questions During Student Problem Solving. Kristy Elizabeth Boyer, William Lahti, Robert Phillips, Michael D. Wallis, Mladen A. Vouk, James C. Lester. Proceedings of the 41st ACM Technical Symposium on Computer Science Education (SIGCSE), Milwaukee, Wisconsin, 2010, pp. 460-464. [bib]
2009
[10]An Empirically Derived Question Taxonomy for Task-Oriented Tutorial Dialogue. Kristy Elizabeth Boyer, William J Lahti, Robert Phillips, Michael D Wallis, Mladen A Vouk, James C Lester. Proceedings of the Second Workshop on Question Generation, Brighton, United Kingdom, 2009, pp. 9-16. [bib]
[9]Discovering Tutorial Dialogue Strategies with Hidden Markov Models. Kristy Elizabeth Boyer, Eun Young Ha, Michael D. Wallis, Robert Phillips, Mladen A. Vouk, James C. Lester. Proceedings of the 14th International Conference on Artificial Intelligence in Education (AIED), Brighton, United Kingdom, 2009, pp. 141-148. [bib]
[8]Inferring Tutorial Dialogue Structure with Hidden Markov Modeling. Kristy Elizabeth Boyer, Eun Young Ha, Robert Philips, Michael D. Wallis, Mladen A. Vouk, James C. Lester. Proceedings of the 4th Workshop on Innovative Use of NLP for Building Educational Applications (BEA), Boulder, Colorado, 2009, pp. 19-26. [bib]
[7]Modeling Dialogue Structure with Adjacency Pair Analysis and Hidden Markov Models. Kristy Elizabeth Boyer, Robert Phillips, Eun Young Ha, Michael D. Wallis, Mladen A. Vouk, James C. Lester. Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics and Human Language Technology (NAACL HLT), Boulder, Colorado, 2009, pp. 49-52. [bib]
[6]The Impact of Instructor Initiative on Student Learning: A Tutoring Study. Kristy Elizabeth Boyer, Robert Phillips, Michael D. Wallis, Mladen A. Vouk, James C. Lester. Proceedings of the 40th ACM Technical Symposium on Computer Science Education (SIGCSE), Chattanooga, Tennessee, 2009, pp. 14-18. [bib]
[5]Investigating the Role of Motivation in Computer Science Education through One-on-One Tutoring. Kristy Elizabeth Boyer, Robert Phillips, Michael D. Wallis, Mladen A. Vouk, James C. Lester. Computer Science Education, vol. 19 no. 2, 2009, pp. 111-136. [bib]
2008
[4]A Development Environment for Distributed Synchronous Collaborative Programming. Kristy Elizabeth Boyer, August A. Dwight, R. Taylor Fondren, Mladen A. Vouk, James C. Lester. Proceedings of the 13th Annual Conference on Innovation and Technology in Computer Science Education (ITiCSE), Madrid, Spain, 2008, pp. 158-162. [bib]
[3]Learner Characteristics and Feedback in Tutorial Dialogue. Kristy Elizabeth Boyer, Robert Philips, Michael D. Wallis, Mladen A. Vouk, James C. Lester. Proceedings of the 3rd Workshop on Innovative Use of NLP for Building Educational Applications (BEA), Columbus, Ohio, 2008, pp. 53-61. [bib]
[2]Balancing Cognitive and Motivational Scaffolding in Tutorial Dialogue. Kristy Elizabeth Boyer, Robert Phillips, Michael D. Wallis, Mladen A. Vouk, James C. Lester. Proceedings of the International Conference on Intelligent Tutoring Systems (ITS), Montréal, Quebec, 2008, pp. 239-249. [bib]
2007
[1]The Influence of Learner Characteristics on Task-Oriented Tutorial Dialogue. Kristy Elizabeth Boyer, Mladen A. Vouk, James C. Lester. Proceedings of the 13th International Conference on Artificial Intelligence in Education (AIED), Marina Del Rey, California, 2007, pp. 365-372. [bib]