- Using LLMs for personalized education has been a goal of technologists and teachers.
- Incorporating interest-based learning into LLM-based tutors, Abhishek Kulkarni, Ph.D., is working to make them more engaging for students.
- The prompts from his system of virtual tutors elicit students’ interests, which are incorporated into the pedagogy.
Technology that allows teaching to be tailored to a student’s needs and learning style — known as personalized education — has long been a goal of technologists and teachers. Recent advances in large language models such as ChatGPT are powering personalized tutors that seem to intuit students’ needs and styles.
Educators are taking notice. Even in 2023, experts asserted that ChatGPT-enabled tutors would become indispensable classroom tools.
“Since ChatGPT has access to far more knowledge than Aristotle ever did, it has great potential for providing tutoring to students to help them learn more than they would otherwise,” wrote Anne Trumbore, chief digital learning officer at Sands Institute for Lifelong Learning at the University of Virginia, in a 2023 article in “The Conversation.”
Of course, these next-generation tutors are only as good as the data upon which they’re trained.
Abhishek Kulkarni, Ph.D., currently a postdoctoral researcher at the University of Florida Digital Worlds Institute, is working to make virtual tutors more effective and engaging for students. Leveraging research done during his doctoral studies at UF’s Department of Computer & Information Science & Engineering, or CISE, he’s using ideas gleaned from a burgeoning field of research called interest-based learning and incorporating them into the systems that are training virtual tutors.
With interest-based learning, tutors who are familiar with their students’ interests ask better questions and are able to connect with students more easily than tutors who simply provide information.
But how do researchers collect and systematize students’ interests?
Take, for example, this interaction from the study between an actual student and a virtual tutor:
AI: Hi there! It’s great to meet you. Before we dive into any educational topics, could you tell me a bit about your interests or hobbies? It might help us connect these with what you’re learning.
Student: I love watching “The Office.” I’ve done like tons of reruns and I still love watching it! Today, I wanted to learn about entropy and the second law of thermodynamics.
AI: That’s awesome! “The Office” is such a classic show, and we can definitely use elements from it to make learning about entropy and the second law of thermodynamics more fun.
Let’s start by understanding what you know about entropy and the second law of thermodynamics. Have you ever come across these concepts before, or is this your first dive into them? Knowing this will help me tailor the explanation to your current understanding.
Student: This will be my first dive into them!
The prompts from the LLM were engineered to seem natural and casual, not dissimilar from what a human tutor would ask. Later in the chat, the AI agent referenced scenes from the student’s favorite show to illustrate the concept of entropy. Student responses were fed back into the system, informing the subsequent prompts, all optimized to create an engaging experience for the student.
“We started by asking if we could somehow integrate AI and try and automate this generation of content,” Kulkarni said.
Kulkarni is lead author of “Towards Pegagogy-Grounded Conversational AI Tutors for Interest-Based Learners,” recently published in the Proceedings of the Eighth Conference on Conversational User Interfaces, an Association for Computing Machinery conference. His co-authors are CISE faculty members Neha Rani, Ph.D., and Sharon Lynn Chu, Ph.D., and CISE doctoral student Sarah Anne Brown.
“When we started this work, there really weren’t any chatbots that specifically were using interest-based learning,” Kulkarni said. “But what we did have was an initial study where we asked human educators to design interest-based lessons and then teach them to students. We analyzed these tutoring sessions to identify what human educators actually do.”
Based on real-world results, the authors identified seven ways tutors used students’ reported interests in their teaching processes and incorporated these very human methods into the LLM tutor.
But how to test the effectiveness?
The prompts and results were examined by both synthetic and human users. Researchers used AI-powered synthetic users to study the results of tutoring sessions between human students and LLM-powered tutors.
“This is something that is being used in research a lot recently, where you have your AI system, your large language model, but you also test it with AI users,” Kulkarni said.
But testing AI systems with virtual AI users might reinforce existing design flaws or cause errors to be missed. Researchers also employed a panel of expert educators who reviewed the chat transcripts and analyzed the pedagogical styles and effectiveness.
“Our panel of educators reviewed the transcripts,” Kulkarni remembered. “We asked them, ‘What do you think of these conversations? Do you think this is actually good pedagogy? How would you improve this process?’”
The resulting data, sourced from humans and otherwise, were fed back into the LLM for further improvement and testing.
Kulkarni said the team will next examine the eventual effectiveness of the pedagogy. The paper specifically limited the researchers’ scope to whether an LLM-powered tutor could adapt to the specific interest-based learning strategies outlined in their research. The results so far are promising.