Jake Whitehill, center, will study which AI Partners will help students collaborate better in the classroom.

Can Artificial Intelligence Help Students Work Better Together? According to Research, the Answer is Yes.

WPI computer scientist looks at better collaboration between AI and humans in today’s classrooms.
October 19, 2020

WPI assistant professor of computer science Jacob Whitehill is collaborating with colleagues at the University of Colorado Boulder to explore how artificially intelligent (AI) teaching agents might help encourage more meaningful collaboration among students in school classrooms.

As part of a five-year, $20 million grant awarded by the National Science Foundation to CU Boulder, Whitehilll and colleagues at nine other institutions will study how to build AI agents that interact with students in small-group collaborative problem-solving settings and thereby foster more enriching learning experiences for students.

“There is so much evidence that students learn best when they learn actively and collaborate with each other in real time,” says Whitehill, who specializes in AI and machine learning. “But that’s not always easy for teachers to achieve, whether the collaboration is happening in a physical classroom or during remote learning.”

Under the grant, of which WPI will receive almost $612,000, Whitehill and his colleagues will work with two public school districts in Colorado, building and testing different forms of “AI Partners” in the classroom—ranging from a simple iPad with a camera and microphone, to animated interactive avatars that perceive and understand students’ language, gestures, facial expressions. The role of the AI Partner will vary depending on the situation: In some settings, the Partner might play the role of another “student” who is learning the curriculum alongside the human students. In other settings, the Partner might be a coach who keeps the students on-topic and makes sure everyone is contributing meaningfully to the conversation.

“The AI Partner might, for example, ask a particular student to share their ideas, particularly if they had been quiet for most of the time,” Whitehill says.

The role of Whitehill’s team will be to use multi-modal machine learning to develop social signal processing algorithms to determine who is talking when, and to estimate the emotion of each person.

WPI professor Jacob Whitehill’s research with CU Boulder stems from his earlier work toward building an Automatic Classroom Observation Recognition neural Network (ACORN) , which analyzes videos of school classrooms and estimates the level of “emotional support” exhibited by students and teachers. ACORN can provide researchers and teachers with a new “lens” with which to observe classrooms objectively, potentially lowering the costs of teacher training.

Many WPI graduate and undergraduate students have helped advance this research since its early stages, Whitehill says, particularly through machine learning research in the areas of emotion recognition, speaker identification, and human behavior recognition.

Once the AI Partners are integrated in these classrooms, Whitehill and his team will be able to collect data on how students interact with them, and then iteratively make them more intelligent and effective. Initially, the AI Partner might be controlled by a human teacher in a backroom (“Wizard of Oz”-style interaction), but over time, it can learn from its human controller what to do when and thereby become more autonomous. Whitehill and his team anticipate that the particular form that the Partner takes is likely also important.

“Students might find an embodied robot creepy, but they might like interacting with an animated avatar on a touchscreen,” he says.

This project represents a shift in how researchers envision AI in the classroom. While earlier work in this field sought to fully automate the teaching process, which Whitehill considers to be infeasible, this project is about human-AI teaming, and how humans and teachers possess complementary abilities. AI Partners can help to magnify teachers’ existing strengths by increasing the number of students in the classroom who receive the real-time feedback they need for optimal learning.

Whitehill also says that this research will be greatly informative even during the COVID-19 pandemic, when many school districts across the country are participating in remote learning. In fact, he says testing agent-student interactions over platforms like Zoom have certain advantages over in-person interactions.

“With Zoom, each student and teacher in the classroom is cleanly separated from each other, and all their audiovisual inputs are channeled through a common software interface. This makes it much easier to analyze their speech, gestures, language, and interactions with each other,” Whitehill says. “In contrast, in normal, in-person classrooms, the interactions are much messier,” since students often sit in all kinds of different positions, might be touching their faces, and work in a noisy environment, which makes it more challenging for the Partner to observe and analyze.

By the end of this research, Whitehill says he hopes to find practical teaching and coaching strategies that AI Partners can execute that work well with students. “It’s not clear at all that the way humans teach would work well for a computer, robot, or avatar,” he says.

While the computational challenges of the project—signal processing in extremely noisy and cluttered settings, real-time control in an uncertain environment, and human-computer interaction for a novel setting—are formidable, Whitehill says the potential rewards make it worth the effort.

“The exciting thing about this project is that we get to completely rethink the role of AI in the classroom,” he says. “My hope is that, through next-generation educational AI, we will be able to stimulate deeper critical thinking and collaboration among students to help them learn better and achieve more.”


–Jessica Messier