Title page for ETD etd-050311-160847

Document Type thesis

Author Name Goldstein, Adam B

Email Address abgoldstein at gmail.com

URN etd-050311-160847

Title Responding to Moments of Learning

Degree MS

Department Computer Science

Advisors
Neil T. Heffernan, Advisor
Ryan S.J.d Baker, Advisor
Sonia Chernova, Reader

Keywords
machine learning
knowledge tracing
intelligent tutoring systems
educational data mining

Date of Presentation/Defense 2011-05-02

Availability unrestricted

Abstract
In the field of Artificial Intelligence in Education, many contributions have been made toward estimating student proficiency in Intelligent Tutoring Systems (cf. Corbett & Anderson, 1995). Although the community is increasingly capable of estimating how much a student knows, this does not shed much light on when the knowledge was acquired. In recent research (Baker, Goldstein, & Heffernan, 2010), we created a model that attempts to answer that exact question. We call the model P(J), for the probability that a student just learned from the last problem they answered. We demonstrated an analysis of changes in P(J) that we call Â“spikinessÂ”, defined as the maximum value of P(J) for a student/knowledge component (KC) pair, divided by the average value of P(J) for that same student/KC pair. Spikiness is directly correlated with final student knowledge, meaning that spikes can be an early predictor of success. It has been shown that both over-practice and under-practice can be detrimental to student learning, so using this model can potentially help bias tutors toward ideal practice schedules.
After demonstrating the validity of the P(J) model in both CMUÂ’s Cognitive Tutor and WPIÂ’s ASSISTments Tutoring System, we conducted a pilot study to test the utility of our model. The experiment included a balanced pre/post-test and three conditions for proficiency assessment tested across 6 knowledge components. In the first condition, students are considered to have mastered a KC after correctly answering 3 questions in a row. The second condition uses Bayesian Knowledge Tracing and accepts a student as proficient once they earn a current knowledge probability (Ln) of 0.95 or higher. Finally, we test P(J), which accepts mastery if a studentÂ’s P(J) value spikes from one problem and the next first response is correct. In this work, we will discuss the details of deriving P(J), our experiment and its results, as well as potential ways this model could be utilized to improve the effectiveness of cognitive mastery learning.

Files
goldstein.pdf

Browse by Author | Browse by Department | Search all available ETDs

Questions? Email etd-questions@wpi.edu
Maintained by webmaster@wpi.edu

Document Type	thesis
Author Name	Goldstein, Adam B
Email Address	abgoldstein at gmail.com
URN	etd-050311-160847
Title	Responding to Moments of Learning
Degree	MS
Department	Computer Science
Advisors	Neil T. Heffernan, Advisor Ryan S.J.d Baker, Advisor Sonia Chernova, Reader
Keywords	machine learning knowledge tracing intelligent tutoring systems educational data mining
Date of Presentation/Defense	2011-05-02
Availability	unrestricted
Abstract In the field of Artificial Intelligence in Education, many contributions have been made toward estimating student proficiency in Intelligent Tutoring Systems (cf. Corbett & Anderson, 1995). Although the community is increasingly capable of estimating how much a student knows, this does not shed much light on when the knowledge was acquired. In recent research (Baker, Goldstein, & Heffernan, 2010), we created a model that attempts to answer that exact question. We call the model P(J), for the probability that a student just learned from the last problem they answered. We demonstrated an analysis of changes in P(J) that we call Â“spikinessÂ”, defined as the maximum value of P(J) for a student/knowledge component (KC) pair, divided by the average value of P(J) for that same student/KC pair. Spikiness is directly correlated with final student knowledge, meaning that spikes can be an early predictor of success. It has been shown that both over-practice and under-practice can be detrimental to student learning, so using this model can potentially help bias tutors toward ideal practice schedules. After demonstrating the validity of the P(J) model in both CMUÂ’s Cognitive Tutor and WPIÂ’s ASSISTments Tutoring System, we conducted a pilot study to test the utility of our model. The experiment included a balanced pre/post-test and three conditions for proficiency assessment tested across 6 knowledge components. In the first condition, students are considered to have mastered a KC after correctly answering 3 questions in a row. The second condition uses Bayesian Knowledge Tracing and accepts a student as proficient once they earn a current knowledge probability (Ln) of 0.95 or higher. Finally, we test P(J), which accepts mastery if a studentÂ’s P(J) value spikes from one problem and the next first response is correct. In this work, we will discuss the details of deriving P(J), our experiment and its results, as well as potential ways this model could be utilized to improve the effectiveness of cognitive mastery learning.
Files	goldstein.pdf