Worcester Polytechnic Institute Electronic Theses and Dissertations Collection

Title page for ETD etd-041913-155347


Document Typemasters report
Author NameBodah, Joshua
URNetd-041913-155347
TitleLeveraging User Testing to Address Learnability Issues for Teachers Using ASSISTments
DegreeMS
DepartmentComputer Science
Advisors
  • Neil Heffernan, Advisor
  • David Brown, Reader
  • Craig Wills, Department Head
  • Keywords
  • assistments
  • user interface
  • controlled experiment
  • a/b testing
  • user testing
  • usability
  • Date of Presentation/Defense2013-04-19
    Availability unrestricted

    Abstract

    The goal of this thesis is to demonstrate how user testing can be used to identify and remediate learnability issues of a web application. Experimentation revolved around ASSISTments (www.assistments.org), an intelligent tutoring web application in which teachers create virtual classrooms where they can assign problem sets to their students and gain valuable data which can be used to make informed decisions. Recent log analysis uncovered very low task completion rates for new users on tasks that were intended to be trivial. Suspecting that this could be due to poor user interface design, user tests were conducted to help identify usability problems. Sessions were analyzed, and changes were made between each user test to address issues found.

    Feedback from user testing led to the implementation of an embedded support system. This support system consisted of a splash page which gave an overview of how the system should be used and a collection of context-sensitive tooltips which tried to give the user instructions on what to do as well as explain various parts of the interface. A randomized control trial was performed to measure the effectiveness of the embedded support. 69 participants were shown one of two interfaces: one with embedded support and one without. Task completion rates were analyzed for each of the groups. We found that the support system was able to influence which links a user clicked. However, although the support system was intended to address poor task completion rates, users in the conditions had similar task completion rates regardless of whether the support system was enabled.

    Files
  • thesis.pdf

  • Browse by Author | Browse by Department | Search all available ETDs

    [WPI] [Library] [Home] [Top]

    Questions? Email etd-questions@wpi.edu
    Maintained by webmaster@wpi.edu