RBE MS Thesis Presentation: Ankur Agrawal

Tuesday, November 13, 2018
2:00 pm to 3:00 pm
Floor/Room #: 
Suite #201, RBE Conference Room

RBE MS Thesis Presentation

Ankur Agrawal

Automating Endoscopic Camera Motion for Teleoperated Minimally Invasive Surgery Using Inverse Reinforcement Learning

Abstract:  During a laparoscopic surgery, an endoscopic camera is used to provide visual feedback of the surgery to the surgeon and is controlled by a skilled assisting surgeon.  However, in robot-assisted teleoperated systems such as the daVinci surgical system, the same control lies with the operating surgeons.  This results in an added task of constantly changing view point of the endoscope which can be disruptive.  The work presented in this thesis aims to provide an approach that results in an intelligent camera control for such systems using machine learning algorithms.  A particular task of pick and place was selected to demonstrate this approach.  To add a layer of intelligence to the endoscope, the task was classified into subtasks representing the intent of the user.  Neural networks with long short term memory cells (LSTMs) were trained to classify the motion of the instruments in the subtasks and a policy was calculated for each subtask using inverse reinforcement learning (IRL).  Since current surgical robots do not enable the movement of the camera and instruments simultaneously, an expert data set was unavailable that could be used to train the models.  Hence, a user study was conducted in which the participants were asked to complete the task of picking and placing a ring on a peg in a 3-D immersive simulation environment created using CHAI libraries.  A virtual reality headset, Oculus Rift, was used during the study to track the head movements of the users to obtain their view points while they performed the task.  This was considered to be expert data and was used to train the algorithm to automate the endoscope motion.  Results for the classification and the rewards calculated using IRL have been discussed.

Thesis Advisor: Prof. Gregory Fischer
Thesis Committee: Prof. Loris Fichera, Prof. Jing Xiao

Tuesday, November 13, 2018
2:00 p.m. - 3:00 p.m.
85 Prescott St., Suite #201, RBE Conference Room

Teresa Hemple