Foundations for Intelligent Surgical Robotics: Simulation, Modeling, and Multi-Modal Data Infrastructure
Abstract: Minimally invasive surgery offers important clinical benefits, but it also imposes significant perceptual, kinematic, and ergonomic challenges on surgeons. Surgical robotic systems help address some of these limitations, yet most existing platforms remain primarily teleoperated and rely little on autonomy or advanced perception. Advancing toward more capable surgical robotic systems requires progress in simulation environments, real-world robot modeling, and high-quality multi-modal data infrastructure. This dissertation presents an integrated investigation spanning these three areas using the da Vinci Research Kit (dVRK).
First, this work develops photorealistic simulation environments for robotic suturing and uses them to study needle tracking and automation algorithms for representative suturing subtasks. These simulation efforts provide a flexible testbed for perception and policy development while also revealing the limitations of simulation-only approaches. Second, motivated by these limitations, this dissertation investigates the physical dVRK platform through dynamic model identification, gravity compensation, and force estimation. A complete pipeline is developed for the dVRK Classic and dVRK-Si patient side manipulators, including symbolic modeling, excitation trajectory generation & execution, data collection, parameter identification, and evaluation on the real robot. Third, this work establishes a time-synchronized multi-modal data collection and post-processing framework for the dVRK, including calibration pipelines, kinematic reprojection, depth estimation, optical flow generation, and construction of a large-scale real-world dataset for downstream perception and learning tasks.
Together, these contributions provide foundational simulation tools, robot modeling methods, and data infrastructure that support future research in surgical robot perception, control, and sim-to-real transfer, and help accelerate progress toward safe and effective automation in surgical robotics.
Advisor: Professor Gregory Fischer
Committee: Professor Loris Fichera, Professor Liaohai Chen, Professor Peter Kazanzides
Zoom link: https://wpi.zoom.us/j/91520501724