Markerless And Model-free Vision-based Robotic Control
Vision-based control techniques offer significant advantages for robotic manipulators operating in unstructured and cluttered environments by enabling closed-loop control using task-relevant visual information. These methods enhance robustness against model inaccuracies, making them particularly effective for robots with complex or variable dynamics, such as soft robots, under-actuated robots, 3D-printed robots, and robots built with inexpensive hardware. By leveraging model-free visual servoing approaches, which learn robot-feature motion models during control, the reliance on explicit robot modeling or proprioceptive sensing is minimized. My research goal is to push the boundaries of purely vision-based control and path planning by utilizing natural visual features along a robot’s body in the image space, without requiring external markers or explicit robot model. To this end, we developed an adaptive visual servoing algorithm that dynamically estimates the robot’s Jacobian using its visual skeleton, enabling convergence to desired configurations. Additionally, we introduced algorithms to track natural visual features on the robot's body for precise control. We have also developed techniques that leverage these visual features as a model-free alternative to traditional motion planning approaches. For the remainder of my PhD, I aim to design a deep learning model to robustly track these visual features during occlusions, ensuring continuity of control even when the robot moves partially out of the visible workspace. We also aim to extend the control scheme to out-of-plane motion, enabling the manipulation of articulated objects in real time. These contributions collectively aim to strive toward robust vision-driven, markerless, model-free robotic manipulation and control in dynamic, cluttered environments.
Advisor: Professor Berk Calli
Committee: Professor Nitin Sanket, Professor Constantinos Chamzas, Professor Chun Kit Ngan
Zoom Link: https://wpi.zoom.us/j/95538290413