Robust Surgical Perception - Toward Autonomous Surgery

Despite the strong capabilities of current surgical robots, such as high precision and dexterity, they are predominantly fully controlled by surgeons during surgeries. Toward the goal of elevating surgical robot autonomy, the capability for real-time, accurate 3D perception of complex surgical environments, and the ability to understand surgical procedures are critical enabling technologies. In this talk, I will discuss a series of our works on developing robust perception frameworks that can perform reconstruction and deformation tracking of surgical scenes, as well as track robot poses, potentially allowing surgeons and robots to better 'see' what's happening during surgery. I will also discuss how we leverage these perception results to control robots for deformable object manipulation.