Surgical Task Automation

Surgical robots, such as Intuitive Surgical’s da Vinci Surgical System, have brought about more efficient surgeries by improving dexterity and reducing surgeon fatigue through teleoperational control. While these systems are already providing great care to patients, they have also opened the door to a variety of research, including surgical task automation. Surgical task automation has furthermore been an increasing area of research in an effort to improve patient throughput, reduce quality-of-care variance among surgeries, and potentially deliver automated surgery in the future. We are developing algorithms and control policies that automate surgical tasks to work towards this future.

Learning to Automate Surgery

Reinforcement Learning (RL) is a machine learning framework for AI systems to solve complex problems. In recent years, success in solving challenging games and robotic manipulation tasks has increased, partly due to collaborative efforts on open-sourced environment simulators like OpenAI's Gym. We present the first open-sourced reinforcement learning environment for surgical robotics, called dVRL, which is functionally equivalent to Gym. dVRL enables prototyping and implementing state-of-art RL algorithms on surgical robotics problems, aiming to introduce autonomous robotic precision and accuracy during surgery. Combining dVRL with the da Vinci Surgical Research Kits network, we enable the surgical robotics community to leverage the newest RL strategies and RL scientists to develop algorithms for autonomous surgery challenges.

Robust Surgical Automation

Robustness is a critical factor in surgical automation since the nature of surgical procedures demands unwavering reliability and resilience. Robust surgical automation should be capable of seamlessly adapting to dynamic surgical scenarios, handling variability in patient anatomy, and quickly recovering from disruptions, minimizing the potential for adverse impacts on the patient's well-being. Achieving robustness in surgical automation is crucial for instilling confidence in both surgeons and patients, as it guarantees consistent and reliable performance, even in challenging or unpredictable circumstances.

Recent efforts in our group such as SURESTEP (Surgical Uncertainty-aware Robust ESTimation TrajEctory Protocol) present frameworks for uncertainty-aware trajectory optimization that enables robust automation by enhancing visual tool tracking accuracy. It optimizes trajectories from any policies to be robust to motion and observation uncertainties commonly encountered in surgical settings.



  • Nikhil Shinde
  • Zih-Yun Chiu
  • *Florian Richter
  • Xiao Liang
  • Neelay Joglekar
  • Soofiyan Atar
  • Chung-Pang Wang
  • Changwei Chen



  • *Ryan Orosco
  • *Emily Funk



Differentiable Rendering-based Pose Estimation for Surgical Robotic Instruments


arXiv preprint arXiv:2503.05953 (2025)
Zekai Liang, Zih-Yun Chiu, Florian Richter, Michael C Yip

Humanoids in Hospitals: A Technical Study of Humanoid Surrogates for Dexterous Medical Interventions


arXiv preprint arXiv:2503.12725 (2025)
Soofiyan Atar, Xiao Liang, Calvin Joyce, Florian Richter, Wood Ricardo, Charles Goldberg, Preetham Suresh, Michael Yip

MEDiC: Autonomous Surgical Robotic Assistance to Maximizing Exposure for Dissection and Cautery


Proc. IEEE International Conference on Robotics and Automation (ICRA) (2025)
Xiao Liang, Chung-Pang Wang, Nikhil Uday Shinde, Fei Liu, Florian Richter, Michael Yip

AutoPeel: Adhesion-aware Safe Peeling Trajectory Optimization for Robotic Wound Care


Proc. IEEE International Conference on Robotics and Automation (ICRA) (2025)
Xiao Liang, Youcheng Zhang, Fei Liu, Florian Richter, Michael Yip BEST PAPER AWARD NOMINATION

JIGGLE: An Active Sensing Framework for Boundary Parameters Estimation in Deformable Surgical Environments


Robotics: Science and Systems (2024)
Nikhil Uday Shinde, Xiao Liang, Fei Liu, Yutong Zhang, Florian Richter, Sylvia Herbert, Michael C Yip

SURESTEP: An Uncertainty-Aware Trajectory Optimization Framework to Enhance Visual Tool Tracking for Robust Surgical Automation


Proc. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (2024)
Nikhil U Shinde, Zih-Yun Chiu, Florian Richter, Jason Lim, Yuheng Zhi, Sylvia Herbert, Michael C Yip

Autonomous Image-to-Grasp Robotic Suturing Using Reliability-Driven Suture Thread Reconstruction


IEEE Robotics and Automation Letters (2025)
Neelay Joglekar, Fei Liu, Florian Richter, Michael C Yip

Bridging Real-to-Sim Gaps through Online Stiffness Optimization with Perception-Enabled Residual Mapping


Proc. Workshop on Integrated Perception, Planning, and Control for Physically and Contextually-Aware Robot Autonomy, IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (2024)
Xiao Liang, Fei Liu, Yutong Zhang, Michael Yip

HemoSet: The First Blood Segmentation Dataset for Automation of Hemostasis Management


Proc. IEEE International Symposium on Medical Robotics (ISMR) (2024)
Albert J Miao, Shan Lin, Jingpei Lu, Florian Richter, Benjamin Ostrander, Emily K Funk, Ryan K Orosco, Michael C Yip

Real-to-Sim Deformable Object Manipulation: Optimizing Physics Models with Residual Mappings for Robotic Surgery


Proc. IEEE International Conference on Robotics and Automation (ICRA) (2024)
Xiao Liang, Fei Liu, Yutong Zhang, Yuelei Li, Shan Lin, Michael Yip

SuPerPM: A Large Deformation-Robust Surgical Perception Framework Based on Deep Point Matching Learned from Physical Constrained Simulation Data


Proc. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (2024)
Shan Lin, Albert J Miao, Ali Alabiad, Fei Liu, Kaiyuan Wang, Jingpei Lu, Florian Richter, Michael C Yip

Robust Surgical Tool Tracking with Pixel-based Probabilities for Projected Geometric Primitives


Proc. IEEE International Conference on Robotics and Automation (ICRA) (2024)
Christopher D'Ambrosia, Florian Richter, Zih-Yun Chiu, Nikhil Shinde, Fei Liu, Henrik I Christensen, Michael C Yip

The current state of autonomous suturing: a systematic review


Surgical Endoscopy (2024)
Benjamin T Ostrander, Daniel Massillon, Leo Meller, Zih-Yun Chiu, Michael Yip, Ryan K Orosco

Suture thread spline reconstruction from endoscopic images for robotic surgery with reliability-driven keypoint detection


Proc. IEEE International Conference on Robotics and Automation (ICRA) (2023)
Neelay Joglekar, Fei Liu, Ryan Orosco, Michael Yip

Real-time constrained 6d object-pose tracking of an in-hand suture needle for minimally invasive robotic surgery


Proc. IEEE International Conference on Robotics and Automation (ICRA) (2023)
Zih-Yun Chiu, Florian Richter, Michael C Yip BEST PAPER AWARD

Artificial intelligence meets medical robotics


Science (2023)
Michael Yip, Septimiu Salcudean, Ken Goldberg, Kaspar Althoefer, Arianna Menciassi, Justin D Opfermann, Axel Krieger, Krithika Swaminathan, Conor J Walsh, He Huang, I-Chieh Lee

BASED: Bundle-adjusting surgical endoscopic dynamic video reconstruction using neural radiance fields


IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) (2025)
Shreya Saha, Sainan Liu, Shan Lin, Jingpei Lu, Michael Yip

Porcine Model for Open Robotic Thyroidectomy


Journal of Robotic Surgery (2022)
Emily Funk, Won Seo Park, Florian Richter, Benjamin T Ostrander, Michael Yip, Philip A Weissbrod, Ryan K Orosco

Semantic-SuPer: A Semantic-aware Surgical Perception Framework for Endoscopic Tissue Identification, Reconstruction, and Tracking


Proc. IEEE International Conference on Robotics and Automation (ICRA) (2022)
Shan Lin, Albert J Miao, Jingpei Lu, Shunkai Yu, Zih-Yun Chiu, Florian Richter, Michael C Yip

Markerless suture needle 6d pose tracking with robust uncertainty estimation for autonomous minimally invasive robotic surgery


Proc. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (2022)
Zih-Yun Chiu, Albert Z Liao, Florian Richter, Bjorn Johnson, Michael C Yip

Real-to-sim registration of deformable soft tissue with position-based dynamics for surgical robot autonomy


Proc. IEEE International Conference on Robotics and Automation (ICRA) (2021)
Fei Liu, Zihan Li, Yunhai Han, Jingpei Lu, Florian Richter, Michael C Yip

Super deep: A surgical perception framework for robotic tissue manipulation using deep learning for feature extraction


Proc. IEEE International Conference on Robotics and Automation (ICRA) (2021)
Jingpei Lu, Ambareesh Jayakumari, Florian Richter, Yang Li, Michael C Yip

Model-predictive control of blood suction for surgical hemostasis using differentiable fluid simulations


Proc. IEEE International Conference on Robotics and Automation (ICRA) (2021)
Jingbin Huang, Fei Liu, Florian Richter, Michael C Yip BEST PAPER AWARD NOMINEE

Optimal multi-manipulator arm placement for maximal dexterity during robotics surgery


Proc. IEEE International Conference on Robotics and Automation (ICRA) (2021)
Mingwei Xu, James Di, Nikhil Das, Michael C Yip

From bench to bedside: The first live robotic surgery on the dVRK to enable remote telesurgery with motion scaling


Proc. IEEE International Symposium on Medical Robotics (ISMR) (2021)
Florian Richter, Emily K Funk, Won Seo Park, Ryan K Orosco, Michael C Yip

Autonomous robotic suction to clear the surgical field for hemostasis using image-based blood flow detection


IEEE Robotics and Automation Letters (2021)
Florian Richter, Shihao Shen, Fei Liu, Jingbin Huang, Emily K Funk, Ryan K Orosco, Michael C Yip

Bimanual regrasping for suture needles using reinforcement learning for rapid motion planning


Proc. IEEE International Conference on Robotics and Automation (ICRA) (2021)
Zih-Yun Chiu, Florian Richter, Emily K Funk, Ryan K Orosco, Michael C Yip

SuPer: A Surgical Perception Framework for Endoscopic Tissue Manipulation with Surgical Robotics


IEEE Robotics and Automation Letters (2020)
Yang Li, Florian Richter, Jingpei Lu, Emily Funk, Ryan Orosco, Jianke Zhu, Michael C Yip

Vibration-based multi-axis force sensing: Design, characterization, and modeling


IEEE Robotics and Automation Letters (2020)
Winnie Kuang, Michael Yip, Jun Zhang

Open-sourced reinforcement learning environments for surgical robotics


arXiv preprint arXiv:1903.02090 (2019)
Florian Richter, Ryan K Orosco, Michael C Yip

A 2D surgical simulation framework for tool-tissue interaction


arXiv preprint arXiv:2010.13936 (2020)
Yunhai Han, Fei Liu, Michael C Yip

Robot Autonomy for Surgery


The Encyclopedia of Medical Robotics (2017)
Michael Yip, Nikhil Das

Autonomous control of continuum robot manipulators for complex cardiac ablation tasks


Journal of Medical Robotics Research (2017)
Michael C Yip, Jake A Sganga, David B Camarillo

Model-less control of a flexible robotic catheter


Workshop on Advances in Flexible Robots for Surgical Interventions, Proc. IEEE International Conference on Robotics and Automation (ICRA) (2014)
Michael C Yip, Paul J. Wang, David B. Camarillo BEST PAPER AWARD

A robust uniaxial force sensor for minimally invasive surgery


IEEE Transactions on Biomedical Engineering (2010)
Michael C Yip, Shelten G Yuen, Robert D Howe

Indirect low-intensity ultrasonic stimulation for tissue engineering


Journal of Tissue Engineering (2010)
Hyoungshin Park, Michael C Yip, Beata Chertok, Joseph Kost, James B Kobler, Robert Langer, Steven M Zeitels

Intra-Operative Laryngoscopic Instrument for Characterizing Vocal Fold Viscoelasticity


Frontiers in Biomedical Devices (2007)
Mark P Ottensmeyer, Michael Yip, Conor J Walsh, James B Kobler, James T Heaton, Steven M Zeitels

Robotic force stabilization for beating heart intracardiac surgery


Proc. Medical Image Computing and Computer-Assisted Intervention (MICCAI) (2009)
Shelten G Yuen, Michael C Yip, Nikolay V Vasilyev, Douglas P Perrin, Pedro J Del Nido, Robert D Howe