Real-to-sim registration of deformable soft tissue with position-based dynamics for surgical robot autonomy

Proc. IEEE International Conference on Robotics and Automation (ICRA), 2021

Fei Liu, Zihan Li, Yunhai Han, Jingpei Lu, Florian Richter, Michael C Yip

Abstract: Autonomy in robotic surgery is very challenging in unstructured environments, especially when interacting with deformable soft tissues. The main difficulty is to generate model-based control methods that account for deformation dynamics during tissue manipulation. Previous works in vision-based perception can capture the geometric changes within the scene, however, model-based controllers integrated with dynamic properties, a more accurate and safe approach, has not been studied before. Considering the mechanic coupling between the robot and the environment, it is crucial to develop a registered, simulated dynamical model. In this work, we propose an online, continuous, real-to-sim registration method to bridge 3D visual perception with position-based dynamics (PBD) modeling of tissues. The PBD method is employed to simulate soft tissue dynamics as well as rigid tool interactions for model-based control. Meanwhile, a vision-based strategy is used to generate 3D reconstructed point cloud surfaces based on real-world manipulation, so as to register and update the simulation. To verify this real-to-sim approach, tissue experiments have been conducted on the da Vinci Research Kit. Our real-to-sim approach successfully reduces registration error online, which is especially important for safety during autonomous control. Moreover, it achieves higher accuracy in occluded areas than fusion-based reconstruction.

Liu et al. (2021) Real-to-sim registration of deformable soft tissue with position-based dynamics for surgical robot autonomy, Proc. IEEE International Conference on Robotics and Automation (ICRA), pp. 12328-12334.

Pub Link: http://ieeexplore.ieee.org/abstract/document/9561177/
arXiv: