Bridging Real-to-Sim Gaps through Online Stiffness Optimization with Perception-Enabled Residual Mapping

Proc. Workshop on Integrated Perception, Planning, and Control for Physically and Contextually-Aware Robot Autonomy, IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2024

Xiao Liang, Fei Liu, Yutong Zhang, Michael Yip

Abstract: Accurate deformable object manipulation (DOM) is an essential component for achieving autonomy in robotic surgery, which involves operations on soft tissue. Many DOM algorithms can be powered by simulation, which ensures realistic deformation by adhering to the governing physical constraints. However, real soft objects in robotic surgery have complex, anisotropic physical parameters and topology that a simulation with simple initialization cannot capture. To use the simulation technique in real surgical tasks, the “real-tosim” gap needs to be properly compensated. In this work, we propose to use an optimization-based residual mapping module to close the positional gap between a physics simulation and perceptual observation. We further use this module to guide an adaptive online estimation of the initialized physics parameters of soft bodies using a position-based dynamics (PBD) simulator. The proposed method is able to produce more realistic soft body deformation. The performance of the proposed mechanism is evaluated in the manipulation of a real thin-shell tissue manipulated by the Da Vinci Surgical System.

Liang et al. (2024) Bridging Real-to-Sim Gaps through Online Stiffness Optimization with Perception-Enabled Residual Mapping, Proc. Workshop on Integrated Perception, Planning, and Control for Physically and Contextually-Aware Robot Autonomy, IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 1-5.

Pub Link:
arXiv: