Augmented reality predictive displays to help mitigate the effects of delayed telesurgery

Proc. IEEE International Conference on Robotics and Automation (ICRA), 2019

Florian Richter, Yifei Zhang, Yuheng Zhi, Ryan K Orosco, Michael C Yip

Abstract: Surgical robots offer the exciting potential for remote telesurgery, but advances are needed to make this technology efficient and accurate to ensure patient safety. Achieving these goals is hindered by the deleterious effects of latency between the remote operator and the bedside robot. Predictive displays have found success in overcoming these effects by giving the operator immediate visual feedback. However, previously developed predictive displays can not be directly applied to telesurgery due to the unique challenges in tracking the 3D geometry of the surgical environment. In this paper, we present the first predictive display for teleoperated surgical robots. The predicted display is stereoscopic, utilizes Augmented Reality (AR) to show the predicted motions alongside the complex tissue found in-situ within surgical environments, and overcomes the challenges in accurately tracking slave-tools in real-time. We call this a Stereoscopic AR Predictive Display (SARPD). To test the SARPD’s performance, we conducted a user study with ten participants on the da Vincitextregistered{} Surgical System. The results showed with statistical significance that using SARPD decreased time to complete task while having no effect on error rates when operating under delay.

Richter et al. (2019) Augmented reality predictive displays to help mitigate the effects of delayed telesurgery, Proc. IEEE International Conference on Robotics and Automation (ICRA), pp. 444-450.

Pub Link: http://ieeexplore.ieee.org/abstract/document/8794051/
arXiv: http://arxiv.org/pdf/1809.08627v3