Pose estimation for robot manipulators via keypoint optimization and sim-to-real transfer

IEEE Robotics and Automation Letters, 2022

Jingpei Lu, Florian Richter, Michael C Yip

Publisher Link: http://ieeexplore.ieee.org/abstract/document/9714837/

Abstract: Keypoint detection is an essential building block for many robotic applications like motion capture and pose estimation. Historically, keypoints are detected using uniquely engineered markers such as checkerboards or fiducials. More recently, deep learning methods have been explored as they have the ability to detect user-defined keypoints in a marker-less manner. However, different manually selected keypoints can have uneven performance when it comes to detection and localization. An example of this can be found on symmetric robotic tools where DNN detectors cannot solve the correspondence problem correctly. In this work, we propose a new and autonomous way to define the keypoint locations that overcomes these challenges. The approach involves finding the optimal set of keypoints on robotic manipulators for robust visual detection and localization. Using a robotic simulator as a medium, our …

Lu et al. (2022) Pose estimation for robot manipulators via keypoint optimization and sim-to-real transfer, IEEE Robotics and Automation Letters, vol. 7, no. 2, pp. 4622-4629.