Pose Relation Transformer Refine Occlusions for Human Pose Estimation

Researchers at Purdue University have developed a method for refining occlusions in human pose estimation. Accurately determining human pose is essential for many robotics applications. However, existing pose estimation methods are limited by poor performance when occlusion occurs. In a single-view camera setup, various occlusions, such as self-occlusion, occlusion by an object, and being out of frame occur, confusing keypoint detectors in pose estimation methods. It is therefore imperative that more inexpensive, accurate, and efficient solutions for human pose estimation are generated.

This novel pose relation transformer developed by Purdue University researchers mitigates the effects of occlusions to provide a more reliable solution for human pose estimation. The technology can be leveraged in conjunction with any existing keypoint detector for very low computational costs. Occluded joints in keypoint detectors tend to have lower confidence and higher errors, but this refinement technology instead reduces errors by replacing joints with the reconstructed joints. Moreover, this pose relation transformer requires no additional end-to-end training or finetuning after being combined with an existing keypoint detector.

Technology Validation:

To demonstrate the effectiveness of the pose relation transformer in refining occluded joints, the pose relation transformer was evaluated on four datasets that cover various occlusion scenarios. It was found that the pose relation transformer improved the performance of existing keypoint detectors and the pose estimation accuracy of existing human pose estimation methods up to 16% with only an additional 5% of parameters, compared to the existing keypoint detectors alone.

Advantages:

-Can be incorporated into a wide variety of systems that require human pose estimation

-Lightweight and inexpensive (transformer uses locations of joints instead of images)

-Significantly reduces errors compared to keypoint detectors alone

-Robust against joint occlusion

Applications:

-Human-robot interaction and hand-object interaction in AR/VR

-Imitation learning for dexterous manipulation

-Demonstration learning

-Robotics system

-Augmented/virtual/mixed reality system

TRL: 4

Intellectual Property:

Keywords: Human pose estimation, Pose Relation Transformer, occlusions refinement, robotics applications, keypoint detector, AR/VR, imitation learning, demonstration learning, mixed reality, computer vision, Artificial Intelligence, Computer Technology, estimation, hands, Machine Learning, Mechanical Engineering, occlusions, pose, prediction, reconstruction