Multi-Spectral Method for First-Person View Hand Segmentation during Activity Actions with Objects and Tools

Researchers at Purdue University have developed a new method for hand segmentation in augmented reality and virtual reality to address the challenge in robot-human interaction where machines do not often understand the way human hands grip infinitely complex objects. Current technologies do not adapt to every possible degree of freedom of hand movement, nor recognize or classify grasping and using of objects with accuracy. The Purdue University approach overcomes these limitations therefore allowing the camera to distinguish various characteristics of human hands. Researchers were able to obtain seventy percent accuracy in comparison between manual hand motion and computer vision in a test involving individuals clasping thirty-six objects and using thirty different tools, where over four hundred thousand frames were collected. This resulted in a four percent improvement in Intersection over Union (IoU) with thirty-percent fewer parameters compared to similar machine learning architectures.

Advantages:

-Rapid human robotic communication

-Automated manufacturing

-Improved performance

-Fewer parameters than other architectures

Potential Applications:

-Machine Learning

-Pedestrian Detection

-Biological Image Processing

-Autonomous Vehicles

TRL: 4

Intellectual Property:

Keywords: Hand segmentation, augmented reality, virtual reality, robot-human interaction, machine learning architectures, automated manufacturing, computer vision, gesture recognition, hand tracking, human-robot interaction, Augmented Reality, Automotive, Computer Programming, Computer Technology, Computer Vision, Data Processing, Data Visualization, Hand, Human Body Communication, Machine Learning, Mechanical Engineering, Motion Capture, Pattern Recognition, Pixels, Robotic Task Planning, Scalable Video Coding, Sequential Task Authoring, Thermal, Training, Virtual Reality