InstruMentAR: Auto-Generation of AR Tutorials for Operating Digital Instruments Through Recording Embodied Demonstration

Video tutorials have become a primary means by which people learn how to operate new equipment. Augmented reality (AR) has been emerging in recent years as a means of making these tutorials more immersive for users. Augmented reality tutorials directly superimpose visual guidance on the physical equipment. However, producing these AR tutorials can be both time consuming and frustrating, as creators must constantly alternate between interacting with virtual elements (such as arrows and text boxes) and operating the physical equipment itself. Other methods of AR tutorial production utilize over-head camera systems to track the movement of physical objects and create virtual instructions, but these techniques are limited by the risk of visual occlusion due to the small screen size of most digital devices on which tutorials will be displayed.

To overcome the labor intensity, cognitive load, and visual occlusion issues with these methods, researchers at Purdue University have developed a new method of producing AR tutorials. Their system, called InstruMentAR, combines direct user input, gestural tracking, and voice recordings with a groundbreaking processing system to automatically generate smooth, effective tutorials InstruMentAR eliminates the need for tedious virtual object corrections and ensures a clear line of sight for users at all times. It is also designed with learners in mind. With automatic transitions from one module to the next and feedback when it detects possible errors, InstruMentAR guides learners step by step. Using InstruMentAR, companies will save time and money producing AR tutorials for operating their equipment, while also ensuring that their employees or customers will always have access to clear, high-quality tutorials. With this technology, users will be able to learn any new operating skills with ease.

Technology Validation:

According to the published article, researchers performed a study with a mock-up user interface to test their hand sensor's ability to accurately detect user motion. These trials showed 84-92% success rates with no errors. They also tested InstruMentAR's authoring abilities using an experimental set of users. This test showed an almost 50% reduction in total time to author an AR tutorial compared to a predetermined baseline. In a survey with their test users after the fact, 60% responded that the automated system was "very intuitive and easy to use."

Advantages:

-No need for constant monitoring and adjustment of virtual objects (in terms of size, orientation, location, etc.)

-No need to switch back and forth between virtual interaction and physical manipulation; seamlessly combines these two essential elements of tutorial creation

-No risk of visual occlusion, as may be seen in overhead camera setups

-Automatically generates tutorials

-Streamlined AR tutorial creation is easy, intuitive, and faster than current methods

Applications:

-Useful in AR tutorial creation, with particular emphasis on overlayed tutorials on the small screens of digital devices/machinery.

TRL: 6

Intellectual Property:

Keywords: Augmented reality tutorials, AR tutorial creation, InstruMentAR, gestural tracking AR, automated AR tutorials, virtual guidance, AR overlayed tutorials, operating digital instruments AR, reduce AR tutorial authoring time, Purdue University AR, AR, AR tutorials, Augmented Reality, augmented reality tutorials, authoring, Computer Technology, digital device operation, tutorial, tutorial creation