Multimodal Interface for the Perception of Visual Data by Blind Users

At present, if we look at blind or visually impaired (BVI) people, there is no suitable substitute technology to enable this population to interpret visual scientific data from lab experimentation in real time, such as performing light microscopy, spectrometry, chemical reactions, and using other scientific instruments. Current technologies for this purpose include tactile paper, which is limited because it is not real time, has to be made in advance, requires more staff, and does not allow independent work for blind students.

To address this challenge, Purdue University researchers developed a real-time multimodal image perception system to transform scientific data to something more useful. Image features such as size, color, shape, intensity, location, texture, and opacity are individually mapped through three different sensory channels of hearing, haptics, and vibrotactility. Experimental results have confirmed that this real-time approach provides greater accuracy in recognizing and analyzing objects within images compared to tactile papers. Participants have also shown better analytic performance with this technology compared to other one-sensory standard interfaces. Although this technology may take more time when exploring an image, it also allows higher accuracy during the actual analysis of that information.

Thus, this technology has potential use during lab experimentation for BVI people, which would greatly enhance information interpretation. It would allow advancement in careers in medicine, biology, chemistry, and others alike that has so far been restricted for this population.

Advantages:

-Real-time interface

-Multiple image features

-Three sensory channels

Potential Applications:

-University, government, and corporate labs

TRL: 4

Intellectual Property:

Keywords: Real-time image perception, Multimodal perception system, Visually impaired scientific data, Blind people lab experimentation, Assistive technology BVI, Scientific data interpretation, Hearing haptics vibrotactility, Multimodal sensory channels, Accessible scientific instruments, Real-time sensory feedback