Sensory Substitution

Can sensory data be fed through unusual sensory channels?

And can the brain learn to extract the meaning of such information streams?

Yes and yes. Sensory substitution is a non-invasive technique for circumventing the loss of one sense by feeding its information through another channel. We leveraged this technique to develop a non-invasive, low-cost method to allow people with deafness or hearing impairments to perceive auditory information via small vibrations on their skin.

The original form factor in my laboratory was a vest with vibratory motors stitched into it. It captured sounds and converted them to patterns of vibration on the skin. We had great results in the laboratory and scientific community, which led to my presentation at TED:

The project quickly became funded by venture capitalists and spun out of my lab as a company, Neosensory.

For the larger theoretical ideas behind this project, see a 3 minute video on BigThink: Welcome to Your Future Brain.

With a fantastic team of Silicon Valley engineers, we shrunk the hardware down from a vest to a wristband, which I’m pleased to say is now on wrists all over the world. See our Neosensory blog for the ways this has touched people’s lives.

For a bit of our latest research, see Perrotta, Asgeirsdottir, Eagleman (2021). Deciphering sounds through patterns of vibration on the skin. Neuroscience. 

Beyond deafness, we’ve also launched products for high frequency loss, tinnitus, and balance, with many more product lines coming soon.

The overarching goal of our work is to better understand how sensory streams can be re-packaged into atypical sensory channels to restore perception or give new perceptions. Beyond applications for hearing, this research will contribute to a generalized framework for designing devices to send any kinds of information to the brain for sensory processing via atypical sensory modalities. Such a framework will feed both rehabilitative and assistive technologies

For more on the principles of neural plasticity that underlie this work, please check out my Pulitzer-prize-nominated book on brain plasticity, Livewired.