https://arstechnica.com/science/2021/05/neural-implant-lets-paralyzed-person-type-by-imagining-writing/ Skip to main content * Biz & IT * Tech * Science * Policy * Cars * Gaming & Culture * Store * Forums Subscribe [ ] Close Navigate * Store * Subscribe * Videos * Features * Reviews * RSS Feeds * Mobile Site * About Ars * Staff Directory * Contact Us * Advertise with Ars * Reprints Filter by topic * Biz & IT * Tech * Science * Policy * Cars * Gaming & Culture * Store * Forums Settings Front page layout Grid List Site theme Black on white White on black Sign in Comment activity Sign up or login to join the discussions! [ ] [ ] [Submit] [ ] Stay logged in | Having trouble? Sign up to comment and more Sign up Touchless typing -- Neural implant lets paralyzed person type by imagining writing A paralyzed individual hit 90 characters per minute, 99% accuracy. John Timmer - May 12, 2021 5:03 pm UTC An artist's schematic of the system. Enlarge / An artist's schematic of the system. Nature reader comments 61 with 47 posters participating Share this story * Share on Facebook * Share on Twitter * Share on Reddit Elon Musk's Neuralink has been making waves on the technology side of neural implants, but it hasn't yet shown how we might actually use implants. For now, demonstrating the promise of implants remains in the hands of the academic community. Further Reading The big advance in Elon Musk's Pong-playing monkey is what you can't see This week, the academic community provided a rather impressive example of the promise of neural implants. Using an implant, a paralyzed individual managed to type out roughly 90 characters per minute simply by imagining that he was writing those characters out by hand. Dreaming is doing Previous attempts at providing typing capabilities to paralyzed people via implants have involved giving subjects a virtual keyboard and letting them maneuver a cursor with their mind. The process is effective but slow, and it requires the user's full attention, as the subject has to track the progress of the cursor and determine when to perform the equivalent of a key press. It also requires the user to spend the time to learn how to control the system. But there are other possible routes to getting characters out of the brain and onto the page. Somewhere in our writing thought process, we form the intention of using a specific character, and using an implant to track this intention could potentially work. Unfortunately, the process is not especially well understood. Downstream of that intention, a decision is transmitted to the motor cortex, where it's translated into actions. Again, there's an intent stage, where the motor cortex determines it will form the letter (by typing or writing, for example), which is then translated into the specific muscle motions required to perform the action. These processes are much better understood, and they're what the research team targeted for their new work. Advertisement Specifically, the researchers placed two implants in the premotor cortex of a paralyzed person. This area is thought to be involved in forming the intentions to perform movements. Catching these intentions is much more likely to produce a clear signal than catching the movements themselves, which are likely to be complex (any movement involves multiple muscles) and depend on context (where your hand is relative to the page you're writing on, etc.). With the implants in the right place, the researchers asked the participant to imagine writing letters on a page and recorded the neural activity as he did so. What was that again? Altogether, there were roughly 200 electrodes in the participant's premotor cortex. Not all of them were informative for letter-writing. But for those that were, the authors performed a principal component analysis, which identified the features of the neural recordings that differed the most when various letters were imagined. Converting these recordings into a two-dimensional plot, it was obvious that the activity seen when writing a single character always clustered together. And physically similar characters--p and b, for example, or h, n, and r--formed clusters near each other. (The researchers also asked the participant to do punctuation marks like a comma and question mark and used a > to indicate a space and a tilde for a period.) Overall, the researchers found they could decipher the appropriate character with an accuracy of a bit over 94 percent, but the system required a relatively slow analysis after the neural data was recorded. To get things working in real time, the researchers trained a recurrent neural network to estimate the probability of a signal corresponding to each letter. Despite working with a relatively small amount of data (only 242 sentences' worth of characters), the system worked remarkably well. The lag between the thought and a character appearing on screen was only about half a second, and the participant was able to produce about 90 characters per minute, easily topping the previous record for implant-driven typing, which was about 25 characters per minute. The raw error rate was only about 5 percent, and applying a system like a typing autocorrect could drop the error rate down to only 1 percent. Advertisement The tests were all done with prepared sentences. Once the system was validated, however, the researchers asked the participant to type out free-form answers to questions. Here, the speed went down a bit (to 75 characters a minute) and errors went up to 2 percent after autocorrection, but the system still worked. Not even a prototype As the researchers themselves put it, this "is not yet a complete, clinically viable system." To begin with, it has only been used in a single individual, so we have no idea how well it might work for others. The simplified alphabet used here doesn't contain any digits, capital letters, or most forms of punctuation. And the behavior of the implants changes over time, perhaps because of minor shifts relative to the neurons they read or the build-up of scar tissue, so the system had to be recalibrated regularly--at least once per week to maintain a tolerable error rate. That said, the system shows a very significant speed boost compared to previous implant-driven systems, and the accuracy is quite good. The system also has the potential to be similar to touch-typing, in that a user doesn't have to actually visually focus on letter production, allowing more normal interactions with the user's surroundings. The letter issue might be solved in part by using an alternate alphabet designed by the researchers, in which all the letters are defined by dissimilar patterns of strokes. There's a lot of potential here. The experiments also provide a reminder of the potential of these implants more generally and why companies might start finding the technology worth commercializing. Nature, 2021. DOI: 10.1038/s41586-021-03506-2 (About DOIs). reader comments 61 with 47 posters participating Share this story * Share on Facebook * Share on Twitter * Share on Reddit John Timmer John became Ars Technica's science editor in 2007 after spending 15 years doing biology research at places like Berkeley and Cornell. Email jtimmer@arstechnica.com // Twitter @j_timmer Advertisement You must login or create an account to comment. Channel Ars Technica - Previous story Next story - Related Stories Sponsored Stories Powered by Today on Ars * Store * Subscribe * About Us * RSS Feeds * View Mobile Site * Contact Us * Staff * Advertise with us * Reprints Newsletter Signup Join the Ars Orbital Transmission mailing list to get weekly updates delivered to your inbox. Sign me up - CNMN Collection WIRED Media Group (c) 2021 Conde Nast. All rights reserved. Use of and/or registration on any portion of this site constitutes acceptance of our User Agreement (updated 1/1/20) and Privacy Policy and Cookie Statement (updated 1/1 /20) and Ars Technica Addendum (effective 8/21/2018). Ars may earn compensation on sales from links on this site. Read our affiliate link policy. Your California Privacy Rights | Do Not Sell My Personal Information The material on this site may not be reproduced, distributed, transmitted, cached or otherwise used, except with the prior written permission of Conde Nast. Ad Choices