https://news.mit.edu/2024/mouth-based-touchpad-augmental-0605 Skip to content | Massachusetts Institute of Technology MIT Top Menu| * Education * Research * Innovation * Admissions + Aid * Campus Life * News * Alumni * About MIT * More | Search MIT Search websites, locations, and people [ ] See More Results Suggestions or feedback? MIT News | Massachusetts Institute of Technology Subscribe to MIT News newsletter Browse Enter keywords to search for news articles: [ ] Submit Browse By Topics View All - Explore: * Machine learning * Social justice * Startups * Black holes * Classes and programs Departments View All - Explore: * Aeronautics and Astronautics * Brain and Cognitive Sciences * Architecture * Political Science * Mechanical Engineering Centers, Labs, & Programs View All - Explore: * Abdul Latif Jameel Poverty Action Lab (J-PAL) * Picower Institute for Learning and Memory * Media Lab * Lincoln Laboratory Schools * School of Architecture + Planning * School of Engineering * School of Humanities, Arts, and Social Sciences * Sloan School of Management * School of Science * MIT Schwarzman College of Computing View all news coverage of MIT in the media - Listen to audio content from MIT News - Subscribe to MIT newsletter - Close Breadcrumb 1. MIT News 2. Mouth-based touchpad enables people living with paralysis to interact with computers Mouth-based touchpad enables people living with paralysis to interact with computers The startup Augmental allows users to operate phones and other devices using their tongue, mouth, and head gestures. Watch Video Zach Winn | MIT News Publication Date: June 5, 2024 Press Inquiries Press Contact: Abby Abazorius Email: abbya@mit.edu Phone: 617-253-2709 MIT News Office Media Download A rendering of the device, like a mouth retainer with a gold sensor on the palate | Download Image Caption: The MouthPad allows users to interact with phones and computers using their tongue and other head gestures. Credits: Image: Courtesy of Augmental *Terms of Use: Images for download on the MIT News office website are made available to non-commercial entities, press and the general public under a Creative Commons Attribution Non-Commercial No Derivatives license. You may not alter the images provided, other than to crop them to size. A credit line must be used when reproducing images; if one is not provided below, credit the images to "MIT." Close A rendering of the device, like a mouth retainer with a gold sensor on the palate. Caption: The MouthPad allows users to interact with phones and computers using their tongue and other head gestures. Credits: Image: Courtesy of Augmental Previous image Next image When Tomas Vega SM '19 was 5 years old, he began to stutter. The experience gave him an appreciation for the adversity that can come with a disability. It also showed him the power of technology. "A keyboard and a mouse were outlets," Vega says. "They allowed me to be fluent in the things I did. I was able to transcend my limitations in a way, so I became obsessed with human augmentation and with the concept of cyborgs. I also gained empathy. I think we all have empathy, but we apply it according to our own experiences." Vega has been using technology to augment human capabilities ever since. He began programming when he was 12. In high school, he helped people manage disabilities including hand impairments and multiple sclerosis. In college, first at the University of California at Berkeley and then at MIT, Vega built technologies that helped people with disabilities live more independently. Today Vega is the co-founder and CEO of Augmental, a startup deploying technology that lets people with movement impairments seamlessly interact with their personal computational devices. Augmental's first product is the MouthPad, which allows users to control their computer, smartphone, or tablet through tongue and head movements. The MouthPad's pressure-sensitive touch pad sits on the roof of the mouth, and, working with a pair of motion sensors, translates tongue and head gestures into cursor scrolling and clicks in real time via Bluetooth. "We have a big chunk of the brain that is devoted to controlling the position of the tongue," Vega explains. "The tongue comprises eight muscles, and most of the muscle fibers are slow-twitch, which means they don't fatigue as quickly. So, I thought why don't we leverage all of that?" People with spinal cord injuries are already using the MouthPad every day to interact with their favorite devices independently. One of Augmental's users, who is living with quadriplegia and studying math and computer science in college, says the device has helped her write math formulas and study in the library -- use cases where other assistive speech-based devices weren't appropriate. "She can now take notes in class, she can play games with her friends," Vega says. "She is more independent. Her mom told us that getting the MouthPad was the most significant moment since her injury." That's the ultimate goal of Augmental: to improve the accessibility of technologies that have become an integral part of our lives. "We hope that a person with a severe hand impairment can be as competent using a phone or tablet as somebody using their hands," Vega says. Video thumbnail Play video Making computers more accessible In 2012, as a first-year student at UC Berkeley, Vega met his eventual Augmental co-founder, Corten Singer. That year, he told Singer he was determined to join the Media Lab as a graduate student, something he achieved four years later when he joined the Media Lab's Fluid Interfaces research group run by Pattie Maes, MIT's Germeshausen Professor of Media Arts and Sciences. "I only applied to one program for grad school, and that was the Media Lab," Vega says. "I thought it was the only place where I could do what I wanted to do, which is augmenting human ability." At the Media Lab, Vega took classes in microfabrication, signal processing, and electronics. He also developed wearable devices to help people access information online, improve their sleep, and regulate their emotions. "At the Media Lab, I was able to apply my engineering and neuroscience background to build stuff, which is what I love doing the most," Vega says. "I describe the Media Lab as Disneyland for makers. I was able to just play, and to explore without fear." Vega had gravitated toward the idea of a brain-machine interface, but an internship at Neuralink made him seek out a different solution. "A brain implant has the highest potential for helping people in the future, but I saw a number of limitations that pushed me from working on it right now," Vega says. "One is the long timeline for development. I've made so many friends over the past years that needed a solution yesterday." At MIT, he decided to build a solution with all the potential of a brain implant but without the limitations. In his last semester at MIT, Vega built what he describes as "a lollipop with a bunch of sensors" to test the mouth as a medium for computer interaction. It worked beautifully. "At that point, I called Corten, my co-founder, and said, 'I think this has the potential to change so many lives,'" Vega says. "It could also change the way humans interact with computers in the future." Vega used MIT resources including the Venture Mentoring Service, the MIT I-Corps program, and received crucial early funding from MIT's E14 Fund. Augmental was officially born when Vega graduated from MIT at the end of 2019. Augmental generates each MouthPad design using a 3D model based on a scan of the user's mouth. The team then 3-D prints the retainer using dental-grade materials and adds the electronic components. With the MouthPad, users can scroll up, down, left, and right by sliding their tongue. They can also right click by doing a sipping gesture and left click by pressing on their palate. For people with less control of their tongue, bites, clenches, and other gestures can be used, and people with more neck control can use head-tracking to move the cursor on their screen. "Our hope is to create an interface that is multimodal, so you can choose what works for you," Vega says. "We want to be accommodating to every condition." Scaling the MouthPad Many of Augmental's current users have spinal cord injuries, with some users unable to move their hands and others unable to move their heads. Gamers and programmers have also used the device. The company's most frequent users interact with the MouthPad every day for up to nine hours. "It's amazing because it means that it has really seamlessly integrated into their lives, and they are finding lots of value in our solution," Vega says. Augmental is hoping to gain U.S. Food and Drug Administration clearance over the next year to help users do things like control wheelchairs and robotic arms. FDA clearance will also unlock insurance reimbursements for users, which will make the product more accessible. Augmental is already working on the next version of its system, which will respond to whispers and even more subtle movements of internal speech organs. "That's crucial to our early customer segment because a lot of them have lost or have impaired lung function," Vega says. Vega is also encouraged by progress in AI agents and the hardware that goes with them. No matter how the digital world evolves, Vega believes Augmental can be a tool that can benefit everyone. "What we hope to provide one day is an always-available, robust, and private interface to intelligence," Vega says. "We think that this is the most expressive, wearable, hands-free input system that humans have created." Share this news article on: * X * Facebook * LinkedIn * Reddit * Print Related Links * Augmental * Video: "The tongue controlled touchpad helping those with physical impairments access the online world" * Media Lab * School of Architecture and Planning Related Topics * Research * Assistive technology * Computer science and technology * Human-computer interaction * Innovation and Entrepreneurship (I&E) * Alumni/ae * Machine learning * Media Lab * Startups * Artificial intelligence * School of Architecture and Planning Related Articles Arnav Kapur, a researcher in the Fluid Interfaces group at the MIT Media Lab, demonstrates the AlterEgo project. Computer system transcribes words users "speak silently" Katy Olesnavage of MIT is a $15,000 Lemelson-MIT "Cure it!" Graduate Winner for her method to design a better prosthetic foot. Collegiate inventors awarded 2017 Lemelson-MIT Student Prize Researchers at the MIT Media Lab have created a finger-worn device with a built-in camera that can convert text to speech for the visually impaired. Finger-mounted reading device for the blind Previous item Next item More MIT News At left, a chip micrograph labeled as about 6 millimeters wide with areas labeled "pixel array," "amplifiers and adcs," and "digital controls and readout." At right, an inset shows four square pixels, each about 8 microns wide, which are mostly black with a thin strip on the left labeled "circuits." With programmable pixels, novel sensor improves imaging of neural activity New camera chip design allows for optimizing each pixel's timing to maximize signal-to-noise ratio when tracking real-time visual indicator of neural voltage. Read full story - Nergis Mavalvala sits at her desk with bookshelves in the background Featured video: Researchers discuss queer visibility in academia In "Scientific InQueery," LGBTQ+ MIT faculty and graduate students describe finding community and living their authentic lives in the research enterprise. Read full story - A DNA strand is trapped inside a piece of amber. Scientists preserve DNA in an amber-like polymer With their "T-REX" method, DNA embedded in the polymer could be used for long-term storage of genomes or digital data such as photos and music. Read full story - Headshot of Bob Prior Bob Prior: A deep legacy of cultivating books at the MIT Press After 36 years and hundreds of titles, the executive editor reflects on his career as a "champion of rigorous and brilliant scholarship." Read full story - A woman speaks to a packed room of people. A large screen behind her displays a blue slide with the words, "Starting foundational AI for brain research." Symposium highlights scale of mental health crisis and novel methods of diagnosis and treatment Co-hosted by the McGovern Institute, MIT Open Learning, and others, the symposium stressed emerging technologies in advancing understanding of mental health and neurological conditions. Read full story - A person looks into a head-shaped opening, which reveals many people walking Just thinking about a location activates mental maps in the brain MIT neuroscientists have found that the brain uses the same cognitive representations whether navigating through space physically or mentally. Read full story - * More news on MIT News homepage - More about MIT News at Massachusetts Institute of Technology This website is managed by the MIT News Office, part of the Institute Office of Communications. News by Schools/College: * School of Architecture and Planning * School of Engineering * School of Humanities, Arts, and Social Sciences * MIT Sloan School of Management * School of Science * MIT Schwarzman College of Computing Resources: * About the MIT News Office * MIT News Press Center * Terms of Use * Press Inquiries * Filming Guidelines * RSS Feeds Tools: * Subscribe to MIT Daily/Weekly * Subscribe to press releases * Submit campus news * Guidelines for campus news contributors * Guidelines on generative AI Massachusetts Institute of Technology MIT Top Level Links: * Education * Research * Innovation * Admissions + Aid * Campus Life * News * Alumni * About MIT * Join us in building a better world. Massachusetts Institute of Technology 77 Massachusetts Avenue, Cambridge, MA, USA Recommended Links: * Visit * Map (opens in new window) * Events (opens in new window) * People (opens in new window) * Careers (opens in new window) * Contact * Privacy * Accessibility * + Social Media Hub + MIT on X + MIT on Facebook + MIT on YouTube + MIT on Instagram