https://pubmed.ncbi.nlm.nih.gov/27801867/ This site needs JavaScript to work properly. Please enable it to take advantage of the complete set of features! Clipboard, Search History, and several other advanced features are temporarily unavailable. Skip to main page content U.S. flag An official website of the United States government Here's how you know Dot gov The .gov means it's official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you're on a federal government site. Https The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely. NIH NLM Logo Log in Show account info Close Account Logged in as: username * Dashboard * Publications * Account settings * Log out Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation pubmed logo Search: [ ] Search Advanced Clipboard User Guide Save Email Send to * Clipboard * My Bibliography * Collections * Citation manager Display options Display options Format [Abstract] Save citation to file Format: [Summary (text) ] Create file Cancel Email citation Subject: 1 selected item: 27801867 - PubMed To: [ ] Format: [Summary ] [ ] MeSH and other data Send email Cancel Add to Collections * ( ) Create a new collection * (*) Add to an existing collection Name your collection: [ ] Name must be less than 100 characters Choose a collection: [] Unable to load your collection due to an error Please try again Add Cancel Add to My Bibliography * ( ) My Bibliography Unable to load your delegates due to an error Please try again Add Cancel Your saved search Name of saved search: [ ] Search terms: [ ] Test search terms Would you like email updates of new search results? Saved Search Alert Radio Buttons * (*) Yes * ( ) No Email: (change) Frequency: [Monthly] Which day? [The first Sunday ] Which day? [Sunday ] Report format: [Summary ] Send at most: [5 items ] [ ] Send even when there aren't any new results Optional text in email: [ ] Save Cancel Create a file for external citation management software Create file Cancel Your RSS Feed Name of RSS Feed: [ ] Number of items displayed: [15 ] Create RSS Cancel RSS Link [ ] Copy Full text links full text provider logo Multidisciplinary Digital Publishing Institute (MDPI) Free PMC article Full text links Actions Cite Favorites Display options Display options Format [Abstract] Share * * * Permalink [https://pubmed.ncbi.]Copy Page navigation * Title & authors * Abstract * Conflict of interest statement * Figures * Similar articles * Cited by * References * MeSH terms * LinkOut - more resources [Title & authors ] Sensors (Basel) Actions * Search in PubMed * Search in NLM Catalog * Add to Search . 2016 Oct 29;16(11):1812. doi: 10.3390/s16111812. Towards Contactless Silent Speech Recognition Based on Detection of Active and Visible Articulators Using IR-UWB Radar Young Hoon Shin^ 1 2 , Jiwon Seo^ 3 4 Affiliations Expand Affiliations * ^1 School of Integrated Technology, College of Engineering, Yonsei University, 85 Songdogwahak-ro, Yeonsu-gu, Incheon 21983, Korea. yh.s@yonsei.ac.kr. * ^2 Yonsei Institute of Convergence Technology, Yonsei University, 85 Songdogwahak-ro, Yeonsu-gu, Incheon 21983, Korea. yh.s@yonsei.ac.kr. * ^3 School of Integrated Technology, College of Engineering, Yonsei University, 85 Songdogwahak-ro, Yeonsu-gu, Incheon 21983, Korea. jiwon.seo@yonsei.ac.kr. * ^4 Yonsei Institute of Convergence Technology, Yonsei University, 85 Songdogwahak-ro, Yeonsu-gu, Incheon 21983, Korea. jiwon.seo@yonsei.ac.kr. * PMID: 27801867 * PMCID: PMC5134471 * DOI: 10.3390/s16111812 Free PMC article Item in Clipboard Towards Contactless Silent Speech Recognition Based on Detection of Active and Visible Articulators Using IR-UWB Radar Young Hoon Shin et al. Sensors (Basel). 2016. Free PMC article Show details Display options Display options Format [Abstract] Sensors (Basel) Actions * Search in PubMed * Search in NLM Catalog * Add to Search . 2016 Oct 29;16(11):1812. doi: 10.3390/s16111812. Authors Young Hoon Shin^ 1 2 , Jiwon Seo^ 3 4 Affiliations * ^1 School of Integrated Technology, College of Engineering, Yonsei University, 85 Songdogwahak-ro, Yeonsu-gu, Incheon 21983, Korea. yh.s@yonsei.ac.kr. * ^2 Yonsei Institute of Convergence Technology, Yonsei University, 85 Songdogwahak-ro, Yeonsu-gu, Incheon 21983, Korea. yh.s@yonsei.ac.kr. * ^3 School of Integrated Technology, College of Engineering, Yonsei University, 85 Songdogwahak-ro, Yeonsu-gu, Incheon 21983, Korea. jiwon.seo@yonsei.ac.kr. * ^4 Yonsei Institute of Convergence Technology, Yonsei University, 85 Songdogwahak-ro, Yeonsu-gu, Incheon 21983, Korea. jiwon.seo@yonsei.ac.kr. * PMID: 27801867 * PMCID: PMC5134471 * DOI: 10.3390/s16111812 Item in Clipboard Full text links Cite Display options Display options Format [Abstract] Abstract People with hearing or speaking disabilities are deprived of the benefits of conventional speech recognition technology because it is based on acoustic signals. Recent research has focused on silent speech recognition systems that are based on the motions of a speaker's vocal tract and articulators. Because most silent speech recognition systems use contact sensors that are very inconvenient to users or optical systems that are susceptible to environmental interference, a contactless and robust solution is hence required. Toward this objective, this paper presents a series of signal processing algorithms for a contactless silent speech recognition system using an impulse radio ultra-wide band (IR-UWB) radar. The IR-UWB radar is used to remotely and wirelessly detect motions of the lips and jaw. In order to extract the necessary features of lip and jaw motions from the received radar signals, we propose a feature extraction algorithm. The proposed algorithm noticeably improved speech recognition performance compared to the existing algorithm during our word recognition test with five speakers. We also propose a speech activity detection algorithm to automatically select speech segments from continuous input signals. Thus, speech recognition processing is performed only when speech segments are detected. Our testbed consists of commercial off-the-shelf radar products, and the proposed algorithms are readily applicable without designing specialized radar hardware for silent speech processing. Keywords: IR-UWB radar; articulators' detection; contactless silent speech recognition. Conflict of interest statement The authors declare no conflict of interest. Figures Figure 1 Figure 1 IR-UWB-radar-based silent speech recognition testbed:... Figure 1 IR-UWB-radar-based silent speech recognition testbed: ( a ) Font view; ( b )... Figure 1 IR-UWB-radar-based silent speech recognition testbed: (a) Font view; (b) Side view with a user. The transmitted signal is reflected by multiple points on and inside the face. IR-UWB radar signals can penetrate the skin. Figure 2 Figure 2 Block diagram of the signal... Figure 2 Block diagram of the signal processing flow of the proposed system. Figure 2 Block diagram of the signal processing flow of the proposed system. Figure 3 Figure 3 Examples of raw received radar... Figure 3 Examples of raw received radar signals corresponding to: ( a ) silent pronunciation... Figure 3 Examples of raw received radar signals corresponding to: (a) silent pronunciation of the word "two"; (b) silent pronunciation of the word "five". The approximate beginning time (about 0.4 s) and end time (about 1.1 s) of the pronunciation of "two" is clearly visible, but they are not very clear for "five" in this raw data. Figure 4 Figure 4 Examples of clutter-reduced signals corresponding... Figure 4 Examples of clutter-reduced signals corresponding to: ( a ) silent pronunciation of the... Figure 4 Examples of clutter-reduced signals corresponding to: (a) silent pronunciation of the word "two"; (b) silent pronunciation of the word "five". The raw radar data is the same as the data in Figure 3. Figure 5 Figure 5 Example clean maps obtained by... Figure 5 Example clean maps obtained by the conventional CLEAN algorithm corresponding to: ( a... Figure 5 Example clean maps obtained by the conventional CLEAN algorithm corresponding to: (a) silent pronunciation of the word "two"; (b) silent pronunciation of the word "five". The raw radar data is the same as the data in Figure 3 and Figure 4. Figure 6 Figure 6 Example clean maps obtained by... Figure 6 Example clean maps obtained by the short-template-based CLEAN algorithm: ( a ) silent... Figure 6 Example clean maps obtained by the short-template-based CLEAN algorithm: (a) silent pronunciation of the word "two"; (b) silent pronunciation of the word "five". The raw radar data is the same as the data in Figure 3, Figure 4 and Figure 5. Unlike the raw data in Figure 3b, the approximate beginning time (about 0.3 s) and end time (about 1.1 s) of the pronunciation of "five" is now clearly visible. Figure 7 Figure 7 Examples of the variance of... Figure 7 Examples of the variance of normalized signal amplitude (raw variance data and smoothed... Figure 7 Examples of the variance of normalized signal amplitude (raw variance data and smoothed data): (a) silent pronunciation of the word "two"; (b) silent pronunciation of the word "five". The raw radar data is the same as the data in Figure 3, Figure 4, Figure 5 and Figure 6. The values above the threshold (horizontal dashed line) indicate the general motion of the speaker. The data between the vertical lines are stored for further processing. Figure 8 Figure 8 Example clean maps with the... Figure 8 Example clean maps with the distance thresholds (horizontal dashed lines): ( a )... Figure 8 Example clean maps with the distance thresholds (horizontal dashed lines): (a) silent pronunciation of the word "two"; (b) silent pronunciation of the word "five". The raw radar data is the same as the data in Figure 3, Figure 4, Figure 5, Figure 6 and Figure 7. Both data within the distance thresholds represent articulator motion, and thus the data segments between the vertical lines are stored. Figure 9 Figure 9 Illustration of the distance matrix... Figure 9 Illustration of the distance matrix and alignment path of the MD-DTW algorithm for... Figure 9 Illustration of the distance matrix and alignment path of the MD-DTW algorithm for two features. Each (i, j) element of the matrix contains a distance value calculated by Equation (8). The alignment path in gray is the path having the minimal total distance value. Figure 10 Figure 10 Comparison of precision, recall, and... Figure 10 Comparison of precision, recall, and F-measure of word recognition with five speakers. Each... Figure 10 Comparison of precision, recall, and F-measure of word recognition with five speakers. Each narrow bar indicates the result of each speaker, and each wide bar and corresponding number indicates the average value over five speakers. All figures (10) See this image and copyright information in PMC Similar articles * Short-Range Vital Signs Sensing Based on EEMD and CWT Using IR-UWB Radar. Hu X, Jin T. Hu X, et al. Sensors (Basel). 2016 Nov 30;16 (12):2025. doi: 10.3390/s16122025. Sensors (Basel). 2016. PMID: 27916877 Free PMC article. * Respiration Based Non-Invasive Approach for Emotion Recognition Using Impulse Radio Ultra Wide Band Radar and Machine Learning. Siddiqui HUR, Shahzad HF, Saleem AA, Khan Khakwani AB, Rustam F, Lee E, Ashraf I, Dudley S. Siddiqui HUR, et al. Sensors (Basel). 2021 Dec 13;21(24):8336. doi: 10.3390/s21248336. Sensors (Basel). 2021. PMID: 34960430 Free PMC article. * Contactless Simultaneous Breathing and Heart Rate Detections in Physical Activity Using IR-UWB Radars. Zhang X, Yang X, Ding Y, Wang Y, Zhou J, Zhang L. Zhang X, et al. Sensors (Basel). 2021 Aug 16;21(16):5503. doi: 10.3390/s21165503. Sensors (Basel). 2021. PMID: 34450945 Free PMC article. * Feature Extraction and Reconstruction by Using 2D-VMD Based on Carrier-Free UWB Radar Application in Human Motion Recognition. Jiang L, Zhou X, Che L, Rong S, Wen H. Jiang L, et al. Sensors (Basel). 2019 Apr 26;19(9):1962. doi: 10.3390/s19091962. Sensors (Basel). 2019. PMID: 31027374 Free PMC article. * An Overview of Signal Processing Techniques for Remote Health Monitoring Using Impulse Radio UWB Transceiver. Khan F, Ghaffar A, Khan N, Cho SH. Khan F, et al. Sensors (Basel). 2020 Apr 27;20(9):2479. doi: 10.3390/s20092479. Sensors (Basel). 2020. PMID: 32349382 Free PMC article. Review. See all similar articles Cited by 5 articles * Silent speech command word recognition using stepped frequency continuous wave radar. Wagner C, Schaffer P, Amini Digehsara P, Barhold M, Plettemeier D, Birkholz P. Wagner C, et al. Sci Rep. 2022 Mar 9;12(1):4192. doi: 10.1038/s41598-022-07842-9. Sci Rep. 2022. PMID: 35273225 Free PMC article. * Exploring Silent Speech Interfaces Based on Frequency-Modulated Continuous-Wave Radar. Ferreira D, Silva S, Curado F, Teixeira A. Ferreira D, et al. Sensors (Basel). 2022 Jan 14;22(2):649. doi: 10.3390/s22020649. Sensors (Basel). 2022. PMID: 35062610 Free PMC article. * Biosignal Sensors and Deep Learning-Based Speech Recognition: A Review. Lee W, Seong JJ, Ozlu B, Shim BS, Marakhimov A, Lee S. Lee W, et al. Sensors (Basel). 2021 Feb 17;21(4):1399. doi: 10.3390/ s21041399. Sensors (Basel). 2021. PMID: 33671282 Free PMC article. Review. * Lane Detection Method with Impulse Radio Ultra-Wideband Radar and Metal Lane Reflectors. Kim DH. Kim DH. Sensors (Basel). 2020 Jan 6;20(1):324. doi: 10.3390/s20010324. Sensors (Basel). 2020. PMID: 31935964 Free PMC article. * Silent Speech Recognition as an Alternative Communication Device for Persons with Laryngectomy. Meltzner GS, Heaton JT, Deng Y, De Luca G, Roy SH, Kline JC. Meltzner GS, et al. IEEE/ACM Trans Audio Speech Lang Process. 2017 Dec;25(12):2386-2398. doi: 10.1109/TASLP.2017.2740000. Epub 2017 Nov 28. IEEE/ACM Trans Audio Speech Lang Process. 2017. PMID: 29552581 Free PMC article. References 1. 1. Juang B.-H., Rabiner L.R. Encyclopedia of Language & Linguistics. 2nd ed. Elsevier; Boston, MA, USA: 2006. Speech Recognition, Automatic: History; pp. 806-819. 2. 1. Denby B., Schultz T., Honda K., Hueber T., Gilbert J.M., Brumberg J.S. Silent speech interfaces. Speech Commun. 2009; 52:270-287. doi: 10.1016/j.specom.2009.08.002. - DOI 3. 1. Schultz T., Wand M. Modeling coarticulation in EMG-based continuous speech recognition. Speech Commun. 2010; 52:341-353. doi: 10.1016/j.specom.2009.12.002. - DOI 4. 1. Wand M., Schulte C., Janke M., Schultz T. Array-based Electromyographic Silent Speech Interface; Proceedings of the 6th International Conference on Bio-Inspired Systems and Signal Processing (BIOSIGNALS); Barcelona, Spain. 11-14 February 2013; pp. 89-96. 5. 1. Wand M., Himmelsbach A., Heistermann T., Janke M., Schultz T. Artifact Removal Algorithm for an EMG-Based Silent Speech Interface; Proceedings of the 35th Annual Conference of the IEEE Engineering in Medicine and Biology Society; Osaka, Japan. 3-7 July 2013; pp. 5750-5753. - PubMed Show all 47 references MeSH terms * Algorithms Actions + Search in PubMed + Search in MeSH + Add to Search * Biosensing Techniques / methods* Actions + Search in PubMed + Search in MeSH + Add to Search * Equipment Design Actions + Search in PubMed + Search in MeSH + Add to Search * Humans Actions + Search in PubMed + Search in MeSH + Add to Search * Radar* Actions + Search in PubMed + Search in MeSH + Add to Search * Signal Processing, Computer-Assisted Actions + Search in PubMed + Search in MeSH + Add to Search * Speech* Actions + Search in PubMed + Search in MeSH + Add to Search LinkOut - more resources * Full Text Sources + Europe PubMed Central + Multidisciplinary Digital Publishing Institute (MDPI) + PubMed Central * Other Literature Sources + scite Smart Citations Full text links [x] full text provider logo Multidisciplinary Digital Publishing Institute (MDPI) Free PMC article [x] Cite Copy Download .nbib Format: [NLM] Send To * Clipboard * Email * Save * My Bibliography * Collections * Citation Manager [x] NCBI Literature Resources MeSH PMC Bookshelf Disclaimer Follow NCBI Connect with NLM * * * National Library of Medicine 8600 Rockville Pike Bethesda, MD 20894 Web Policies FOIA HHS Vulnerability Disclosure Help Accessibility Careers * NLM * NIH * HHS * USA.gov