https://github.com/snakers4/silero-models Skip to content Sign up * Product + Features + Mobile + Actions + Codespaces + Packages + Security + Code review + Issues + Integrations + GitHub Sponsors + Customer stories * Team * Enterprise * Explore + Explore GitHub + Learn and contribute + Topics + Collections + Trending + Skills + GitHub Sponsors + Open source guides + Connect with others + The ReadME Project + Events + Community forum + GitHub Education + GitHub Stars program * Marketplace * Pricing + Plans + Compare plans + Contact Sales + Education [ ] * # In this repository All GitHub | Jump to | * No suggested jump to results * # In this repository All GitHub | Jump to | * # In this user All GitHub | Jump to | * # In this repository All GitHub | Jump to | Sign in Sign up {{ message }} snakers4 / silero-models Public * * Notifications * Fork 152 * Star 2.4k Silero Models: pre-trained speech-to-text, text-to-speech and text-enhancement models made embarrassingly simple License View license 2.4k stars 152 forks Star Notifications * Code * Issues 5 * Pull requests 1 * Discussions * Actions * Wiki * Security * Insights More * Code * Issues * Pull requests * Discussions * Actions * Wiki * Security * Insights snakers4/silero-models This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository. master Switch branches/tags [ ] Branches Tags Could not load branches Nothing to show {{ refName }} default View all branches Could not load tags Nothing to show {{ refName }} default View all tags 9 branches 6 tags Code Latest commit @snakers4 snakers4 Update setup.cfg ... 555024c Jun 12, 2022 Update setup.cfg 555024c Git stats * 251 commits Files Permalink Failed to load latest commit information. Type Name Latest commit message Commit time .github Small fix for build_delpoy workflow Feb 28, 2022 files Add logo Sep 27, 2020 src/silero Fix models.yml loading Jun 12, 2022 .gitignore Migrated to standard python package template Feb 22, 2022 CODE_OF_CONDUCT.md Create CODE_OF_CONDUCT.md Sep 17, 2020 LICENSE Change license to CC BY-NC-SA Apr 23, 2022 README.md Update README.md Jun 6, 2022 changelog.md Add changelog Jun 6, 2022 colab_utils.py pip: Fix broken directory bug Feb 27, 2022 examples.ipynb pip: Fix broken directory bug Feb 27, 2022 examples_te.ipynb Fix yaml file Oct 6, 2021 examples_tts.ipynb Fx Jun 6, 2022 hubconf.py pip: Fix broken directory bug Feb 27, 2022 models.yml Fx Jun 6, 2022 pyproject.toml Migrated to standard python package template Feb 22, 2022 requirements.txt Migrated to standard python package template Feb 22, 2022 setup.cfg Update setup.cfg Jun 12, 2022 View code [ ] Silero Models Installation and Basics Speech-To-Text Dependencies PyTorch ONNX TensorFlow Text-To-Speech Models and Speakers V3 Dependencies PyTorch Standalone Use SSML Indic languages Example Supported languages Text-Enhancement Dependencies Standalone Use FAQ Wiki Performance and Quality Adding new Languages Contact Get in Touch Commercial Inquiries Citations Further reading English Chinese Russian Donations README.md Mailing list : test Mailing list : test License: CC BY-NC 4.0 Donations Backers Sponsors Build and Deploy to PyPI PyPI version header * Silero Models + Installation and Basics + Speech-To-Text o Dependencies o PyTorch o ONNX o TensorFlow + Text-To-Speech o Models and Speakers o Dependencies o PyTorch o Standalone Use o SSML o Indic languages + Text-Enhancement o Dependencies o Standalone Use + FAQ o Wiki o Performance and Quality o Adding new Languages + Contact o Get in Touch o Commercial Inquiries + Citations + Further reading o English o Chinese o Russian + Donations Silero Models Silero Models: pre-trained enterprise-grade STT / TTS models and benchmarks. Enterprise-grade STT made refreshingly simple (seriously, see benchmarks). We provide quality comparable to Google's STT (and sometimes even better) and we are not Google. As a bonus: * No Kaldi; * No compilation; * No 20-step instructions; Also we have published TTS models that satisfy the following criteria: * One-line usage; * A large library of voices; * A fully end-to-end pipeline; * Naturally sounding speech; * No GPU or training required; * Minimalism and lack of dependencies; * Faster than real-time on one CPU thread (!!!); * Support for 16kHz and 8kHz out of the box; Also we have published a model for text repunctuation and recapitalization that: * Inserts capital letters and basic punctuation marks (dot, comma, hyphen, question mark, exclamation mark, dash for Russian); * Works for 4 languages (Russian, English, German, Spanish) and can be extended; * By design is domain agnostic and is not based on any hard-coded rules; * Has non-trivial metrics and succeeds in the task of improving text readability; Installation and Basics You can basically use our models in 3 flavours: * Via PyTorch Hub: torch.hub.load(); * Via pip: pip install silero and then import silero; * Via caching the required models and utils manually and modifying if necessary; Models are downloaded on demand both by pip and PyTorch Hub. If you need caching, do it manually or via invoking a necessary model once (it will be downloaded to a cache folder). Please see these docs for more information. PyTorch Hub and pip package are based on the same code. Hence all examples, historically based on torch.hub.load can be used with a pip-package via this basic change: # before torch.hub.load(repo_or_dir='snakers4/silero-models', model='silero_stt', # or silero_tts or silero_te **kwargs) # after from silero import silero_stt, silero_tts, silero_te silero_stt(**kwargs) Speech-To-Text All of the provided models are listed in the models.yml file. Any meta-data and newer versions will be added there. Screenshot_1 Currently we provide the following checkpoints: PyTorch ONNX Quantization Quality Colab English (en_v6) [?] [?] [?] link Open In Colab English (en_v5) [?] [?] [?] link Open In Colab German (de_v4) [?] [?] [?] link Open In Colab English (en_v3) [?] [?] [?] link Open In Colab German (de_v3) [?] [?] [?] link Open In Colab German (de_v1) [?] [?] [?] link Open In Colab Spanish (es_v1) [?] [?] [?] link Open In Colab Ukrainian (ua_v3) [?] [?] [?] N/A Open In Colab Model flavours: jit jit jit jit jit_q jit_q onnx onnx onnx onnx xsmall small large xlarge xsmall small xsmall small large xlarge English [?] [?] [?] [?] [?] en_v6 English [?] [?] [?] [?] [?] en_v5 English [?] [?] en_v4_0 English [?] [?] [?] [?] [?] [?] [?] [?] en_v3 German [?] [?] de_v4 German [?] de_v3 German [?] [?] de_v1 Spanish [?] [?] es_v1 Ukrainian [?] [?] [?] ua_v3 Dependencies * All examples: + torch, 1.8+ (used to clone the repo in tf and onnx examples), breaking changes for version older than 1.6 + torchaudio, latest version bound to PyTorch should work + omegaconf, latest just should work * Additional for ONNX examples: + onnx, latest just should work + onnxruntime, latest just should work * Additional for TensorFlow examples: + tensorflow, latest just should work + tensorflow_hub, latest just should work Please see the provided Colab for details for each example below. All examples are maintained to work with the latest major packaged versions of the installed libraries. PyTorch Open In Colab Open on Torch Hub import torch import zipfile import torchaudio from glob import glob device = torch.device('cpu') # gpu also works, but our models are fast enough for CPU model, decoder, utils = torch.hub.load(repo_or_dir='snakers4/silero-models', model='silero_stt', language='en', # also available 'de', 'es' device=device) (read_batch, split_into_batches, read_audio, prepare_model_input) = utils # see function signature for details # download a single file, any format compatible with TorchAudio torch.hub.download_url_to_file('https://opus-codec.org/static/examples/samples/speech_orig.wav', dst ='speech_orig.wav', progress=True) test_files = glob('speech_orig.wav') batches = split_into_batches(test_files, batch_size=10) input = prepare_model_input(read_batch(batches[0]), device=device) output = model(input) for example in output: print(decoder(example.cpu())) ONNX Open In Colab You can run our model everywhere, where you can import the ONNX model or run ONNX runtime. import onnx import torch import onnxruntime from omegaconf import OmegaConf language = 'en' # also available 'de', 'es' # load provided utils _, decoder, utils = torch.hub.load(repo_or_dir='snakers4/silero-models', model='silero_stt', language=language) (read_batch, split_into_batches, read_audio, prepare_model_input) = utils # see available models torch.hub.download_url_to_file('https://raw.githubusercontent.com/snakers4/silero-models/master/models.yml', 'models.yml') models = OmegaConf.load('models.yml') available_languages = list(models.stt_models.keys()) assert language in available_languages # load the actual ONNX model torch.hub.download_url_to_file(models.stt_models.en.latest.onnx, 'model.onnx', progress=True) onnx_model = onnx.load('model.onnx') onnx.checker.check_model(onnx_model) ort_session = onnxruntime.InferenceSession('model.onnx') # download a single file, any format compatible with TorchAudio torch.hub.download_url_to_file('https://opus-codec.org/static/examples/samples/speech_orig.wav', dst ='speech_orig.wav', progress=True) test_files = ['speech_orig.wav'] batches = split_into_batches(test_files, batch_size=10) input = prepare_model_input(read_batch(batches[0])) # actual onnx inference and decoding onnx_input = input.detach().cpu().numpy() ort_inputs = {'input': onnx_input} ort_outs = ort_session.run(None, ort_inputs) decoded = decoder(torch.Tensor(ort_outs[0])[0]) print(decoded) TensorFlow Open In Colab SavedModel example import os import torch import subprocess import tensorflow as tf import tensorflow_hub as tf_hub from omegaconf import OmegaConf language = 'en' # also available 'de', 'es' # load provided utils using torch.hub for brevity _, decoder, utils = torch.hub.load(repo_or_dir='snakers4/silero-models', model='silero_stt', language=language) (read_batch, split_into_batches, read_audio, prepare_model_input) = utils # see available models torch.hub.download_url_to_file('https://raw.githubusercontent.com/snakers4/silero-models/master/models.yml', 'models.yml') models = OmegaConf.load('models.yml') available_languages = list(models.stt_models.keys()) assert language in available_languages # load the actual tf model torch.hub.download_url_to_file(models.stt_models.en.latest.tf, 'tf_model.tar.gz') subprocess.run('rm -rf tf_model && mkdir tf_model && tar xzfv tf_model.tar.gz -C tf_model', shell=True, check=True) tf_model = tf.saved_model.load('tf_model') # download a single file, any format compatible with TorchAudio torch.hub.download_url_to_file('https://opus-codec.org/static/examples/samples/speech_orig.wav', dst ='speech_orig.wav', progress=True) test_files = ['speech_orig.wav'] batches = split_into_batches(test_files, batch_size=10) input = prepare_model_input(read_batch(batches[0])) # tf inference res = tf_model.signatures["serving_default"](tf.constant(input.numpy()))['output_0'] print(decoder(torch.Tensor(res.numpy())[0])) Text-To-Speech Models and Speakers All of the provided models are listed in the models.yml file. Any meta-data and newer versions will be added there. V3 V3 models support SSML. Also see Colab examples for main SSML tag usage. ID Speakers Auto-stress Language SR Colab aidar, baya, ru 8000, Open v3_1_ru kseniya, xenia, yes (Russian) 24000, In eugene, random 48000 Colab en_0, en_1, ..., en 8000, Open v3_en en_117, random no (English) 24000, In 48000 Colab tamil_female, ..., en 8000, Open v3_en_indic assamese_male, no (English) 24000, In random 48000 Colab eva_k, ..., 8000, Open v3_de karlsson, random no de (German) 24000, In 48000 Colab es_0, es_1, es_2, es 8000, Open v3_es random no (Spanish) 24000, In 48000 Colab fr_0, ..., fr_5, 8000, Open v3_fr random no fr (French) 24000, In 48000 Colab 8000, Open v3_tt dilyara no tt (Tatar) 24000, In 48000 Colab ua 8000, Open v3_ua mykyta, random no (Ukrainian) 24000, In 48000 Colab 8000, Open v3_uz dilnavoz no uz (Uzbek) 24000, In 48000 Colab erdni, delghir, xal 8000, Open v3_xal random no (Kalmyk) 24000, In 48000 Colab hindi_male, indic 8000, Open v3_indic hindi_female, ..., no (Hindi, 24000, In random Telugu, 48000 Colab ...) aidar, baya, ru 8000, Open ru_v3 kseniya, xenia, yes (Russian) 24000, In random 48000 Colab Dependencies Basic dependencies for colab examples: * torch, 1.10+; * torchaudio, latest version bound to PyTorch should work (required only because models are hosted together with STT, not required for work); * omegaconf, latest (can be removed as well, if you do not load all of the configs); PyTorch Open In Colab Open on Torch Hub # V3 import torch language = 'ru' model_id = 'v3_1_ru' sample_rate = 48000 speaker = 'xenia' device = torch.device('cpu') model, example_text = torch.hub.load(repo_or_dir='snakers4/silero-models', model='silero_tts', language=language, speaker=model_id) model.to(device) # gpu or cpu audio = model.apply_tts(text=example_text, speaker=speaker, sample_rate=sample_rate) Standalone Use * Standalone usage just requires PyTorch 1.10+ and python standard library; * Please see the detailed examples in Colab; # V3 import os import torch device = torch.device('cpu') torch.set_num_threads(4) local_file = 'model.pt' if not os.path.isfile(local_file): torch.hub.download_url_to_file('https://models.silero.ai/models/tts/ru/v3_1_ru.pt', local_file) model = torch.package.PackageImporter(local_file).load_pickle("tts_models", "model") model.to(device) example_text = 'V nedrakh tundry vydry v g+etrakh t+yriat v viodra iadra kedrov.' sample_rate = 48000 speaker='baya' audio_paths = model.save_wav(text=example_text, speaker=speaker, sample_rate=sample_rate) SSML Check out our TTS Wiki page. Indic languages Example (!!!) All input sentences should be romanized to ISO format using aksharamukha tool. An example for hindi: # V3 import torch from aksharamukha import transliterate # Loading model model, example_text = torch.hub.load(repo_or_dir='snakers4/silero-models', model='silero_tts', language='indic', speaker='v3_indic') orig_text = "prsidd kbiir adhyetaa, purussottm agrvaal kaa yh shodh aalekh, us raamaanNd kii khoj krtaa hai" roman_text = transliterate.process('Devanagari', 'ISO', orig_text) print(roman_text) audio = model.apply_tts(roman_text, speaker='hindi_male') Supported languages Language Speakers Romanization function hindi hindi_female, transliterate.process('Devanagari', hindi_male 'ISO', orig_text) malayalam malayalam_female, transliterate.process('Malayalam', malayalam_male 'ISO', orig_text) manipuri manipuri_female transliterate.process('Bengali', 'ISO', orig_text) bengali bengali_female, transliterate.process('Bengali', 'ISO', bengali_male orig_text) rajasthani rajasthani_female, transliterate.process('Devanagari', rajasthani_female 'ISO', orig_text) tamil_female, transliterate.process('Tamil', 'ISO', tamil tamil_male orig_text, pre_options= ['TamilTranscribe']) telugu telugu_female, transliterate.process('Telugu', 'ISO', telugu_male orig_text) gujarati gujarati_female, transliterate.process('Gujarati', gujarati_male 'ISO', orig_text) kannada kannada_female, transliterate.process('Kannada', 'ISO', kannada_male orig_text) Text-Enhancement Languages Quantization Quality Colab 'en', 'de', 'ru', 'es' [?] link Open In Colab Dependencies Basic dependencies for colab examples: * torch, 1.9+; * pyyaml, but it's installed with torch itself Standalone Use * Standalone usage just requires PyTorch 1.9+ and python standard library; * Please see the detailed examples in Colab; import torch model, example_texts, languages, punct, apply_te = torch.hub.load(repo_or_dir='snakers4/silero-models', model='silero_te') input_text = input('Enter input text\n') apply_te(input_text, lan='en') FAQ Wiki Also check out our wiki. Performance and Quality Please refer to this wiki sections: * Quality Benchmarks * Performance Benchmarks Adding new Languages Please refer here. Contact Get in Touch Try our models, create an issue, join our chat, email us, read our news. Commercial Inquiries Please see our wiki and tiers for relevant information and email us. Citations @misc{Silero Models, author = {Silero Team}, title = {Silero Models: pre-trained enterprise-grade STT / TTS models and benchmarks}, year = {2021}, publisher = {GitHub}, journal = {GitHub repository}, howpublished = {\url{https://github.com/snakers4/silero-models}}, commit = {insert_some_commit_here}, email = {hello@silero.ai} } Further reading English * STT: + Towards an Imagenet Moment For Speech-To-Text - link + A Speech-To-Text Practitioners Criticisms of Industry and Academia - link + Modern Google-level STT Models Released - link * TTS: + Our new public speech synthesis in super-high quality, 10x faster and more stable - link + High-Quality Text-to-Speech Made Accessible, Simple and Fast - link * VAD: + One Voice Detector to Rule Them All - link + Modern Portable Voice Activity Detector Released - link * Text Enhancement: + We have published a model for text repunctuation and recapitalization for four languages - link Chinese * STT: + Mai Xiang Yu Yin Shi Bie Ling Yu De ImageNet Shi Ke - link + Yu Yin Ling Yu Xue Zhu Jie He Gong Ye Jie De Qi Zong Zui - link Russian * STT + Nashi servisy dlia besplatnogo raspoznavaniia rechi stali luchshe i udobnee - link + Telegram-bot Silero besplatno perevodit rech' v tekst - link + Besplatnoe raspoznavanie rechi dlia vsekh zhelaiushchikh - link + Poslednie obnovleniia modelei raspoznavaniia rechi iz Silero Models - link + Szhimaem transformery: prostye, universal'nye i prikladnye sposoby cdelat' ikh kompaktnymi i bystrymi - link + Ul'timativnoe sravnenie sistem raspoznavaniia rechi: Ashmanov, Google, Sber, Silero, Tinkoff, Yandex - link + My opublikovali sovremennye STT modeli sravnimye po kachestvu s Google - link + Ponizhaem bar'ery na vkhod v raspoznavanie rechi - link + Ogromnyi otkrytyi dataset russkoi rechi versiia 1.0 - link + Naskol'ko Bystroi Mozhno Sdelat' Sistemu STT? - link + Nasha sistema Speech-To-Text - link + Speech To Text - link * TTS: + Teper' nash sintez na 20 iazykakh - link + Teper' nash publichnyi sintez v super-vysokom kachestve, v 10 raz bystree i bez detskikh boliachek - link + Sinteziruem golos babushki, dedushki i Lenina + novosti nashego publichnogo sinteza - link + My sdelali nash publichnyi sintez rechi eshche luchshe - link + My Opublikovali Kachestvennyi, Prostoi, Dostupnyi i Bystryi Sintez Rechi - link * VAD: + A ty ispol'zuesh' VAD? Chto eto takoe i zachem on nuzhen - link + Modeli dlia Detektsii Rechi, Chisel i Raspoznavaniia Iazykov - link + My opublikovali sovremennyi Voice Activity Detector i ne tol'ko -link * Text Enhancement: + Vosstanovlenie znakov punktuatsii i zaglavnykh bukv -- teper' i na dlinnykh tekstakh - link + My opublikovali model', rasstavliaiushchuiu znaki prepinaniia i zaglavnye bukvy v tekste na chetyrekh iazykakh - link Donations Please use the "sponsor" button. About Silero Models: pre-trained speech-to-text, text-to-speech and text-enhancement models made embarrassingly simple Topics text-to-speech german speech pytorch tts speech-synthesis english speech-recognition spanish colab speech-to-text pretrained-models stt asr capitalization onnx stt-benchmark tts-models torch-hub repunctuation Resources Readme License View license Code of conduct Code of conduct Stars 2.4k stars Watchers 60 watching Forks 152 forks Releases 5 v0.4.1 Latest Jun 12, 2022 + 4 releases Sponsor this project * open_collective opencollective.com/open_stt Packages 0 No packages published Contributors 10 * @snakers4 * @Islanna * @evrrn * @Abhi011999 * @adamnsandle * @rominf * @axenov * @kartikeyporwal * @teague-lasser * @slgero Languages * Jupyter Notebook 73.6% * Python 26.4% * (c) 2022 GitHub, Inc. * Terms * Privacy * Security * Status * Docs * Contact GitHub * Pricing * API * Training * Blog * About You can't perform that action at this time. You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.