https://github.com/ggerganov/whisper.cpp/discussions/167 Skip to content Toggle navigation Sign up * Product + Actions Automate any workflow + Packages Host and manage packages + Security Find and fix vulnerabilities + Codespaces Instant dev environments + Copilot Write better code with AI + Code review Manage code changes + Issues Plan and track work + Discussions Collaborate outside of code + Explore + All features + Documentation + GitHub Skills + Blog * Solutions + For + Enterprise + Teams + Startups + Education + By Solution + CI/CD & Automation + DevOps + DevSecOps + Case Studies + Customer Stories + Resources * Open Source + GitHub Sponsors Fund open source developers + The ReadME Project GitHub community articles + Repositories + Topics + Trending + Collections * Pricing [ ] * # In this repository All GitHub | Jump to | * No suggested jump to results * # In this repository All GitHub | Jump to | * # In this user All GitHub | Jump to | * # In this repository All GitHub | Jump to | Sign in Sign up {{ message }} ggerganov / whisper.cpp Public * Notifications * Fork 190 * Star 2.8k * Code * Issues 34 * Pull requests 5 * Discussions * Actions * Projects 0 * Security * Insights More * Code * Issues * Pull requests * Discussions * Actions * Projects * Security * Insights Talk = GPT-2 + Whisper + WASM #167 ggerganov started this conversation in Show and tell Talk = GPT-2 + Whisper + WASM #167 @ggerganov ggerganov Nov 17, 2022 * 8 comments * 7 replies Return to top Discussion options * Quote reply edited * {{editor}}'s edit {{actor}} deleted this content . {{editor}}'s edit Loading... Sorry, something went wrong. [199] ggerganov Nov 17, 2022 Maintainer - I just had an awesome idea: Make a web-page that: * Listens when someone speaks * Transcribes the words using WASM Whisper * Generates a new sentence using WASM GPT-2 * Uses Web Speech API to synthesise the speech and play it on the speakers. All of this running locally in the browser - no server required I have all the ingredients and I think the performance is just enough. I just have to put it together. The total data that the page will have to load on startup (probably using Fetch API) is: * 74 MB for the Whisper tiny.en model * 240 MB for the GPT-2 small model * Web Speech API is built-in in modern browsers I think it will be very fun because you could talk to the web-page or even add extra devices that talk to each other only through the mic and the speakers. For example, you simply open the page on your phone and tablet and put them next to each other - listen to them talk about something Any ideas to make this even more fun? --------------------------------------------------------------------- Update: This is now fully functional at: https://whisper.ggerganov.com/talk/ Source code is here: https://github.com/ggerganov/whisper.cpp/tree/ master/examples/talk.wasm Looking for beta testers, feedback and ideas for improvement! talk-2.mp4 Beta Was this translation helpful? Give feedback. 17 You must be logged in to vote 6 4 3 [?] 17 2 Replies: 8 comments * 7 replies * Oldest * Newest * Top Comment options * Quote reply [713] eschmidbauer Nov 17, 2022 - this sounds really fun! Beta Was this translation helpful? Give feedback. 1 You must be logged in to vote 0 replies Comment options * Quote reply [199] ggerganov Nov 20, 2022 Maintainer Author - So.. this is turning out to be even better than I expected talk-0.mp4 Beta Was this translation helpful? Give feedback. 4 You must be logged in to vote 3 0 replies Comment options * Quote reply edited * {{editor}}'s edit {{actor}} deleted this content . {{editor}}'s edit Loading... Sorry, something went wrong. [292] Vuizur Nov 20, 2022 - These results are extremely impressive! I recently tried to implement something similar in Python, only not locally, but instead using different online APIs, but it felt worse than your demo video because Whisper is much better than the free Google Speech Recognition API (and your optimized version runs significantly better on CPU than the standard Whisper Python lib I tried) :). Beta Was this translation helpful? Give feedback. 1 You must be logged in to vote 1 [?] 1 0 replies Comment options * Quote reply edited * {{editor}}'s edit {{actor}} deleted this content . {{editor}}'s edit Loading... Sorry, something went wrong. [199] ggerganov Nov 24, 2022 Maintainer Author - And here is a less cringe video to demonstrate the capabilities of this implementation: talk-tech-demo-0-lq.mp4 These are 2 Chrome tabs talking and being nice to each other using the microphone and the speakers of a Macbook. Beta Was this translation helpful? Give feedback. 4 You must be logged in to vote 0 replies Comment options * Quote reply [870] beartell Nov 25, 2022 - There is an error on browser Firefox and Chrome.. resim Beta Was this translation helpful? Give feedback. 1 You must be logged in to vote 1 reply @ggerganov Comment options * Quote reply ggerganov Nov 27, 2022 Maintainer Author - Likely, you haven't enabled cross-origin isolation on your HTTP server. For more information, see my #88 (comment) Beta Was this translation helpful? Give feedback. Comment options * Quote reply [870] beartell Nov 27, 2022 - Amazing solution. Works like a charm ;) Pazar, Kasim 27, 2022 13:17 +03 tarihinde, Georgi Gerganov ***@***.***> sunu yazdi: Likely, you haven't enabled cross-origin isolation on your HTTP server. For more information, see my #88 (comment) -- Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: ***@***.***> Beta Was this translation helpful? Give feedback. 1 You must be logged in to vote [?] 1 0 replies Comment options * Quote reply edited * {{editor}}'s edit {{actor}} deleted this content . {{editor}}'s edit Loading... Sorry, something went wrong. [818] aklos Dec 7, 2022 - This is great, been trying to make the same thing except through terminal. Why GPT-2 instead of the GPT-3 text-davinci-003 model? Beta Was this translation helpful? Give feedback. 1 You must be logged in to vote 6 replies Show 1 previous reply @ggerganov Comment options * Quote reply ggerganov Dec 7, 2022 Maintainer Author - To use GPT-3, you have to make API requests and pay for that. With GPT-2, it runs on your computer for free. Beta Was this translation helpful? Give feedback. 3 @kingofdelphi Comment options * Quote reply kingofdelphi Dec 7, 2022 - @ggerganov I meant the recently released one, I think it's free, I've seen some scripts and trying those. I mean, the newer chatbot is pretty advanced compared to the dull GPT-2. Very stark difference. Beta Was this translation helpful? Give feedback. @aklos Comment options * Quote reply edited * {{editor}}'s edit {{actor}} deleted this content . {{editor}}'s edit Loading... Sorry, something went wrong. aklos Dec 7, 2022 - @kingofdelphi I'm pretty sure OpenAI might ban people who use workarounds to use the ChatGPT API. They haven't released a public API yet, so anything you do for that right now would be probably violating ToS. @ggerganov To be fair though, the API requests are cheap enough that I still prefer to use GPT-3 for personal apps. It might be a good idea to allow people to switch to GPT-3 (using their own API key) if they want to. Beta Was this translation helpful? Give feedback. @Anx2k Comment options * Quote reply Anx2k Dec 7, 2022 - I wonder if running something like GPT-JT would be possible: https://www.together.xyz/blog/ releasing-v1-of-gpt-jt-powered-by-open-source-ai Really the GPT-2 part is the only kind of disappointing aspect, as Whisper is great and even the speech synth is more than acceptable, but GPT-2 is giving me Eliza flashbacks. ;) Beta Was this translation helpful? Give feedback. @ggerganov Comment options * Quote reply ggerganov Dec 7, 2022 Maintainer Author - You won't be able to load GPT-JT in a web-page - it's too big. You can easily run talk in the command-line using GPT-J or GPT-JT, but then you won't have speech synthesiser. But I'm pretty sure somebody will hack together something very soon. Beta Was this translation helpful? Give feedback. Comment options * Quote reply [361] waynenilsen Dec 7, 2022 - should use hot mic method hold down space to talk so you can take longer pauses while you think about what to say release spacebar to translate speech to text Beta Was this translation helpful? Give feedback. 1 You must be logged in to vote 1 0 replies Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment Category Show and tell Labels ideas Interesting ideas for experimentation 8 participants @ggerganov @beartell @waynenilsen @kingofdelphi @eschmidbauer @aklos @Vuizur @Anx2k Converted from issue This discussion was converted from issue #154 on November 21, 2022 20:39. Add heading text Add bold text, Add italic text, Add a quote, Add code, Insert Link Link Text [ ] URL [ ] Add Add a link, Add a bulleted list, Add a numbered list, Add a task list, Directly mention a user or team Reference an issue or pull request Add heading text Add bold text, Add italic text, Add a bulleted list, Add a numbered list, Add a task list, 1 reacted with thumbs up emoji 1 reacted with thumbs down emoji 1 reacted with laugh emoji 1 reacted with hooray emoji 1 reacted with confused emoji [?] 1 reacted with heart emoji 1 reacted with rocket emoji 1 reacted with eyes emoji Footer (c) 2022 GitHub, Inc. Footer navigation * Terms * Privacy * Security * Status * Docs * Contact GitHub * Pricing * API * Training * Blog * About You can't perform that action at this time. You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.