[HN Gopher] Show HN: Learn a language quickly by practising spea...
___________________________________________________________________
Show HN: Learn a language quickly by practising speaking with AI
Hi guys, Hope everyone is well. This app was borne out of my own
frustration. I thought that I was terrible at learning languages at
school, since I didn't become conversational in French after 5
years of study. However, I later traveled with some French friends
and, in just under 3 weeks, I was able to hold a reasonable
conversation. I realized that there's no substitute for speaking to
native speakers. I tried to adopt this approach for other
languages, but it's much harder to find people to practise with
when you aren't travelling. I started using iTalki to meet people
from different countries and chat to them. It quickly became very
expensive and time-consuming to schedule the calls, so I gave up.
I made PrettyPolly so that anyone can easily practice speaking 26
languages orally. The app uses ChatGPT (amongst other tools) to
allow you to practice speaking whenever you want. It also generates
a fluency score for each conversation so that you have an objective
way of tracking progress. It's free to use (up to 15 conversations
per month). I've found that using it once or twice per day is
plenty, and you'll be amazed at how much you will pick up in a
week. I've added some FAQs here in case useful -
https://www.prettypolly.app/learn Would really appreciate any
feedback. Let me know if you have any questions, issues or
suggestions. Thanks, Chris
Author : cwbuilds
Score : 89 points
Date : 2023-08-02 16:47 UTC (6 hours ago)
(HTM) web link (www.prettypolly.app)
(TXT) w3m dump (www.prettypolly.app)
| drdaeman wrote:
| Hmm, just had a weird speech-to-text hiccup.
|
| I've asked to teach me how to construct past tense in Spanish. It
| gave me a few examples (Spanish and English translation right
| next to it, text-to-speech engine hadn't realized that some words
| were in English, though, and pronounced them in Spanish manner),
| then suggested I try to form my own sentences. So I've replied
| "Yo entendi sus ejemplos!" (uh, I hope I've got it right) but
| somehow it transcribed it as "You: Suscribete para no perderte
| nuestros proximos videos." While I think I've stuttered and may
| be mispronounced something, it surely didn't sound anywhere close
| to that.
|
| Being an LLM, it didn't bat an eye and asked me about the name of
| my channel :)
|
| But I'm wondering if there was some request-response mismatch
| with the Speech-to-text API you're using and I've somehow got a
| phrase intended for someone else.
| cwbuilds wrote:
| Thanks for the feedback. Yeah - it does seem to have
| hallucinated a little tonight. It normally happens when I time
| the microphone wrong and send some blank audio, then it seems
| to want to magic up some sort of phrase from nowhere. Am
| working on a fix for this. Any chance you could try again and
| let me know if it happens again? Ofc, no worries if busy.
| Thanks again
| larsiusprime wrote:
| Yes I have a question, when is your next round!
| cwbuilds wrote:
| Aha none planned! Let me get through the MVP launch first!
| GenericDev wrote:
| This is really slick!
|
| I really appreciate you having Japanese as a language choice.
| I've been dying to hone my japanese skills with AI in lieu of
| reading grammar/vocab books etc.
|
| I definitely intend to use your product more, because it is
| definitely scratching that itch!
|
| A couple of things that might help me personally would be the
| ability to cancel my current response and redo it, as frequently
| I will stutter or pause on a thought and wish I could restart
| over.
|
| In that same vein I would love to see the output of what I'm
| saying in real-time. If only to gauge and understand what the
| application "thinks" I'm saying and to potentially correct and
| miscommunications from spoken input.
|
| I would also love the chance to role play or have the tool help
| me learn contextual role playing. For example in Japanese keigo
| is a skill I just haven't mastered. I would love a tool to help
| put me in positions where I need to use keigo, and then be
| corrected where I'm failing.
|
| Regardless, this is absolutely fantastic! Thank you so much for
| sharing! You're awesome :)
| cwbuilds wrote:
| Thanks - glad it's useful for your Japanese. Do you like it
| being in the Japanese writing system or would you prefer
| phoenetical English? Wasn't sure which to do on that one.
|
| Thanks for the feedback. The processing point has come up a
| couple of times so I'll definitely be adding that, and I'll
| look into doing some contextual scenarios. If your language is
| good enough, you could try prompting it to act as, eg a waiter
| in a restauraunt, but that's far from ideal for beginners!
|
| Sure - realtime transcription makes a lot of sense. I'll look
| into it.
| drdaeman wrote:
| Absolutely, please keep Japanese output the way it is - it's
| the only approach that makes sense. Romanization loses a lot
| of information and makes things harder (except possibly for
| the absolute beginners), not easier.
|
| However, it'd be extra cool if there'd be an option to
| display furigana - in case user had forgotten or doesn't know
| some particular kanji. Or maybe just throw an option to
| replay the audio.
| GenericDev wrote:
| Oh! I didn't know that. I'm really new to how to interact
| with AI. I'll try that out.
|
| Please please please keep it in Japanese. Phonetic english
| for Japanese is awful. Hiragana only for Japanese is really
| not ideal.
|
| The current output is absolutely perfect.
|
| Also, I think it's really cool I don't need an account to try
| that out. Very cash money of you.
| cwbuilds wrote:
| Ahah hear you loud and clear - no phoenetical words. Of
| course - hope it continues to help you learn!
| infomiho wrote:
| I kept speaking Croatian while I was practicing Spanish and it
| underatood me. What kind of magic is this? Really cool project!
| cwbuilds wrote:
| Ahah the witches of the internet are pulling the strings
| [deleted]
| shazar wrote:
| Just tried a few languages and there seems to be a problem with
| German speech to text. When I say "Hallo" it just thinks I said
| "Untertitel der Amara.org-Community" :-)
|
| English and Spanish seemed to be functioning OK though, but I
| really wanted to test my German.
| sno6 wrote:
| The reason for this is quite interesting and is due to the data
| that Whisper (the transcription service) was trained on:
| https://github.com/openai/whisper/discussions/928
| cwbuilds wrote:
| Thanks for the comment. Sometimes if you toggle the microphone
| quickly without saying anything, it will halucinate something.
| If you could, is there any chance you could try again in German
| and let me know if it's still not working?
| shazar wrote:
| Curious about this, why do you think it hallucinates? Isn't
| that supposed to be the direct output from speech
| recognition?
|
| PS: Just happened with Spanish now as well, which was working
| fine. (You: Subtitulos realizados por la comunidad de
| Amara.org)
| shazar wrote:
| No, still not working. Also tried a bunch of other languages
| and they all seemed to work except Turkish, which also spits
| out something related to subtitles...
| cwbuilds wrote:
| Ah darn - okay, thanks for the feedback. I'll find a way to
| stop it from doing this.
| anhner wrote:
| Awesome project! I had fun talking to it. Thanks for sharing. Can
| you give some hints as to what technologies you used?
| [deleted]
| acd wrote:
| Very good app, how so you sign up for premium service?
| cwbuilds wrote:
| Thanks! Once you hit the conversation limit (15 per month) it
| will automatically take you to the upgrade page. If you really
| fancy paying, feel free to mash the start conversation button
| aha. Otherwise, you can just enjoy the free convos for a while
| :)
| kleene_op wrote:
| Very cool!
|
| I could see this tool to be very useful in my Japanese studies. I
| only have two problems with it:
|
| - AI feedback can sometimes be pretty overwhelming. Maybe you
| could implement a level selector (beginner, intermediate and
| expert?) so that the AI would switch to using shorter/simpler or
| longer/harder sentences accordingly?
|
| - The speech to text works fine most of the time, but sometimes
| it gets a few words wrong. Could there be a way to correct the
| last sentence input?
| cwbuilds wrote:
| Thanks for the feedback. Yes, the beginner, intermediate,
| advanced point has come up so I'll look at adding something
| like that.
|
| Just to check that I understand, do you mean changing the text
| manually when it gets it wrong? If so, sure - I can look into
| it.
| kleene_op wrote:
| I think manually editing your last sentence would be cheating
| in the context of a speaking app. Entirely erasing the last
| sentence of the user (as well as the AI's reply, if any) and
| having him repeat the sentence he had in mind would be better
| because it would force him to speak more clearly.
|
| (Also an undo button would be easier to implement than a
| whole system to edit previous sentences, right?)
| cwbuilds wrote:
| Yeah, agreed that makes much more sense!
| sno6 wrote:
| This looks great thanks for sharing. I've been using a similar
| app on iOS and it's definitely been helping with my German.
| https://apps.apple.com/us/app/langs-ai-language-learning/id1...
| cwbuilds wrote:
| Thanks for the comment and for sharing that app - I'll check it
| out.
| explosion-s wrote:
| Will you make this open source please?
| cwbuilds wrote:
| I can definitely look into it. Do you have any suggestions for
| how to do it well?
| yetanother-1 wrote:
| When open sourcing your work, look first into licensing and
| make sure to understand what each license offers and what
| implications it has.
|
| Then you can chose to push it to github or similar, but
| please prepare a good readme.md file.
| xcdzvyn wrote:
| Upload the source code? :")
| myownpetard wrote:
| What is the fluency score based on?
| cwbuilds wrote:
| Good question. I looked through literature to see whether
| anyone had come up with any good ways of measuring fluency, and
| there are a couple out there.
|
| The key indicators that kept coming up were the number of
| unique words used and the speed at which you speak, so they're
| involved. I'm hesitant to give the full equation, because I
| thought it might be better to leave some mystery so we aren't
| consciously trying to game it while we learn.
|
| It is by no means perfect, but I think it's useful to have some
| objective metric to track progress.
| myownpetard wrote:
| Cool, thanks for the response! I'm enjoying using the site.
|
| One followup: does it take into account grammatical errors,
| incorrect conjugations and that kind of thing?
| cwbuilds wrote:
| Glad to hear it!
|
| Sort of. It will try to understand you even if you get the
| grammar wrong (a bit like a human would), but if it really
| has no clue what you're trying to say, it will ask you to
| repeat. I think it's best that way as it mimics learning in
| real life. If you keep getting your point across, over
| time, your grammar will slowly improve. The only evidence I
| have for this is anecdotal from learning Romanian with it
| myself.
| camjw wrote:
| This is really super cool. I wonder if you could use elevenlabs
| or something similar to make the voice more legit because it
| feels at the moment like I'm speaking to a satnav (in French).
|
| I also got fluency score None which, let's be real, is pretty
| accurate. But I feel like I need a bit more to help me learn.
|
| The microphone interaction is really nice.
| cwbuilds wrote:
| Aha thanks man. Yeah - agreed that there's lots of work to be
| done on the speech.
|
| Makes sense - getting from 0 to 1 can be a bit of a struggle
| with this method, so could be good to find some ways to make
| that easier.
| mahathu wrote:
| I like the Anki shoutout.
|
| I personally would be cautious learning an extra language through
| a tool like this though, especially for a huge list of supposedly
| supported languages like this. Chat-GPT gets basic concepts like
| verb suffixes wrong, that is, explaining those concepts and
| generating example sentences, when I was trying it out a few
| months ago. I wouldn't mind conversations feeling a bit awkward
| or unrealistic, (EDIT: as they do here), but I wouldn't trust AI
| tools atm to get sentence structure, spelling and pronunciation
| (!) all completely right which is imperative for such a tool to
| function (and this one didn't for the two languages I tested).
|
| If we assume the AI is already at that level, it would be trivial
| to have it show translations to user's native languages instead
| of only to English and excluding a number of potential revenue-
| generators for the capitalist machine ("learners").
|
| Your prompt leaked in the text box when I pressed the Start
| Conversation button multiple times, and the site seems to be
| getting HN-hugged-of-death right now.
|
| Also, adding to the person who mentioned this being close
| sourced, if a "deep-rooted desire not to look foolish in front of
| others" was among my reasons not to talk to a stranger in an
| extra language, I don't think I'd be any more likely speaking
| into a microphone, alone, in a room, like I'm conducting my
| biannual mandatory TOEFL(c) test every two years.[1]
|
| [1]: This one is actually worse because you're in a room with 10+
| others simultaneously talking into a computer screen, not talking
| to a person but just having your voice recorded, and it's
| outrageous ETS is getting away with charging people like 200
| bucks (getting to see feedback for your grade is extra)
| cwbuilds wrote:
| Anki is such a great product.
|
| That makes sense. I do understand the worries about learning
| from AI. It's worth saying that different software is used for
| transcribing what you say (rather than ChatGPT), and that's
| been trained on lots of data (including YouTube videos) of
| people speaking that language, not to say that it could never
| get it wrong. But I definitely hear what you're saying about
| ChatGPT sentence structure on the less popular languages.
|
| Good point about open sourcing. I definitely agree.
| Serenacula wrote:
| I actually think I disagree with this. Yes it'd be incorrect
| some of the time, but that is also true of humans. Provided it
| gets it right most of the time and is reasonably fluent, I
| don't think that minor errors would matter much.
|
| This seems much more a tool for practicing the stuff you learn
| in another context to me, rather than a way of learning in
| itself. You can use it without the social pressure of stumbling
| slowly through sentences with another person.
| obscur wrote:
| Seems cool! I know basic Spanish from Duolingo but not enough to
| understand what it says to after a few rounds so my suggestion is
| that there is a side-by-side translated version of the
| conversation.
| cwbuilds wrote:
| That makes a lot of sense! I've been awkwardly having a Google
| Translate window open with it, but agree that it would be much
| less onerous to have it built in. Thanks for the feedback.
| kansumine wrote:
| Thanks for sharing! I'm a bit confused with the fluency score,
| what does it mean exactly? It would be good to have some feedback
| and pointers for improvement, otherwise I don't see myself using
| it for long.
| cwbuilds wrote:
| Thanks for the feedback. The number of unique words you use and
| the rate at which you speak contribute to the fluency score.
| I'm a little reluctant to give the full equation, because it
| prevents us from trying to game it.
|
| That's useful feedback on pointers for improvement - I'll see
| what can be done.
| kalid wrote:
| I've been meaning to brush up on Spanish and the app worked great
| to just hop in! The best part about using GPT is being able to
| talk about whatever is on your mind.
|
| I noticed the transcription APIs clean up my mistakes (which is
| good for the transcript) but I wonder if there is a way to show
| the raw and corrected transcript to help guide corrections?
| Either way, it's already useful as is, looking forward to using
| it more.
| blackkettle wrote:
| You could do something two pass and collect forced alignment
| confidence scores similar to what is fine with whisperx. The OP
| would need to incorporate that into the app but it's a pretty
| standard approach.
| cwbuilds wrote:
| Thanks - I'll look into this
| kian wrote:
| I'll second wishing I could see the raw transcript vs the
| cleaned up version (perhaps vertically over one another in
| different colors?), and also how fantastically magical it is at
| allowing me to come right in and converse naturally.
|
| Great job!
| cwbuilds wrote:
| Thanks for the comment and glad it's useful! Agreed - I love
| the flexibility to talk to it like it's almost just another
| human, but with vast knowledge. Okay - that makes sense - I can
| look into adding some more of the raw transcript. Thanks for
| the feedback.
| kalid wrote:
| Happy to help! To give some context: there was a GPT prompt
| which was something like "Act as a language tutor. I'll have
| a conversation with you, and you reply back. If I make a
| grammar mistake, keep the conversation going, but put the
| correction in a parenthetical statement."
|
| I was using that for a bit, but typing in another language
| was enough friction (vs. speaking) that I didn't use it that
| much.
|
| For this, audio is perfect -- all I need is the gentle
| reminders about what to fix, and why. GPT could probably have
| a parenthetical part in the official transcript too.
|
| Anyway, hope that helps give context.
| cwbuilds wrote:
| Agreed. There's a huge chasm between being able to
| read/write and being able to speak. Speaking is just more
| fun as well imo. I'll have a think about ways that it could
| provide some qualitative feedback. Thanks again
| neilyoungsdog wrote:
| Do Cantonese please! It's such an underrepresented but widely
| used Chinese language...
| cwbuilds wrote:
| Sure - will do. I had to create a shortlist for the MVP, but
| Cantonese will definitely be coming soon.
| bironran wrote:
| The Hebrew one is so wrong... both grammar and pronunciation.
| cwbuilds wrote:
| Darn, okay - I'll work out how to improve this. Thanks for the
| feedback.
| netsharc wrote:
| It needs a "Processing..." notification, I clicked "Start
| conversation" and it takes a while before it says anything. Same
| with "Start speaking" -> I said something, then clicked "Stop
| speaking" but the page seemed unresponsive. I repeated it, and
| when the transcript appeared I ended up answering the question,
| getting another question, and repeating the first answer.
|
| Also after 3-4 sentences the questions got a bit "dull-person-at-
| a-party" small-talk... "What are you doing today?" / "I'm
| programming." / "What programming language are you learning?" /
| [I didn't say I was learning...] "Java." / "Oh, interesting. Java
| is a popular programming language. What do you like about Java?".
| cwbuilds wrote:
| Thanks for the feedback. I'll add in a processing-style
| spinner. Sorry that it was a bit slow when you first used it.
| Do you mind if I ask what device you're using?
| ollysb wrote:
| This my experience on an iphone 12. The UX doesn't really
| work for me yet but I absolutely love the concept, will keep
| an eye on it (would sign up for a mailist if you have one).
| cwbuilds wrote:
| Thanks for that. I did have some trouble getting it working
| on iOS so maybe I didn't iron out all the bugs. I'll get
| working on a fix
| venk12 wrote:
| Amazing app. I am trying to learn dutch with this. It pushes you
| to converse in the language of choice. I also tried to get
| translation in english. Could be a good feature to understand new
| words and form better sentences.
| cwbuilds wrote:
| Thanks man! Yes, definitely - will look at adding translation
| Serenacula wrote:
| Wow, this is so cool! xD I think this is the strongest usecase
| for AI I've seen so far. I'd love some more details on how you
| made it - how are you handling the speech to text? It's so clean,
| I was amazed it could understand my stuttering mess lol.
| cwbuilds wrote:
| Thanks! Speech-to-text is Whisper API, although that seems to
| have caused a couple of errors tonight so am considering moving
| away from it. Will have to have a think and do a lot of
| testing.
|
| I feel your pain, I've also been a stuttering mess in Romanian.
| avn2109 wrote:
| Yeah this is great, shocking how reliable the speech-to-text is
| even for my unconventional accent.
|
| Imho UX would be improved by a larger text box, more clear
| demarcation between "stuff I've said" and "stuff the bot said,"
| similar to how iMessage or Whatsapp have two-column message
| layouts or blue-vs-grey or both.
|
| Also maybe some "hold spacebar to activate mic" keybindings
| would not go amiss, similar to how Zoom or Gmeet makes the mic
| hot if you're muted but hold spacebar. That would save a lot of
| choppy/distracting clicking. (Or possibly just make the mic hot
| all the time when the bot isn't talking?)
| legel wrote:
| Marvelous! One of the best applications of a ChatGPT backend that
| I've seen yet. It's even remarkable to talk in your own native
| language to ChatGPT, but there is just so much room to build out
| the next great language learning app here.
| cwbuilds wrote:
| Thanks for the kind words!
| da39a3ee wrote:
| I love the idea, and just had a successful conversation with it
| for a few minutes in Portuguese. Nice work!
|
| For me in that language I need to work on understanding native
| speakers, so currently it speaks too slowly for what I personally
| need (it's too "easy"). I did ask it twice to speed up but it
| didn't seem able to - I guess it's hardcoded in your text-to-
| speech code? Is that something you can add an option for?
| cwbuilds wrote:
| Awesome! Yes, it's hard-coded to basically be in 'beginner'
| mode at the moment, but I was considering allowing the user to
| select intermediate or advanced, which it sounds like would be
| more useful for you. Thanks for the feedback!
| da39a3ee wrote:
| I would love to try it on advanced if you get time to do
| that. I honestly think I'd use it a lot. If adding it to the
| UI takes time I'd be absolutely happy with a URL query
| parameter prettypolly.app?level=advanced if that's easier.
| cwbuilds wrote:
| It probs wouldn't take much longer to just add a slider or
| something too. If you don't mind me asking, what are you
| planning to use it for? Interested to hear some advanced
| use cases
___________________________________________________________________
(page generated 2023-08-02 23:02 UTC)