Post AzqmrAdCBEbnH3cwIS by knowuh@mastodon.social
(DIR) More posts by knowuh@mastodon.social
(DIR) Post #Azqj23dw6P7kyBuRwO by futurebird@sauropods.win
2025-11-02T22:50:27Z
0 likes, 0 repeats
I use speech to text in apple's notepad on my phone. Since the most recent OS update I've noticed a big jump in quality. Wondering if the text is getting brushed up by an LLM?For example: I narrated a paragraph about ants and it got the word "integument" correct. (I fully expected it to be wrong. "In Ted, good man")This happens to be one of the few things I think LLMs do well --but I also would like to know about all the water I'm destroying.
(DIR) Post #Azqk0XWU8fLf11GRxQ by canacar@ioc.exchange
2025-11-02T23:01:20Z
0 likes, 0 repeats
@futurebird I dont have an iPhone around to check but is may be doing local processing. There should be (privacy) settings about transcript use of cloud, and you should also be able to tell how it does when network/WiFi is off. If it is using a local LLM then the main cost would be your battery charge.
(DIR) Post #Azql8pKRv9AUVh9NEu by futurebird@sauropods.win
2025-11-02T23:14:02Z
0 likes, 0 repeats
@canacar I'm experiencing stress from wondering where exactly this tech is being injected all of the time. This might drive me off the iPhone at last, although I did just get my screen fixed and felt like I ought to be good for two more years.
(DIR) Post #AzqmrAdCBEbnH3cwIS by knowuh@mastodon.social
2025-11-02T23:33:14Z
0 likes, 0 repeats
@futurebird @canacar Apple has made a big deal about having to “opt-in” to AI related services that happen off device. I am certain that the text to speech model is something like whisper running on your phone.Quantized inference is relatively small and efficient. Most of the energy demands are on the training side.
(DIR) Post #Azqnnha9pLT0GX7FtQ by knowuh@mastodon.social
2025-11-02T23:42:55Z
0 likes, 0 repeats
@futurebird @canacar https://support.apple.com/en-us/121115“To get started with Apple Intelligence features on your compatible iPhone, iPad, Mac, or Apple Vision Pro, update your device to the latest software version, and ensure you have Apple Intelligence turned on under Settings > Apple Intelligence & Siri.”
(DIR) Post #AzqnnitKxWP2KJ65QW by futurebird@sauropods.win
2025-11-02T23:43:52Z
0 likes, 0 repeats
@knowuh @canacar It's still asking me to turn that on and I keep ignoring it.
(DIR) Post #AzqoCKxL5UKJKchpWi by tsturm@famichiki.jp
2025-11-02T23:48:17Z
0 likes, 1 repeats
@futurebird @canacar Apple is doing that on the phone, it's a small pre-trained AI model that lives in the "neural engine" or whatever they call that part of the CPU.There are supposedly some new iOS services that will ask nicely if they can take your data offsite for better processing, but that should be all opt-in.
(DIR) Post #AzqoEfskFxoAoMfhzs by futurebird@sauropods.win
2025-11-02T23:48:46Z
0 likes, 0 repeats
@tsturm @canacar This sounds ... fine?
(DIR) Post #AzqoRkuuqcQWBwvOTY by tsturm@famichiki.jp
2025-11-02T23:51:05Z
0 likes, 0 repeats
@futurebird @canacar That's how I understand it. Apple's AI push seems to have fizzled out a bit. They will certainly try and expand Siri with more features that require a data-center, but speech recognition should still be all in device.
(DIR) Post #AzqoYr4MN7QBIej9Hc by futurebird@sauropods.win
2025-11-02T23:52:24Z
0 likes, 0 repeats
@tsturm @canacar This is literally the best thing to come out of this whole "tech cycle" It's amazing to be able to dictate text and have it be correct, with nice punctuation and only a few little things to fix.
(DIR) Post #AzqpXMCXkjkJEGP9JA by futurebird@sauropods.win
2025-11-03T00:03:14Z
0 likes, 0 repeats
@apLundell Why isn't functional speech to text more of a big deal? Seems like a massive tech win that could change workflows all over the place.
(DIR) Post #AzqptlPghCZznGEl8a by jannem@fosstodon.org
2025-11-03T00:05:48Z
0 likes, 1 repeats
@tsturm @futurebird @canacar I believe the Android speech recognition and translation functions are also local models. And the Firefox language translator is also entirely local.There's lots of truly useful and significant tools coming from recent ML advances (one, Alphafold, got a Nobel prize), but they're not LLM chatbots and don't get all this public recognition or money.
(DIR) Post #Azqq2BjEjwNdj3Ok52 by futurebird@sauropods.win
2025-11-03T00:08:52Z
0 likes, 0 repeats
@jannem @tsturm @canacar It's gonna change more lives than chatbots for the better. Think of the applications to teaching things like writing. Think about me writing more because I get sick of typing and now I can go for a walk instead and keep going.
(DIR) Post #Azqr3o85YHiP4FMHNg by piofthings@mastodon.social
2025-11-03T00:20:22Z
0 likes, 0 repeats
@futurebird two different but related things - Transformers are vastly more efficient than previous Neural Networks and were actually discovered while trying to generate more efficient transaltion models. Second, LLMs were enabled because of Transformers but you don’t need LLMs for everything. Niche models can be built for niche requirements. Because Transformers enable huge amounts of parallelism, they are (mis)used to build LLMs in hopes of making all knowing oracle AI!
(DIR) Post #Azqy4T3G6VTZfFMRoO by canacar@ioc.exchange
2025-11-02T23:35:29Z
0 likes, 1 repeats
@rom yes, Apple has been good at making sure user has knowledge and control over how these things work. I expect local processing, as smaller LLMs can run on newer phones, but Apple also recently announced "Private Cloud Compute" which is supposed to offload compute intensive tasks to cloud while preserving privacy. Not sure which one this is, but turning off all network connectivity and checking if the transcription quality changes should help.@futurebird It seems Apple is not immune to pushing the LLM hype into their devices. They did delay rolling out "LLM Siri" and had papers about LLMs inability to do reasoning, so their devices may still be a better choice in that regard, especially considering how MS and Google are fully on board with this.
(DIR) Post #AzqzAsGhMT9dZ3GZ3g by gdupont@framapiaf.org
2025-11-03T01:51:16Z
0 likes, 0 repeats
@futurebird@canacar Can't you just disconnect from any network (telco & wifi) and try? Does it still work offline?
(DIR) Post #AzqzEu9EJbj4YHh87s by futurebird@sauropods.win
2025-11-03T01:52:01Z
0 likes, 0 repeats
@gdupont @canacarI will try this and see. I'm annoyed I didn't think of it.
(DIR) Post #AzqzxCXdI0s4LAEOP2 by Phosphenes@mastodon.social
2025-11-03T01:30:06Z
0 likes, 0 repeats
@jannem @tsturm @futurebird @canacar None of those accomplishments were by generative AI were they? All the crappy AI is generative. Translation and speech recognition are just mapping, no hallucinations.
(DIR) Post #AzqzxDmChK7YAe3Xkm by jannem@fosstodon.org
2025-11-03T01:55:39Z
0 likes, 0 repeats
@Phosphenes @tsturm @futurebird @canacar "Generative ai" is a misnomer. Some useful tools use the same kind of architecture as image or video generators (text to speech for instance), and some use the same kind of transformer architecture as chatbots.But that's all implementation details. That's not important (it's like arguing what language was used to write a specific program). What matters is what it's used for, and by whom.
(DIR) Post #AzqzxF0m6dN207sh6W by futurebird@sauropods.win
2025-11-03T01:59:54Z
0 likes, 0 repeats
@jannem @Phosphenes @tsturm @canacar I think the "generative" adjective is about if one is using the training data to correctly match, or to extrapolate.Sometimes when I do dictation there is a loud noise or I mumble: I get a bunch of nonsense. The new nonsense is much more like normal sentences. It's doing a better job guessing in that way. I said a sentence and it only has a few sounds so it gives me a sentence (the wrong one)But this same improvement lets it get words right more often.
(DIR) Post #Azr1FvLdXS9uTK6z4a by futurebird@sauropods.win
2025-11-03T02:14:39Z
0 likes, 0 repeats
@gdupont @canacar “this is a test of the speech to text function to see if it’s able to work when my phone is not connected to the Internet. Given a context, such as writing about ants can it understand a word like integument the outer EXO skeleton of an ant.”That’s not bad at all!
(DIR) Post #Azr1U1vJt618v5Wp28 by futurebird@sauropods.win
2025-11-03T02:17:11Z
0 likes, 1 repeats
@knowuh @canacar @gdupont pointed out I could just disconnect from internet and wifi and test it again. So I did. And it works great! It really must be doing most of the work locally, including the more fancy stuff where it goes back and fixes words as you add more context. That makes me very happy because I like this feature.
(DIR) Post #Azr41VaKpMpUNUnvuK by gdupont@framapiaf.org
2025-11-03T02:45:38Z
0 likes, 0 repeats
@futurebird @knowuh @canacar Great to confirm (some of) the models are local. Side note: This doesn't warrantee the data is not shared when online or that bigger remote models are not used when online. But that's already better than most android cases I guess.
(DIR) Post #Azr44ILmUSyuCSKFk0 by gdupont@framapiaf.org
2025-11-03T02:46:09Z
0 likes, 0 repeats
@futurebird @canacar Sometimes the obvious is in plain sight ;-)
(DIR) Post #Azr4GJ1JpEHNkjTvmK by futurebird@sauropods.win
2025-11-03T02:48:21Z
0 likes, 0 repeats
@gdupont @canacar Don't know why it thinks I shouted the EXO part of exoskeleton... but, I'll take it.
(DIR) Post #Azr4Mg0FHz7gln19Rg by gdupont@framapiaf.org
2025-11-03T02:49:27Z
0 likes, 0 repeats
@futurebird @canacar Named entity collision?https://en.wikipedia.org/wiki/Exo(that would be funny)