[HN Gopher] Apple's new Transformer-powered predictive text model
___________________________________________________________________
Apple's new Transformer-powered predictive text model
Author : nojito
Score : 458 points
Date : 2023-09-17 03:00 UTC (20 hours ago)
(HTM) web link (jackcook.com)
(TXT) w3m dump (jackcook.com)
| jackcook wrote:
| Wow, it was quite the surprise to wake up to seeing this post
| near the top of HN! I wrote the post, happy to answer questions
| if anyone is wondering about any details.
| dkga wrote:
| Well, that was pretty cool. Plus, I got to know about the way
| processes communicate via xpc and that opened a whole new
| rabbit whole!
| callalex wrote:
| I was surprised to see the direct references to GPT2 by name.
| ShamelessC wrote:
| It's not an official Apple piece or anything.
| Angostura wrote:
| I think they mean surprising to see gpt directly referenced
| in file names.
| furyofantares wrote:
| I don't see that anywhere.
| rrsp wrote:
| 'Most of the layers within each decoder block have names
| like gpt2_transformer_layer_3d'
| ilaksh wrote:
| [flagged]
| TheRoque wrote:
| I know it's infuriating but that's how marketing works, and
| since they are a successful company, idk why they would stop
| using it. Just like some people say "RTX" instead of RayTracing
| now, it's quite a success for the company if it managed to
| replace a technical term in the mind of most consumers
| ilaksh wrote:
| I don't blame the marketing people, I blame Apple fans for
| adopting the marketing terms as if they were real technical
| terms. It's ridiculous.
| JimDabell wrote:
| "Transformer" is a real technical term. It comes from the
| _Attention is All You Need_ paper published by Google. It's
| not an Apple marketing term.
| [deleted]
| deprecative wrote:
| [flagged]
| dchest wrote:
| What? Transformer model is a standard industry term for a
| particular machine learning model architecture:
| https://en.wikipedia.org/wiki/Transformer_(machine_learning_...
|
| At 34M parameters, it's not very large. A LLM can be a
| transformer model, but also it can have a different
| architecture (e.g. RNN, such as RWKV).
|
| Apple has been using RNNs for many things, I believe it's the
| first time they're shipping an on-device transformer model.
| bobbylarrybobby wrote:
| This is the one case where the technical term and Apple's
| marketing actually do happen to overlap! These _are_
| transformer models. Apple didn 't invent them, but their
| description of them is technically accurate. "LLM" is the
| dumbed-down term for the masses, describing only the "what" but
| not the "how".
| ilaksh wrote:
| I know what a transformer model is. But Apple knows the
| normal technical term is LLM. They are deliberately avoiding
| it to try to create a new category for themselves, because
| their model's "intelligence" is very poor when compared to
| other LLMs.
| mortureb wrote:
| An LLM is a transformer model. That's the standard, industry
| term. What are you going on about?
| ilaksh wrote:
| I am very aware of that.
|
| Nowhere does the press release say LLM. That's because it's
| not very large or smart and when compared to most LLMs'
| "intellectual" performance, it looks bad. By avoiding that
| term, they are successfully sidestepping an important aspect
| of the evaluation of this system in the eyes of many Apple
| fans.
| ShrigmaMale wrote:
| you want to compare llama2, inflection1, gpt4 et al. to
| apple's typing assistant model that is designed to run
| locally and in a tiny scope? large language model literally
| isn't accurate for something that's two orders of magnitude
| off what the term usually describes. if anything, give
| apple points for not hyping up "LLM technology" or whatever
| to boost their share price like every other tech-adjacent
| company is rn.
| l33t7332273 wrote:
| > Nowhere does the press release say LLM. That's because
| it's not very large
|
| So your gripe is that they don't have a LLM and didn't say
| they did?
| codeflo wrote:
| > It's a small, fairly dumb LLM.
|
| I have no idea whether you really don't know what "LLM" stands
| for or are just trolling. Not calling it an LLM actually _is_
| more honest marketing precisely because it's small, and
| "transformer-based model" is a precise technical description
| that everybody with even superficial knowledge of the field
| understood immediately.
|
| And I'm actually onboard with most of your other examples. I'd
| include "all-day battery life" as another empty marketing term
| that doesn't really say much.
| ilaksh wrote:
| I know that LLM stands for large language model. It is still
| that type of model even though that final parameter count is
| not very large. They are deliberately avoid "LLM" because it
| doesn't compare favorably with most.
| Me1000 wrote:
| It's an autocomplete model, it's not designed to compare
| favorably to LLMs.
|
| And a transformer model is a specific type of LLM. You
| could also build a language model using a RNN. There's
| nothing deceptive here.
| codeflo wrote:
| > It is still that type of model
|
| "That type of model" are called transformers, not "like an
| LLM but small". The fact that you didn't know that doesn't
| make it wrong.
| ilaksh wrote:
| [flagged]
| nurettin wrote:
| This model wasn't advertised by apple, someone who is
| probably not even affiliated with apple wrote a piece on
| how he found, analyzed and activated a small neural network
| inside of an obscure folder. There are no misguided "apple
| fans" to incorrect here.
| danjc wrote:
| Inline predictive text is an anti-feature that disrupts your flow
| of thought. I've tried but cannot understand how anyone could
| actually want this.
|
| Edit: I was referring to writing prose where you're making
| creative decisions rather than code which is closer to a
| technical document.
|
| In code things are different. Traditional intellisense is usually
| just filtering a small set of possibilities. For example, auto
| completion of a reference to a variable.
|
| This is different to code prediction where for example, a
| variable name will be suggested.
|
| I can see value in code prediciton ala Copilot but personally
| don't use it.
| dchest wrote:
| I loved it in OpenOffice Writer in early 2000s: type a couple
| of letters and press Enter to complete. Same with autocomplete
| when programming.
| [deleted]
| Roark66 wrote:
| My favourite input app on android does real time autocorrect by
| displaying 3 most likely predictions on a bar above the
| keyboard. So it is not really inline.
|
| With programming IDEs when I can I configure them to display
| suggestions below the text I type, but if there is no such
| setting I don't find inline autocomplete bothersome at all. (as
| long as it displays it's prediction in dark shade of grey or
| another color sufficiently different from the text I typed,
| also there has to be a special key to accept the suggestion,
| like tab, no "enter accept")
| codeflo wrote:
| And interesting test. I didn't see a mention of the temperature
| setting used. Temperature controls the probability to pick a
| token that isn't the top prediction, which leads to more
| creative/less robotic results.
|
| For actual input prediction, you probably want the temperature to
| be zero. But even a model as good as GPT-3 becomes very boring
| and repetitive with those settings.
| smpanaro wrote:
| There is an input for temperature in the CPU model. If you can
| find and hook the call to predict you can probably see what is
| being passed.
|
| Interestingly the Neural Engine version of the model does not
| take a temperature input, but it does output the raw embeddings
| in addition to the logits.
| jackcook wrote:
| I used greedy sampling (temperature 0) for all of them. Since I
| didn't have access to logits/probabilities for Apple's model, I
| wasn't able to do anything else in a way that would be fair.
| ricardobeat wrote:
| I don't think you can make a fair comparison like this. The
| examples at the end are essentially praising GPT-2 for
| hallucinating - is it better when it's suggesting completely
| irrelevant text to your sentences? Apple's approach can't
| generate full sentences on its own, but that's not the goal
| anyway.
| IndySun wrote:
| Apple's current autocorrect is skewed towards verbal, not typo,
| errors. Apple wants your voice. Implementing this deliberately
| worse change also gave Apple a path to 'improve autocorrect'.
| Comments are correct, T9 was based on the keyboard and common
| mistyped keys, simple, and effective.
| [deleted]
| tpowell wrote:
| Is it possible for it to improve based on experience/iMessage
| history or is it locked in? I'd love to opt-in to training it on
| my previous convos...
| ComputerGuru wrote:
| It already does and I hate it. If a person you're speaking with
| misspells a word, it'll happy suggest^w forcibly autocorrect a
| similar word to that misspelling if it was recent enough.
| mcintyre1994 wrote:
| Do they do that across conversations, eg if I misspell
| something while texting you does it recommend my misspelling
| when you're talking to other people?
| petesergeant wrote:
| Yes. Ever since I went to the Maldives it wants to
| capitalize "male" and I have to work quite hard to get it
| to not correct "Yes" to "Yea", and "Thailand" to "Thailnd".
| Hate it.
| layer8 wrote:
| In _Settings_ there is _Reset Keyboard Dictionary_ to
| revert to the default dictionary.
| petesergeant wrote:
| Thank you!!
| layer8 wrote:
| Googling for something like "iphone wrong autocorrect"
| helps in such cases. ;)
| pjerem wrote:
| Yes
| artursapek wrote:
| This is called zoomer slang support
| coder543 wrote:
| I think tpowell was asking about the all-new autocorrect
| system. Do you have information on how that works, or were
| you just talking about the old system that is being deleted
| on Monday?
| ComputerGuru wrote:
| I have no evidence or reason to believe that the inputs to
| the corpus of valid word sources for the new system changed
| - most likely only the prediction model did. This
| misfeature only recently shipped (iOS 16, I think) and I'm
| sure Cupertino views it as a net gain.
| dkga wrote:
| Beyond fixing iPhone's autocorrect as widely mentioned below, I
| wonder if in future versions Apple will end up switching to
| phi-1.5 or other models that are much smaller but trained on
| higher quality data.
|
| Would also be cool if they trained their own copilot for Xcode,
| given their obviously enturmes code base in Swift and Obj-C.
| dchest wrote:
| I'd guess they already trained it on high quality (and highly
| censored) data. It's GPT-2-like architecture, but not GPT-2
| weights.
|
| phi-1.5 is 1.3 BILLION, this one is a lot smaller at 34
| million.
| fragmede wrote:
| What's "high quality" refer to here? The amount of txt spk I
| use on my iphone keyboard, vs words that are in the dictionary,
| is heavily skewed in the direction of short words that a corpus
| consisting of English text is likely to omit.
| egwor wrote:
| My hypothesis is that this question, "What's 'high quality'
| refer to here?", is a big challenge and the answer depends
| upon the audience. Here are a few examples:
|
| - kids use one level of slang
|
| - older ages groups using another (and it might mean
| something quite different)
|
| - communications for professional wouldn't use slang and
| would use formal messages
|
| - different cultures/regions will use different phrases for
| the same thing (and some colloquialisms are quite
| sophisticated like Cockney rhyming slang)
|
| - on twitter when out of space you might then shorten words
|
| The thing that really 'riles me up' / 'get on my wick' /
| 'annoys me' / 'is immensely frustrating' / 'is an opportunity
| for a sales feature' [trying to demonstrate the different
| phrasing depending upon audience] is when the auto correct
| 'fixes' grammar incorrectly. It needs to be clever enough to
| realise the correct use of there/their/they're and its/it
| is/it's.
|
| I'll leave with a relevant poem:
| https://allpoetry.com/poem/4010351-They-re-There-with-
| Their-...
| basisword wrote:
| You use text speak on an iPhone? How? I find it impossible
| due to autocorrect. Either it will correct with the full word
| or it will correct with a wrong word making it more efficient
| to just type in full.
| tuukkah wrote:
| Turn off autocorrect: https://support.apple.com/en-
| us/HT207525
| basisword wrote:
| My problem with this (I've tried it) is that the iOS
| keyboard still seems to try to guess which key you meant
| to type. So you hit "o" and it thinks you probably meant
| "p" and inserts that. Last I checked this still occurred
| with autocorrect off and made it even worse than using
| autocorrect.
| astrange wrote:
| Why? So you can use text speak? Bit silly.
| tuukkah wrote:
| In general so that what you write is not mutilated by
| software that doesn't know the words you use. If there
| are typos, it's not difficult for the receiver to
| "autocorrect" while reading, but when autocorrect
| miscorrects, it's not easy for the writer to notice how
| the message changed nor for the reader to guess what was
| there before the "fixes".
| kayodelycaon wrote:
| That assumes your typing and spelling accuracy is high
| and the other person reading has significant proficiency
| in your language.
|
| My typing and spelling accuracy is horrendous to the
| point I use autocorrect on my laptop. For me, it's an
| important accessibility feature. I also use dictation for
| single words.
| tuukkah wrote:
| Normal spellcheck still exists, highlights typos and lets
| you pick from suggested fixes.
|
| Anyway, I'm not saying you shouldn't use whatever works
| best for you but one size doesn't fit all.
| macNchz wrote:
| I've had autocorrect disabled on my iPhone for years--for
| whatever reason it really bothers me when something I
| just typed magically changes into something I didn't want
| to write. I like that what I type is _exactly_ what
| appears, not the computer's guess at my intent. It's one
| of the first things I disable when setting up a new Mac
| as well.
|
| It took a few months to get used to it, but I developed
| typo-correction muscle memory pretty quickly (RIP 3D
| Touch, it was awesome for text editing). Plus I think
| that the copyedit re-read that I give to things I write
| on my phone is helpful above and beyond just catching
| typos. So be it if I make the occasional typo!
| dkga wrote:
| I'm one of the "lucky few" that (a) really needs
| autocorrect because I can never seem to type correctly in
| mobile phone keyboards and (b) really hates autocorrect
| because often it either gets me wrong or worse, changes
| legitimate words into awkward and incorrect words - like
| the other day I typed something pretty mundane that got
| changed into a more... vulgar word.
|
| I got a little reprieve with swiping text, but still,
| autocorrect should be much better.
| ComputerGuru wrote:
| Just give me the option of not having autocorrect _change the
| previous word_ when I am typing a new one. I don't move on to the
| next word unless the current one was correct or I have already
| corrected it. Why does it assume I'm stupid? It even does this
| when I've picked the completion for the prior word or swiped to
| type and then made manual corrections. It's painfully obvious
| that it shouldn't be touched again.
| [deleted]
| kristopolous wrote:
| Apple has assumed its users are hapless morons for many years.
| Most people seem to prefer it this way.
| jeffbee wrote:
| And then deleting BOTH words with a single backspace. This is
| its most irritating behavior.
| LeoPanthera wrote:
| Settings > General > Keyboard > Delete Slide-to-Type by Word
| = Off
| willis936 wrote:
| That changes more behavior than what GP wants.
| barathr wrote:
| We might ask ourselves at some point: if the point of simple text
| prediction is to speed up text entry, and not generate large
| amounts of text from scratch, then that's a sign the input
| interfaces are the bottleneck. If we had a way to get text from
| our brains to the computer faster, we wouldn't need this kind of
| prediction.
| r0m4n0 wrote:
| iPhone's auto complete and form fills drive me absolutely mad.
| It's honestly just basic stuff that should have just worked a
| decade ago. I'm not even begging them for LLMs or some fancy ML,
| just learn basic words I type out all the time. For example my
| full email address? how about my last name? This seems like basic
| stuff
| ezfe wrote:
| My iPhone has always suggested those things...
| r0m4n0 wrote:
| Sometimes on obvious fields like an email input on a web page
| (not all though). Not in this comment box for example
| blindriver wrote:
| I don't want better predictive text. I need better autocorrect.
| Something happened about 6 years ago where the quality of the
| autocorrect fell off the roof, and it's been absolutely terrible
| since then. I spend too much of my time fighting with either
| mispelling, flipping to the wrong word even though I spelled the
| word properly, etc. It has made typing on my iPhone an unpleasant
| experience and I need it to change. That's just about the only
| thing better between the iPhone and Android for me.
| Roark66 wrote:
| For me stock autocorrect was always pretty much unusable, but I
| have no real complaints about the app I use. I'm bilingual and
| I type in both languages a lot. I remember when I got my first
| smartphone I bought SwiftKey app (later bought by Microsoft)
| that made language switching very easy by just swiping the
| space. Later the app got so good I just left it on default
| setting and it would still recommend the right words. Since I
| bought the app it has been always one if first things I
| installed on a new android phone (I think I had it on an iPhone
| 3g back in the day too, but I can't remember for sure).
|
| I recently realised how much of a difference it makes when you
| type quickly as I got an extra work phone with stock android I
| have to use occasionally.
|
| There is also a feature of this app I use only occasionally,
| but when I do its great. It let's you type by moving your
| finger between letters in one continously motion. When one
| describes it seems rather weird, but it's really one of the
| best features of the app. You can write really fast this way,
| the only difficulty it has is with very uncommon words.
| audunw wrote:
| > It let's you type by moving your finger between letters in
| one continously motion.
|
| FYI, this has been included in stock iOS keyboard for a while
| now.
| recursion wrote:
| And also stock Android.
| atraac wrote:
| I also use SwiftKey and can highly recommend. I use Polish
| and English, sometimes within the same message and native
| keyboard was an absolute shitshow for me. It would literally
| never correct properly, sometimes make up non existing words
| etc. My only pain point with SwiftKey is that since last
| update it crashes often, I hope they might actually fix it
| someday. GBoard on iOS was forgotten long time ago so that's
| a no go...
| kahnclusions wrote:
| Totally agree. I love my iPhone but autocorrect is absolutely
| the most painful and most infuriating part of the experience. I
| am _constantly_ fighting with the autocorrect... sometimes
| repeating the same word 3 times because I've typed it correctly
| but iPhone really wants to change it.
|
| Same with proper nouns. If I deliberately start a word with a
| capital letter it's probably a proper noun and I know how it's
| spelled, don't correct me.
|
| Fix this shit, Apple, it's the one part of iPhone I really
| hate.
| olah_1 wrote:
| Seconding that autocorrect is inexcusably bad on iPhone.
| happymellon wrote:
| Pretty sure my typing on T9 was faster and more accurate that
| the current state of keyboards.
| squeaky-clean wrote:
| I use SwiftKey and it's much better than the Apple one. But it
| does require giving Microsoft access to most of what you write.
| willis936 wrote:
| It does seem absurd that the options are "completely broken
| keyboard features" and "literal keylogger". Meanwhile we're
| paying for many transistors for NN engines and getting worse
| results than if they were never leveraged at all.
|
| Is this what progress looks like?
| nixpulvis wrote:
| How about just fixing the damn spellchecker on iOS?
|
| I can never press on underlined words that are the last word
| before a linebreak while typing.
|
| It's been broken for years. YEARS!
| [deleted]
| cmehdy wrote:
| I've had a similar experience with autocorrect on Android
| across multiple phones (lg, samsung) and keyboards (gboard,
| swiftkey) around the same time, I'm curious to know whether
| it's just my own bias being validated or something happened.
| All predictions have become completely stupid (switching
| languages mid sentence when they were robust in the past,
| finding least consistent suggestion first, suggesting very
| random terms I've never seen like "Gaylene" when I swiped to
| write "happened", etc).
| nprateem wrote:
| Same here. I get all sorts of nonsensical suggestions when
| swiping now that I never used to get. It changed a few years
| pre-covid, so yeah, 5-6 years ago. I assumed it was stunting
| to do with a list lawsuit against samsung or something
| colordrops wrote:
| Same. I suspect everyone switched algorithms at some point
| because it was "state of the art", but actually a step
| backwards in reality.
| gleenn wrote:
| Maybe they were optimizing battery or some other metric
| that wasn't necessarily prediction performance.
| gkoller wrote:
| Typing on Android was perfect when Swype was still around.
| GBoard and SwiftKey were worse and seem to actually decline
| in accuracy till this day.
| cja wrote:
| Swype was the best. Google should buy it and release it as
| the new version of Gboard
| freeone3000 wrote:
| Swype was acquired by Nuance which was acquired by
| Microsoft. Its competitor, SwiftKey, was also acquired by
| Microsoft.
| kevingadd wrote:
| I reset my gboard training data every 6-12 months since if
| I don't it seems to steadily get worse.
| navigate8310 wrote:
| I can vouch for this from personal experience. Back when I
| had my Nexus 4, the built-in keyboard surpassed the current
| Gboard by a long shot. I'm not sure what happened along the
| way, but the spell-checking capabilities of the completely
| offline and lightweight keyboard that Google shipped back in
| those days were miles ahead.
| tpowell wrote:
| This made me realize I was conflating these in my head, and I
| wholeheartedly agree. I had a 12 Pro Max 256GB until last year,
| and autocorrect was great/fine/never noticed issues. Fed up
| with heavy phones, I purchased a 14 Plus 512GB (very similar
| physical size) and set it up from scratch while migrating
| whatever is included in iCloud (Photos, iMessage etc).
| Autocorrect has been bad from the start on the 14 Plus, and has
| improved little in nearly a year. Typing feels _tedious_ now,
| where it felt like a strength previously. The phone as been a
| joy to use otherwise.
| deergomoo wrote:
| > Something happened about 6 years ago where the quality of the
| autocorrect fell off the roof, and it's been absolutely
| terrible since then
|
| That was when they first changed to a machine learning-based
| model. I'm hoping this one is better.
| bboygravity wrote:
| You could get a phone with a physical keyboard so you would
| barely need auto-correct in the first place :p
|
| Source: I have one :p
| [deleted]
| dchest wrote:
| It's powered by the same thing. They even mention "Even more
| accurate autocorrect" in iOS 17 on the product page.
| tasogare wrote:
| [dead]
| layer8 wrote:
| I don't understand why people still put up with Apple's
| autocorrect instead of just turning it off. It's so wrong so
| often as to be utterly useless.
| M4v3R wrote:
| Because it depends on the person. For me autocorrect is 99%
| spot on and so it's a valid trade off for me to deal with
| correcting the remaining 1%. My wife on the other hand cannot
| live with it so she turned it off. And every time I type
| something on her phone I immediately feel slower because I
| know each of my key presses have to be much more deliberate.
| dbbk wrote:
| In recent years mine has gone completely crazy and started
| autocorrecting to random words that don't exist. I think
| somehow over time it can get corrupted.
| paganel wrote:
| Look at the positive side, it made you type less on your phone
| (strangely enough, I had to fight with my SE's autocorrect
| three times in order to input "positive")
| jonplackett wrote:
| Totally agree.
|
| And I'd like to add that the editing experience is also dire
| and getting worse.
|
| Just trying to put the cursor where I want it to fix an error
| earlier in the sentence in ios17 is a massive pain in the ass.
|
| You tap somewhere and it continually selects entire words - I
| just want to put the cursor there dammit!
| texuf wrote:
| Press and hold the space bar to place the cursor exactly
| where you want it. It took me way too long to learn this.
| brrwind wrote:
| Thank you for mentioning this! I thought Apple had
| eliminated this editing mode when they got rid of Force
| Touch. So happy to learn that it's still available but a
| little frustrated I'm only learning this now.
| [deleted]
| jamiek88 wrote:
| If you hold down 'space' the cursor becomes Freeform. Works
| much better.
| artursapek wrote:
| Yeah I noticed this starting a year or two ago, it drives me
| crazy
|
| Also, getting the copy/paste menu to show takes a very long
| time
| jonplackett wrote:
| Yeah there's just so much delay. Editing/writing text is
| just plain tedious.
| artursapek wrote:
| Agreed!
| benreesman wrote:
| IMHO people don't often enough think through what the Google
| search type-ahead would be if it had 100s of millis per BPE
| token and a demanding cost structure.
|
| No one credible, least of all a guy like me who had been an
| enthusiastic amateur for a few years, is saying the stuff
| hadn't gotten pretty wild recently.
|
| But unprecedented? Nah.
| pentae wrote:
| Same with voice to text translation. It's absolutely, utterly,
| worthless. I end up having to go and edit half the message.
| Apple can't even get basic things right
| criddell wrote:
| I use voice to text all the time on my iPhone and find it to
| be astonishingly good. I'm a 50 year old man who grew up in
| Southwestern Ontario so that gives you an idea of my accent.
| kaba0 wrote:
| With ios 17 it also greatly improved (at least the input,
| mechanism itself).
| jamiedumont wrote:
| I broke my right (dominant) collarbone on Wednesday so I've
| entered the realm of the temporarily disabled user.
|
| So far I prefer iOS because the predictive text above the
| keyboard (in iOS 16 too) and in particular the speech to text
| feature has been so good!
|
| The only egregious error that I have encountered has been
| confusion around the phrase "into" when I mean "in two". An
| understandable mistake.
| messe wrote:
| Out of curiosity, what accent do you have? In my experience
| that can have a significant impact on how good speed to
| text works for you.
| jamiedumont wrote:
| Fairly neutral British. Born and raised in Guernsey so
| have a complete absence of regional accent without the
| very forced enunciation that comes from elocution lessons
| / Queens English. Admittedly I probably represent the
| best-case subject.
| messe wrote:
| > Admittedly I probably represent the best-case subject.
|
| I don't think you could do any better unless you were
| from Cupertino. With an Irish accent, my experience has
| been... okay with Apples text to speech, it's certainly
| better than others, but I still have to make a conscious
| effort to enunciate quite differently to my normal
| speech.
|
| OpenAI's whisper has really impressed me though, and
| transcribes almost everything perfectly, even if I throw
| in phrases or words from other languages part way through
| the conversation.
| AmericanChopper wrote:
| I regularly use 4 languages on my iPhone and my autocorrect is
| completely broken. I can't be bothered switching between
| keyboards, so the English keyboard has just completely lost the
| plot...
| therein wrote:
| Yup same, that's why I am sticking with Android as well. I
| want multi-language auto-correct for my preferred languages
| _on the English QWERTY keyboard_.
| P-Nuts wrote:
| I want the opposite! I know which language I'm typing but
| my phone still randomly decides I've written the (French
| for tea) instead of the.
| Me1000 wrote:
| Word (or multi-word) prediction is a great starting place for
| an autocorrect model. If the keyboard I'm using knows the
| probability of all the possible next tokens I could type, then
| you can start making the tap targets for those keys bigger. And
| since the model is super cheap to run, you can replay the last
| n keyboard taps to simulate a different correction being made.
| That's at least how I'd make autocorrect better if I worked on
| the keyboard team.
|
| Anecdotally, I've heard autocorrect is much better on iOS 17.
| But I won't update until the general release next week.
| deergomoo wrote:
| > then you can start making the tap targets for those keys
| bigger
|
| That's how autocorrect has worked since day one, it was just
| using a simpler heuristic-based prediction rather than a
| machine learning model.
| willis936 wrote:
| That's how it should be, but as others have implied and I
| claim outirght: the heuristic-based prediction has been
| broken for years.
|
| The solution badly needed isn't a complicated black box
| model when we know a simple one is good enough.
| aidos wrote:
| Maybe it's better and I just need to keep typing so it can
| figure out what I mean.
|
| In the last couple of weeks using the beta I've found myself
| fighting it more than ever. After years of typing by tapping
| at the screen, I've switched to swiping the words in because
| it seems to be more reliable.
| QuadrupleA wrote:
| Everyone's asking tech details and "how", but I wonder about the
| "why". Do we want LLMs to always write for us, or whisper in our
| ear what to say? By design LLMs tend toward the most commonplace,
| mainstream ideas and ways of saying things. They're not much for
| originality or human idiosyncracy. Are we engineering a bland
| world full of pablum?
| runjake wrote:
| Because at work I'm typing the same bland things all the time
| in documents and communications. I appreciate stuff like the
| predictive word stuff in Google Docs. It's helpful because
| business language is expected to be normalized and boring.
|
| On the other side of that token, the average language abilities
| of the average American office worker are pretty low so I'm
| assuming they view this as an enhanced AutoCorrect and they
| appreciate it because it makes them look less dumb.
|
| I agree with your point though. And to answer your last
| question, unfortunately I think the answer is "yes".
| _a_a_a_ wrote:
| > because business language is expected to be normalized and
| boring
|
| why?
| harshaw wrote:
| I might say it a bit differently. The amazon writing style
| is to avoid flowery prose, weasel words (maybe, perhaps,
| etc), give precise dates, use data, etc. I'd love a model
| that I could hand to engineers to help them write in this
| style.
| Zambyte wrote:
| > Because at work I'm typing the same bland things all the
| time in documents and communications.
|
| This problem can be solved without text prediction by
| building a personal knowledge base with hyperlinking[0] and
| backlinking[1] for discovery, and transclusion[2] for
| automating writing the same bland things. How I org in 2023
| by Nick Anderson[3] goes over a great workflow for this. The
| advantage of this is that all of the words that you share are
| actually words that you wrote, instead of sharing words that
| Google suggested you share.
|
| [0] https://en.wikipedia.org/wiki/Hyperlink [1]
| https://en.wikipedia.org/wiki/Backlink [2]
| https://en.wikipedia.org/wiki/Transclusion [3]
| https://cmdln.org/2023/03/25/how-i-org-in-2023/
| runjake wrote:
| I use macros for frequently-reused messaging but I meant
| business communication itself. It's essentially a different
| dialect of English meant to be "inclusive", unambiguous,
| and not prone to misinterpretation.
|
| Anyway, I like and use the feature daily at work. So do
| lots and lots of other people.
| shiponfire wrote:
| What an obtuse way of saying use links and quote replies
| Zambyte wrote:
| Thanks!
| vore wrote:
| On the other side of that token, the average language
| abilities of the average American office worker are pretty
| low so I'm assuming they view this as an enhanced AutoCorrect
| and they appreciate it because it makes them look less dumb.
|
| Awfully presumptuous, don't you think?
| dsco wrote:
| I would also assume this to be largely true, mainly because
| of how language in media has gone from being formal and
| informative to casual and less expressive.
| vore wrote:
| I agree there has been a style shift, I don't think
| writing has gotten any less informative. I personally
| don't like formal writing, but either way it has no
| bearing on whether or not ideas can be gotten across
| effectively. Substance over style!
| vore wrote:
| If I'm writing an email to my coworkers, what I really care
| about is the information I'm getting across and not really the
| presentation of it. If LLM autocomplete can speed up writing a
| totally functional email then how bland it is really isn't at
| the top of my mind.
| dclowd9901 wrote:
| Maybe not always (I assume an annoyed person could turn this
| feature off), but I think the general trend is, yes, especially
| for boring rote writing we have to get done for work or school
| that doesn't require a tremendous amount of creativity.
|
| I really look forward to using GPT to help me throw together
| RFCs, documentation, announcement letters, daily standup write
| ups and other artifacts like that that prevent me from getting
| actual work done.
| andrei_says_ wrote:
| Quite likely yes in the context of repetitive daily routine
| communication.
| brookst wrote:
| Most human communication is bland, and people who make a point
| of being unpredictable and shocking are usually pretty
| annoying.
|
| Think of it like spellcheck : the vast majority of the time it
| produces desired results, but if you really want to type
| bjPvc9fQ, you certainly can.
| mtkhaos wrote:
| This throws out the possibility of high quality content.
| quest88 wrote:
| Is this an apt analogy? Don't LLMs train off these bland
| humans you mention? Wouldn't LLMs then also be bland?
| jsolson wrote:
| Yes, but sometimes that's what's called for. I think I may
| have found my answer to where LLMs might be useful for
| short communications: softening something that might be
| interpreted as "curt" or even "rude" while not really
| changing the message.
| davio wrote:
| That's my mental model. First we had calculators. Then
| spellcheck. Now we have automated "let me google that for
| you"
| QuadrupleA wrote:
| Well it's not so much about deliberately / affectedly being
| original and weird (which is annoying), but just leaving some
| space for natural idiosyncratic ways of writing.
| avianlyric wrote:
| As someone with Dyslexia, who struggles a lot with written
| communication, and frequently finds myself fighting with
| spellcheckers, trying to get them to provide the correct
| correction. I'm very excited for these types of autocomplete
| systems. They're like spellcheckers, except they also use
| context to produce better results, and frequently remove the
| need to play "guess the right misspelling" to get a normal
| spellcheck to provide good suggestions.
|
| I'll accept the risk of blandness, if it means that written
| communication finally becomes "easy" for me to participate in.
| crazygringo wrote:
| It's not LLM's writing for us, it's just autocomplete.
|
| If the suggestion doesn't match what you were already planning
| on saying, you just ignore it.
|
| The human desire to be original and authentic is always going
| to be stronger.
|
| (It's much less effort to ignore it and keep typing your
| original thought, than it is to think about it, compare with
| what you were going to say, decide its version is better, and
| then accept it.)
| fulladder wrote:
| > The human desire to be original and authentic is always
| going to be stronger.
|
| Authenticity didn't start to become a thing people cared
| about until about the early 1990s, and it didn't blow up and
| take over the mainstream culture until the 2000s and 2010s.
|
| Prior to the 1990s, there was a lot of interest in
| professionalism. People spoke and wrote in overly formal,
| jargony ways that they perceived as being a marker for
| competence in some specialized professional domain. Being too
| honest/authentic would have been seen as unsophisticated and
| lower class.
|
| Just pointing out that what people strive to emulate can
| change over time. Once LLM writing becomes commonplace, it
| will probably become trendy to write in the exact opposite
| way that an LLM does!
| DiogenesKynikos wrote:
| One of the most famous American essays of the 19th Century
| is about authenticity.
|
| _Self-Reliance_ , by Ralph Waldo Emerson (1841):
| https://en.wikipedia.org/wiki/Self-Reliance
| CharlesW wrote:
| > _Authenticity didn 't start to become a thing people
| cared about until about the early 1990s..._
|
| In your lifetime, but...
|
| _Another common theme in the philosophy of both Dostoevsky
| and Descartes is the idea of authenticity. Dostoevsky
| believed that people must live authentically, in accordance
| with their true selves, and that this is essential for
| their happiness and well-being. Descartes, on the other
| hand, argued that the path to knowledge and certainty
| begins with the rejection of all previous beliefs and the
| adoption of a completely new and authentic perspective._ 1
|
| And of course, the U.S. counterculture movement of the
| 1960s was deeply preoccupied with authenticity:
|
| _You note three themes underlying the American hippie
| experience: authenticity, individualism and community. Why
| did these concepts stand out, and what did they mean in the
| context of the hippie /counterculture movement? / W.R.:
| Although hippies often disagreed about beliefs and
| practices, they shared a desire to be authentic. Members of
| the counterculture condemned mainstream society for being
| conformist, rule-driven and uptight. Authenticity meant
| "doing your own thing." Because freaks distrusted both
| society and government, individual decisions were applauded
| as the most authentic._2
|
| 1 https://vocal.media/geeks/freedom-authenticity-and-the-
| mind
|
| 2 https://www.washington.edu/news/2015/08/17/uw-historian-
| will...
| WirelessGigabit wrote:
| 2 problems with suggestions:
|
| 1) there is a sort order to them. And we don't know how the
| 'recommendations' work. What is a recommendation? Why is
| Google recommending something over something else?
|
| 2) it bogs down creativity. You end up not thinking and
| accepting the suggestion as 'good' enough.
| Tagbert wrote:
| Most writing does not need creativity. It is wrote
| communication. If you want to put some creativity into
| something or write something where you want to write with
| some personality, then turn off the predictions or use it
| just for spell check.
| maroonblazer wrote:
| My job requires me to correspond with dozens of people over
| email, Teams, and Slack every day. We're all trying to get
| work done and need our communications to be as succinct as
| possible. Sure, occasionally I might dress it up to add
| some humor, but that's ~1% of cases. An AI, with access to
| my entire corpus of work-related communication, could
| likely very easily predict most of my communications, since
| they fall into a small set of categories.
|
| "When can I expect to get $workproduct?"
|
| "Here's when I can commit to getting you $workproduct"
|
| "What's the estimated date for $milestone?"
|
| "Here's the project plan for $initiative"
|
| "I can't make this meeting, can you be sure to record it?"
|
| I welcome any tool that can predict what I want to type and
| does it for me. I'm not sure if it's my imagination or not
| but Outlook and Teams seem to have gotten better in this
| regard. I'll take more of that.
| nojs wrote:
| > it bogs down creativity. You end up not thinking and
| accepting the suggestion as 'good' enough.
|
| This is why I stopped using copilot autocomplete in my IDE.
| Once you see a suggestion, you can't make your brain un-see
| it.
| jsolson wrote:
| I find they're useful as a tool for achieving a particular
| structure or tone that isn't "my voice," including populating
| boilerplate from bullet points. Sometimes I'll go back and
| revisit everything they've produced, but they let me put
| something tolerable in place early. This lets me focus on the
| meaty parts of the text sooner than I'd otherwise find myself
| capable of.
|
| I personally don't find them useful for quick / short /
| informal communication like email, or at least not yet.
| [deleted]
| tempaccount420 wrote:
| I don't think people write much original thought on phones
| anyway.
| tmpX7dMeXU wrote:
| Yeah. I can't wait to see what your parents said about
| desktop computers.
| dartos wrote:
| Most sort messages people send on Apple devices aren't meant to
| be original, just to quickly convey some information.
| dwringer wrote:
| One approach that I've been using with a local LLM, mostly
| brainstorming for my friends and my dungeons and dragons
| sessions, is to set up a prompt to be completed with some
| certain detail of storyline, character background, etc., Then
| running that prompt maybe 50 or 100 times and keeping informal
| statistics on the different results that come out. In such an
| application, it can be used as much for inspiration of how
| _not_ to sound bland, commonplace, or derivative. You can pick
| the one in 100 oddity that strikes you as interesting, or
| simply make sure not to use any of the outcomes that the LLM
| came up with.
| MetaMonk wrote:
| LLMs will by necessity but unintentionally enforce phonotactics
| but more at the sentence / thought level.
| shawnz wrote:
| > Are we engineering a bland world full of pablum?
|
| Like any other technological advancement, we're freeing
| ourselves from the burden of wasting mental energy manually
| doing those commonplace unoriginal tasks which can be easily
| automated, so there's more bandwidth left for human beings to
| focus specifically on the things which can't be
| elicash wrote:
| "Bing Chat's" implementation already allows you to select more
| creative generation of text. It's just a radio button option.
| There are also different technical solutions for the LLM to
| select which word to generate that allow either for more
| interesting, or more predictable, words.
|
| This isn't to say a human element doesn't have a ton to offer!
| Just to say that we aren't necessarily engineering a bland
| world of pablum, either.
| QuadrupleA wrote:
| That's just temperature, which evens out the random
| probability a little of the N most probable next words. It's
| still vastly favoring the N most common ones based on the
| training corpus, and will have a hard time producing uncommon
| ones.
|
| E.g. try asking an LLM to name a real, non-famous person. The
| internet and it's training corpus is full of regular people,
| but you won't have much luck - they're statistically too
| uncommon to remember.
| fulladder wrote:
| Early versions of GPT would tell me about myself, but July
| of this year it was saying that it could not comment on a
| private individual. I'm not at all famous, but there's
| plenty of writing by and about me in the common training
| datasets.
|
| So, I don't agree that there's insufficient data for it to
| remember a random person. This was obviously a conscious
| decision, probably in response to situations like this one
| last spring: https://www.businessinsider.com/chatgpt-ai-
| made-up-sexual-ha...
| brookst wrote:
| > That's just temperature, which evens out the random
| probability a little of the N most probable next words.
|
| Source? A MS exec said that creative and precise are GPT4
| but balanced is a not (or not 100%):
| https://twitter.com/MParakhin/status/1693579775590224097
| QuadrupleA wrote:
| I was assuming those Bing Chat settings are temperature-
| based, those types of "creative/precise" controls usually
| are - but perhaps there's more to it.
| xp84 wrote:
| I would be skeptical of the results of that experiment,
| just because I assume the minders of the big LLMs have
| attempted to make it deeply uncomfortable with discussing
| anything that might be "personal." For a fun time, ask
| ChatGPT who was executed (as in capital punishment) in the
| US in a certain long-past year. It responds with a bunch of
| rubbish about privacy and about how an LLM can't be
| completely sure about stuff so it wouldn't be ok to
| speculate for fear of tarnishing someone's reputation, as
| though the person executed 15 years ago is going to sue
| OpenAI for sharing their name and what crime they were
| publicly convicted of and killed for.
| tambourine_man wrote:
| It's an important question, but one we're not addressing in so
| many areas. We're 8 billion in the world and more culturally
| homogeneous than ever.
|
| I think and hope the pendulum will swing back eventually, but
| my guess is that it's still got some way to go before that.
| babl-yc wrote:
| I disabled Gmail Smart Compose for this reason; I felt like it
| was putting words into my mouth suggesting entire sentences for
| the email.
|
| I'm much more open to using transformers as a better auto-
| correct, where it's one word at a time and uses the first
| letter as a filter. Especially on a tiny phone keyboard on the
| go.
| quadcore wrote:
| Those who still carefully craft their text messages will stand
| out.
| Zinu wrote:
| The example at the end made me wonder if Apple's model is
| actually better than GPT2 for text prediction. It generated
| garbage, but all that garbage made somewhat sense in the context
| of only the word "Today".
|
| Whereas GPT2 hallucinated random stuff about the US government. A
| text prediction model should predict what the user wanted to
| type, so if you evaluate the models based on that, GPT2 actually
| performed horribly, since the user showed zero intent in talking
| about the US.
| stavros wrote:
| It seems obvious to me that it's not, because if you asked a
| human to guess what comes after "today" in a text, they'd never
| say "probably some gibberish about a day a day".
| ricardobeat wrote:
| Garbage in, garbage out? The preceding text is gibberish, so
| the prediction will be worse. Presumably they also only show
| completions with a much higher confidence threshold.
| coldtea wrote:
| Maybe: "Today was fine. Since I've retired, I'm taking my
| life a day a day".
|
| Or maybe I wanted to express myself in the timeless words of
| the poets:
|
| "A day, a day of glory! A day that ends our woe! A day that
| tells of triumph. Against our vanquished foe!"
|
| "Rose is a rose is a rose is a rose. Loveliness extreme.
| Extra gaiters. Loveliness extreme."
|
| "A-well now, everybody's heard about the bird, everybody's
| heard about the bird, About the bird, the bird, bird bird
| bird, Haven't you heard about the bird? Don't you know that
| the bird's the word?"
| rayshan wrote:
| It's hallucination if you hate it, and creativity if you like
| it.
| cubefox wrote:
| The example at the end sounds just like the predictions you get
| from normal phone keyboards in the last couple of years, which
| presumably don't use a modern GPT-style language model. A bit
| disappointing.
| xp84 wrote:
| Seriously disappointing. I was expecting that it would not
| produce total gibberish. It acts like it's a Markov chain,
| and only considers the last 1-2 words. Identical to the
| currently-shipping thing that we've had for the past however-
| many years.
| BoorishBears wrote:
| People trying to draw this comparison proves making good
| products is harder than it seems...
|
| The default goal everyone is assuming is spitting out the
| longest correct sequence possible.
|
| But in reality the mental cost of a _wildly wrong_
| prediction is much higher than the mental cost of a
| _slightly wrong_ one, so what you 'd train the model for is
| sequences of a few words at most being with higher
| confidence.
|
| Most people can/will tune out slightly wrong words
| especially as they get a feel for what autocorrect is good
| and bad at.
|
| If you unleash the full range of tokens GPT 2 can normally
| output, you'll constantly be blasting out words they didn't
| expect.
|
| --
|
| The fact your long sequence prediction got better doesn't
| matter because the UI is autocomplete not "auto-write":
| they're still expecting to drive, and a smart but noisy
| copilot is worse than a dumb but lazy one in that case.
|
| I wouldn't be surprised if they trained the model to an
| effective context window of just a few hundred tokens with
| that in mind
| bloppe wrote:
| GPT-2 saw "today," and thought "this must be news copy" and
| generated more news copy. Given a few more words, it could have
| narrowed down the context. The Apple suggestions aren't even
| grammatically correct, seemingly no different from the shallow
| statistical completion we've already had for years, so it's
| weird that they branded it in lofty AI terms
| fh9302 wrote:
| Autocorrect doesn't suggest whole sentences so it is
| irrelevant if the remaining sentence is gibberish or not.
| firewolf34 wrote:
| Maybe it should though, given we have the power to do so
| (at least, with just a lil more power)
| shiponfire wrote:
| Yes but it also looks no better than the existing autocomplete
| they have, in which case why use a battery-draining midLM?
|
| "Today is a good day for you to be able to do it more than i
| just a few weeks to get a new."
| xp84 wrote:
| I want to know if this will be used to improve all the places
| where Apple devices attempt to interpret what you might mean to
| type, including the swipe keyboard. I've been suffering for years
| with their terrible, unusable swipe typing. You can't even get it
| to swipe "I love you" because it always prioritizes "your" over
| you, regardless of the context. I've even experimented extremely
| slowly and taken screen recordings to ensure the swipes are
| accurate. There are a couple of other completely astonishing
| common words that it 100% of the time gets wrong. I guess GPT3
| would probably tax the battery, but on the other hand, I'd like
| to have the freedom to try it, because it would let me finish
| tasks much easier if text entry wasn't like fighting with an
| insane overconfident toddler. Honestly, I don't know what
| happened. Text entry with iPhones 10 years ago was far less
| infuriating.
| pmcf wrote:
| You have to be ducking kidding. Works great for me.
| tornato7 wrote:
| I love you I love you I love you I love you
|
| Seems to work fine. Though I typically use Gboard because the
| swipe typing is much better.
|
| I find the most restrictive thing about Apple's autocorrect and
| speech to text to be the limited vocabulary. Once you start
| using any industry terms it completely fails.
| gorenb wrote:
| why are you using a <p> tag
| demarq wrote:
| I've been arguing this is the way AI should be deployed. Rather
| than trying to sell ai as an end to end solution, just let it do
| the small part that it can reliably do. It's cost effective for
| the host, and valuable for the user. win win engineering!
| tempaccount420 wrote:
| It makes a lot more economical sense to cloud-host big models
| like GPT-4 and optimize them for the hardware they run on. But
| for small models, sure, let the user run them locally,
| eliminating network latency.
| rgbrgb wrote:
| I wouldn't want to send everything I type to apple's auto
| suggest server. For me it's quite important that this is not
| apple's approach to privacy.
| apozem wrote:
| I mean, I get why all these startups are trying to sell AI as a
| panacea. It's an exciting technology and someone has to figure
| out its limits.
|
| That said, as a user, it is nice to see LLMs used in a small,
| discrete, undeniably useful way like this. No flashy promises,
| nothing new for me to learn- it's just autocorrect, but better.
| astrange wrote:
| It's not an LLM. The first L stands for large and this isn't.
| flangola7 wrote:
| What's the threshold to be called Large?
| brookst wrote:
| As soon as it exceeds medium.
| [deleted]
| chaxor wrote:
| UnilmCtrl seems to imply some dependence on Sochers CTRL models,
| but I wouldn't be surprised if it's just another name collision.
| Would be interesting if it were, as that would likely mean they
| have been working on releasing this for a few more years than the
| "chat-gpt craze". I am a bit biased, but I always have more
| respect if someone has worked in the NLP field for at least ~8
| years, rather than jumping on after Sparrow and RLHF.
| flakiness wrote:
| For text input support, being boring feels like a feature not a
| bug. For writing on mobile, you'll be writing a lot of boring
| short messages. That's said, it'd be cool if it can precondition
| with the app name - You'll probably write something more
| interesting on Pages vs on iMessage.
| [deleted]
| [deleted]
| Toutouxc wrote:
| Meanwhile my language and dozens of others with hundreds of
| millions of speakers still don't get the predictions at all,
| don't support multi-language typing and don't support swipe
| typing. Typing on a $1000 iPhone in one of these languages is
| still a 2007 experience.
| qingcharles wrote:
| Meanwhile, Microsoft managed to add Inuktitut to its products,
| despite only 30K speakers in the world.
|
| I think the Czech government needs to intervene. I believe
| that's what happened with getting Welsh into Windows too?
| Krasnol wrote:
| Well, maybe you're not the target group for this product and
| company and should buy one from the competition? Also tell it
| to your friends.
|
| I don't think companies like Apple listen to anything but the
| numbers.
| Toutouxc wrote:
| I'm an adult and I bought an iPhone knowing there are pros
| and cons. Some cons are more absurd than others and this is
| one of them. Otherwise I'm very happy with my Mini.
| tgv wrote:
| Modern language processing relies on pretty large corpora, and
| to bootstrap it requires a part to be of high quality and
| annotated, although for spelling correction you don't need
| annotation. Or you can go the GPT-3 route and use really huge
| corpora. If that doesn't exist for your language, and for many
| smaller languages it doesn't, it won't get better than 2007.
| corbezzoli wrote:
| Indonesian is spoken by 200M people (45M of which as first
| language), I bet they could find some text for it. When
| joined with the Malaysian family, it brings the total
| speakers to 290M.
| bobthepanda wrote:
| My understanding is that the languages with better support
| have the advantage of official bodies that have section-by-
| section translations of all their documents; the EU into
| its 24 official languages and the UN with its six, for
| starters.
| tgv wrote:
| Building corpora has been an ongoing research activity
| since the 1990s. Many countries lacked the academic
| programs and support to build them. Often, the material
| isn't even there. E.g., wikipedia is a great source, but
| many "national" wikis are rather empty. The Bahasa wiki has
| 1/10th of the articles of the English one. They also seem
| to be shorter.
|
| But also, Apple doesn't do much research. MS, however, has
| always had an active NLP research and support program, if
| only because of the importance of Word. But Apple depends
| on the research available from third parties.
| fauigerzigerk wrote:
| I'm not sure about that. It doesn't require a huge corpus to
| learn the structural features of a language. If you have that
| plus an English dictionary for the language, an English
| language model should transfer pretty well to any language.
| [deleted]
| Roark66 wrote:
| What language is it?
| Toutouxc wrote:
| In my case it's Czech, but it's the same for many others,
| even much more common.
| tasogare wrote:
| [dead]
| post-it wrote:
| My biggest multilingual gripe on iOS is that highlighting text
| in, say, Czech and pressing Translate on the popup just tells
| you to fuck off.
|
| Like, sure, don't support the language, but at least give me a
| Google Translate link then.
| cianmm wrote:
| They do this for cycling directions in Apple Maps too.
|
| Come on Apple, you KNOW you don't support cycling directions
| in Ireland - stop getting my hopes up every time I ask for
| directions. Just only show the button when you're in a geo
| that supports the feature.
| PartiallyTyped wrote:
| Same for Danish :(
| chongli wrote:
| Did you enable the Czech-English dictionary under Settings ->
| General -> Dictionary?
|
| You can also add Czech as a language under Settings ->
| General -> Language & Region -> Add Language. This will let
| you switch your keyboard to Czech when typing and enable the
| language on websites and stuff. You can also switch the whole
| phone to Czech if you like, just by moving Czech to the top
| of the list of preferred languages.
| post-it wrote:
| I do have the keyboard enabled, but the long press
| translation feature is a separate thing.
| a1o wrote:
| For comparison, by simply using Gboard in iPhone you don't
| need that and can type English and Czech in the same
| keyboard.
| SSLy wrote:
| Same with SwiftKey, if for any reason you'd rather not
| have your keyboard from GOOGL.
| jval43 wrote:
| Even worse, on the Apple Watch you cannot input text _at all_
| in one of these languages, as you cannot (!) turn autocorrect
| off on the watch.
|
| They managed to fit a full QWERTY keyboard on the screen yet
| made it completely useless.
| doug_life wrote:
| The thing I find most interning is that typing (in English)
| on the microscopic Apple Watch keyboard seems to have better
| results than typing on my phone keyboard. I don't know what
| the watch keyboard does differently but it seems to read my
| brain while my phone does the opposite.
| smpanaro wrote:
| I've been looking at these files too and have another data point
| for unilm.bundle being the new text prediction.
|
| If you take an iOS simulator, turn off "Settings > General >
| Keyboard > Predictive", reboot it and then watch the console logs
| as you turn that switch back on, you'll see the "kbd" process
| load the models out of that bundle.
| 01851159997 wrote:
| [flagged]
| 01851159997 wrote:
| [flagged]
| toxik wrote:
| The example output reads exactly like the existing output, I have
| had it get caught in exactly that cycle.
| dchest wrote:
| Even large language models with billions of parameters get
| caught in the cycle. You don't usually see it exposed to users
| because there are sampling tricks applied, such as
| repetition/frequency penalty.
| cubefox wrote:
| Such penalties would only exist for fine-tuned models. Base
| models have only the temperature setting. As the example at
| the end shows, even GPT-2 seems "smarter" than the Apple
| model, probably because of the number of parameters.
| dchest wrote:
| There's no such thing as "base models have only the
| temperature setting". Models do not have sampling settings
| (temperature, repetition penalty, etc), the sampling code
| does, which obviously you can use on any model.
|
| For example, here's a function from llama.cpp that applies
| repetition penalty: https://github.com/ggerganov/llama.cpp/
| blob/master/llama.cpp...
|
| Here's the one from transformers: https://github.com/huggin
| gface/transformers/blob/0a55d9f7376...
|
| To summarize how they work: you keep some number of
| previously generated tokens, and once you get logits that
| you want to sample a new token from, you find the logits
| for existing tokens and multiply them by a penalty, thus
| lowering the probability of the corresponding tokens.
| cubefox wrote:
| I don't think such penalties were applied to GPT-2 or
| even GPT-3, yet they weren't repetitive like that.
| dchest wrote:
| Yes, they are applied. Here's OpenAI doc which describes
| how to set various sampling parameters for GPT-3:
|
| https://platform.openai.com/docs/api-
| reference/completions/c...
|
| See presence_penalty and frequency_penalty.
|
| Sampling techniques is one of important arts of LLMs,
| you'll can find a lot of papers on them.
|
| In general, smaller are more prone to repetition, but you
| can get caught in it even with larger models.
| dchest wrote:
| To clarify: I meant that in general they are commonly
| applied, but in this case they weren't as the author
| confirmed. The repetition, of course, doesn't happen all
| the time.
| jackcook wrote:
| Yes, you're right, I should have mentioned it in the
| post, but I used pure greedy sampling for the GPT-2
| outputs since I couldn't do anything but that for the
| Apple model. So temperature was set to zero, and there
| was no repetition penalty.
| toxik wrote:
| I don't agree, it's not very common for LLMs to get stuck in
| loops simply because loops are not commonly observed in the
| datasets.
| dchest wrote:
| Try running llama.cpp with 0 temperature and without
| repetition penalty and you'll sometimes get caught in a
| loop.
|
| Or, if you're okay with a smaller LLM, go here, set
| temperature to zero and enjoy repetition:
| https://transformer.huggingface.co/doc/distil-gpt2
|
| https://imgur.com/a/hF1Ewk4
| avianlyric wrote:
| That's where you're wrong. Raw LLM very often get stuck in
| loops, usually extremely small loops. There's a big chunk
| of infrastructure that exists on the output end of any
| production LLM that exists explicitly for the purpose of
| preventing loops.
|
| That post-processing infrastructure can use all kinds of
| mechanisms to prevent loops and induce more useful output.
| With the most basic simply systems simply refusing to
| select any output token that already in the input, to more
| complex stochastic process that explore the tree of
| possible outputs, to find branches whose overall result is
| improved by choosing less optimal immediate steps.
|
| The vast majority of what make Chat GPT different to
| simpler GPT-3 models is this complex post-processing phase
| that allows designers to push and pull on the behaviour of
| the pre-baked static model underlying the chat interface.
| keyle wrote:
| In typical Apple fashion, when everyone's going bigger and
| stronger, they're going in a different direction... With
| optimising to the smallest model that can run all day without
| draining your battery.
|
| I love that they're almost never first to market but they find a
| way to distil value than others don't. It's the almond milk of
| technology.
| jaimex2 wrote:
| In typical Apple fanboy fashion they are oblivious to what
| Google has been doing for over two years now.
| kkielhofner wrote:
| Apple put neural in their silicon a full four years before
| Google did:
|
| https://www.apple.com/newsroom/2017/09/the-future-is-here-
| ip...
|
| https://blog.research.google/2021/11/improved-on-device-
| ml-o...
|
| Apple has just been more methodical to the rest of the
| ecosystem - essentially waiting to understand use-cases
| before fully embracing it across their ecosystem from Apple
| Silicon in Mac with neural and `device=mps`, CoreML, and now
| more-or-less full force with their ML studio now in Xcode.
|
| Many apps didn't wait for this and Snapchat, as one example,
| has been successfully taking advantage of Apple Neural since
| the early early days.
|
| There are many reasons Apple has been the world's most
| valuable company for over a decade. One certainly is that
| they have a prescient ability to predict where the market is
| going, even if they have to use their ever-building position
| to drag things that way. As demonstrated by the facts and
| references I provided above, not to mention the iPhone
| essentially re-defining what a smartphone is way back when
| Blackberry was the current market leader.
| lern_too_spel wrote:
| > Apple put neural in their silicon a full four years
| before Google did:
|
| You're confusing when Google started designing its own
| mobile SOC with when Android devices (including Google's)
| first started using neural network accelerators, which
| happened months earlier on Android than on iPhones.
|
| https://www.qualcomm.com/news/onq/2017/01/tensorflow-
| machine...
|
| As for GP's claim, Android has indeed had what the iPhone
| is now getting for two years:
| https://blog.research.google/2021/10/grammar-correction-
| as-y...
| kkielhofner wrote:
| > You're confusing when Google started designing its own
| mobile SOC with when Android devices (including Google's)
| first started using neural network accelerators, which
| happened months earlier on Android than on iPhones.
|
| Apple (famously) doesn't announce until new
| functionality/devices are actually available. When they
| announce real people in the real world get it within a
| week if not less.
|
| A press release from Qualcomm that came a few months
| before Apple actually got it in people's hands only
| further demonstrates that Apple was working on this long
| before Qualcomm included it in the chipsets that lag on
| the market from Android device manufacturers.
|
| > As for GP's claim, Android has indeed had what the
| iPhone is now getting for two years
|
| You meant to say that Pixel had a variant at that time
| with Gboard. I'm not going to bother to do the research
| on Gboard on iOS but I suspect it was also available on
| iOS with Gboard at the same time or soon thereafter. If
| not that's on the Gboard team at Google for not
| supporting neural on iOS and Apple hardware that came
| far, far sooner.
|
| On a somewhat-negative Apple take they are infamous for
| being, ummm, "inspired by" successful apps and add-ons in
| terms of what makes it to iOS. I wouldn't be surprised in
| the least if Gboard vs built-in iOS keyboard is another
| one of these cases.
| lern_too_spel wrote:
| > A press release from Qualcomm that came a few months
| before Apple actually got it in people's hands
|
| This press release was for a new Tensorflow release for
| devices that had been shipping for many months before
| that. It was _in people 's hands_ long before Apple even
| announced anything similar.
|
| > If not that's on the Gboard team at Google for not
| supporting neural on iOS and Apple hardware that came
| far, far sooner.
|
| Once again, Apple's hardware came later than Qualcomm's
| hardware. Pixel phones come with a lot of features that
| could be implemented on iOS, but why should Google go
| through the work of porting it only to promote a
| competitor's inferior platform? For example, Google Maps
| with navigation shipped years earlier on Android. Grammar
| check with a transformer model is now on all Android
| devices in Gboard (for more than a year and a half) and
| still no iOS devices.
| kkielhofner wrote:
| All this thread has taught me is that Google/Apple
| religious zealotry is a very real thing.
|
| What's interesting about this "debate" is my original
| links very, very, very clearly show the reality: Google
| being excited about launching the Pixel 6 (in 2021) with
| Tensor silicon - a first for them.
|
| All anyone on this thread has done since is refuse to
| acknowledge how obvious and clear that is while
| deflecting and throwing things at the wall.
| flangola7 wrote:
| How does Snapchat use it?
| kkielhofner wrote:
| https://qz.com/1005879/snapchat-quietly-revealed-how-it-
| can-...
| smoldesu wrote:
| The linked paper makes no reference to CoreML or Apple in
| general. It seems to be CPU-accelerated on all platforms,
| iPhone included.
|
| Do you have another source that goes into Apple's
| implementation?
| kkielhofner wrote:
| I'm pretty fatigued on constantly providing references
| and sources in this thread but an example of what they've
| made availably publicly (second result Googling "snap
| coreml"):
|
| https://github.com/snap-research/EfficientFormer
| lern_too_spel wrote:
| Your linked source says Snapchat has been using CoreML
| for neural network acceleration since 2022, which is not
| "since the early days" as you had originally claimed.
| kkielhofner wrote:
| The link was provided to refute "Snap uses CPU". The repo
| is from 2022 but do you really think Snap publishes
| breaking code and papers on new, novel, and highly
| competitive functionality? The snap-research GH org
| wasn't even created until 2020. Do you really think they
| weren't doing anything before that because it's not on
| Github?
|
| This thread has been exhausting, I suspect due to the
| religious war that is Apple/Google. As I've said time and
| time again - if you're genuinely curious Google is your
| friend here and it takes seconds to find any number of
| apps, projects, etc using CoreML in 2017.
| smoldesu wrote:
| It's important to ask, because your sources are not
| referring to the same thing. The first is a reference to
| training techniques that have nothing to do with CoreML
| or Apple hardware. The second thing you've linked is a
| transformer model from 2022 that was ported to CoreML
| (alongside Pytorch, ONNX and 10+ other execution
| providers).
|
| It's extremely unclear how any of these sources
| corroborate your claim, particularly the first link. It's
| why I asked for clarification.
| kkielhofner wrote:
| Is it really so hard to believe that the technical
| details of Snap's implementation just may not have been
| made publicly available at the time they deployed it?
| It's not as though they're known for running a breaking
| engineering blog like some smaller companies... Speaking
| of which:
|
| In my top comment I said "Snapchat, as one example". Feel
| free to search around to find others that were more
| transparent on implementation details, such as this
| random weight loss app from 2017:
|
| https://loseitblog.com/2017/11/10/on-device-image-
| recognitio...
|
| They actually include the following sentence: "While we
| are still awaiting the official launch of Tensorflow
| Lite, Apple released CoreML with iOS 11."
|
| And another:
|
| https://dl.acm.org/doi/abs/10.1145/3132787.3132815
|
| If you're genuinely curious there are plenty of
| references of apps using CoreML in 2017 (again, Google)
| but at this point I'm pretty confident you're just
| (lazily) trying to win an argument.
| [deleted]
___________________________________________________________________
(page generated 2023-09-17 23:01 UTC)