[HN Gopher] Ferret: A Multimodal Large Language Model
___________________________________________________________________
Ferret: A Multimodal Large Language Model
Author : weirdcat
Score : 407 points
Date : 2023-12-23 16:19 UTC (6 hours ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| aaronbrethorst wrote:
| Can someone define the term "MLLM"?
| schaefer wrote:
| Multimodal Large Language Model
| pests wrote:
| why not LLMM?
| CharlesW wrote:
| Because the first word is "multimodal" :^) and also because
| MLLM is the established initialism.
| pests wrote:
| My point was the phrase already contains the word model.
| Why are we calling it a multimodel LL model? Why not just
| add multi to the existing model?
| notdisliked wrote:
| Multimod _a_ l, not multimodel. Multimodal referring to
| the different possible modes of input (text, picture)
| into the model.
| TrueDuality wrote:
| Modalities and models are not the same thing.
| sva_ wrote:
| Modal, not model
| chaos_emergent wrote:
| MultimodAl, not multimodEl
| astrange wrote:
| There is something like a "multimodel LLM" but it's
| called MoE ("mixture of experts").
| replygirl wrote:
| what's a language multimodal model
| bbor wrote:
| Ok our options Multimodal large language
| model Large multimodal language model Large
| language multimodal model Large language model
| (multimodal)
|
| I prefer 1, because this is a multimodal type of an
| existing technique already referred to as LLM. If I was
| king, I'd do Omnimodal Linguistic Minds, but no one asks me
| such things, thank god
| n2d4 wrote:
| I mean, if we want to be silly, what about "Language
| model (large, multimodal"? :)
| rain_iwakura wrote:
| The bikeshed color argument never ceases to be relevant.
| Would you say "large language model multimodal"? I doubt
| it.
| Tempest1981 wrote:
| Also, is FERRET an acronym?
| Someone wrote:
| I would guess it's wordplay on other models being named after
| animals (llama, vicuna) and figurative use of "ferret".
|
| https://en.m.wiktionary.org/wiki/ferret: _"3. (figurative) A
| diligent searcher"_
| CamperBob2 wrote:
| The language model works by delegating tasks to smaller
| language models and overcharging them for GPU time.
| ZeroCool2u wrote:
| We're watching Apple fill the moat in.
| FredPret wrote:
| How so?
| colesantiago wrote:
| Running Multimodal LLMs on device and offline, i.e LLMKit for
| free equaling GPT-3.5 / 4 then Google will follow on Android.
|
| Ability to download / update tiny models from Apple and
| Google as they improve, a la Google Maps.
|
| No need for web services like ChatGPT.
| FredPret wrote:
| So Apple is filling in ChatGPT's moat then, not their own?
| Pardon my confusion
| colesantiago wrote:
| Yes, it looks like Apple is going after everyone and
| anyone that has a web based LLM, ChatGPT, Poe, Claude,
| etc. via developer kits LLMKit that can work offline.
|
| This will only work if their models (even their tiny or
| even medium / base models) equal (or are better than)
| GPT-3.5 / 4.
|
| From there, Google will follow Apple in doing this
| offline / local LLM play with Gemini.
|
| OpenAI's ChatGPT moat will certainly shrink a bit unless
| they release another powerful multimodal model.
| turnsout wrote:
| Apple's moat has been and continues to be their insanely
| large installed base of high-margin hardware devices.
| Meanwhile, LLMs are rapidly becoming so commoditized that
| consumers are already expecting them to be built-in to
| every product. Eventually LLMs will be like spell check--
| completely standard and undifferentiated.
|
| If OpenAI wants to survive, they will need to expand way
| beyond their current business model of charging for
| access to an LLM. The logical place for them to go would
| be custom chipsets or ARM/RISCV IP blocks for inference.
| CharlesW wrote:
| I believe that's the point the parent commenter was
| trying to make, although as the leaked Google document
| noted, " _[Google has]_ no moat and neither does OpenAI
| ".
|
| This is more evidence that Apple is investing in building
| a MLLM as good as anything OpenAI and Google can build,
| albeit in a more Apple-y way (privacy-first, licensed
| content, etc.).
| m3kw9 wrote:
| OpenAI can just copy this.
| yreg wrote:
| They cannot integrate it deeply into Apple's platforms.
| colesantiago wrote:
| Hence OpenAI is looking at hardware themselves.
|
| https://www.reuters.com/technology/chatgpt-owner-openai-
| is-e...
|
| https://techcrunch.com/2023/09/27/openai-is-reportedly-in-
| ta...
| nicce wrote:
| It does not matter much if they do not make competitor
| for iPhones and get consumers to choose them. Because
| consumers keep buying iPhones and will have then Apple
| hardware.
|
| And they cannot bring software ecosystem for their
| hardware without Google, at least easily.
| smoldesu wrote:
| It doesn't matter because Apple would not offer OpenAI
| actual integration terms, period. Their only option is to
| create an alternative hardware platform, because Apple
| fights tooth-and-nail to pick and choose what software
| iPhone users are allowed to run.
| nicce wrote:
| That is what I was saying...
| philistine wrote:
| Never bet against the phone. Right now it's an essential
| component of any winning move in tech.
| smoldesu wrote:
| If you _do_ bet on the phone, make sure you only bet on
| the first-party OEM. API depreciation, exclusive
| entitlements, user restrictions, payment-processing or
| flat-out Sherlocking are all realistic and credible
| threats to your business.
|
| As they say, the house always wins.
| daralthus wrote:
| Don't think they have AR Glasses just yet.
| pridkett wrote:
| Yes, OpenAI can copy this, but they'll still have less of a
| moat. That's the problem with moats, once they're gone even
| if you copy what others do, you don't have a moat anymore.
|
| Think of it in a physical sense. OpenAI is a high walled
| castle surrounded by a physical moat. This protects them and
| their business model. Apple comes along and builds a super
| tall tower right next to the moat. They can now see into
| OpenAI's castle, fire arrows, catapult in a giant wooden
| badger, etc. Even if Open AI copies the design of Apple's
| really tall tower and built it behind the moat and castle
| walls, it wouldn't do much because Apple still would be able
| to get stuff over the moat and walls. The moat doesn't matter
| anymore for the most part. The castle (OpenAI) can be
| compromised and needs bigger walls, relocating to someplace
| with a bigger, or a way of attacking the tower (Apple).
| Copying doesn't really accomplish any of those three.
| tomrod wrote:
| Here it comes!
| jonahbenton wrote:
| Dig the moat out, I think you mean ;)
| CaptainOfCoit wrote:
| Maybe the abstract of the paper is a better introduction to what
| this is:
|
| > We introduce Ferret, a new Multimodal Large Language Model
| (MLLM) capable of understanding spatial referring of any shape or
| granularity within an image and accurately grounding open-
| vocabulary descriptions. To unify referring and grounding in the
| LLM paradigm, Ferret employs a novel and powerful hybrid region
| representation that integrates discrete coordinates and
| continuous features jointly to represent a region in the image.
| To extract the continuous features of versatile regions, we
| propose a spatial-aware visual sampler, adept at handling varying
| sparsity across different shapes. Consequently, Ferret can accept
| diverse region inputs, such as points, bounding boxes, and free-
| form shapes. To bolster the desired capability of Ferret, we
| curate GRIT, a comprehensive refer-and-ground instruction tuning
| dataset including 1.1M samples that contain rich hierarchical
| spatial knowledge, with 95K hard negative data to promote model
| robustness. The resulting model not only achieves superior
| performance in classical referring and grounding tasks, but also
| greatly outperforms existing MLLMs in region-based and
| localization-demanded multimodal chatting. Our evaluations also
| reveal a significantly improved capability of describing image
| details and a remarkable alleviation in object hallucination.
|
| https://arxiv.org/abs/2310.07704
| s3p wrote:
| Is it just me or did they include as many buzzwords as possible
| in technical writing?
| barbecue_sauce wrote:
| >>spatial referring
|
| I can't seem to nail down the meaning of this phrase on its
| own. All the search results seem to turn up are "spatial
| referring expressions".
| TrueDuality wrote:
| I'm just inferring myself, but I believe it's referring to
| discussing things in the foreground / background or in a
| specific location in the provided image (such as top right,
| behind the tree, etc) in user queries.
| lukasb wrote:
| It sounds like the "region inputs" are raster or vector
| inputs. So I'm imagining highlighting a region of the photo
| with my finger and having it tell me "that's the Duomo in
| Florence."
| devinprater wrote:
| This is going to be great for accessibility! Imagine being
| blind and loading up a video game and using this to figure out
| what's around, having everything described locally. I mean, um,
| well that's what I'd use it for anyway. But knowing Apple, we
| won't be able to prompt the LLM directly so that probably won't
| happen until 5 years from now.
| MBCook wrote:
| The Magnifier app on iOS can already describe whatever you
| point your phone at in iOS 17.
|
| It's not going to know an orc from a health potion, but
| they're certainly working on the idea in the everyday stuff
| domain.
| samstave wrote:
| This will make Drone-based AI image context for behavior
| extremely powerful - especially when aspects of that MLLM
| handling for spatial-sitrep extremely precise for autonomous
| movement, then ultimately for decision making WRT interacting
| with humans (positive interactions and negative interactions).
|
| Is it just me, or doesnt this MLLM seem particularly useful for
| flying objects with vision?
| SushiHippie wrote:
| > Usage and License Notices: The data, and code is intended and
| licensed for research use only. They are also restricted to uses
| that follow the license agreement of LLaMA, Vicuna and GPT-4. The
| dataset is CC BY NC 4.0 (allowing only non-commercial use) and
| models trained using the dataset should not be used outside of
| research purposes.
|
| Wait, how did "GPT-4" get in there?
| simonw wrote:
| Presumably because GPT-4 generated training data was used
| somewhere along the line - maybe by Vicuna.
| adastra22 wrote:
| Lawyers.
| owenversteeg wrote:
| Huh, interesting, that's Apple just openly saying that GPT-4
| was used in the training.
| freedomben wrote:
| > Usage and License Notices: The data, and code is intended and
| licensed for research use only.
| echelon wrote:
| Boo.
|
| But what do we expect from these giants? They're not going to
| create fertile ground for new competition. The only businesses
| they foster are those living under thumb and paying tax.
|
| I guess I at least hoped for "commoditize the compliments"
| here. Make Google and OpenAI broadly less special.
| cyanydeez wrote:
| it's more likely it's all "stolen" and this is CYA
| MBCook wrote:
| I seriously doubt that. I'm sure Apple got the rights to
| whatever they need, it's not like they're short on money.
|
| But the fact that they licensed it doesn't mean that
| license can be transferred to other people. So it may be
| that they can only release it for research under the terms
| of the licenses they got.
| dbish wrote:
| Many big "open source" releases in the AI community recently
| are not licensed for commercial use. Not really OSS at that
| point (ex:fuyu model from adept)
| fragmede wrote:
| I think the term should be "model available" rather than open
| source.
| yreg wrote:
| I really hope Apple releases an iPhone with a good on-device
| private LLM assistant, perhaps next year. Their hardware is well-
| positioned for it.
|
| It could make me get a new phone outside of my usual ~4 year
| cycle. Siri is almost unusable for me.
| CaptainOfCoit wrote:
| You're unlikely to get a better experience with Siri if she
| becomes equipped with a 7B or 13B LLM, unless Apple figured out
| something revolutionary.
| s3p wrote:
| Why would that be?
| ghqst wrote:
| Have you ever actually used Siri?
| jurmous wrote:
| Released 2 days ago by Apple, a research paper on methods to
| run larger llms on iPhones.
|
| https://www.macrumors.com/2023/12/21/apple-ai-researchers-
| ru... https://arxiv.org/pdf/2312.11514.pdf
| reissbaker wrote:
| The paper was definitely cool but doesn't allow you to run
| particularly large LLMs on iPhones. It allows you to run a
| certain kind of LLM (sparse ReLU based LLMs) whose weights
| are somewhere less than 2x RAM. So, 7b Falcon works, but
| the competitive-with-gpt-3.5-turbo LLMs are still out of
| reach (and aren't ReLU based, although maybe that could
| change in the future). And nothing is competitive with
| GPT-4 right now.
|
| Of course in the long run I think it will happen -- smaller
| and more efficient models are getting better regularly, and
| Apple can also just ship their new iPhones with larger
| amounts of RAM. But I'd be very surprised if there was
| GPT-4 level intelligence running locally on an iPhone
| within the next couple years -- that sized model is so big
| right now even with significant memory optimizations, and I
| think distilling it down to iPhone size would be very hard
| even if you had access to the weights (and Apple doesn't).
| More likely there will be small models that run locally,
| but that fall back to large models running on servers
| somewhere for complex tasks, at least for the next couple
| years.
| SpaceManNabs wrote:
| > And nothing is competitive with GPT-4 right now.
|
| You mean nothing available? Or you mean nothing that
| public knows exists? The answers to those two questions
| are different. There are definitely products that aren't
| available but the public knows exist and are upcoming
| that are in GPT-4's ballpark.
| olddustytrail wrote:
| Easy to claim but harder to prove. Name one.
| mikhailt wrote:
| Yea but it's likely to be better than the current
| iteration of Siri even in that state.
|
| They can still outsource to a much larger LLMs on their
| servers for anything that can't be done locally like they
| do now.
| bbor wrote:
| Note that "using an LLM" doesn't just mean "plugging user
| queries straight into an LLM". Enhancing Siri will probably
| be an ensemble project.
| dbish wrote:
| This is part of the mismatch between comparing
| Alexa/Siri/Cortana to a chat based LLM right now. If you
| just want chat and info retrieval, today's LLMs are way
| better then the conversational dialogue, search, and q&a
| capabilities any of those assistants have. But, if you want
| relatively predicable task completion for things like smart
| home control, timers and alarms, real time weather updates,
| etc. (basic but frequently used interactions) or any
| integration with your phone or computer directly, there's a
| lot to do that isn't solved yet in the LLM space and is
| more of a system integration and action choice problem that
| the existing assistants have been hammering away at for
| years.
| bbor wrote:
| I would argue that "info retrieval" is also something the
| LLM space has yet to yet to solve to a human level of
| reliability, but I think your comment is right on. I see
| this all as part of the greater symbolic vs. stochastic
| (/neat v scruffy) dynamics
| spookthesunset wrote:
| I would hope that it would at last get you out of the
| hell that is "what the heck did I name that stupid
| light?". Device naming is, in my opinion, the worst part
| of any of the voice based home assistant things.
|
| Is it "porch led", "porch light" or "outdoor light"? Or
| is "outdoor light" actually the one in the front yard?
| What is the one by my kids nightstand? And what routine
| name do I use to set the mood for watching a movie?
|
| I would hope a properly trained llm with an awareness of
| my devices and their locations would allow for more
| verbal ambiguity when controlling things.
| nexuist wrote:
| Siri is really quite dumb. I am confident that a 7B model
| would be able to provide better responses in over 90% of user
| queries. I can't even get Siri to reliably set a timer.
| CaptainOfCoit wrote:
| Yes, Siri is really dumb. But so is every 7B/13B model out
| there too.
| kossTKR wrote:
| Eh no, 7B Mistral / Deepseek would certainly almost
| already be able to function as a super Siri, but probably
| something closer to PHI-2 + the new MLX apple
| optimisations. Have you tried those?
| https://twitter.com/awnihannun/status/1735355067673526360
|
| If trained on an iPhone API + documentation and given a
| little web access it would blow absolutely everything out
| of the water.
|
| If they can already create -basic- Python/Swift/JS/rust
| apps that sets timers, save things, create lists, how's
| that too dumb for being a Siri replacement? They just
| have to give it access to an iPhone/Web Api like
| ChatGPT's code analysis tool.
|
| So if you ask it "hey siri do this, this and this", it
| will create a script, then run it on the internal API, or
| fetch an article then work on that etc.
|
| I know it's still logically "dumb" but i'm not trying to
| play game theoretical scenarios with my phone or do logic
| tests or advanced math (yet).
| skygazer wrote:
| That sounds amazing and also the jailbreak of it via
| adversarial voice prompting sounds like a horrific
| vulnerability.
| kossTKR wrote:
| True but you could make the api restricted, having
| certain routes completely locked, some requiring double
| checks, some requiring on screen approval or face-id,
| throttling outside fetches, only being able to run get
| and not etc, no financial app control etc.
|
| But yeah "hey siri transfer all of my funds to eric", or
| "hey siri group all of my photos where i'm nude and send
| them to jack" are new almost sci fi vectors.
| bugglebeetle wrote:
| Depends on if they implement some form of function
| calling, really. If something like a 7B Mistral fine-tune
| had access to search and various applications, I imagine
| it would perform fine and better than Siri.
| sigmar wrote:
| Ask perplexity7B-online anything and then compare it to
| siri. https://labs.perplexity.ai/
| aaronbrethorst wrote:
| Rumors suggest they're gearing up to make iOS 18 an AI focused
| release. It'll be interesting to see if they offer different
| capabilities for online/offline scenarios, or if their
| offerings are strictly offline.
|
| Here's one story to offer some context. There are others.
| https://archive.is/en3VL
| para_parolu wrote:
| I really hope they will make siri usable. In the current
| state it's only good for fixed phrases. And even then it
| fails time to time
| behnamoh wrote:
| I hope they get rid of it completely. People on r/locallama
| and others have made much better assistants using GPT which
| use iOS APIs to control the phone. It's ridiculous that
| Apple still hasn't done anything useful regarding Siri.
| michelb wrote:
| It's been in an 'AI' rewrite for a while now. Pretty sure
| we'll see something next year.
| astrange wrote:
| It's been rewritten several times, people largely don't
| notice because they don't try using it in different
| languages. And of course because they want to seem savvy
| so they repeat impressions from other people's posts, not
| realizing those posts are years old.
| ben_w wrote:
| I'd be (pleasantly) surprised; look at how long it takes
| them to allow system apps to get replaced with
| downloadable 3rd party.
|
| And even then, the activation keyword is likely to be
| whatever Apple says. Similar logic as 3rd party
| keyboards, don't want user input to get stuck on _even
| merely potentially_ untrustworthy or buggy code.
| spookthesunset wrote:
| Correct me if I'm wrong but isn't the activation word
| "special" in that it has to work in very low power
| states? I always assumed that is why the wake words are
| fixed, because said words need to "fit" within a very
| small power budget.
| ben_w wrote:
| Could be -- I've heard rumours along those lines, but
| none came with evidence.
| skygazer wrote:
| https://machinelearning.apple.com/research/hey-siri
| (2017)
|
| The rumors were true.
| darthrupert wrote:
| That's not due to incompetence; that's due to not seeing
| any reason to do it.
| ben_w wrote:
| I was not intending to imply otherwise.
| behnamoh wrote:
| > Rumors suggest they're gearing up to make iOS 18 an AI
| focused release.
|
| Don't underestimate Apple at disappointing enthusiasts like
| you and me. We've been hearing many awesome stories about the
| next thing Apple will do, only to realize their marketing
| team chose to keep it for future iOS/MBP/iPhone generations
| to keep the profits high.
| baz00 wrote:
| I think your expectations are wrong. They sit there in
| silence with a few leaks here and there and some github
| projects, people speculate and get all excited about
| extrapolating those things. Then Apple deliver what works
| which may or may not be related to it.
|
| What they don't do is sell you a lie a year before release
| then deliver shit (like every other fucking vendor).
| zdragnar wrote:
| They're plenty capable of delivering garbage. Certain
| year models of the MacBook pro were inherently faulty.
| I've had the displeasure of having two of them bought for
| me at work.
|
| All of Apple's windows software (iTunes, Safari, etc) has
| been, at best, a barely working port.
|
| I'm assuming they are putting a lot more thought and care
| into it than the touchbar, crappy keyboards and the rest,
| but I'm also not holding out much hope either.
| baz00 wrote:
| Since they got rid of Jony, it's been great. That's all
| I'm saying.
| mpweiher wrote:
| Absolutely!
|
| The hardware, that is.
|
| With software, their Come to Jesus moment is still in the
| future.
|
| (To me, Swift is sort of the Jony correlate on the
| software side. Doesn't fit perfectly, of course, but very
| similar "we are perfect who cares about evidence la la la
| I can't hear you" vibes and results)
| behnamoh wrote:
| Yes, they have treated macOS like a toy. It's time they
| made it a real OS.
| pjerem wrote:
| Hello MacBook Pro 2016 !
| scarface_74 wrote:
| And that Apple /// that Apple released in 1980 was also
| garbage...
| MBCook wrote:
| Apple can screw up, no question. But they don't do the
| two-year hype cycle thing that just about everyone else
| does in technology (or video games).
|
| It's incredibly rare for Apple to publicly talk about
| things that won't be selling extremely soon.
|
| The iPhone had to be pronounced because it was going to
| show up on the FCC website, and obviously Apple wanted to
| control the message. I suspect the Vision Pro may be
| similar, but they also wanted developers to start getting
| ready so they would have software day one.
|
| The only thing I can think of that Apple pre-announced
| and failed at was the Air Power mat. They said it would
| be coming out soon after and had to push that a couple
| times before finally canceling it.
|
| Other than that small exception, if modern (post jobs
| return) Apple announces something is coming, it will come
| out and be quite close to what they say.
|
| They don't pull a Humane AI, Segway, Cyberpunk 2077, or
| No Man's Sky.
| behnamoh wrote:
| > What they don't do is sell you a lie a year before
| release then deliver shit (like every other fucking
| vendor).
|
| If you're referring to Google, then you're right. But
| OpenAI has consistently delivered what they announced
| pretty quickly. Same with Microsoft. To think that Apple
| somehow has a secret sauce that helps them surprise
| everyone is an illusion. They've had 3 years now to show
| their interest in LLMs, but they're just too conservative
| to "think different" anymore.
| cpill wrote:
| I think LLMs are to inconsistent for Apple's taste, I
| mean that in both senses. Their perfectionism won't risk
| bad output, ever, which is impossible for LLMs
| darthrupert wrote:
| My expectation of Apple is that they lurk in the shadows,
| looking at what others do while perfecting their own thing.
| Then on the day of release, they'll be a decade ahead of
| competition.
|
| They've done this a dozen times already.
| antiframe wrote:
| Which dozen times has Apple released something decades
| ahead of the competition? I'm blanking on 4-12.
| xerxes249 wrote:
| 64-bit phones is the easy one.
| cj wrote:
| Spotify / Apple Music
|
| Netflix + Hulu / Apple TV+
|
| Generic Earbuds / AirPods
|
| Meta Quest / Apple Vision Pro
|
| (The last one being a hopeful wish)
| cromwellian wrote:
| ? AppleTV and Apple Music are not decades ahead of
| anything. AirPods are way better than the existing
| Bluetooth headsets that were on the market.
| Someone wrote:
| Running a LLM on-device alongside other apps (i.e. without
| it taking up all phone resources), and it being reasonably
| fast may well require more powerful hardware than they ever
| sold.
|
| A voice assistant that takes 3 seconds to reply and then
| takes half a second per word is a nice demo, but not a
| product Apple wants to sell.
|
| And yes, some people will say they rather have that than
| nothing on their hardware, but "the Internet" would say iOS
| 18 is slow, eats battery life, etc, damaging Apple's brand.
| dzhiurgis wrote:
| This. If there is slightest chance model can say "poop" -
| they'll can it
| ignoramous wrote:
| > _marketing team chose to keep it for future iOS
| /MBP/iPhone generations to keep the profits high._
|
| VisionPro is nice. I can see costs coming down over a
| period of time. Also, we've been waiting long enough for
| that _AI_ car.
| georgespencer wrote:
| > We've been hearing many awesome stories about the next
| thing Apple will do, only to realize their marketing team
| chose to keep it for future iOS/MBP/iPhone generations to
| keep the profits high.
|
| I believe it's more likely we've heard awesome stories
| about things Apple will do in the future, only to realize
| that the average HN commenter is incapable of understanding
| that such stories are contextless leaks, and that it is far
| more likely you are operating with incomplete information
| than Apple's "marketing team" is holding things back for
| future "iOS/MBP/iPhone generations" to keep their profits
| high.
|
| I know it's more fun to vomit dumb conspiracies onto the
| internet, but consider changing "realize" to something
| which conveys equivocation, because your theory about
| Apple's marketing team holding back mature technology in
| order to benefit future devices - in addition to being
| predicated on leaks and rumors, and risibly inane when you
| consider that such action would create a significant attack
| vector for competitors - is as equivocal as the belief that
| Trump is on a secret mission to destroy a global cabal of
| pedophiles.
| 0x1ceb00da wrote:
| If they do it right, it might make me switch from Android.
| I've never used iOS before and the only thing I'm able to use
| Google assistant for is setting alarms, and it can't even
| delete the alarm I created just now.
| bbor wrote:
| I really, really doubt it for one reason: I'm convinced Apple
| is still terrified of that "Amazon Alexa tells child to stick a
| penny in a socket" story, and will hamstring themselves in an
| attempt to have their agential cake and eat it too
| thebruce87m wrote:
| They are right to be careful, they are held to a much higher
| standard than their competitors.
|
| Pixel phones have had emergency call issues for years across
| multiple models but they just get a pass. Apple would be
| crucified for this.
| astrange wrote:
| Sounds like a regulator issue. Doing emergency calls is a
| phone's #1 job, they shouldn't be allowing them to be sold
| if they don't work.
| Baldbvrhunter wrote:
| yet in 15+ years I have never used any of mine for that
| astrange wrote:
| Good for you. It's the only thing phones are required to
| be able to do in the US even if they don't have a working
| SIM or you haven't paid the phone bill.
|
| Well, I guess they're also not allowed to cause RF
| interference or randomly catch fire.
| behnamoh wrote:
| Apple is all about a controlled pleasant experience, it
| doesn't matter if it doesn't give you shiny new things; most
| Apple customers don't even know those shiny new things exist,
| so they keep spreading the word that "Apple is so easy and
| simple."
|
| The idea of having an unpredictable LLM in the ecosystem is
| Apple's worst nightmare. I bet they will overly restrict it
| to the point that it stops being a general purpose LLM and
| becomes a neutered obedient LLM that always acts according to
| Apple's rules.
|
| Also, it doesn't help that ALL the authors of this Apple
| paper are chinese. It raises questions about how Apple will
| handle political debates with its LLM.
| astrange wrote:
| > Also, it doesn't help that ALL the authors of this Apple
| paper are chinese. It raises questions about how Apple will
| handle political debates with its LLM.
|
| The CCP thinks it owns all Chinese people on Earth, but
| that doesn't mean you have to agree with them!
| hmottestad wrote:
| With iOS 17 they added a tiny little LLM to the predictive
| typing. I have the newest and greatest iPhone but I feel that I
| very rarely see it in action. I must assume that it's just too
| slow at to keep up with my typing at the moment. Or it's just
| not large enough to give very many useful suggestions.
| KMnO4 wrote:
| Is tiny LLM an oxymoron? I believe Apple has told us it's a
| transformer language model, but not specifically a LLM.
| catoc wrote:
| Yeah, they meant a TLM
| 0x1ceb00da wrote:
| That's print('Hello, world!')
| ben_w wrote:
| It's like how the "New Forest" is really old now: even
| small LLMs are (from what I've seen which isn't exhaustive)
| large compared to Markov language models.
| astrange wrote:
| There's no difference. An LLM is just a transformer
| language model that's "large".
| dontlaugh wrote:
| It's probably why autocomplete got drastically worse, to the
| point I'm considering turning it off entirely.
|
| Most "AI" features are so incredibly fragile they're not
| worth deploying.
| jachee wrote:
| Autocomplete got worse because the new system in iOS17
| didn't retain training data from prior versions. It reset
| everyone back to untrained. I've been manually correcting
| specialized language I use daily (e.g. "iRacing") on my 12
| (non-pro) since iOS17 release, and now it gets it correct
| 99.5% of the time.
|
| So, rather than turning it off, manually correct the
| incorrect completions and use the suggested words bar
| frequently and it _will_ learn how you type. It's just
| having to start over after tossing out several OSes worth
| of training that makes it _feel_ worse.
| alphabettsy wrote:
| It seems to have reset, but I find it's actually much
| better than before after some intervention/training when I
| first updated.
| wenc wrote:
| It's a GPT2 model. It hasn't changed the autocomplete
| experience that much (occasionally I'll see a word
| completion).
| mrbonner wrote:
| Really? Typing in my iPhone 12 Pro has become a nightmare. I
| suspect it is because of predictive typing ML shit. It
| happens all the freaking time now. The symptom is that my
| whole device just froze for a few seconds while the next word
| is squeezed out. How do I turn it off?
| evantbyrne wrote:
| Compared to Android, the iOS keyboard has always been a
| nightmare. However, wI feel it has been causing me fewer
| issues within the past month-ish. Has it been updated
| recently?
| fragmede wrote:
| Thankfully, Gboard is available on iOS.
| cedws wrote:
| What are you going to do with it?
| fnordpiglet wrote:
| The auto correct is already backed by a smallish LLM, FYI.
|
| https://jackcook.com/2023/09/08/predictive-text.html
| blululu wrote:
| And it is a serious quality regression IMO. The dictionary is
| too small and misses/messes up a ton of basic words.
| fnordpiglet wrote:
| It's a place holder for iOS 18's expansion. It's 0.1 of the
| LLM in iOS. And the other prior implementation was so
| ducking bad that I'm not sure how you would observe such a
| regression.
| scosman wrote:
| SLM? :)
| zitterbewegung wrote:
| Next year releases of macOS / iOS are rumored to have LLMs as a
| feature .
| spaceman_2020 wrote:
| GPT-4 voice is so, so good. Really what you would want a voice
| tool to be like. I can talk to it like a normal human being,
| unlike issuing specific commands loudly as with Siri.
| Rucadi wrote:
| I wonder if these models are trained to have some kind of
| identification in case you use them for non-research purposes for
| example.
|
| "Tell me who is your manufacturer" for example
| chefandy wrote:
| From Bard:
|
| _My situation is a bit unique, so the term "manufacturer"
| might not be the most accurate way to describe who created me.
| Here's a breakdown of what you need to know:
| Developed by Google AI: I was created by a team of researchers
| and engineers at Google AI, specializing in language models and
| artificial intelligence. Trained on a massive dataset:
| My knowledge and abilities come from being trained on a massive
| dataset of text and code, containing books, articles, code, and
| other forms of information. Continuously learning and
| evolving: I'm still under development, constantly learning and
| improving as I interact with users and process new information.
|
| So, while I don't have a single manufacturer in the traditional
| sense, I'm the result of collaboration and advancement in AI
| research and development at Google.
|
| I hope this helps clarify things! Let me know if you have any
| other questions._
| SpaceManNabs wrote:
| Why was this downvoted? It didn't answer the question, but it
| showed that there is a sort of imprint that GP was asking
| about.
|
| And it saves everyone a tab's worth of effort.
| MBCook wrote:
| Usually I would almost automatically vote down a comment
| where someone just stuck something into a LLM and pasted
| the output. It almost never adds to the discussion.
|
| However in the case that we're talking about the kind of
| output generated by the LLM in some circumstance, it can be
| instructive. Like you noted this is a perfect example.
| behnamoh wrote:
| Easy to get rid of that by a little fine tuning and system
| prompting.
| smoldesu wrote:
| > FERRET is trained on 8 A100 GPUs with 80GB memory.
|
| Huh, even Apple isn't capable of escaping the CUDA trap. Funny to
| see them go from moral enemies with Nvidia to partially-dependent
| on them...
| ssijak wrote:
| I guess they also have Samsung fridges in the offices..
| smoldesu wrote:
| I don't get it, does Apple also make fridges now?
| p_j_w wrote:
| No, they don't build compute clusters either.
| ayewo wrote:
| They are implying that even though Apple is a wealthy
| consumer hardware company that has major spats with nVidia
| and Samsung, it doesn't always make economic sense to make
| tools they might need _in-house_ when they can simply buy
| them from a rival.
|
| So rather than invest engineering resources to re-imagine
| the fridge, they can simply buy them from established
| manufacturers that make household appliances like Samsung,
| Sony etc.
| lern_too_spel wrote:
| Apple doesn't make silly charts saying they make better
| refrigerators.
| airstrike wrote:
| Because they don't sell refrigerators
| lern_too_spel wrote:
| The point of this thread is that even though Apple makes
| silly charts saying how good their hardware is at ML,
| they use products that their silly charts say aren't as
| good.
|
| There is no such hypocrisy if they use Samsung
| refrigerators.
| astrange wrote:
| ML inference and training are not the same task.
| smoldesu wrote:
| ...and doing _either_ of those things with CUDA is
| impossible on Mac. Why? Because Apple burned their bridge
| with Nvidia and threw a temper tantrum, that 's why. Now
| Nvidia _can 't_ support MacOS, even if they wanted.
|
| That's kinda the point of my original comment. Apple
| claims to know what's best, but contradict themselves
| through their own actions. We wouldn't be in awkward
| situations like this if Apple didn't staunchly box-out
| competitors and force customers to follow them or abandon
| the ecosystem. It's almost vindicating for people like
| me, who left MacOS because of these pointless decisions.
| MBCook wrote:
| Not only that, don't they buy display panels and maybe
| even storage or RAM chips from Samsung?
|
| Once two giant companies are dealing with each other it
| can get really complicated to cut everything off.
| causal wrote:
| And probably Intel processors and Linux in their datacenters.
| cryogenicfire wrote:
| Well apple was a prime Intel client for years until they
| released M1, and ARM on the cloud isn't really a thing for
| now... Ultimately it's all about what makes the most sense
| for what will make the most money, and on a datacenter that
| means x86 with Linux/Unix
| amelius wrote:
| And they use CAD software running on Windows (it simply
| doesn't exist on MacOS)
| cryogenicfire wrote:
| I feel like Apple is only testing the waters with AI right now,
| but perhaps if they get involved enough they'll spend money on
| their own compute infrastructure? Nvidia is kind of the king at
| GPU compute right now, and developing comparable hardware is no
| small or cheap task, but I think Apple is in a very good
| position to be able to make it work---if they decide to invest
| in it. But honestly, as far as corporate feud goes, I feel like
| companies will happily suck it up if it makes some process
| cheaper and/or easier
| MBCook wrote:
| > But honestly, as far as corporate feud goes, I feel like
| companies will happily suck it up if it makes some process
| cheaper and/or easier
|
| That's what I think is going on. Apple hated being on the
| hook for Nvidia's terrible drivers and chipset/heat problems
| that ended up causing a ton of warranty repairs.
|
| In this case they're not a partner, they're just a normal
| customer like everyone else. And if Intel comes out with a
| better AI training card tomorrow Apple can switch over
| without any worry.
|
| They're not at the mercy of Nvidia like they were with
| graphics chips. They're just choosing (what I assume to be)
| the best off the shelf hardware for what they need.
| amelius wrote:
| Dependent is a strong word. At the end of the day all these DL
| models run on any hardware, and you can easily swap out one
| type of hardware for another perhaps with some small
| performance impact. They're commodities, basically.
| tambourine_man wrote:
| > FERRET is trained on 8 A100 GPUs
|
| So Apple uses NVidia internally. Not surprising, but doesn't bode
| well for A Series. Dogfooding.
|
| [edit] I meant M series, Apple Silicon
| hhh wrote:
| Why would they dogfood Apple Silicon for training models? Seems
| like a waste of developer time to me.
| tambourine_man wrote:
| Apple doesn't even sell NVidia cards on their Mac Pros. Are
| they training it on Linux?
|
| I think Apple would strive to be great at all computing
| related tasks. "Oh, Macs are not good for that, you should
| get a PC" should make them sad and worried.
|
| AI/LLM is the new hot thing. If people are using Windows or
| Linux, you're loosing momentum, hearts and minds... and
| sales, obviously.
| Gorgor wrote:
| But no one is training these kinds of models on their
| personal device. You need compute clusters for that. And
| they will probably run Linux. I'd be surprised if Microsoft
| trains their large models in anything else than Linux
| clusters.
| tambourine_man wrote:
| Apple used to sell servers. I don't thing they should
| settle for "just use Linux" in such and important field.
| MBCook wrote:
| Why does the OS matter for training models?
|
| Apple would want to train models as fast as they could.
| Nvidia provides an off the shelf solution they can just
| buy and use for a very reasonable price and sell on the
| second hand market.
|
| If they wanted to use their own hardware they would
| either need more of it, which would cost a lot and divert
| production from sellable devices; or they would need to
| make special chips with much bigger neural engines, which
| would cost even more.
|
| Also Apple uses public clouds for service stuff. They may
| not even own any hardware and just be renting it from
| AWS/Azure/GCP for training.
| SpaceManNabs wrote:
| > But no one is training these kinds of models on their
| personal device
|
| on-device transfer learning/fine tuning is def a thing
| for privacy and data federation reasons. Part of the
| reason why model distillation was so hot a few years ago.
| hermannj314 wrote:
| If a train with a GM diesel engine delivers raw materials
| to a Ford factory for making F150s, you would conclude that
| consumers whould start driving trains to work?
|
| Is that your argument?
| tambourine_man wrote:
| Not at all, just that the engineers at Ford would be more
| proud if the train used their own diesel engine. And that
| this kind of thing affects public perception. "Ford is
| not for heavy duty"
| causal wrote:
| Don't think that follows. I doubt all their cloud services
| run on Apple hardware. They make consumer devices.
| nicolas_17 wrote:
| AFAIK all their cloud services run on x86 hardware with
| Linux, _including_ Xcode Cloud (which runs macOS in a
| QEMU VM, in a way that only Apple can technically and
| legally do).
| threeseed wrote:
| It has always been x86 Linux going back to the NeXT days.
|
| Today they have large scale Kubernetes ands some legacy
| Mesos clusters.
| bjtitus wrote:
| This
|
| They ceded the data center environment years ago.
| nicolas_17 wrote:
| Apple doesn't even _support_ NVidia cards on their Mac
| Pros. The technical details are above my head, but the way
| Apple M* chips handle PCIe make them incompatible with GPUs
| and other accelerator cards. Whether you use macOS or
| Linux.
| tjohns wrote:
| Apple doesn't even make rack-mount server hardware. It's
| not that surprising.
|
| Apple makes very capable, efficient devices for end users
| and content producers. End users do not normally need to
| train new models.
| danieldk wrote:
| They do make rack mount hardware:
|
| https://www.apple.com/shop/buy-mac/mac-pro/rack
|
| Don't know if that qualifies as server :).
| threeseed wrote:
| > Are they training it on Linux
|
| Yes. Apple has always run their servers on Linux e.g. App
| Store, iTunes.
|
| And training isn't right now an end user activity but
| something reserved for server farms.
| shepherdjerred wrote:
| > I think Apple would strive to be great at all computing
| related tasks. "Oh, Macs are not good for that, you should
| get a PC" should make them sad and worried.
|
| What percent of Apple's customers train models? Does it
| even crack 1%?
|
| Apple already fails for many types of computing, e.g. any
| workflow that requires Windows or AAA gaming.
| sxg wrote:
| By "A series" are you referring to the Nvidia A100 or the Apple
| A-series iPhone/iPad chips? If the latter, I don't think you
| can draw that conclusion. Training has memory and processor
| requirements that are very different from inference. You don't
| need iPhones and iPads to train models--you need them to run
| models. These are two very different things.
| tambourine_man wrote:
| Apple Silicon, sorry for the ambiguity. Apple sells Macs too.
| That's where I'd hope they would train their models.
| woke_neolib wrote:
| Apple apparently uses Google Cloud, so it's that or TPUs!
| tambourine_man wrote:
| They use many clouds. But LLM should be their core business
| and they usually don't outsource that.
| blackoil wrote:
| That's not how s/w or any development work. Even if there
| is a team working on M4 SuperUltra++ which competes with
| H100, in meanwhile, s/w team will continue to use what is
| available which may be Intel/AMD PCs with Nvidia GPUs or
| GoogleCloud/Azure/AWS.
| tambourine_man wrote:
| Sure, but there's also the possibility that they aren't
| developing the SuperUltra++
| gooob wrote:
| yeah, aren't the new M3 chips supposed to be really good for ML
| training?
| cryogenicfire wrote:
| I feel like Apple is only testing the waters with AI right now,
| but perhaps if they get involved enough they'll spend money on
| their own compute infrastructure? Nvidia is kind of the king at
| GPU compute right now, and developing comparable hardware is no
| small or cheap task, but I think Apple is in a very good
| position to be able to make it work---if they decide to invest
| in it. But honestly, as far as corporate feud goes, I feel like
| companies will happily suck it up if it makes some process
| cheaper and/or easier
| tambourine_man wrote:
| I think you're completely correct, but if they were caught
| off guard by the AI train, they shouldn't be testing the
| waters now. It should be treated as an existential threat.
| cryogenicfire wrote:
| I've always wondered why Apple is never in the conversation
| of modern AI, like until now it almost feels like they've
| been just watching from the sidelines without taking part
| in all the commotion
|
| Maybe they can just afford to observe before making major
| decisions on the direction of the company... For a company
| like Apple, I feel like they won't lose their customer-base
| just because they are taking the AI race slowly, in fact
| Apple has often been late to introducing very common
| feature
| cryogenicfire wrote:
| I hit reply before I finished typing XD
|
| ... Anyways, my point being that Apple will gladly
| introduce a polished product a couple of years after
| everyone else has already done it, and their target
| audience will still applaud their work and give them
| money. Apple for some reason simply _can_ afford to test
| the water
| fragmede wrote:
| You can edit your comments shortly after you make them.
| hmottestad wrote:
| Apple had a falling out with Nvidia a number of years ago. I
| believe they were using chips from Nvidia in their MacBook
| Pros, the first to come with both integrated and discrete
| graphics, but the solder between the chips and the motherboard
| kept cracking and a rather large number of MacBooks needed to
| be repaired.
|
| https://techcrunch.com/2008/12/09/scientists-nvidia-put-faul...
| dcchambers wrote:
| As long as the inference can be done locally on their chips I
| don't at all think it's a big deal to train models on
| Nvidia/other hardware.
|
| Are all the iCloud servers running on Apple silicon? I assumed
| they were running on standard rack mounted hardware.
| tambourine_man wrote:
| I think Apple considers cloud infrastructure a necessary evil
| and a commodity.
|
| AI isn't, yet at least, and I don't think they can afford to
| treat it as such.
| andy99 wrote:
| One big plus if this takes off as a base model is the abundance
| of weasel family animals to use in naming the derivatives.
| Ermine, marten, fisher, ... I'd like to call Wolverine. Llama
| didn't have much room for some interesting variety beyond alpaca
| and vicuna.
| behnamoh wrote:
| Yes, because that's the main concern and limitation in the LLM
| community. /s
|
| If anything, I think people should use meaningful and relevant
| names, or invent new ones.
| cpressland wrote:
| Finally, some decent competition for Not Hotdog!
| slau wrote:
| I think you just put a smile on Tim Anglade's face by
| mentioning this.
|
| https://news.ycombinator.com/item?id=14636228
| Jackson__ wrote:
| >Ferret: A Multimodal Large Language Model
|
| What I thought when reading the title: A new base model trained
| from the ground up on multimodal input, on hundreds to thousands
| of GPUS
|
| The reality: A finetune of Vicuna, trained on 8xA100, which
| already is a finetune of Llama 13b. Then it further goes on to
| re-use some parts of LLava, which is an existing multimodal
| project already built upon Vicuna. It's not really as exciting as
| one might think from the title, in my opinion.
| foxhop wrote:
| Thanks for the summary.
| basiccalendar74 wrote:
| this seems like a good but small research project by a research
| team in Apple. far away from what product teams are working on
| for next generation of apple products.
| orenlindsey wrote:
| Has anyone actually run this yet?
| devinprater wrote:
| They're already going multi-modal? Holy crap, if google can't
| deliver in the accessibility space for this (image descriptions
| better than "the logo for the company"), then I'll definitely go
| back to Apple. I mean I do hope Apple cleans out bugs and makes
| VoiceOver feel like it won't fall over if I breathed hard, but
| their image descriptions, even without an LLM, are already clean
| and clear. More like "A green logo on a black background", where
| Google is, like I said, more like "The logo for the company." I
| guess it's kinda what we get when AI is crowdsourced rather than
| given good, high quality data to work with.
| zitterbewegung wrote:
| Honestly if they are coming out with a paper now Apple has
| probably been working on it for a year or two at minimum . Next
| year releases of macOS / iOS are rumored to have LLMs as a
| feature .
| beoberha wrote:
| I don't mean to discount this work, but this particular model
| is the product of a few months of work tops. It's effectively
| LLava with different training data, targeted at a specific
| use case. While I'm sure there is a significant effort at
| multimodal LLMs within Apple, this is just a tiny corner of
| it.
| jonplackett wrote:
| Presumable because this is Conda none of this can be run on any
| Apple hardware despite people managing to get M processors to do
| a bit of dabbling with AI?
| _visgean wrote:
| > because this is Conda none of this can be run on any Apple
| hardware
|
| conda supports m1? https://www.anaconda.com/blog/new-release-
| anaconda-distribut...
| jonplackett wrote:
| Did not know that!
| adt wrote:
| Old paper (Oct/2023), but the weights are new (Dec/2023):
|
| https://lifearchitect.ai/models-table/
| halyconWays wrote:
| I'm glad Apple invented AI. Now they'll put a fancy new name on
| it and consumers will believe it.
| moneycantbuy wrote:
| anyone know what is the best open source model that allows
| commercial use and can run locally on an iphone?
| mandelken wrote:
| Mistral 7B is pretty good and the instruct v0.2 runs on my
| iPhone through MLC Chat.
|
| However, the ChatGPT4 app is much better in usability: better
| model, multi-modal with text/vision/speech and better UI.
| hackernewds wrote:
| gpt 4 allows commercial use?
| satvikpendem wrote:
| Why wouldn't it? They sell the API for a reason.
| WhitneyLand wrote:
| Yes and no.
|
| You can use it commercially but there are some
| restrictions, including some of a competitive nature, like
| using the output to train new LLMs. This is the restriction
| that Bytedance (Tiktok) was recently banned for violating.
| BrutalCoding wrote:
| I've made an example app for a Flutter plugin I created that
| can do this.
|
| Open-source, runs natively on all major platforms. I shared
| videos showing it on my iPad Mini, Pixel 7, iPhone 12, Surface
| Pro (Win 10 & Ubuntu Jellyfish) and Macs (Intel & M archs).
|
| By all means, it's not a finished app. I simply wanted to use
| on-device AI stuff in Flutter so I started with porting over
| llama.cpp, and later on I'll tinker with porting over whatever
| is the state of the art (whisper.cpp, bark.cpp etc).
|
| Repo: https://github.com/BrutalCoding/aub.ai
|
| For any of your Apple devices, use this:
| https://testflight.apple.com/join/XuTpIgyY
|
| App is compatible with any GGUF files, but it must be in the
| ChatML prompt format otherwise the chat UI/bubbles probably
| gets funky. I haven't made it customizable yet, after all -
| it's just an example app of the plugin. But I am actively
| working on it to nail my vision.
|
| Cheers, Daniel
| shrimpx wrote:
| Apple has been looking sleepy on LLMs, but they've been
| consistently evolving their hardware+software AI stack, without
| much glitzy advertising. I think they could blow away
| Microsoft/OpenAI and Google, if suddenly a new iOS release makes
| the OpenAI/Bard chatbox look laughably antiquated. They're also a
| threat to Nvidia, if a significant swath of AI usage switches
| over to Apple hardware. Arm and TSMC would stand to win.
| harryVic wrote:
| Can you give an example? I switched to android because i use
| personal assistant a lot while driving and siri was absolutely
| horrible.
| shrimpx wrote:
| - FaceID
|
| - Facial recognition in Photos
|
| - "Memories" in Photos
|
| - iOS keyboard autocomplete using LLMs. I am bilingual and
| noticed in the latest iOS it now does multi-language
| autocomplete and you no longer have to manually switch
| languages.
|
| - Event detection for Calendar
|
| - Depth Fusion in the iOS camera app, using ML to take
| crisper photos
|
| - Probably others...
|
| The crazy thing is most/all of these run on the device.
| pkage wrote:
| I just wish you could turn the multilingual keyboard off--I
| find that I usually only type in one language at a time and
| having the autocomplete recommend the wrong languages is
| quite frustrating
| shrimpx wrote:
| That's true, I have found that mildly annoying sometimes.
| But most of the time it's a win. It was really annoying
| manually switching modes over and over when typing in
| mixed-language, which I do fairly often. It'd be great if
| there was a setting though.
| Affric wrote:
| Don't TSMC make Nvidia's chips too?
| shrimpx wrote:
| Yup! TSMC wins either way.
| madeofpalk wrote:
| I doubt Apple's going to make some big ChatGPT-style chatbot.
| They're "just" going to use the same tech to drive iterative
| (good!) improvements to their products, like Siri and keyboard
| auto-complete.
| shrimpx wrote:
| Yeah. Siri supports text input already, anyway. Siri is their
| ChatGPT-style bot that's going to keep improving.
| fbdab103 wrote:
| I would challenge the keyboard autocomplete. I find the Apple
| suggestions to be frustratingly poor vs my experience on
| Android.
| Booourns wrote:
| Out of curiosity have you experienced their autocorrect on
| iOS 17 because that's when they updated to be LLM based?
| georgespencer wrote:
| Transformer based autocomplete on iOS 17 feels just as
| bad -- but in different ways -- as its previous
| incarnation to me.
| simonair wrote:
| Are you tapping the keys or swiping over those that make
| up the word you want to type? In my experience, tapping
| has always been and remained poor but swiping is getting
| better and better with every iOS version.
| theferalrobot wrote:
| Given Apple's track record on anything AI related and the
| terrible state they keep CoreML that not only seems
| extraordinarily unlikely, it would take a lot of time to win
| developer trust and that I just don't see happening.
| mark_l_watson wrote:
| I have enjoyed working with CoreML over the last few years.
| Please share what you didn't like about it.
| hosh wrote:
| Apple doesn't have to win developer trust or build an AI
| platform. They just have to build a compelling consumer
| product that can only function with AI, and they are better
| equipped to do that than Google or Microsoft. It remains to
| be seen if OpenAI will go that route instead of a business
| built on training and providing access to foundational
| models.
| amitprasad wrote:
| Also relevant: LLM in a flash: Efficient Large Language Model
| Inference with Limited Memory
|
| Apple seems to be gearing up for significant advances in on-
| device inference using this LLMs
|
| https://arxiv.org/abs/2312.11514
| Thorrez wrote:
| Does Apple know that ferrets are illegal in California?
|
| https://www.legalizeferrets.org/
___________________________________________________________________
(page generated 2023-12-23 23:00 UTC)