[HN Gopher] CoreNet: A library for training deep neural networks
___________________________________________________________________
CoreNet: A library for training deep neural networks
Author : rocauc
Score : 450 points
Date : 2024-04-24 01:26 UTC (21 hours ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| gnabgib wrote:
| h1: _CoreNet: A library for training deep neural networks_
| symlinkk wrote:
| Pretty funny that Apple engineers use Homebrew too.
| guywithabike wrote:
| Why is it funny? Homebrew is the de facto standard terminal
| packaging tool for macOS.
| AceJohnny2 wrote:
| <cries in MacPorts>
| TMWNN wrote:
| I also use MacPorts, but certainly have often noticed that
| Homebrew has some package that MacPorts doesn't.
|
| I guess there's nothing stopping me from moving to Homebrew
| other than familiarity.
| fastball wrote:
| I used MacPorts a decade ago, but at some point realized
| that Homebrew had more packages that were kept
| consistently up-to-date. Switched and never looked back.
| nicolas_t wrote:
| I switched away back to macports when homebrew decided to
| get rid of formula options. To be honest, I always find
| homebrew frustrating, it feels that they've often made
| technical decisions that are not necessarily the best but
| they've been much more successful at marketing themselves
| than macports.
| pnw_throwaway wrote:
| If I'm reading the formula docs right, only homebrew-core
| packages don't support it (due to CI not testing them).
| That part does suck, though.
|
| Other taps, like homebrew-ffmpeg, offer a ton of options.
| nicolas_t wrote:
| oh, I actually hadn't realized that this is what they
| settled on in the end. ffmpeg is the quintessential
| package where options make sense so good that that's
| still supported.
|
| The other issue I experienced with homebrew around that
| time were related to having different versions of openssl
| installed because I had some old codebase I had to run
| (and for performance reasons didn't want to use docker).
| But that's definitely a edge case.
| detourdog wrote:
| I haven't looked at Homebrew since that got started. The
| philosophical difference at that time was using macports
| and having a consistent and managed */local/ collection
| of tools with self contained dependencies vs. adding new
| tools with dependencies tied to the current Mac OS
| release.
|
| I still use MacPorts for that reason and it is easy
| enough to create a local portfile for whatever isn't in
| Macports.
|
| I find this to be the easy way to manage networked
| development computers.
| ramesh31 wrote:
| >Why is it funny? Homebrew is the de facto standard terminal
| packaging tool for macOS.
|
| It's funny because a multi-trillion dollar company can't be
| bothered to release a native package manager or an official
| binary repository for their OS after decades of pleading from
| developers.
| etse wrote:
| Well, without charging for it, right?
| 2muchcoffeeman wrote:
| They should do it to become the de facto platform for
| programming.
| astrange wrote:
| They did, they sponsored MacPorts. (And then Swift Package
| Manager.)
| Tagbert wrote:
| So you want them to Sherlock Homebrew?
| TillE wrote:
| "Sherlocking" can be unfortunate for a developer, but
| it's odd to view it as an inherently bad thing. A package
| manager is a core OS feature, even Microsoft has WinGet
| now.
| fragmede wrote:
| it's odd to feel empathetic when someone has their
| livelihood taken from them?
| Someone wrote:
| > A package manager is a core OS feature
|
| It has become a core OS feature. Historically, you see
| the set of core OS features expand tremendously. Back in
| the 80's drawing lines and circles wasn't even a core OS
| feature (not on many home computers, and certainly not on
| early PCs), bit-mapped fonts were third part add-ons for
| a while, vector-based fonts were an Adobe add-on
| (https://en.wikipedia.org/wiki/Adobe_Type_Manager),
| printer drivers were third party, etc.
|
| I think that's natural. As lower layers become
| commodities (try making money selling an OS that only
| manages memory and processes), OS sellers have to add
| higher layer stuff to their products to make money on
| them.
|
| As to Sherlocking, big companies cannot do well there in
| the eyes of "the angry internet":
|
| - don't release feature F: "They don't even support F out
| of the box. On the competitor's product, you get that for
| free"
|
| - release a minimal implementation: "They have F, but it
| doesn't do F1, F2, or F3"
|
| - release a fairly full implementation: "Sherlocking!"
| and/or nitpicking about their engineering choices.
| randomdata wrote:
| They released "App Store" for the average Joe. We can all
| agree it is not suitable for power users, but at the same
| time what would power users gain over existing solutions if
| they were to introduce something?
| katbyte wrote:
| You can brew install mas (I think) and then
| install/manage Mac store stuff via the cli pretty easily
| photonbeam wrote:
| I hear a lot about people moving to nix-darwin, is it popular
| or am I showing my own bubble
| jallmann wrote:
| I use nixpkgs on MacOS, is nix-darwin is a different
| project?
|
| I love Nix but it probably has too many rough edges for the
| typical homebrew user.
| tymscar wrote:
| Its a different complementary thing. It lets you define
| your macos settings the same way you would on nixos
| firecall wrote:
| I've never heard of it until now, but will check it out!
| :-)
| armadsen wrote:
| I'm a full-time Mac and iOS developer, have been for almost
| 20 years, and this is the first I've heard of it. Might
| just be _my_ bubble, but I don't think it's a huge thing
| yet. (I'm going to check it out now!)
| pyinstallwoes wrote:
| I never even heard of nix-Darwin. Interesting.
| sevagh wrote:
| Apple should do like this library, re-release Homebrew with
| their own name on the README and people would lap it up.
| andreygrehov wrote:
| What hardware would one need to have for the CoreNet to train
| efficiently?
| mxwsn wrote:
| Built on top of pytorch.
| buildbot wrote:
| Does this support training on Apple silicon? It's not very clear
| unless I missed something in the README.
| zmk5 wrote:
| I believe the MLX examples allow for it. Seems like a general
| purpose framework rather than a Mac specific one.
| gbickford wrote:
| I couldn't find any training code in the MXL examples.
| blackeyeblitzar wrote:
| Would such a capability (training) be useful for anything other
| than small scale experimentation? Apple doesn't make server
| products anymore and even when they did, they were overpriced.
| Unless they have private Apple silicon based servers for their
| own training needs?
| MBCook wrote:
| There are an insane number of Apple Silicon devices out
| there.
|
| If your product runs on an iPhone or iPad, I'm sure this is
| great.
|
| If you only ever want to run on 4090s or other server stuff,
| yeah this probably isn't that interesting.
|
| Maybe it's a good design for the tools or something, I have
| no experience to know. Maybe someone else can build off it.
|
| But it makes sense Apple is releasing tools to make stuff
| that works better on Apple platforms.
| blackeyeblitzar wrote:
| I can understand the inference part being useful and
| practical for Apple devs. I'm just wondering about the
| training part, for which there Apple silicon devices don't
| seem very useful.
| rgbrgb wrote:
| I've seen several people fine tune mistral 7B on
| MacBooks.
| spmurrayzzz wrote:
| My M2 Max significantly outperforms my 3090 Ti for
| training a Mistral-7B LoRA. Its sort of a case-by-case
| situation though, as it depends on how optimized the CUDA
| kernels happen to be for whatever workload you're doing
| (i.e. for inference, theres a big delta between standard
| transformers vs exllamav2, apple silicon may outperform
| the former, but certainly not the latter).
| jjtheblunt wrote:
| Isn't the current Mac Pro available in rack mount form?
|
| https://www.apple.com/mac-pro/
| donavanm wrote:
| > Unless they have private Apple silicon based servers for
| their own training needs?
|
| Id be SHOCKED if so. Its been 15 years, but I was there when
| xserve died. Priorities were iphone > other mobile devices
| >>> laptops > displays & desktops >>> literally anything
| else. When xserve died we still needed osx for OD & similar.
| Teams moved on to 3P rack mount trays of mac minis as a stop
| gap. Any internal support/preference for server style
| hardware was a lolwut response. Externally I see no reason to
| suspect thats changed.
| gbickford wrote:
| > Relationship with CVNets
|
| > CoreNet evolved from CVNets, to encompass a broader range of
| applications beyond computer vision. Its expansion facilitated
| the training of foundational models, including LLMs.
|
| We can expect it to have grown from here:
| https://apple.github.io/ml-cvnets/index.html
|
| It looks like a mid-level implementations of training and
| inference. You can see in their "default_trainer.py"[1] that the
| engine uses Tensors from torch but implements its own training
| method. They implement their own LR scheduler and optimizer; the
| caller can optionally use Adam from torch.
|
| It's an interesting (maybe very Apple) choice to build from the
| ground up instead of partnering with existing frameworks to
| provide first class support in them.
|
| The MLX examples seem to be inference only at this point. It does
| look like this might be a landing ground for more MLX specific
| implementations: e.g.
| https://github.com/apple/corenet/blob/5b50eca42bc97f6146b812...
|
| It will be interesting to see how it tracks over the next year;
| especially with their recent acquisitions:
|
| Datakalab https://news.ycombinator.com/item?id=40114350
|
| DarwinAI https://news.ycombinator.com/item?id=39709835
|
| 1:
| https://github.com/apple/corenet/blob/main/corenet/engine/de...
| blackeyeblitzar wrote:
| > It looks like a mid-level implementations of training and
| inference
|
| I'm not familiar with how any of this works but what does state
| of the art training look like? Almost no models release their
| training source code or data sets or pre processing or
| evaluation code. So is it known what the high level
| implementation even is?
| spott wrote:
| https://github.com/NVIDIA/Megatron-LM
|
| This is probably a good baseline to start thinking about LLM
| training at scale.
| error9348 wrote:
| The interface looks very Apple as well. Looks like you create a
| config file, and you already have a model in mind with the
| hyperparameters and it provides a simple interface. How useful
| is this to researchers trying to hack the model architecture?
|
| One example:
| https://github.com/apple/corenet/tree/main/projects/clip#tra...
| sigmoid10 wrote:
| Not much. But if you just want to adapt/optimize hyperparams,
| this is a useful approach. So I can certainly see a possible,
| less technical audience. If you actually want to hack and
| adapt architectures it's probably not worth it.
| davedx wrote:
| > It's an interesting (maybe very Apple) choice to build from
| the ground up instead of partnering with existing frameworks to
| provide first class support in them.
|
| It smells of a somewhat panicked attempt to prepare for WWDC to
| me. Apple has really dropped the ball on AI and now they're
| trying to catch up.
| pizza wrote:
| Wouldn't WWDC-related endeavors be more product-facing? I'm
| not so sure this has to do with their efforts to incorporate
| ai into products, and tbh I would say their ai research has
| been pretty strong generally speaking.
| davedx wrote:
| I expect that a lot of WWDC will be Apple trying to get
| more developers to build AI products for their platforms,
| because at the moment, Apple products don't have much AI.
| The other tech companies have integrated user facing LLM
| products into a significant part of their ecosystem -
| Google and Microsoft have them up front and center in
| search. Apple's AI offerings for end users are what
| exactly? The camera photos app that does minor tweaks to
| photos (composing from multiple frames). What else actually
| is there in the first party ecosystem that significantly
| leverages AI? Siri is still the same trash it's been for
| the last 10 years - in fact IMO it's become even less
| useful, often refusing to even do web searches for me. (I
| WANT Siri to work very well).
|
| So because their first party AI products are so non-
| existent, I think WWDC is a desperate attempt by Apple to
| get third party developers to build compelling AI products.
| I say desperate because they're already a year behind the
| competition in this space.
|
| (I can imagine they'll be trying to get developers to build
| Vision Pro software too, though I hear sales there have
| collapsed so again, way too little, too late)
| niek_pas wrote:
| I'm not sure what you mean by "AI products", and why you
| think Apple needs them for their platforms.
| tzakrajs wrote:
| They have tons of computer vision, NN inference and
| natural language processing in their products. It's
| reductive to say Apple products don't have much AI.
| wokwokwok wrote:
| Can you be more specific?
|
| What AI products are present in other ecosystems (eg.
| Android, Samsung, whatever) and missing from Apple?
|
| Honest question: I find the platform distinction largely
| meaningless in most cases apart from "what your phone
| looks like" and "can you side load apps"...
| lynx23 wrote:
| I am guessing you are not familiar with the AI-powered
| vision features that already ship since a few years.
| Mostly accessibility related, so I am not surprised you
| missed it.
| devinprater wrote:
| Yep. Google, the AI company, only recently launched image
| descriptions in TalkBack, which VoiceOver has had for
| years now. Google still doesn't have Screen Recognition,
| which basically does OCR and image/UI classification to
| make inaccessible apps more accessible.
| matthewmacleod wrote:
| For one thing, I can search for any text I've ever take a
| photo of. Finding a picture of someone I took 20+ years
| ago by searching for a single work I remember on their
| t-shirt is pretty cool, and is all done on-device.
|
| I think it's important to remember that there are a bunch
| of actual useful AI-driven features out there that aren't
| just GenAI chatbots.
| audunw wrote:
| I don't get the idea that Apple dropped the ball on AI. They
| were fairly early with adding neural engine hardware to their
| chips and have been using ML extensively on-device for a long
| time now
|
| They haven't put an LLM assistant out there. But they don't
| make their own search engine either so I don't think "online
| LLM assistant" is something they'll ever put much effort into
| unless it's part of a bigger effort to launch their own AI-
| based search engine as well.
|
| As for generative AI I don't think the quality is up to a
| level that would be reasonable for Apple.
|
| The only area where i would expect Apple to keep up is the
| kind of Copilot integration Microsoft is working on. And we
| know Apple is working on on-device AI assistant, and probably
| have for a long time. It'll be launched when they can get
| good quality results on-device. Something nobody else has
| achieved anyway, so we can't say that they're behind anyone
| yet.
| chrsw wrote:
| >I don't get the idea that Apple dropped the ball on AI.
|
| That's the public perception. Maybe due to them not getting
| in on a quick cash grab off the LLM hype wave?
| fauigerzigerk wrote:
| I share this perception for two reasons:
|
| 1) Siri
|
| 2) Dearth of published AI research
| chrsw wrote:
| I agree with 1. For 2, have they ever been a company big
| into research? They're very consumer focused and it can
| take time to integrate new tech into consumer products at
| scale. Especially the way Apple likes to do it: polished
| and seamlessly integrated into the rest of their
| ecosystem.
| fauigerzigerk wrote:
| I would say not doing AI research (or buying another big
| company that does) is tantamount to dropping the ball on
| AI, if it turns out that AI is a capability they should
| have had and must have to succeed.
|
| You could argue that publishing research is not the same
| thing as doing it. But they don't seem to have done much
| of it until fairly recently.
|
| I agree that Apple does less research than other big tech
| companies. But they do it where they think it matters.
| Their M-series CPUs are more than just integration and
| polishing. And they have been doing some research in
| health AI as well, I think.
| jldugger wrote:
| > Dearth of published AI research
|
| https://machinelearning.apple.com/research seems to have
| too many publications to be considered a "dearth" IMO.
| fauigerzigerk wrote:
| Dearth relative to Apple's size and relative to the
| amount of research that competitors have been doing.
|
| But I think part of the problem is that Apple simply
| hasn't focused on the tasks and the methods and the
| people that have now turned out to be so impactful.
|
| They have clearly been course correcting for a while now
| as some of the more recent papers show, and they have
| done successful research in areas such as health AI.
| talldayo wrote:
| > They were fairly early with adding neural engine hardware
| to their chips
|
| If that's all it takes to stay ahead of the curve, then
| Rockchip and Qualcomm are arguably right up there alongside
| them. _Tons_ of vendors shipped their own AI silicon, and
| of those vendors, it seems like Nvidia is the only one that
| shipped anything truly usable. Medium-sized LLMs, Stable
| Diffusion and probably even stuff like OAI Whisper is
| faster run on Apple 's GPUs than their AI coprocessor.
| wtallis wrote:
| > and of those vendors, it seems like Nvidia is the only
| one that shipped anything truly usable. Medium-sized
| LLMs, Stable Diffusion and probably even stuff like OAI
| Whisper is faster run on Apple's GPUs than their AI
| coprocessor.
|
| Be careful not to have NVIDIA-shaped tunnel vision.
| Performance isn't the whole story. It's very telling that
| approximately _everybody_ making SoCs for battery powered
| devices (phones, tablets, laptops) has implemented an AI
| coprocessor that 's separate from the GPU. NVIDIA may
| take exception, but the industry consensus is that GPUs
| aren't always the right solution to every AI/ML-related
| problem.
| talldayo wrote:
| Ideally, you're right. Realistically, Apple has to choose
| between using their powerful silicon (the GPU) for high-
| quality results or their weaker silicon (the Neural
| engine) for lower-power inference. Devices that are
| designed around a single power profile (eg. desktop GPUs)
| can integrate the AI logic _into_ the GPU and have _both_
| high-quality and high-speed inference. iPhones gotta
| choose one or the other.
|
| There's not _nothing_ you can run on that Neural Engine,
| but it 's absolutely being misunderstood relative to the
| AI applications people are excited for today. Again; if
| chucking a few TOPS of optimized AI compute onto a mobile
| chipset is all we needed, then everyone would be running
| float16 Llama on their smartphone already. Very clearly,
| something must change.
| jldugger wrote:
| > they don't make their own search engine
|
| Curious then, why they keep recruiting search engineers[1].
| And why they run a web crawler[2]. And why typing "Taylor
| Swift" into safari offers a Siri Suggested website before
| Google.
|
| I guess what people mean by search engine is "show ads
| alongside web search to as many people as possible"?
|
| 1: https://jobs.apple.com/en-us/details/200548043/aiml-
| senior-s... 2: https://support.apple.com/en-us/HT204683
| throw0101c wrote:
| > _Apple has really dropped the ball on AI and now they 're
| trying to catch up._
|
| Apple put a neural engine on-die in the A11 back in 2017:
|
| * https://en.wikipedia.org/wiki/Apple_A11#Neural_Engine
|
| The A-derived M-series chips had them from the beginning in
| 2020:
|
| * https://en.wikipedia.org/wiki/Apple_M1#Other_features
|
| Seems like they've been doing machine learning for a while
| now.
| jdewerd wrote:
| They've been using them, too. Auto OCR so selecting text in
| images Just Works, image enhancements, Siri. I'm sure LLM
| Siri is in the works. Scanning your photos for CSAM. Let's
| hope that last one is more reliable than Siri :/
| thealistra wrote:
| Wasn't csam ultimately rolled back? And wasn't it not Ai
| based but hash based?
| zitterbewegung wrote:
| What you say is true about the project but both PyTorch works
| on Mace and Tensorflow was ported to Macs by Apple
| _aavaa_ wrote:
| They were originally available only as binaries, have they
| released the code changes required or upstreamed them yet?
| miki123211 wrote:
| What's the advantage of using this over something like
| Huggingface Transformers, possibly with the MPS backend?
| pshc wrote:
| "MLX examples demonstrate how to run CoreNet models efficiently
| on Apple Silicon. Please find further information in the
| README.md file within the corresponding example directory."
|
| _> mlx_example /clip: ... an example to convert CoreNet's CLIP
| model implementation to MLX's CLIP example with some customized
| modification._ - FP16 Base variant: 60% speedup
| over PyTorch - FP16 Huge variant: 12% speedup
|
| _> mlx_example /open_elm: ... an MLX port of OpenELM model
| trained with CoreNet. MLX is an Apple deep learning framework
| similar in spirit to PyTorch, which is optimized for Apple
| Silicon based hardware._
|
| Seems like an advantage is extra speedups thanks to
| specialization for Apple Silicon. This might be the most power-
| efficient DNN training framework (for small models) out there.
| But we won't really know until someone benchmarks it.
| HarHarVeryFunny wrote:
| OpenELM (ELM = Efficient Language Models) has an unfortunate
| name clash with another LLM-related open source project.
|
| https://github.com/CarperAI/OpenELM (ELM = Evolution through
| Large Models)
| jaimex2 wrote:
| Nothing, its basically pytorch with an Apple logo.
| upbeat_general wrote:
| The implementation seems to be pretty clean and modular here
| where transformers (and diffusers) isn't, unless you take their
| modules standalone.
|
| This repo has a lot of handy utilities but also a bunch of
| clean implementations of common models, metrics, etc.
|
| In other words, this is more for writing new models rather than
| inference.
| coder543 wrote:
| They also mention in the README:
|
| > CatLIP: CLIP-level Visual Recognition Accuracy with 2.7x Faster
| Pre-training on Web-scale Image-Text Data
|
| This is the first I'm hearing of that, and the link seems broken.
| huac wrote:
| cat's out of the bag, too early?
| simonw wrote:
| The link should go here I think:
| https://github.com/apple/corenet/tree/main/projects/catlip
| seanvelasco wrote:
| somewhat related, i came across this, mlx examples for openai
| clip: https://github.com/ml-explore/mlx-examples/tree/main/clip
|
| curious to know how fast catlip is. the above using openai clip
| is already fast.
| ipsum2 wrote:
| It's interesting that Apple also actively develops
| https://github.com/apple/axlearn, which is a library on top of
| Jax. Seems like half the ML teams at Apple use PyTorch, and the
| other half uses Jax. Maybe its split between Google Cloud and
| AWS?
| josephg wrote:
| In my experience, this is pretty normal in large companies like
| Apple. Coordination costs are real. Unless there's a good
| reason to standardize on a single tool, its usually easier for
| teams to just pick whichever tool makes the most sense based on
| the problem they're solving and what the team has experience
| with.
| tomComb wrote:
| Big companies like Apple yes, but not Apple
| te_chris wrote:
| I don't know as haven't worked there, but have always heard
| Apple described more as a series of companies/startups than one
| coherent entity like Meta or whatever. Each is allowed a large
| degree of autonomy from what I've heard.
| flawn wrote:
| aka Google some years ago (don't know about now...)
| leodriesch wrote:
| How does this compare to MLX? As far as I understand MLX is
| equivalent to PyTorch but optimized for Apple Silicon.
|
| Is this meant for training MLX models in a distributed manner? Or
| what is its purpose?
| simonw wrote:
| It looks like MLX is a part of this initiative.
| https://github.com/apple/corenet lists "MLX examples" as one of
| the components being released in April.
| dagmx wrote:
| Just skimming the README it looks like it's a layer above MLX.
| So looks like a framework around it to ease ML
| ipsum2 wrote:
| It's a layer on top of PyTorch, and it has code to translate
| PyTorch models into MLX.
| Mandelmus wrote:
| So, is CoreNet the equivalent of Keras, whereas MLX is the
| Jax/PyTorch equivalent?
| hmottestad wrote:
| Sounds reasonable. Apple writes the following about MLX:
| "The design of MLX is inspired by frameworks like NumPy,
| PyTorch, Jax, and ArrayFire."
| ipsum2 wrote:
| Not quite. The closest equivalent would be something like
| fairseq. It's config (yaml) driven.
| reader9274 wrote:
| As mentioned in the "mlx_examples/open_elm": "MLX is an Apple
| deep learning framework similar in spirit to PyTorch, which is
| optimized for Apple Silicon based hardware."
| jn2clark wrote:
| I would love an LLM agent that could generate small api examples
| (reliably) from a repo like this for the various different models
| and ways to use them.
| benob wrote:
| > OpenELM: An Efficient Language Model Family with Open-source
| Training and Inference Framework https://arxiv.org/abs/2404.14619
|
| Apple is pushing for _open_ information on LLM training? World is
| changing...
| tzakrajs wrote:
| We are all starting to better understand the ethos of their
| engineering teams more generally.
| javcasas wrote:
| Looks at Apple: CoreNet Looks at Microsoft: Net Core
|
| My inner trademark troll demands a bucket of popcorn.
| pixl97 wrote:
| Heh, when I saw this post this is the first thing I thought.
| orena wrote:
| The style is not very different than
| NeMo(nvidia)/fairseq(Facebook)/espent(oss) etc..
| m3kw9 wrote:
| Ok, why would anyone use this when you have industry standard
| methods already?
| RivieraKid wrote:
| What library would you recommend for neural net training and
| inference on Apple M1? I want to use it from C++ or maybe Rust.
| The neural net will have 5M params at most.
| the_king wrote:
| I would use Pytorch as your starting point. Its metal backend
| is pretty quick on Apple Silicon, and it's the most widely used
| library for everyone from hackers to foundation model builders.
___________________________________________________________________
(page generated 2024-04-24 23:01 UTC)