[HN Gopher] Inside the M4 Apple Neural Engine, Part 1: Reverse E...
___________________________________________________________________
Inside the M4 Apple Neural Engine, Part 1: Reverse Engineering
Author : zdw
Score : 232 points
Date : 2026-03-01 17:11 UTC (1 days ago)
(HTM) web link (maderix.substack.com)
(TXT) w3m dump (maderix.substack.com)
| poszlem wrote:
| Genuine question, not trying to throw a shade or anything, but
| are those cores actually useful with the state of apple
| intelligence being what it is?
| rahkiin wrote:
| They are also used by ML models that are deeply integrated in
| macos and ios without you knowing. Like object and text
| detection in images.
| geerlingguy wrote:
| And help in Photos, Final Cut Pro, and other apps.
| willis936 wrote:
| I wish they would (or wouldn't if they are) hook it up to the
| ios keyboard.
| esafak wrote:
| You can convert your own ML models to MLX to use them; Apple
| Intelligence is not the only application.
| nullstyle wrote:
| MLX does not run on NPUs AFAIK; just gpu and cpu. You have to
| use CoreML to officially run code on the neural engine.
| mirsadm wrote:
| Even then there is no transparency on how it decides what
| runs on the ANE/GPU etc
| sroussey wrote:
| Correct. OS level stuff get first priority, so you can't
| count on using it.
| znagengast wrote:
| Turns out third party actually gets priority for ANE
| llm_nerd wrote:
| https://dennisforbes.ca/blog/microblog/2026/02/apple-neural-...
| malshe wrote:
| This is a nice article. Thanks for sharing.
| stetrain wrote:
| Apple's OSes run a lot of local ML models for many tasks that
| aren't branded as Apple Intelligence, and they have done so for
| many years now.
| dagmx wrote:
| If you strip away the branding, Apple has and continues to ship
| a ton of algorithms that likely use the ANE and end users can
| use CoreML to do the same.
|
| Just some things that people will likely take for granted that
| IIRC Apple have said use the ANE or at least would likely
| benefit from it: object recognition, subject extraction from
| images and video, content analysis, ARKit, spam detection,
| audio transcription.
| sroussey wrote:
| Don't forget FaceID and many of the image manipulation.
|
| And while everyone else went to more powerful giant LLMs,
| Apple moved most of Siri from the cloud to your device.
| Though they do use both (which you can see when Siri corrects
| itself during transcription--you get the local Siri version
| corrected later by the cloud version).
| love2read wrote:
| This article was clearly written by a human (and AI) but still
| has a few "LLMisms" such as:
|
| - The key insight - [CoreML] doesn't XXX. It YYY.
|
| With that being said, this is a highly informative article that I
| enjoyed thoroughly! :)
|
| The article links to their own Github repo:
| https://github.com/maderix/ANE
| walthamstow wrote:
| We've got about a year before so many people are interacting
| with LLMs on a daily basis that its style starts to reverse
| infect human speech and writing
| Angostura wrote:
| My honest take? You're probably right
| sholladay wrote:
| You are absolutely right.
|
| Here is why you are correct:
|
| - I see what you did there.
|
| - You are always right.
| pixl97 wrote:
| This said, there were people that talked like this before
| LLMs, it didn't develop this whole cloth.
| DrScientist wrote:
| Exactly. LLM's are mimics.
|
| People seem to be going around pointing out that people
| talk like parrots, when in reality it's parrots talk like
| people.
| pixl97 wrote:
| I mean, it's both.
|
| Did you develop your own whole language at any point to
| describe the entire world? No, you, me, and society mimic
| what is around us.
|
| Humans have the advantage, at least at this point, of
| being a continuous learning device so we adapt and change
| with the language use around us.
| pcrh wrote:
| The article above doesn't read well, at all.
|
| It's not my subject, but it reads as a list of things.
| There's little exposition.
| dylan604 wrote:
| Gawd Damn LISTICLES!!!! And all of those articles that
| list in bullet points at the top of the article the
| summary of the article. And all of those people saying
| they don't want to read exposition, just give me the
| bullet points.
| baxtr wrote:
| Great insight - Would you like to try and identify some
| specific "AI-isms" that you've noticed creeping into your own
| writing or your colleagues' emails lately?
| gogopromptless wrote:
| It's already happened to me. I've started to have dreams
| where instead of some sort of interpersonal struggle the
| entire dream is just a chatbot UI viewport and I'm arguing
| with an LLM streaming the responses in. Which is super trippy
| when I become aware its a dream. In the old days I'd dream
| about playing chess against myself and lose which was quite
| bizzare feeling because my brain was running both players.
| But thats totally normal compared to having my brain pretend
| to be an LLM inside a dream.
| rafram wrote:
| Also the Prior Art section, which has telltale repetition of
| useless verbs like "documenting," "providing insight into," and
| "confirming" on each line. This was definitely AI-written, at
| least in part.
| tzs wrote:
| Below are the items from that section. How should they be
| written to not look like an AI?
|
| > hollance/neural-engine -- Matthijs Hollemans' comprehensive
| community documentation of ANE behavior, performance
| characteristics, and supported operations. The single best
| existing resource on ANE.
|
| > mdaiter/ane -- Early reverse engineering with working
| Python and Objective-C samples, documenting the ANECompiler
| framework and IOKit dispatch.
|
| > eiln/ane -- A reverse-engineered Linux driver for ANE
| (Asahi Linux project), providing insight into the kernel-
| level interface.
|
| > apple/ml-ane-transformers -- Apple's own reference
| implementation of transformers optimized for ANE, confirming
| design patterns like channel-first layout and 1x1 conv
| preference.
| mattlangston wrote:
| The future is bright for software engineers.
|
| The big takeaway isn't reverse engineering the ANE per se, but
| what Manjeet could do with his software engineering skills when
| accelerated by AI.
|
| This is a good example of the present state of software
| engineering. Not future state - present state.
| Octoth0rpe wrote:
| Part 2 has benchmarks: https://maderix.substack.com/p/inside-
| the-m4-apple-neural-en...
|
| 6.6 FLOPS/W, plus the ability to completely turn off when not in
| use, so 0W at idle.
| AceJohnny2 wrote:
| But not 38 TOPS that Apple claims, with the weak explanation of
|
| > _Apple's "38 TOPS INT8" is computed as 19 TFLOPS FP16 x 2,
| following the industry convention of counting INT8 operations
| as 2x the FP16 rate. But the hardware doesn't actually execute
| INT8 operations twice as fast._
|
| Why would Apple follow that convention when the hardware
| explicitly doesn't seems like a more straight-faced lie that I
| expect from Apple
| kamranjon wrote:
| I have always wondered if the neural engine could be used for
| training - pretty excited for part 3 of this to see if the juice
| is actually worth the squeeze
| juancn wrote:
| In principle most if not all inference hardware should be
| usable for training.
|
| Efficiency is the question.
| eleventyseven wrote:
| > Throughout this series, "we" refers to maderix (human) and
| Claude Opus 4.6 (by Anthropic) working as a pair. The reverse
| engineering, benchmarking, and training code were developed
| collaboratively
|
| Sure, "collaboratively." Why would I ever trust a vibe coded
| analysis? How do I, a non expert in this niche, know that Opus
| isn't pulling a fast one on both of us? LLMs write convincing
| bullshit that even fools experts. Have you manually verified each
| fact in this piece? I doubt it. Thanks for the disclaimer, it
| saved me from having to read it.
| withinboredom wrote:
| Claude likes to hide bad benchmarks from you, so it will show
| you where you are clearly winning. You even see some weird
| benchmarks in the article.
| Anonbrit wrote:
| Humans also write endless amounts of convincing bullshit, and
| have done since time immemorial. False papers and faked results
| have been a growing scourge in academia before LLMs were a
| thing, and that's just counting the intentional fraud - the
| reproducibility crisis in science, especially medical and
| psychological science, affects even the best designed and well
| intentioned of studies.
|
| Humans also make mistakes and assumptions while reverse
| engineering, so it will always need more engineers to go
| through the results, test things
| LatencyKills wrote:
| I worked on the Xcode team for years and know the lengths Apple
| goes to make this stuff difficult to figure out.
|
| I just wanted to say that you've done an excellent job and am
| looking forward to the 3rd installment.
| RetpolineDrama wrote:
| >I worked on the Xcode team for years
|
| Why did you guys remove the ability to detach the console and
| move it to another window?
| daoistmonk wrote:
| Tangential: Is anyone doing something similar to accelerate the
| support matrix of Linux on anything higher than M2?
| GeekyBear wrote:
| The recent news is that Apple is supposedly replacing the Core ML
| framework with an updated version that will make it easier to
| integrate third party LLMs into your apps.
|
| > the company is also planning a few other software-based AI
| upgrades, including a new framework called Core AI. The idea is
| to replace the long-existing Core ML with something a bit more
| modern.
|
| https://www.bloomberg.com/news/newsletters/2026-03-01/apple-...
| behnamoh wrote:
| It's insane that the source code of ANE is not available _even to
| the MLX team_ , possibly one of the reasons Awni (MLX project
| head) left Apple.
| FL33TW00D wrote:
| Unreadable Claude slop
| mayhemducks wrote:
| I never realized just how much hardware engineering Apple
| dedicated to enabling people to type faster with their thumbs!
| giancarlostoro wrote:
| Reverse Engineering with AI is only going to get better. I have
| seen some crazy things friends of mine have done with Claude
| alone. Let's just says SaaS isn't the only industry that could
| one day suffer.
| msie wrote:
| I remember the good old days when Apple was desperate for
| developers and produced great documentation and there were a lot
| of great 3rd-party books too. You can't just give out awards in
| hopes that someone will make that great app.
| pstuart wrote:
| Yeah, the Inside Macintosh guides were epic.
| ericol wrote:
| > human intuition driving the exploration
|
| This, a thousand times this.
|
| For me, what AI brings is augmented humans. Just as we don't
| calculate on paper anymore, what is the reason of doing things by
| hand when a machine in X times better.
|
| Want to code by hand, as artisans of old? Suit yourself.
|
| I, for one, love the smell of burning chrome.
| pklausler wrote:
| If "AI" were doing anything more than repeating content from
| the web without attribution, I might agree with you.
| grey-area wrote:
| If only they could fix the iOS autocomplete, which is getting
| worse with every iteration.
| zozbot234 wrote:
| Much of this information we already knew the very basics of from
| documentation of the M1/M2 ANE as accessed via bare-metal from
| Asahi Linux, but it's nice to see confirmation and it being
| explored in further depth. Note that according to OP Parts 1/2
| for very large matmuls CoreML adds little to no overhead compared
| to the lower-level interface, so there seems to be plenty of
| scope for supporting ANE for prefill in local AI frameworks.
| Decode is generally memory-bandwidth limited unless context is
| very large, and the ANE requires special handling (converting
| from matmul to 1x1 convolution as described here is wasteful of
| memory bandwidth, as is potentially dequantizing to INT8/FP16 in
| memory) so it's less of a clear win.
| notepad0x90 wrote:
| I've been guilty of this myself, but every other comment here is
| like "What about <insert something unrelated to the topic but
| related to apple>".
| blobbers wrote:
| Can someone help me understand when these neural engines kick in
| in open source software?
|
| I typically use python ML libraries like lightgbm, sklearn,
| xgboost etc.
|
| I also use numpy for large correlation matrices, covariance etc.
|
| Are these operations accelerated? Is there a simple way to
| benchmark?
|
| I see a lot of benchmarks on what look like C functions, but
| today in my jobs I rely on higher level libraries. I don't know
| if they perform any better on apple HW, and unless they have a
| flag like use_ane I'm inclined to think they do better.
|
| Of course chatgpt suggested I benchmark an Intel Mac vs. newer
| apple silicon. Thanks chatgpt, there's a reason people still hate
| AI.
| zozbot234 wrote:
| > when these neural engines kick in in open source software?
|
| It mostly doesn't because NPUs are bespoke and vendor-specific
| (which incents neglect by software devs working on open source
| numerics and ML/AI infrastructure), and the Apple ANE is no
| exception. Part of this effort is most likely about fixing that
| for the specific case of the Apple ANE.
| blobbers wrote:
| Part of which effort? The Reverse engineering is so it can be
| used blog article?
|
| I just think: great it seems like I'm paying for a hardware
| accelerator that makes Siri go faster. And I use siri on my
| laptop exactly 0 times in the last infinite years.
___________________________________________________________________
(page generated 2026-03-02 23:00 UTC)