[HN Gopher] Windows ML is generally available
___________________________________________________________________
Windows ML is generally available
Author : sorenjan
Score : 56 points
Date : 2025-09-25 20:11 UTC (2 days ago)
(HTM) web link (blogs.windows.com)
(TXT) w3m dump (blogs.windows.com)
| MrCoffee7 wrote:
| How does Windows ML compare to just using something like Ollama
| plus an LLM that you download to your device (which seems like it
| would be much simpler)? What are the privacy implications of
| using Windows ML with respect to how much of your data it is
| sending back to Microsoft?
| wmf wrote:
| Ollama doesn't support NPUs.
| pjmlp wrote:
| Windows ML is an abstraction to use local LLM models across
| CPU, GPU or NPU, making the code independent of the actual
| hardware.
|
| It is the evolution of DirectX for ML, previously known as
| DirectML.
| gloyoyo wrote:
| I think this is very neat. So many possibilities.
| catgary wrote:
| It's kind of a bummer because this is the exact same playbook
| as DirectX, which ended up being a giant headache for the
| games industry, and now everyone is falling for it again.
| thanksgiving wrote:
| Exactly, any work you do on top of this makes your work
| hostage to Windows.
| adzm wrote:
| Yeah DirectX ended up being a giant headache but there were
| times in its history where it was the easiest api to use
| and very high performance. DirectX came about because the
| alternatives at the time were, frankly, awful.
| Flere-Imsaho wrote:
| OpenGL (the main competition to DirectX) really wasn't
| that bad in the fixed-function days. Everything fell
| apart when nVidia / AMD came up with their own standards
| for GPU programming.
|
| DirectX was nice in that the documentation, and
| example/sample code was excellent.
| sorenjan wrote:
| What's stopping you from using ONNX models on other
| platforms? A hardware agnostic abstraction to make it
| easier for consumers to actually use their inference
| capable hardware seems like a good idea, and exactly the
| kind of stuff I think an operating system should provide.
|
| > Call the Windows ML APIs to initialize EPs [Execution
| Providers], and then load any ONNX model and start
| inferencing in just a few lines of code.
| kg wrote:
| I would be curious to see whether it's a common opinion
| that DirectX was a bad thing for the games industry. It was
| preceded by a patchwork of messy graphics/audio/input APIs,
| many of them proprietary, and when it started to gain
| prominence, Linux gaming was mostly a mirage.
|
| A lot of people still choose to build games on Direct3D 11
| or even 9 for convenience, and now thanks to Proton games
| built that way run fine on Linux and Steam Deck. Plus
| technologies like shadercross and mojoshader mean that
| those HLSL shaders are fairly portable, though that comes
| at the cost of a pile of weird hacks.
|
| One good thing is that one of the console vendors now
| supports Vulkan, so building your game around Vulkan gives
| you a head start on console and means your game will run on
| Windows, Linux and Mac (though the last one requires some
| effort via something like MoltenVK) - but this is a
| relatively new thing. It's great to see either way, since
| in the past the consoles all used bespoke graphics APIs
| (except XBox, which used customized DirectX).
|
| An OpenGL-based renderer would have historically been even
| more of an albatross when porting to consoles than DX,
| since (aside from some short-lived, semi-broken support on
| PS3) native high-performance OpenGL has never been a
| feature on anything other than Linux and Mac. In comparison
| DirectX has been native on XBox since the beginning, and
| that was a boon in the XBox 360 era when it was the
| dominant console.
|
| IMO historically picking a graphics API has always been
| about tradeoffs, and realities favored DirectX until at
| least the end of the XBox 360 era, if not longer than that.
| hbcondo714 wrote:
| > Windows ML is the built-in AI inferencing runtime optimized for
| on-device model inference...lets both new and experienced
| developers build AI-powered apps
|
| This sounds equivalent to Apple's announcement last week about
| _opening up access for any developer to tap into the on-device
| large language model at the core of Apple Intelligence_ [1]
|
| No matter the device, this is a win-win for developers making &
| consumers getting privacy-focused apps
|
| [1] https://www.apple.com/newsroom/2025/09/new-apple-
| intelligenc...
| pjmlp wrote:
| This is the evolution of Direct ML, taking into account the
| issues with it being too focused on C++, like anything DirectX.
|
| Thus C#, C++ and Python support as WinRT projections on top of
| the new API.
| mey wrote:
| Which should also make it pretty easy to drop Java JNI on top
| of it.
| politelemon wrote:
| I'm not seeing the equivalence. Isn't the announcement here to
| let you run any model?
| coffeecoders wrote:
| Ollama started out as the clean "just run a model locally" story,
| but now it's branching into paid web search, which muddies the
| open-source appeal. Windows ML goes the other direction: deep OS
| integration, but at the cost of tying your stack tightly to the
| Windows ecosystem, very reminiscent of DirectX.
|
| Now, the real question is whether vLLM/ONNX or just running
| straight on CUDA/ROCm are the only alternatives or we are all
| trading one vendor lock-in with another.
| storus wrote:
| How is this going to support custom layers like variations of
| (flash) attention that every company seems to introduce? Would it
| mean one won't be able to run a specific model (or only have its
| bastardized version) until MS implements it in the runtime?
| daemonologist wrote:
| System ONNX might be quite nice for Windows applications,
| _provided_ the backends are actually reliable on most systems.
| AMD currently has three options for example (ROCm, MIGraphX, and
| Vitis), and I 've never gotten any of them to work. (Although
| MIGraphX looks to be no longer marked as experimental so maybe I
| should give it another try.)
___________________________________________________________________
(page generated 2025-09-27 23:00 UTC)