[HN Gopher] Proposed Windows NT sync driver brings big Wine/Prot...
___________________________________________________________________
Proposed Windows NT sync driver brings big Wine/Proton performance
improvements
Author : neogodless
Score : 175 points
Date : 2024-01-25 15:42 UTC (7 hours ago)
(HTM) web link (www.gamingonlinux.com)
(TXT) w3m dump (www.gamingonlinux.com)
| shmerl wrote:
| It's roughly comparable to existing esync and fsync solutions in
| performance, but the main benefit would be that Wine could
| finally upstream all this and there won't be a need to drag
| around out of tree patches.
| the8472 wrote:
| AIUI fsync is built on futex_waitv which has been upstreamed.
| So this has to be more than that.
|
| Edit: ah, it's about wine upstream, not kernel upstream. they
| won't take esync/fsync because those have some missing
| features.
|
| https://github.com/torvalds/linux/commit/a0eb2da92b715d0c97b...
| mjevans wrote:
| futex_waitv was only a small part of the whole FUTEX2
| package.
| https://www.phoronix.com/news/FUTEX2-futex_waitv-v2 ...
| https://www.phoronix.com/search/FUTEX2 ...
| https://www.phoronix.com/news/Futex2-System-Call-RFC
| https://www.phoronix.com/news/FUTEX2-LPC-2020
| https://www.phoronix.com/news/FUTEX2-2021-Still-WIP
| https://www.phoronix.com/news/FUTEX2-Bits-In-Locking-Core
|
| Though none of the articles so far have bothered looking
| up... 'NtPulseEvent() or the "wait-for-all" mode of
| NtWaitForMultipleObjects()'
|
| Not many great hits for NtPulseEvent() but the first result I
| got says "Function sets event to signaled state, releases all
| (or one - dependly of EVENT_TYPE) waiting threads, and resets
| event to non-signaled state. If they're no waiting threads,
| NtPulseEvent just clear event state."
|
| NtWaitForMultipleObjects() https://learn.microsoft.com/en-
| us/windows/win32/api/synchapi...
|
| After reading this, I'm still unclear about what's missing. A
| guess is the wakeup on timeout?
| poizan42 wrote:
| For the public documentation for NtPulseEvent, see
| PulseEvent (which just calls NtPulseEvent):
| https://learn.microsoft.com/en-
| us/windows/win32/api/winbase/...
| shmerl wrote:
| Yeah, it's not upstreamed in Wine, you can check the linked
| video which explains why (same problem as with esync
| basically, some Windows logic like wait on all and nt pulse
| event doesn't work with it).
|
| The funny part is that those are almost never used anyway, so
| it's more of a formality on the Wine part to be always
| correct. But at least it's moving somewhere now possibly.
| derefr wrote:
| Seems weird to upstream this support all as one shared kernel
| interface, modelled as only "NT emulation."
|
| I'm surprised the kernel would be okay with this impl. It reminds
| me of when OpenVZ tried to upstream their big blob of container
| logic. The kernel maintainers forced them to part it out into
| individual patches so it could be reworked (and we ended up with
| cgroups and namespaces.)
|
| If Linux is missing certain kernel-provided sync primitives, why
| not just implement them all as regular Linux syscalls for every
| userland app to use (similar to how e.g. futexes were introduced)
| rather then crowding them all into some compatibility chardev
| with an ABI arcane enough that it would only ever be talked to by
| one library?
| arka2147483647 wrote:
| I imagine the point is not to implement some species of sync
| primitives currently not supported by Linux, but to implement
| exactly what windows does, so one does not need to emulate them
| in userspace.
| the8472 wrote:
| The slides[0] (page 17) mention alternative solutions. One of
| them is faster userspace switching. That sounds a bit like
| something google has proposed before with
| switch_to/FUTEX_SWAP[1]
|
| [0]
| https://lpc.events/event/17/contributions/1517/attachments/1...
| [1] https://lwn.net/Articles/824409/
| cwillu wrote:
| This _is_ broken up into individual patches, with documentation
| and intent discussed (briefly) in each patch. It's not a big
| blob of undocumented and opaque container logic being dumped in
| one giant patch.
| derefr wrote:
| Sure, but all of these patches still have a common implicit
| "framework" they're introducing ABI surface to -- the ntsync
| char device. That "framework" is axiomatic to the patches.
|
| And _that 's_ the part that made OpenVZ unacceptable to
| integrate into the kernel -- the axiomatic framework being
| introduced by the patch series. No matter how you refactor
| the OpenVZ patches, they'd still just be a set of patches
| each proposing one _component_ of an _already-externally-
| designed_ higher-level feature, that can only exist if all
| the patches are accepted. (See the relevance?)
|
| My understanding of how things _should_ go in Linux kernel
| development, is that when submitting a single patch to LKML,
| 1. the _motivating user-story_ should first be explained, and
| 2. the patch that follows should then be the most idiomatic
| way to _solve that single problem_ within the existing
| architecture the Linux kernel. This will then 3. start a
| discussion, where any "specialization" of the design from
| its normative "obvious" form may be proposed, and
| justifications for this argued.
|
| When submitting a patch _series_ , each patch must go through
| this process independently. If there is any common framework
| to the patches, it can only be because the "obvious way" (by
| the consensus of all participants) to solve patch 2's user-
| story, with patch 1 already in place in a dev branch, is to
| refactor the logic of patch 1 and patch 2 together.
|
| In other words, it should be up to the _core kernel
| maintainers_ -- not the contributor -- to propose an in-
| kernel architecture like "this set of operations introduced
| over the last few development patch series, would make a lot
| of sense to expose as messages passed into a single character
| device." And such an architecture should be proposed because
| it's _what 's best for the kernel_, rather than because
| having it that way would make the contributor's life easier
| in implementing their library.
| wtallis wrote:
| Wine needs a high-performance implementation of sync primitives
| that even Microsoft has deprecated and discouraged, because
| there are real programs out there using those Windows APIs that
| Linux users want to run.
|
| Most of what Wine wants added to the kernel should not have a
| scope of use beyond Wine, which is why the Wine devs have put
| so much effort into trying to solve these problems in
| userspace.
| zeusk wrote:
| Which sync primitive has Microsoft deprecated and
| discouraged?
|
| WaitForMultipleObjects and Nt(Set|Reset|Pulse)Event are used
| throughout Windows source code
| dblohm7 wrote:
| It's still used, but PulseEvent has been discouraged for a
| long time.
|
| [1] https://learn.microsoft.com/en-
| us/windows/win32/api/winbase/...
|
| [2] https://devblogs.microsoft.com/oldnewthing/20050105-00/
| ?p=36...
| sumtechguy wrote:
| I had a wince proj that someone had used that thing and
| did not understand the 'pulse' part of that. Where it
| could go off but if nothing was waiting it would not
| signal. So they were really confused why sometimes their
| home grown event queue would be delayed until another
| event went off and it would drain their queue.
| zaphirplane wrote:
| I though Cgroup originated from google employed engineers
|
| It's a shame openVz pioneered container, yet docker ate their
| lunch
| dusted wrote:
| I used OpenVZ for a long time before Docker came along, it
| was certainly nicer than what I did previously (run
| everything right on the host) but, of the two, I'd pick
| Docker every time. Dockerfiles, images, layers, thinking of
| containers as "application instances", the networking, the
| cli, the speed, there's nothign I miss from OpenVZ, even if
| it was good for what it was, the way Docker improved on it
| made a very big difference, in a positive way.
| antod wrote:
| Docker didn't bring container running features or kernel
| features for running containers (they just used LXC at first
| didn't they?), Docker brought an image layering and
| distribution set of features for single process containers.
|
| OpenVZ (like LXC) was more of an OS level VM alternative type
| of container. Docker was more of a developer oriented
| distribution/isolation system than OpenVZ/LXC/etc which were
| more of a sysadmin oriented hosting system. In the very early
| days Docker didn't make much sense to me until I realised
| that.
| rrdharan wrote:
| cgroups was built by Google
|
| https://en.wikipedia.org/wiki/Cgroups
| jonhohle wrote:
| Perhaps the operations could have all been added to `ioctl`.
| asylteltine wrote:
| Wow that's huge. I hope this makes it way into proton. It's
| getting to the point where windows is going to be mostly
| irrelevant for gaming, and ideally, for everything!
|
| Mac is a better experience for every single thing other than
| gaming (that may change soon too). My steam deck is an amazing
| mobile gaming device. I'm hoping one day I can actually just
| steamos on my gaming pc and dump my last vestigial windows
| install.
| thomastjeffery wrote:
| Proton is a configuration system for wine.
|
| The only big problem left is kernel-aware anticheat. Ideally,
| we would just stop allowing game companies to hijack the
| windows kernel in the first place...
| ronsor wrote:
| Perhaps we can run (emulate?) the kernel drivers in user-
| space.
| jsheard wrote:
| That's known in the business as "bypassing the anti-cheat"
| and will get you banned the moment the anti-cheat provider
| notices what you're doing, even if you aren't actually
| cheating. If you tamper with, sandbox or subvert the AC in
| any way then they _have to_ assume you 're cheating since
| any signals they get from the AC can no longer be trusted.
| shmerl wrote:
| If they don't trust you, you shouldn't trust them either.
| Their anti-cheat is essentially malware.
| thomastjeffery wrote:
| Yes, but the more important point is that you are forced
| into a position of compromise. The only options are for
| you to accept their malware, or for you to not
| participate. There is no practical way to convince them
| to remove their malware requirement.
| shmerl wrote:
| My point is, you that you shouldn't participate, becasue
| why would you trust them not to use that malware for
| malicious purpose.
| bitwize wrote:
| Because you want to play Valorant, that's why.
|
| This is like when a guy said to RMS, "I have to use
| proprietary video editing software to do my job" and RMS
| was like "Then find another job."
|
| IF LINUX CANNOT RUN THE SOFTWARE PEOPLE WANT TO RUN, IT
| IS A POOR OPERATING SYSTEM.
|
| We've been telling open source folks this since the 90s.
| It's time for them to listen and learn.
| viraptor wrote:
| Even if not available upstream, I'd love it if SteamDeck included
| that patch in kernel+proton. Those are some massive gains, even
| for their narrow scope.
| Sakos wrote:
| It's mouthwatering how good the improvements are. I can only
| hope somebody at Valve has taken notice and is hurriedly
| pushing it through the process.
| johnny22 wrote:
| they're the same people in this regard. The patch author
| works for codeweavers and does work related to proton.
| ornornor wrote:
| Speaking of that, is there a resource to figure out what GPUs
| work well in Linux? I loathe my workstation that is running a
| nvidia Quadro P620, the proprietary drivers are a joke... Looking
| to switch but it's very hard to figure out the options based on a
| budget and Linux compatibility.
|
| Any pointers?
| aappleby wrote:
| I have a cheap RX 6400 and it is flawless in both Wayland and
| X.
| rpcope1 wrote:
| I have one, and I sure wouldn't recommend it beyond being a
| simple display driver. The real killer on that card in
| particular is that it, so far as I can tell, does not have
| any (?) video encoding/decoding acceleration whatsoever, and
| generally the support on AMD cards is mediocre at best until
| you step up to very high power cards.
|
| That's actually one of the painful parts about video on
| Linux, is that if you've got something that needs GPU
| acceleration via libav, you've got three (or four) not so
| nice choices: nvidia, which has good hardware support but is
| hit or miss if it actually works outside of their proprietary
| nvenc, especially for libav and vaapi; AMD, which seems to
| lack hardware for this on low power cards, or it's poor where
| it exists; Intel, where the ARC cards don't work well (or at
| all) if you lack resizable BAR (which means you can't use
| anything with any age as the host). Intel's integrated
| graphics is probably the best choice, but still I'd really
| like a discrete GPU with good Linux h264, vp9 and av1
| encoding and decoding support, as it doesn't seem to exist
| today.
| jeroenhd wrote:
| The 6400 lacks video encoders, but it still has decoders
| for H.264 and H.265. VP9 decoding should also work, but
| it's not listed on AMD's website per se. To use these
| capabilities on Linux, you will need to install the correct
| VA-API drivers, though, and Firefox is still reluctant to
| enable hardware video decoding on Linux.
|
| The resizable BAR problem is annoying, but that's not
| limited to AMD, either. I'm not 100% sure about Intel, but
| if resizable BAR doesn't affect performance, that only
| makes me think they left some easy performance win on the
| table.
|
| I have to agree on Intel QuickSync, though, that's hands
| down the best video encode/decode solution for consumers.
| AV1 encoders are very recent additions to the GPU
| landscape, though, so you won't be able to get those on a
| budget. AV1 decoders are easier to obtain, and H.264/VP9
| especially so.
| johnny22 wrote:
| what do you mean by "a joke"? I chose not to buy nvidia due to
| the proprietary drivers, but I didn't think they were "a joke".
| In the next yearish we're likely to see decently performing
| open drivers via nouveau (in kernel) and nvk (in mesa) which
| will be really nice for fitting in with the everything else,
| but I doubt it will reach the same performance level as the
| proprietary drivers.
| ornornor wrote:
| I mean they don't work well. At all.
|
| I get graphical glitches in gnome, it crashes GPU accelerated
| browser tabs, my logs are full of "FAULT_PTE
| ACCESS_TYPE_READ" errors, it freezes the displays for 30
| seconds seemingly randomly. And that's with the latest
| drivers available for Linux and the latest available 6.5
| kernel.
| johnny22 wrote:
| Isn't that why GPU accel isn't turned on by default still?
| Or did that change. I myself went to AMD for my new laptop,
| but didn't have those problems on my current laptop with a
| 960M.
| machomaster wrote:
| Amd is really your only choice. You will be fine in normal
| usage/gaming, but no bueno if you want to do ML since AMD
| naturally don't support the market leader CUDA.
| shmerl wrote:
| No very good resources for that, but you can check GPU stats
| here:
|
| https://www.gamingonlinux.com/users/statistics/#GPUModel-top
|
| As a rule of a thumb, use AMD.
| avhception wrote:
| Anecdotal, but I've got a Radeon RX 7600 at the moment and was
| using an RX 580 before. Just an hour ago I played Witcher3
| through Steam/Proton/Wine on my 4k TV. I haven't done any
| tinkering with drivers or anything, it just works out of the
| box even with my stupid self-compiled Gentoo kernel. The RX 580
| went into the secondary box running OpenSUSE and works equally
| well there.
| nitwit005 wrote:
| Wine code I've actually looked at. I was curious how they
| implemented some of it.
|
| I suppose a driver makes sense, but I'm curious how many recent
| programs actually use things like PulseEvent given the warnings
| Microsoft has put on it. People seem to have moved to the
| pthread-like APIs in recent code I've seen.
| trentnelson wrote:
| I'm curious if anyone knows how the new threadpool primitives
| introduced in Vista fair in Wine/proton? Do games simply not use
| any of these APIs? (That's surprisingly, the threadpool stuff is
| easily one of the best pieces with no comprehensive counterpart
| on other platforms.)
| bhaney wrote:
| Looking forward to this. Getting Msync[1] for wine on macOS has
| made a huge difference there. It'll be great to have a similar
| realm of improvement in wine on Linux as well. We're nearly at
| the point where I'd rather run most Windows software on any
| platform other than Windows.
|
| [1] https://github.com/marzent/wine-msync
| zoeysmithe wrote:
| From the comments the big gains are from using wineserver in
| userland but valve/steam/deck/proton uses fsync which is already
| a kernel api, so there then won't be these big gains.
|
| I'm very suspicious vale/proton/wine slept on 2-3x performance
| boosts. It sounds like this problem is already solved, but this
| implementation might be more elegant, but its the same
| performance out of edge cases.
___________________________________________________________________
(page generated 2024-01-25 23:01 UTC)