[HN Gopher] Bug in reader/writer locks in Windows API
       ___________________________________________________________________
        
       Bug in reader/writer locks in Windows API
        
       Author : Georgelemental
       Score  : 200 points
       Date   : 2024-03-03 15:59 UTC (7 hours ago)
        
 (HTM) web link (old.reddit.com)
 (TXT) w3m dump (old.reddit.com)
        
       | mastax wrote:
       | I understand why this is the case but it's also extremely
       | frustrating:
       | 
       | > It is extremely difficult for programmer-users to report bugs
       | against the Windows API (we're supposed to direct you to Feedback
       | Hub, but you may as well transmit your message into deep space).
       | I've filed OS-49268777 "SRWLOCK can deadlock after an exclusive
       | owner has released ownership and several reader threads are
       | attempting to acquire shared ownership together" with a slightly
       | reduced repro.
       | 
       | > Thanks for doing your homework and creating a self-contained
       | repro, plus pre-emptively exonerating the STL. I've filed this OS
       | bug as a special favor - bug reports are usually off-topic for
       | r/cpp. The microsoft/STL GitHub repo is the proper channel for
       | reporting STL misbehavior; it would have been acceptable here
       | even though the root cause is in the Windows API because this
       | situation is so rare. If you see STL misbehavior but it's clearly
       | due to a compiler bug, reporting compiler bugs directly to VS
       | Developer Community is the proper thing to do.
        
         | adgjlsfhk1 wrote:
         | The best way I know of to report bugs to windows is report them
         | as documentation bugs. see for example
         | https://github.com/MicrosoftDocs/cpp-docs/pull/3526.
        
           | IshKebab wrote:
           | Clever!
        
           | ddlsmurf wrote:
           | It looks like it was declared in Nov 21, and in May 23 they
           | merged in the "fix" by adding "it's approximate" to the docs
           | ( https://github.com/MicrosoftDocs/cpp-
           | docs/commit/447b5d8a781... ), so not sure this is the best
           | approach for actual bugs.
        
         | 1over137 wrote:
         | Nice to know that it's not just Apple to whom reporting bugs in
         | hopeless. :)
        
           | masklinn wrote:
           | I generally assume that's the case for any large company.
           | 
           | Sometimes I get pleasantly surprised, but generally speaking
           | the internal incentives are skewed against, the primary focus
           | is whatever the roadmap is followed by tickets from paying
           | clients, public bugs generally have a very low hit ratio so
           | they're unrewarding, unless you manage to snipe one of the
           | company's employees (either nerd-snipe or interest / shock
           | them enough to raise the issue internally) it's like playing
           | the lottery.
        
             | jansan wrote:
             | > I generally assume that's the case for any large company.
             | 
             | Despite Google partly losing its marbles recently,
             | reporting bugs to Chromium still works very well.
        
               | bee_rider wrote:
               | I wonder if the open source element of the project keeps
               | them "honest" to some extent?
        
             | generic92034 wrote:
             | > I generally assume that's the case for any large company.
             | 
             | What works to some degree are dedicated maintenance teams
             | providing development support. If their main task is fixing
             | bugs and they are evaluated on this basis, support tickets
             | reporting real bugs have a good chance to receive the
             | required attention.
             | 
             | However, there is always the temptation for management to
             | redirect resources from those teams. But at least in the
             | B2B area costly customer escalations can remind management
             | of the importance of good maintenance.
        
           | tgv wrote:
           | Depends on the team, I think. I've had no luck at all with
           | the OS/framework and "core" apps bugs, except getting the
           | report that it has been fixed, and I should install the new
           | version of the OS. Only to find out the bug wasn't solved.
           | 
           | The Logic team, however, has been helpful, and in one or two
           | cases (that were discussed in musician's forums) went out of
           | their way.
        
           | Dalewyn wrote:
           | The amount of attention and care a bug report or a piece of
           | feedback receives is inversely proportional to how easy it is
           | to file it.
        
         | jcalvinowens wrote:
         | > It is extremely difficult for programmer-users to report bugs
         | against the Windows API
         | 
         | I can't imagine living in this hell. When I find bugs in Linux,
         | I E-mail the actual engineers directly and get responses in
         | under 24 hours:
         | https://lore.kernel.org/lkml/Zcb3_fdyJWUlZQci@gmail.com/
        
           | iknowstuff wrote:
           | Thats pretty cool. No comments left in the code after the
           | revert though. Are they relying on their minds and commit
           | history as documentation of invariants? How do they prevent
           | the same mistake from being made again in the future?
        
           | charcircuit wrote:
           | Microsoft engineers have emails too.
        
             | dieortin wrote:
             | And how would you know what email to contact in a case like
             | this one? It's not about "having email"
        
               | charcircuit wrote:
               | You can either go down the route of finding the team that
               | owns the code and then contacting someone on that team,
               | or by contacting someone you know from the company to
               | look it up for you or report the bug on your behalf.
        
         | TillE wrote:
         | Like the comment says, filing an issue on GitHub would've also
         | eventually led to a proper internal report, so the system isn't
         | wholly reliant on one guy trawling Reddit.
         | 
         | I have many complaints about Win32 (having half a dozen
         | different error code types, for one), but outright _bugs_ are
         | really rare.
        
       | bingo3131 wrote:
       | Misleading title?
       | 
       | This is a Windows API bug with the slim reader/writer (SRW)
       | locks. It's just that the bug was discovered via
       | std::shared_mutex as that is implemented using SRW locks.
       | 
       | SRW locks: https://learn.microsoft.com/en-
       | us/windows/win32/sync/slim-re...
       | 
       | Confirmation from a Microsoft employee that the bug has been
       | raised internally with the Windows API team:
       | https://old.reddit.com/r/cpp/comments/1b55686/maybe_possible...
        
         | dang wrote:
         | Ok, I've changed the title to say that. Thanks!
        
         | tialaramex wrote:
         | Yes, for example if you make a Rust std::sync:RWLock on
         | Windows, it will literally be SRWLock because Microsoft
         | advertises this API as having exactly the behaviour Rust wants
         | so why would you build something worse instead ?
         | 
         | Rust's Mutex on Windows is also an SRWLock but it can't hit
         | this bug because it deliberately only uses the exclusive
         | locking.
        
           | iknowstuff wrote:
           | So it doesn't use a shared lock when .read() is used?
        
             | zozbot234 wrote:
             | Rust has both Mutex and RWLock. The Mutex only uses
             | exclusive locks, there's no distinction between "read" and
             | "write".
        
               | iknowstuff wrote:
               | Oh sorry my morning brain thought the comment was all
               | about RWLock
        
       | amluto wrote:
       | Once upon a time, you could buy various things from MS that came
       | with support incidents. I had an MSDN subscription that came with
       | two per year. Using an incident got you an actual support
       | engineer who would be helpful and escalate issues if necessary.
       | And, if your issue turned out to be a real bug of any
       | significance in an MS product, your support incident would be
       | credited back.
       | 
       | This was great for developers (real support was available, and
       | spam was discouraged), and it was great for MS (they found real
       | issues that affected paying customers and they got feedback to
       | improve their documentation).
       | 
       | I wonder whether this program still exists. I have the impression
       | the overall quality of MS documentation has declined.
        
         | twisteriffic wrote:
         | That's Premier support, though now with a bunch of extra layers
         | of contractors who can't do anything other than follow a
         | script, capture traces and wait on the next level. In my
         | experience it's been a time consuming waste of effort. I've had
         | a lot better luck harvesting names and business cards from the
         | teams at conferences and demo days, then going direct to the
         | source.
        
         | sebazzz wrote:
         | Generally as far as first line support goes, everything is
         | outsourced to India. Cheaper, on paper.
         | 
         | Anectodically, I tried to get support for something unrelated:
         | I'm trying to use IMAP sync support in Outlook.com, but it
         | refuses to work properly with iCloud-IMAP but doesn't give any
         | error message either. As a paying M365 subscriber I expect
         | proper support. I tried over five times to get support, and
         | every time it ended in frustration. Every time I got someone
         | who either doesn't understand to product, says IMAP support is
         | not available and deprecated (it is not! I was able to set-up a
         | different IMAP provider just fine), or I got redirected to the
         | Windows or Office support team, who then couldn't help me and
         | closed my ticket.
        
           | Someone1234 wrote:
           | Often times it just feels like they try to extend the ticket
           | out as long as possible asking irrelevant questions, then as
           | you said redirect you or close the ticket after wasting an
           | appropriate amount of time.
           | 
           | I cannot prove it, but I bet there is an internal number of
           | replies before they can close without penalty.
        
           | rwmj wrote:
           | If it's any consolation, our company pays for 20,000 GMail
           | licenses and we seem unable to escalate any issue at all to
           | Google, even major issues such as their clearly not working
           | spam filtering, emails being lost, incorrect deduplication of
           | emails, or their random IMAP throttling.
        
             | amluto wrote:
             | Is there any vendor of actually good email-as-a-service?
             | O365/Exchange/Outlook/Hotmail is a mess. Google has a
             | support problem. Fastmail has an offline email problem.
             | iCloud is not obviously suitable for professional use.
             | 
             | What's left?
        
               | Avamander wrote:
               | There are multiple providers out there that have their
               | own stacks (Google, Microsoft, Tutanota, Protonmail to
               | some extent and others) and there are those that use more
               | common combinations (basically just managed mail-in-a-
               | box). Nothing perfect though, so pick your poison.
               | 
               | Some issues are also due to the ecosystem itself.
               | Avoiding POP3 goes a long way for example.
        
         | marcosdumay wrote:
         | Well, maybe last century.
         | 
         | At any point of this century, the response to those support
         | incidents from MS was always to deny a problem, and if it's a
         | known one, to try to gaslight the customer in a direction
         | contrary to solving it.
         | 
         | I have seen organizations lose way too many people-hours trying
         | to satisfy the MS support and apply what it recommended. That
         | when the real solution was often reachable in a hour or two of
         | research on 3rd party knowledge bases.
        
           | amluto wrote:
           | ISTR a got an actual useful outcome from a support incident
           | related to a bug in the runtime libraries shipped with Visual
           | Studio 2005. Support gave me a hotfix.
           | 
           | Sadly the hotfix had its own little bug: installing it took
           | about 20 hours. Office productivity was rather low for the
           | rest of the day. I don't know everything that goes on under
           | the hood with Microsoft's installers, but wow they're slow.
           | 
           | I, blissfully, don't use Visual Studio on Windows any more :)
        
         | kkert wrote:
         | I actually remember reporting a crypto bug way back through a
         | similar program in Win98, where certain keys would not work if
         | the highest bit was set in the key material. It got fixed
        
         | userbinator wrote:
         | You can still pay MS for support. Whether you'll get the answer
         | you want, as the others have noted in sibling comments, is
         | highly variable.
        
       | AndrewStephens wrote:
       | Subtle bugs in Reader/Writer locks do not surprise me. I worked
       | on an in-house implementation based on Win32 (before C++11 and
       | std::shared_mutex) and my recollection is that although the
       | implementation sounds simple it is exceedingly easy to make
       | subtle mistakes.
       | 
       | The experience left me with such a bad feeling for shared locks
       | that I tend to avoid them unless absolutely required. When I last
       | tested std::shared_mutex, performance was so poor compared to
       | std::mutex that double-buffering the data protected by a simple
       | mutex was much faster.
       | 
       | This was a great post by the original Redditor.
        
         | convivialdingo wrote:
         | I did some work with testing a cross platform in-house library
         | for read-write locking.
         | 
         | We tested cmpxchg16b and found the performance was terrible
         | with more than 4 cores.
         | 
         | Ended up using spin-locks similar to Linux kernel RCU.
        
         | tialaramex wrote:
         | SRWLock seems like a situation where it's small enough for you
         | to construct proofs that what you did is correct, and important
         | enough that the enormous expense of such proof is justified.
         | 
         | The C++ std::mutex provided in Microsoft's STL is ludicrously
         | big, whereas SRWLocks are the same size as a pointer. In one
         | sense being ludicrously big is great - less risk the std::mutex
         | will mistakenly share a cache line with your unrelated data,
         | but for most people this makes std::mutex annoyingly expensive.
        
         | SunlitCat wrote:
         | Sad thing is, Reader / Writer locks are pretty tempting to use
         | as they appear to be lightweight.
        
         | userbinator wrote:
         | It has been my experience that anything related to concurrency
         | can be full of subtle edge cases. I tend to avoid it completely
         | unless absolutely necessary.
        
       | chrisjj wrote:
       | > Bug in reader/writer locks in Windows API
       | 
       | Correction: Bug in reader/writer locks in Windows
       | 
       | Not in API.
        
         | valleyer wrote:
         | Thank you. Blaming the API is like saying there's a problem
         | with your gas pedal after you blew a head gasket.
        
       | loeg wrote:
       | Some reddit comments mention it reproduces back to Vista (2008).
       | I am kind of shocked no one has noticed this bug in that time. I
       | guess under typical rwlock usage you just get random instances of
       | shared lockers unable to acquire the lock and no deadlock, but
       | still.
        
         | Diggsey wrote:
         | I think part of the reason is that there's a very similar code
         | pattern which _is_ user error, and avoiding that pattern tends
         | to avoid this pattern as well.
         | 
         | The similar case occurs when you have: - 1+ threads holding a
         | shared lock (Readers) - 1+ threads waiting to acquire an
         | exclusive lock (Pending Writers) - 1+ threads trying to acquire
         | the shared lock (Pending Readers) - 1+ Reader is waiting on a
         | Pending Reader
         | 
         | In this case the Pending Readers will be unable to acquire the
         | shared lock even though it is still in "read mode" because in a
         | fair RW lock Pending Writers are prioritised above Pending
         | Readers so as not to starve the writer side of the lock.
        
         | TillE wrote:
         | Vista is when that API was first implemented.
         | 
         | It's doing a pretty weird thing with the locks, I wonder what
         | the actual use case was. Readers should almost never care about
         | other readers. Typically you just grab the lock, read the
         | thing, and release it. You always have to be super careful
         | about deadlocks if you're holding a lock and also waiting
         | around for something else to happen.
        
       | davekilian wrote:
       | I was wondering how something so basic could go unnoticed for so
       | long. Halfway down the page on OP's link, a user u/rbmm provides
       | a compelling answer: that there are (possibly expected?) cases
       | where a thread trying to acquire the lock in shared mode can
       | accidentally get it in exclusive mode instead. This is due to
       | interleaving of atomic bit test-and-[re]set operations between
       | the (shared mode acquire) thread and the (exclusive mode release)
       | thread running simultaneously.
       | 
       | The repro code holds the shared lock while waiting for all other
       | threads to acquire the shared lock, and thus deadlocks if any of
       | the worker threads accidentally gets an exclusive lock. In
       | "normal" use cases where the lock is used to protect some shared
       | resource, threads holding the lock don't wait for each other, so
       | there is no deadlock.
       | 
       | Interesting stuff!
        
         | zozbot234 wrote:
         | Why does it have to use bit test and set and interleave with
         | other threads, though. AIUI you can use a CAS loop to implement
         | any RMW atomically over word-sized (or double word sized, on
         | many platforms) data. That seems like a no-brainer.
         | 
         | For comparison, the Rust implementation for lightweight RWLocks
         | on futex-capable *nix platforms is here: https://doc.rust-
         | lang.org/stable/src/std/sys/unix/locks/fute... It sets the
         | "reader counter" in the underlying atomic to a special value to
         | signal that the lock is set for exclusive access. So a reader
         | thread acquiring the lock as shared can never result in this
         | kind of bug. Bits are used to signal whether readers or writers
         | are currently waiting on a lock, but this just cannot turn a
         | lock that's acquired for shared access into exclusive, or vice
         | versa.
        
         | userbinator wrote:
         | If that "rbmm" is the same person as I've seen on other sites,
         | and the characteristic non-native English is a clue that it is,
         | he certainly knows his stuff.
         | 
         |  _threads holding the lock don 't wait for each other_
         | 
         | Unless you're doing nested locking.
        
       | Nition wrote:
       | > It is extremely difficult for programmer-users to report bugs
       | against the Windows API (we're supposed to direct you to Feedback
       | Hub, but you may as well transmit your message into deep space).
       | 
       | :'-(
        
         | Avamander wrote:
         | This is an issue with a lot of MS products. I honestly have no
         | clue how much money one would have to throw at MS to get a bug
         | fixed.
         | 
         | For example Microsoft's *own* Pluton-enabled platforms fail
         | Windows' Device Health Attestation checks due to an incomplete
         | chain (https://call4cloud.nl/2023/04/are-you-there-intune-its-
         | me-ha...).
        
           | Nition wrote:
           | I've noticed a lot of big products have a user feedback cycle
           | that goes something like this:
           | 
           | - Create new feedback tracker
           | 
           | - Direct feedback to tracker
           | 
           | - Stop paying any attention to tracker
           | 
           | - Tracker is hundreds of pages of users shouting into the
           | void, and much of it out-of-date
           | 
           | - Delete everything
           | 
           | - Create new feedback tracker...
        
             | Nition wrote:
             | There's nothing worse than having a problem with a product,
             | finding 250 other people on the feedback tracker that have
             | had the same problem over the last three years, and the
             | only official response is some support person on the first
             | page who's saying your feedback is very important to us,
             | and have you tried [troubleshooting that won't work]?
        
               | Avamander wrote:
               | At that scale they have 250 000 people that didn't try
               | the obvious troubleshooting first though. But yes, it's
               | annoying.
        
             | AnthonyMouse wrote:
             | The fundamental problem here is that they have a billion
             | users and most of them don't know what they're talking
             | about. If you create a simple way to contact the company it
             | will soon be full of messages from end users who can't even
             | articulate what their problem _is_ but it 's usually some
             | kind of malware or user error and is definitely not a
             | problem with whatever component they're reporting the issue
             | against.
             | 
             | What you really need is a way to report problems which is
             | _high friction_. You have to submit a git pull request of
             | your ssh public key so you can transmit your bug report via
             | sftp. Now they only get bug reports from people who can
             | figure out how to do that and can actually pay attention to
             | them because it filters out all the spam from people asking
             | Microsoft how to connect their Android to a Mac.
        
               | mschuster91 wrote:
               | Microsoft makes ~16 billion $ a quarter in profit, of
               | which they return ~10 billion $ to their investors.
               | 
               | They could go and spend 200 million a quarter on decent
               | customer support without making too much of a dent in
               | their financial line.
               | 
               | [1] https://www.microsoft.com/en-
               | us/investor/earnings/fy-2023-q2...
        
               | AnthonyMouse wrote:
               | $200M is approximately $0.15/user. How much support do
               | you expect to get for that?
        
               | mschuster91 wrote:
               | How many of these 1 billion users actually need support?
               | Only a tiny fraction.
        
               | AnthonyMouse wrote:
               | Uh... what? If there was a free number you could call to
               | get competent technical support, people would spend their
               | entire day on the phone with it instead of reading
               | documentation or hiring IT staff.
        
         | TexasMick wrote:
         | I've ran into this too, the feedback hub seems to be populated
         | by first line support types.
         | 
         | I also found a bug in a win32 API and the feedback hub told me
         | to reboot my PC
        
         | mook wrote:
         | Unfortunately, the opposite of that is everybody trying to file
         | random things that aren't actionable. See the collection of
         | mail sent to the curl maintainer...
         | 
         | https://github.com/bagder/emails/blob/main/2015/2015-06-08.m...
        
       | userbinator wrote:
       | I'm curious if this also occurs in WINE's implementation.
       | 
       | I also want to test this on my highly customised XP install which
       | has been patched to add the SRW API among other extensions, and
       | where I had also patched the kernel to fix a race condition
       | causing a deadlock in the keyed event API that the SRW
       | implementation is based on (maybe it's this same one, although in
       | Vista+ they changed it significantly; but the same edge case
       | could occur.)
        
         | nightowl_games wrote:
         | How did you patch the kernel? Like how is that possible?
        
           | userbinator wrote:
           | With a hex editor, debugger, and skills that most developers
           | these days seem to lack.
           | 
           | I patched the kernel in memory first, using a kernel
           | debugger, to verify my fix worked before editing the file on
           | disk.
        
           | gorlilla wrote:
           | Windows xp and server 2003 sources (though incomplete AFAIR)
           | were leaked back in 2020.
        
         | zozbot234 wrote:
         | The code is here
         | https://source.winehq.org/source/dlls/ntdll/sync.c#0474 and it
         | uses compare exchange operations throughout, so it should be
         | unaffected.
         | 
         | The ReactOS implementation is more involved
         | https://doxygen.reactos.org/d1/db8/srw_8c_source.html but
         | still, it uses mostly CAS operations both for the shared and
         | the exclusive case. So it should be largely free from issues.
        
       | ww520 wrote:
       | The program has a bug. It's mixing atomic and non-atomic
       | variables in the yield() checking loop. Non-atomic variables have
       | no guarantee on cache consistency for different threads. This can
       | cause the loop to run forever.                   struct
       | ThreadTestData {             int32_t numThreads = 0;
       | std::shared_mutex sharedMutex = {};
       | std::atomic<int32_t> readCounter = 0;         };              //
       | child thread         DoStuff() {
       | data->readCounter.fetch_add(1);             while
       | (data->readCounter.load() != data->numThreads) {
       | std::this_thread::yield();             }         }
       | 
       | The numThreads field is not an atomic variable. It's initialized
       | to 0 and set to 5 in the main thread. Its memory address is then
       | passed to the child threads to be checked in the yielding loop.
       | Since it's non-atomic, there's no memory barrier instruction to
       | force its new value (5) to propagate to all CPU's running the
       | threads. A child thread might get the old value 0. The logic of
       | the yield checking loop using it would never exit.
       | 
       | Since the main thread runs the code in an endless loop, the same
       | numThreads memory allocated on the stack is being set to 0 and 5
       | repeatedly. Some of the child threads can get the old value in
       | one pass of the loop. Thus the hanging.
        
         | userbinator wrote:
         | You seem to have missed the part where an actual MS employee
         | confirmed it was a bug in their API.
        
           | ww520 wrote:
           | He just read OP's code, the C++/STL Standard, and the
           | Microsoft Learn document and made that conclusion. That's a
           | rather haste determination. Unless he read the actual Windows
           | lock implementation and found a bug there, I don't think his
           | conclusion is correct.
        
             | forrestthewoods wrote:
             | > That's a rather haste determination.
             | 
             | I hope you realize how deeply ironic this statement is. If
             | you read the comments you'll find he even produced a
             | slightly reduced repro. And in other comments he tried
             | minor tweaks like the one you suggested.
             | 
             | You have a thesis that the program has a bug. (It doesn't.)
             | Go ahead and test your thesis and report back.
        
         | nemothekid wrote:
         | > _Its memory address is then passed to the child threads to be
         | checked in the yielding loop. Since it 's non-atomic, there's
         | no memory barrier instruction to force its new value (5) to
         | propagate to all CPU's running the threads._
         | 
         | Each core would have to fetch the value from main memory, where
         | it will be undoubtedly 5. There is no valid reordering (at
         | least under x86) that would cause the thread to read 0.
        
           | ww520 wrote:
           | The main thread is running the child thread creation in an
           | endless loop, repeatedly setting numThreads to 0 and to 5,
           | back to 0 and to 5 again. Can the caches of the CPU's
           | consistently keep up with the changes?
        
             | zozbot234 wrote:
             | > repeatedly setting numThreads to 0 and to 5, back to 0
             | and to 5 again.
             | 
             | The reset to 0 and to 5 happens at the start of the loop.
             | There's a happens-before relationship between it and the
             | threads being created, and then again between the threads
             | being joined and the loop cycling back. So there _shouldn
             | 't_ be any data race here.
        
               | ww520 wrote:
               | CPU's not running the main thread don't care about the
               | execution order of the instructions of the main thread.
               | They only see their local caches of the same memory
               | location got changed from 0 to 5, 5 to 0, and back to 5
               | repeatedly. When a new thread lands on a CPU with the old
               | 0 cache value, it will hang.
        
               | zozbot234 wrote:
               | The code runs in what's effectively a single-threaded
               | context. CPUs running other threads will be involved when
               | joining and launching threads, this should suffice as
               | synchronization.
        
               | cesarb wrote:
               | > CPU's not running the main thread don't care about the
               | execution order of the instructions of the main thread.
               | 
               | On x86, they do (the x86 family is unusual in having
               | strong memory ordering), but that's not the issue here.
               | 
               | > They only see their local caches of the same memory
               | location got changed from 0 to 5, 5 to 0, and back to 5
               | repeatedly.
               | 
               | Their local caches of that memory see only a 5, since at
               | the moment they read that cache line, the value in memory
               | is 5; the operating system ensures that the write of the
               | 5 value by the main thread is flushed to memory[ _]
               | before the main thread starts the child thread, and also
               | that the cache of the child thread does not have stale
               | data from before that moment. That memory location is
               | only set back to 0 after all the child threads have
               | exited, so there 's no instant where the child thread
               | could read a 0 on that location from main memory into its
               | cache.
               | 
               | > When a new thread lands on a CPU with the old 0 cache
               | value, it will hang.
               | 
               | When a new thread lands on a CPU core with an old 0 cache
               | value for that memory location (which could happen if
               | that CPU core had been running the main thread, and the
               | main thread was migrated to another CPU core before it
               | could set it back to 5), it will still see a 5 at that
               | memory location, because the operating system invalidates
               | the cache of a CPU core when necessary before starting a
               | new thread on it.
               | 
               | [_] Actually, it only has to be flushed as far as the
               | last level cache, or the "point of unification" in ARM
               | terminology; I simplified a lot in this explanation.
        
         | ot wrote:
         | `numThreads` is written before the child threads that read it
         | are started, so there is an explicit happens-before
         | relationship and no data race. Before `numThreads` is reset,
         | the child thread are joined.
         | 
         | There is no bug in the program, it is legal to use non-atomic
         | variables across threads as long as they're correctly
         | sequenced.
        
           | ww520 wrote:
           | See my last paragraph above.
        
         | forrestthewoods wrote:
         | Your assessment is completely, totally, and verifiably wrong.
         | It's obnoxious, but not surprising, how many commenters have
         | chimed in with "program has bug BLAH" when they provably did
         | not try to run the program themselves.
         | 
         | Because if you did attempt to run the program you'd find that
         | changing numThreads to a constexpr makes no difference.
        
         | cesarb wrote:
         | > there's no memory barrier instruction to force its new value
         | (5) to propagate to all CPU's running the threads.
         | 
         | The equivalent of the memory barrier instructions is there, but
         | it's hidden within the operating system code which creates and
         | initializes a new thread. That is, the operating system ensures
         | that the value in the current CPU (in this case, 5) is
         | propagated to the CPU running the newly started thread, before
         | the thread start routine (in this case, DoStuff) is called. The
         | value is not modified while the child threads are running (it
         | waits for the child threads to exit before clearing the value),
         | so there's no chance of the child threads seeing the value
         | being set back to zero.
        
           | ww520 wrote:
           | Based on the Windows CreateThread API [1], it doesn't say
           | anything about memory synchronization guarantee. Does it do
           | internally?
           | 
           | [1] https://learn.microsoft.com/en-
           | us/windows/win32/api/processt...
        
       ___________________________________________________________________
       (page generated 2024-03-03 23:00 UTC)