[HN Gopher] Async hazard: MMAP is blocking IO
___________________________________________________________________
Async hazard: MMAP is blocking IO
Author : mmastrac
Score : 56 points
Date : 2024-08-21 17:01 UTC (3 days ago)
(HTM) web link (huonw.github.io)
(TXT) w3m dump (huonw.github.io)
| PaulHoule wrote:
| No secret. Reading from memory is synchronous and always has
| been, at least in a normal computer. (Sometimes I think of how
| you could fit a fancy memory controller in a transport triggered
| architecture but that's something different)
| xoranth wrote:
| On Linux, you might be able to use userfaultfd to make it
| async...
| dividuum wrote:
| I don't see how that would work. The memory access causing
| the page fault still blocks, but now another thread handles
| paging in the requested data. So without coordination between
| those two, nothing really changes. Sounds easier to just use
| nonblocking reads directly.
|
| Thanks for the pointer to userfaultfd. Didn't know that
| existed.
| cmrdporcupine wrote:
| Yeah. Part of the problem is that userfaultfd isn't itself
| quite flexible enough. What you might want to do is release
| the faulted thread to do some other work, letting it "know"
| what it can come back later when the data is available, but
| there's no mechanism to make that happen. Instead it's
| going to be entirely blocked until the fault can be
| resolved.
| 01HNNWZ0MV43FF wrote:
| By blocking they mean that it can take ballpark non-volatile
| storage times instead of ballpark RAM times
| xoranth wrote:
| I believe they mean that since it bypasses the (Tokio)
| scheduler, so if you use it in async code you lose the main
| benefit of async code (namely, the scheduler is able to
| switch to some other task while waiting for IO to complete.).
| Basically the same behavior you'd get if you called a
| blocking syscall directly.
| guerrilla wrote:
| GP is aware. mmap makes files act like memory. Memory is
| always synchronous, thus blocking, so mmaped files are always
| blocking. I'm surprised OP even found this surprising. It
| should be completely obvious.
| dataflow wrote:
| At first I thought the title meant the mmap() call _itself_
| blocks, which I figured could be slightly surprising. But
| it seems they 're referring to I/O on the mapped file? I'm
| also baffled, how could it possibly _not_ block?
| usefulcat wrote:
| > how could it possibly not block?
|
| When the bytes being read are already in the cache. Hence
| the later part of the article where the author shows that
| reading mapped memory can be significantly faster.
| magicalhippo wrote:
| It still blocks. It just completes orders of magnitudes
| faster.
| icedchai wrote:
| Do you consider reading from a normal array (one not
| backed by a memory mapped file) to also be blocking?
| okr wrote:
| Ha. Exactly.
| magicalhippo wrote:
| In the languages and platforms I use, absolutely yes. Do
| you have some examples where a normal memory read is
| async?
| icedchai wrote:
| Your definition of blocking is a bit different from my
| own. Synchronous is not always blocking. If the data is
| there, ready to go, there is no "blocking."
|
| If you consider all memory reads to be "blocking", then
| everything must be "blocking". The executable code must,
| after all, be read by the processor. In an extreme case,
| the entire executable could be paged out to disk! This
| interpretation is not what most people mean by
| "blocking."
| magicalhippo wrote:
| Fair point. I guess I conflate the two, because what's
| interesting to me, most of the time, is where does the
| control flow switch.
|
| I never rely on synchronous IO being non-blocking when
| writing regular code (ie not embedded). As such reading
| from cache (non-blocking) vs disk (blocking) doesn't
| matter that much, as such. It's synchronous and that's
| all I need to reason about how it behaves.
|
| If I need it to be non-blocking, ie playing audio from a
| file, then I need to ensure it via other means (pre-
| loading buffer in a background thread, etc etc).
|
| edit: And if I _really_ need it not to block, the buffer
| needs to reside in the non-paged pool. Otherwise it can
| get swapped to disk.
| rbanffy wrote:
| > Do you have some examples where a normal memory read is
| async?
|
| This hints at a way to make it work, but would need the
| compiler (or explicit syntax) to make it clear you want
| to be able to switch to another task when the page fault
| triggers the disk read, and return to a blocking access
| that resolves the read from memory after the IO part is
| concluded.
|
| It could look like a memory read but would include a
| preparation step.
| nemetroid wrote:
| If the memory has been paged to disk, I guess so?
| danarmak wrote:
| No, if the memory-mapped page you're accessing is in RAM,
| then you're just reading the RAM; there is no page fault
| and no syscall and nothing blocks.
|
| You could say that any non-register memory access
| "blocks" but I feel that's needlessly confusing. Normal
| async code doesn't "block" in any relevant sense when it
| accesses the heap.
| magicalhippo wrote:
| When dealing with async I think it is very relevant to
| think of exactly the points where control can be
| switched.
|
| As such a regular memory read is blocking, in that
| control will not switch while you're doing the read (ie
| your not doing anything else while it's copying). This is
| unlike issuing an async read, which is exactly a point
| where control _can_ switch.
|
| edit: As an example, consider synchronous memory copy vs
| asynchronous DMA-based memory copy. From the point of
| view of your thread, the synchronous copying blocks,
| while with the DMA-based copying the thread can do other
| stuff while the copying progresses.
| nemetroid wrote:
| So what is the definition of "blocking" here? That it
| takes more than 1 us?
| danarmak wrote:
| That the process/thread enters kernel mode and then is
| suspended waiting for IO or for some other event. As long
| as the thread is running your code (or, is scheduleable)
| it's not blocked. And then the async implementation can
| ensure your code cooperatively gives up the CPU for other
| code.
| icedchai wrote:
| One trick is to read the file into memory at application
| startup. All data is paged in, so it's hot and ready to
| go: no page faults. In the early 2000's, I worked on a
| near real time system that used memory mapped I/O. At app
| startup, several gigabytes were read into memory. It
| never blocked under normal circumstances (in production)
| since the systems were provisioned with enough memory.
| dataflow wrote:
| But that requires knowing your RAM is big enough to fit
| the file. It can't work in general.
| anonymoushn wrote:
| Well, the OP could probably get their benchmarks to run
| faster if they passed MAP_POPULATE, which would make the
| mmap call block for longer.
|
| In a pedantic sense, the mmap call is already blocking,
| because any system call takes longer than e.g. stuffing
| an sqe onto a queue and then making 0 syscalls, and it
| could take a variable amount of time depending on factors
| beyond that one process's control. I don't think anyone
| actually needs to offload their non-MAP_POPULATE mmaps to
| a separate thread or whatever though.
| jerf wrote:
| The term "blocking" has diverged between various
| communities and it is important to recognize those
| differences or you'll have dozens of people talking past
| each other for hundreds of messages as they all say
| "blocking" and think they mean the same thing, and then get
| very confused and angry at all the other people who are so
| obviously wrong (and in their context, they are) but just
| can't see it.
|
| It is obvious that a given "execution context", which is my
| generalized term for a thread and an async job and anything
| else of a similar nature, when it reaches for a value from
| an mmap'd file will be blocked until it is available.
| However, different communities have different ideas of an
| "execution context".
|
| Threaded language users tend to see them as threads, so
| while a given thread may be blocked the rest of the program
| can generally proceed. (Although historically the full
| story around file operations and what other threads can
| proceed past has been quite complicated.)
|
| Async users on the other hand are surprised here because
| the operation is blocking their entire _executor_ , even
| though in principle it ought to be able to proceed with
| some other context. Because it's invisible to the executor,
| it isn't able to context-switch.
|
| In this case, the threaded world view is reasonably
| "obvious" but it can be non-obvious that a given async
| environment may not be able to task switch and it may
| freeze an entire executor, and since "one executor" is
| still a fairly common scenario, the entire OS process.
|
| (I am expressing no opinion about whether it _must_ block
| an executor. System calls come with a lot of flags nowadays
| and for all I know there 's some way an async executor
| could "catch" a mapped access and have an opportunity to
| switch. I am taking the original article's implicit claim
| that there isn't one happening in their particular
| environment at face value.)
|
| As long as you do not distinguish how various communities
| use the term "blocking", you will get very, very deeply
| nested threads full of arguments about something that, if
| you just are careful with your terminology, isn't
| complicated for anyone from any subculture to understand.
| bhawks wrote:
| Same thing would happen if your memory has been paged to disk
| too.
| mjb wrote:
| I like this point - it's no secret that mmap can make memory
| access cost the same as an IO (swap can too) - but the
| interaction with async schedulers isn't immediately obvious. The
| cost can, sometimes, be even higher than this post says, because
| of write back behavior in Linux.
|
| Mmap is an interesting tool for system builders. It's super
| powerful, and super useful. But it's also kind of dangerous
| because the gap between happy case and worst case performance is
| so large. That makes benchmarking hard, adds to the risk of
| stability bugs, and complicates taming tail latency. It's
| behavior also varies a lot between OSs.
|
| It's also nice to see all the data in this post. Too many systems
| design conversations are just dueling assertions.
| a-dub wrote:
| it outsources buffer management and user thread i/o scheduling
| to the kernel. for some use cases it's a great way to simplify
| implementation or boost performance. for others it may not
| perform as well.
|
| the blog post points (in my mind) at some more general advice
| when programming which is not to mix and match paradigms unless
| you really know what you're doing. if you want to do user space
| async io, cool. if using kernel features tickles your fancy,
| also cool.
|
| mixing both without a deep understanding of what's going on
| under the hood will probably give you trouble.
| rbanffy wrote:
| Making it work asynchronously would require the compiler to
| split the memory access into two parts, a non-blocking IO
| dispatch and a blocking access to the mapped address. The OS
| would need to support that, however, and the language would
| need to keep track of what is a materialised array and what's
| not.
| a-dub wrote:
| as i understand, mmap is only efficient because it can
| leverage hardware support for trapping into the kernel when
| a page needs to be loaded to satisfy an access attempt.
|
| i think adding software indirection to every access in the
| mapped region would be really slow.
|
| i think a better answer would be to impose more structure
| on the planned memory access, then maybe given some
| constraints (like say, "this loop is embarrassingly
| parallel") the system could be smarter about working on the
| stuff in ram first while the rest is loaded in.
| cbsmith wrote:
| I'm surprised this is seen as a liability of mmap rather than a
| cooperative scheduler that isn't using native kernel threads.
| This is the deal you make with the devil when you use
| cooperative scheduling without involving the kernel, so I'm
| surprised it is news to people working with cooperative
| schedulers. These faults can happen even if you never
| explicitly memory map files (particularly since executables and
| shared libraries are often memory mapped into processes), so
| page faults are a blocking hazard for cooperative schedulers
| even without mmap.
|
| The MMU in the hardware is aggressively parallel, and the only
| thread being blocked on the page fault is the one touching the
| page that needs to be swapped in. In reality, you can get
| heavily parallelized IO using mmap (indeed, it works quite well
| when you have a ton of IO you'd like to execute in parallel).
| Retr0id wrote:
| > This is thus a worst case, the impact on real code is unlikely
| to be quite this severe!
|
| I think the actual worst-case would be to read the pages in a
| (pseudo-)random order.
| lowbloodsugar wrote:
| - register
|
| - shadow register
|
| - L1
|
| - L2
|
| - L3
|
| - RAM
|
| - GPU/SPU/RSP
|
| - SSD
|
| - Network
|
| - HDD
|
| The line is drawn depending on what you are doing and how.
|
| Edit: moved Network above HDD. :-)
| tux3 wrote:
| I would put networks above HDDs (depending on how many miles
| you need to send your emails)
| mjb wrote:
| Modern data center networks offer RTTs about 100x lower than
| hard drive latency, and comparable to local SSD. It depends,
| of course, how far over the network you're going, and how
| fast the other side responds, but <100us is very achievable.
| correnos wrote:
| IMO this is a strong argument for proper threads over async: you
| can try and guess what will and won't block as an async framework
| dev, but you'll never fully match reality and you end up wasting
| resources when an executor blocks when you weren't expecting.
| dan-robertson wrote:
| I don't find this argument super strong, fwiw. It could just
| mean 'be wary of doing blocking operations with async, and note
| map makes reading memory blocking (paging in) and writing
| memory blocking (CoW pages)'
|
| I think there are reasons to be wary but to me, debugging comes
| first (this goes two ways though: if you have a single 'actual'
| thread then many races can't happen) because
| debuggers/traces/... work better on non-async code. Performance
| comes second but it's complicated. The big cost with threads is
| heavy context switches and per-thread memory. The big cost with
| async is losing cpu locality (because many syscalls on Linux
| won't lead to your thread yielding, and the core your thread is
| on will likely have more of the relevant information and lots
| of cache to take advantage of when the syscall returns[1]) and
| spending more on coordination. Without io_uring, you end up
| sending out your syscall work (nonblocking fd ops excepted) to
| some thread pool to eventually pick up (likely via some futex)
| load into cache, send to the os on some random core, and then
| send back to you in a way that you will notice such that the
| next step can be (internally) scheduled. It can be hard to keep
| a handle on the latency added by all that indirection. The
| third reason I have to be wary of async is that it can be
| harder to track resource usage when you have a big bag of async
| stuff going on at once. With threads there is some sense in
| which you can limit per-thread cost and then limit the number
| of threads. I find this third reason quite weak.
|
| All that said, it seems pretty clear that async provides a lot
| of value, especially for 'single-threaded' (I use this phrase
| in a loose sense) contexts like JavaScript or Python where you
| can reduce some multithreading pain. And I remain excited for
| io_uring based async to pick up steam.
|
| [1] there's this thing people say about the context switching
| in and out of kernel space for a syscall being very expensive.
| See for example the first graph here:
| https://www.usenix.org/legacy/events/osdi10/tech/full_papers...
| . But I think it isn't really very true these days (maybe
| spectre & co mitigations changed that?) at least on Linux.
| akira2501 wrote:
| > How do other mmap/madvise options influence this (for instance,
| MADV_SEQUENTIAL, MADV_WILLNEED, MADV_POPULATE,
| MADV_POPULATE_READ, mlock)? (Hypothesis: these options will make
| it more likely that data is pre-cached and thus fall into fast
| path more often, but without a guarantee.)
|
| That probably should have been the first thing to try. Too mad
| the mmap2 crate does not expose this.
|
| Also looking at the mmap2 crate, it chooses some rather
| opinionated defaults depending on which function you actually
| call, and it makes accessing things like HUGEPAGE maps somewhat
| difficult.. and for whatever reason includes the MMAP_STACK flag
| when you call through this path.
|
| I feel like a lot of rust authors put faith in crates that, upon
| inspection, are generally poorly designed and do not expose the
| underlying interface properly. It's a bad crutch for the
| language.
| pengaru wrote:
| water is wet
| mwcampbell wrote:
| Nobody automatically knows everything, and we have limited
| energy for drawing inferences based on what we do know. So the
| material covered in this post isn't obvious to everyone in its
| target audience, especially since Rust has had some success in
| making systems programming more approachable to inexperienced
| programmers, which is a good thing.
| malkia wrote:
| WIth mmap you have to be prepared to handle unexpected page fault
| errors due to corrupted volume: Unlike standard read/write, where
| one can handle the issue, now it can happen anywhere the memory
| is mapped - your code, third party library, etc.
|
| It gets even unwieldy, and now you have to add additional
| tracking where access is to be expected. Blindly delegating mmap
| area to any code path that does not have such handling, and you
| would have to deal with these failures.
|
| Maybe that's not the case on Linux/OSX/BSD, but definitely is on
| Windows where you would have it. Also in C/C++ land you have to
| handle this using SEH - e.g. `__try/__except` - standard C++
| handling won't cut it (I guess in other systems these would be
| through some signals (?)).
|
| In any case, it might seem like an easy path to achieve glory,
| yet riddled with complications.
| krilovsky wrote:
| Yes, on POSIX systems you'd get a SIGBUS if the I/O fails or if
| there's no available physical memory to back the mapping.
| jcalvinowens wrote:
| On Linux, if you get a SIGBUS from poking a memory map that
| generally means you'd have certainly gotten -ENOMEM or -EIO
| during an equivalent sequence of syscalls (or been oom-killed,
| if you overcommit). Those are treated as fatal in the vast
| majority of programs, so dying to SIGBUS isn't meaningfully
| different for most usecases.
|
| By your logic, passing a file descriptor to a library is also
| "unwieldy", because the library might not handle -EIO.
|
| You can use MAP_POPULATE|MAP_LOCKED to ensure you get an error
| from mmap() instead of getting killed in the ENOMEM case, if
| you don't overcommit (if you do, you can still be oom-killed).
| You still get SIGBUS beyond EOF, but that's the behavior you
| want: it's equivalent to overrunning a buffer.
|
| The behavior when file size isn't a multiple of PAGE_SIZE is
| legitimately weird (writes to the final page beyond EOF are
| visible to the entire system in memory but never written back
| to the file), but it's intuitive if you understand how the page
| cache works at a high level, and you can avoid it by making the
| size page aligned.
|
| For more complex usecases where you really do want to handle
| these sorts of errors, userfaultfd() gives you all the tools
| you need: https://www.man7.org/linux/man-
| pages/man2/userfaultfd.2.html
|
| EDIT: Initially described MAP_POPULATE wrong.
| loeg wrote:
| > On Linux, if you get a SIGBUS from poking a memory map that
| generally means you'd have certainly gotten -ENOMEM or -EIO
| during an equivalent sequence of syscalls (or been oom-
| killed, if you overcommit). Those are treated as fatal in the
| vast majority of programs, so dying to SIGBUS isn't
| meaningfully different for most usecases.
|
| No. EIO is not instantly fatal in the same way that SIGBUS
| is. It allows for printing an error message, associating the
| error with some context of what failed, and either recovering
| in a degraded state or exiting cleanly. Doing any of this in
| a SIGBUS handler ranges from unwieldy to impossible.
| loeg wrote:
| It is also the case on Linux/BSD.
| dsp_person wrote:
| > One possible implementation might be to literally have the
| operating system allocate a chunk of physical memory and load the
| file into it, byte by byte, right when mmap is called... but this
| is slow, and defeats half the magic of memory mapped IO:
| manipulating files without having to pull them into memory
|
| This doesn't defeat the purpose necessarily. How about for
| example, implementing a text editor: I want the best performance
| by loading the existing file initially (say it is <1MB), and the
| convenience and robustness of any writes to this memory being
| efficiently written to disk.
___________________________________________________________________
(page generated 2024-08-24 23:01 UTC)