[HN Gopher] How fast are Linux pipes anyway?
       ___________________________________________________________________
        
       How fast are Linux pipes anyway?
        
       Author : keepamovin
       Score  : 169 points
       Date   : 2025-06-22 07:27 UTC (15 hours ago)
        
 (HTM) web link (mazzo.li)
 (TXT) w3m dump (mazzo.li)
        
       | aeonik wrote:
       | I feel bad that this doesn't have any comments, the article was
       | really great.
       | 
       | I'd like to use splice more, but the end of the article talked
       | about the security implications and some ABI breaking.
       | 
       | I'm curious to know if long term plans are to keep splice around?
       | 
       | I'd also be curious how hard it would be to patch the default
       | pipe to always use splice for performance improvements.
        
         | amelius wrote:
         | For more comments, see:
         | https://news.ycombinator.com/item?id=44347412
        
       | gkfasdfasdf wrote:
       | Great article, discussed previously on HN:
       | 
       | https://news.ycombinator.com/item?id=31592934 (200 comments)
       | 
       | https://news.ycombinator.com/item?id=37782493 (105 comments)
        
       | gigatexal wrote:
       | This is such a dope article. I love that it comes from time to
       | time.
        
       | layer8 wrote:
       | (2022)
        
       | zh3 wrote:
       | Seared into my soul is the experience porting a linux pipe-based
       | application to Windows, thinking it's all posix and given it's
       | all in memory the performance will be more or less the same. The
       | performance was hideous, even after we found that having pipes
       | waiting for a connection more or less ground windows to a halt.
       | 
       | Some years later this got revisited due to needing to use the
       | same thing under C# on Win10 and while it was better it was still
       | a major embarrassment how big the performance gap was.
        
         | andrewmcwatters wrote:
         | Did you find that you needed interprocess communication to
         | replace the gap?
        
         | asveikau wrote:
         | Some years back Windows added AF_UNIX sockets, I wonder how
         | those would perform relative to Win32 pipes. My guess is
         | better.
        
         | SoftTalker wrote:
         | Well POSIX only defines behavior, not performance. Every
         | platform and OS will have its own performance idiosyncracies.
        
           | klysm wrote:
           | How on earth would POSIX define performance of something like
           | pipes?
        
             | SoftTalker wrote:
             | I was addressing "it's all posix and given it's all in
             | memory the performance will be more or less the same."
             | 
             | Not claiming that POSIX should or could attempt to address
             | performance.
        
       | johnisgood wrote:
       | FWIW there is readv() / writev(), splice(), sendfile(),
       | funopen(), and io_buffer() as well.
       | 
       | splice() is great when transferring data between pipes and UNIX
       | sockets with zero-copy, but it is Linux-only.
       | 
       | splice() is the fastest and most efficient way to transfer data
       | through pipes (on Linux), especially for large volumes. It
       | bypasses memory allocations in userspace (as opposed to
       | read(v)/write(v)), there is no extra buffer management logic,
       | there is no memcpy() or iovec traversal.
       | 
       | Sadly on BSDs, for pipes, readv() / writev() is the most
       | performant way to achieve the same if I am not mistaken. Please
       | correct me if I am wrong.
       | 
       | At any rate, this is a great article.
        
         | messe wrote:
         | > sendfile() is file-to-socket (zero-copy as well), and has
         | very high performance as well, for both Linux and BSDs. It only
         | supports file-to-socket, however, and well, to stay relevant,
         | sendmsg() can't be used with pipes in the general case, it is
         | for UNIX domain sockets, INET sockets, and other socket types.
         | 
         | On Linux, sendfile supports more than just file to socket, as
         | it's implemented using splice. I've used it for file-to-block-
         | device in the past.
        
           | johnisgood wrote:
           | On BSDs probably not, as they don't have splice, but that is
           | good to know. I wonder if on BSDs it really is readv() and
           | writev() that are the fastest way to achieve the same thing
           | as has been done in the article. Maybe I am missing
           | something. I would like to be corrected.
        
             | messe wrote:
             | AFAIK, neither OpenBSD nor NetBSD has sendfile. On FreeBSD,
             | I think you're correct regarding it being file-to-socket
             | only.
        
               | zambal wrote:
               | Indeed, if I'm not mistaken Netflix at least used to use
               | (and commit to kernel) FreeBSD on content servers because
               | of its superior sendfile performance
        
         | tedunangst wrote:
         | Shared memory, like shm_open and fd passing, would be even
         | faster and fully portable.
        
       | lukeh wrote:
       | Does modern Linux have anything close to Doors? I've an embedded
       | application where two processes exchange small amounts of data
       | which are latency sensitive, and I'm wondering if there's
       | anything better than AF_UNIX.
        
       ___________________________________________________________________
       (page generated 2025-06-22 23:00 UTC)