https://lemire.me/blog/2024/01/18/how-much-memory-bandwidth-do-large-amazon-instances-offer/

Skip to content

Daniel Lemire's blog

Daniel Lemire is a computer science professor at the Data Science
Laboratory of the Universite du Quebec (TELUQ) in Montreal. His
research is focused on software performance and data engineering. He
is a techno-optimist and a free-speech advocate.

Menu and widgets

  * My home page
  * My papers
  * My software

Join over 12,500 email subscribers:

[                    ][Go!]
You can follow this blog on telegram.
You can find me on twitter as @lemire or on Mastodon.
Search for: [                    ] [Search]
Support my work!

I do not accept any advertisement. However, you can you can sponsor
my open-source work on GitHub.

Recent Posts

  * How much memory bandwidth do large Amazon instances offer?
  * Estimating your memory bandwidth
  * Implementing the missing sign instruction in AVX-512
  * Science and Technology links (December 30th 2023)
  * Measuring the size of the cache line empirically

Recent Comments

  * Daniel Lemire on How much memory bandwidth do large Amazon
    instances offer?
  * Richard on How much memory bandwidth do large Amazon instances
    offer?
  * AliceBob on Estimating your memory bandwidth
  * Bruce Edmonds on Peer review is an honor-based system
  * Daniel Lemire on Peer review is an honor-based system

Pages

  * A short history of technology
  * About me
  * Book recommendations
  * Cognitive biases
  * Interviews and talks
  * My bets
  * My favorite articles
  * My favorite quotes
  * My rules
  * Newsletter
  * Predictions
  * Privacy Policy
  * Recommended video games
  * Terms of use
  * Write good papers

Archives

Archives [Select Month        ]

Boring stuff

  * Log in
  * Entries feed
  * Comments feed
  * WordPress.org

How much memory bandwidth do large Amazon instances offer?

In my previous post, I described how you can write a C++ program to
estimate your read memory bandwidth. It is not very difficult: you
allocate a large memory region and you read it as fast as you can. To
see how much bandwidth you may have if you use multithreaded
applications, you can use multiple threads, where each thread reads a
section of the large memory region.

The server I used for the blog post, a two-CPU Intel Ice Lake server
has a maximal bandwidth of about 130 GB/s. You can double this amount
of bandwidth with NUMA-aware code, but it will require further
engineering.

But you do not have access to my server. What about a big Amazon
server? So I spun out an r6i.metal instance from Amazon. These
servers can support 128 physical threads, they have 1 terabyte of RAM
(1024 GB) and 6.25 GB/s of network bandwidth.

Running my benchmark program on this Amazon server revealed that they
have about 115 GB/s of read memory bandwidth. That is not counting
NUMA and other sophisticated tricks. Plotting the bandwidth versus
the number of threads used reveals that, once again, you need about
20 threads to maximized memory bandwidth although you get most of it
with only 15 threads.

[bandwidthicelakeaws]

My source code is available.

Published by

[2ca999]

Daniel Lemire

A computer science professor at the University of Quebec (TELUQ).
View all posts by Daniel Lemire

Posted on January 18, 2024January 18, 2024Author Daniel Lemire
Categories  

2 thoughts on "How much memory bandwidth do large Amazon instances
offer?"

 1. [411e48] Richard says:
    January 19, 2024 at 6:05 pm

    Have you considered using the standard stream benchmark for this?
    I'd be interested to compare against their corpus.

    https://www.cs.virginia.edu/stream/

    Reply
     1. [2ca999] Daniel Lemire says:
        January 19, 2024 at 8:05 pm

        I don't think Stream has the same test. They don't measure
        read-only bandwidth.

        Reply

Leave a Reply Cancel reply

Your email address will not be published.

To create code blocks or other preformatted text, indent by four
spaces:

    This will be displayed in a monospaced font. The first four
    spaces will be stripped off, but all other whitespace
    will be preserved.
    
    Markdown is turned off in code blocks:
     [This is not a link](http://example.com)

To create not a block, but an inline code span, use backticks:

Here is some inline `code`.

For more help see http://daringfireball.net/projects/markdown/syntax

          [                                             ]
          [                                             ]
          [                                             ]
          [                                             ]
          [                                             ]
          [                                             ]
          [                                             ]
Comment * [                                             ]

Name * [                              ]

Email * [                              ]

Website [                              ]

[ ] Save my name, email, and website in this browser for the next
time I comment.

Receive Email Notifications? [no, do not subscribe      ] 
[instantly    ]
Or, you can subscribe without commenting.

[Post Comment] 

 [                                             ] 
 [                                             ] 
 [                                             ] 
 [                                             ] 
 [                                             ] 
 [                                             ] 
 [                                             ] 
D[                                             ] 

You may subscribe to this blog by email.

Post navigation

Previous Previous post: Estimating your memory bandwidth
Terms of use Proudly powered by WordPress