[HN Gopher] Power-of-2-Choices in Practice
       ___________________________________________________________________
        
       Power-of-2-Choices in Practice
        
       Author : i0exception
       Score  : 34 points
       Date   : 2021-02-18 18:14 UTC (4 hours ago)
        
 (HTM) web link (engineering.mixpanel.com)
 (TXT) w3m dump (engineering.mixpanel.com)
        
       | vijayjayaram wrote:
       | I'm the author of this post, happy to answer any questions!
        
         | illirik wrote:
         | Great article, always love to see the law of large numbers
         | fail! You mention that the solution "requires all clients to
         | cooperate" - wouldn't it be in the client's best interest to
         | cooperate (maximizes the probability the request is served, as
         | well as minimizes latency)? I suppose a malicious client could
         | deliberately route traffic to a fully-utilized node, but that
         | seems outside the scope of this problem space
        
           | vijayjayaram wrote:
           | Good question! Yes, it's in the interest of clients to
           | cooperate.
           | 
           | Fundamentally, P2C thickens the interface between client and
           | server by leaking the implementation and responsibility of
           | balancing server load onto the client. Again, this works
           | really well if there is a small, static set of clients as we
           | have internally, but not that well if the clients are out in
           | the wild (eg: on mobile devices). It's totally in the
           | client's interest to upgrade, but that's always work and you
           | just can't rely on that. :)
           | 
           | So imo this is a serious tradeoff that must be made before
           | employing P2C. A good hybrid approach (briefly mentioned in
           | the post) is to introduce a proxy that uses P2C. This is
           | fully in your control and is cheaper than having the proxy
           | maintain a full histogram of load on all downstreams.
        
         | dnw wrote:
         | Wonderful use case and love how simple of a heuristic it is to
         | implement.
         | 
         | Diminishing returns beyond choice of 2 is also quite
         | interesting: http://www.cs.tau.ac.il/~azar/box.pdf
        
           | chrisweekly wrote:
           | hieratic?
           | 
           | "of or concerning priests"
        
             | dnw wrote:
             | Thanks, I meant heuristic. Fixed that typo above.
        
       | jeffbee wrote:
       | "It introduces two extra round trips" is an implementation
       | flaw/detail, not an aspect of this strategy. There is no reason
       | that servers cannot periodically advertise their load measures to
       | clients, or staple current load reports to query responses, which
       | the client can cache and use for a period of time.
        
         | bobthebuilders wrote:
         | That is still an extra repsonse time though, you get nothing
         | for free.
        
       ___________________________________________________________________
       (page generated 2021-02-18 23:02 UTC)