From nobody@FreeBSD.org  Tue Oct  7 18:19:38 2008
Return-Path: <nobody@FreeBSD.org>
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id 3B88A1065688
	for <freebsd-gnats-submit@FreeBSD.org>; Tue,  7 Oct 2008 18:19:38 +0000 (UTC)
	(envelope-from nobody@FreeBSD.org)
Received: from www.freebsd.org (www.freebsd.org [IPv6:2001:4f8:fff6::21])
	by mx1.freebsd.org (Postfix) with ESMTP id 294B48FC08
	for <freebsd-gnats-submit@FreeBSD.org>; Tue,  7 Oct 2008 18:19:38 +0000 (UTC)
	(envelope-from nobody@FreeBSD.org)
Received: from www.freebsd.org (localhost [127.0.0.1])
	by www.freebsd.org (8.14.3/8.14.3) with ESMTP id m97IJb6r018686
	for <freebsd-gnats-submit@FreeBSD.org>; Tue, 7 Oct 2008 18:19:37 GMT
	(envelope-from nobody@www.freebsd.org)
Received: (from nobody@localhost)
	by www.freebsd.org (8.14.3/8.14.3/Submit) id m97IJblE018682;
	Tue, 7 Oct 2008 18:19:37 GMT
	(envelope-from nobody)
Message-Id: <200810071819.m97IJblE018682@www.freebsd.org>
Date: Tue, 7 Oct 2008 18:19:37 GMT
From: Renaud Lienhart <renaud@vmware.com>
To: freebsd-gnats-submit@FreeBSD.org
Subject: The TCP bandwidth gets squeezed every time tcp_xmit_bandwidth_limit() kicks in
X-Send-Pr-Version: www-3.1
X-GNATS-Notify:

>Number:         127928
>Category:       kern
>Synopsis:       [tcp] [patch] TCP bandwidth gets squeezed every time tcp_xmit_bandwidth_limit() kicks in
>Confidential:   no
>Severity:       non-critical
>Priority:       medium
>Responsible:    andre
>State:          patched
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Tue Oct 07 18:20:01 UTC 2008
>Closed-Date:    
>Last-Modified:  Sat Aug 14 22:53:41 UTC 2010
>Originator:     Renaud Lienhart
>Release:        FreeBSD 6.2+
>Organization:
VMware, Inc.
>Environment:
ESX 3.x
>Description:
FreeBSD 6,7 & 8 have a bug in their tcp_xmit_bandwidth_limit() function.

The problem is that the bandwidth calculation is a 1/16th weigthed
average and that the saved bandwidth gets reset every time the RTT
threshold is crossed over.  This means that the bandwidth (and thus the
B-D product, and thus the sending window) is briefly squeezed whenever
the mechanism kicks in and it takes a bit of time to reach sane values again.

The threshold by default is 10ms.  With HZ=100, this means that a RTT of
0 tick disables the mechanism and a RTT of 1 tick activates it.  Because
of the poor tick granularity, the RTT is of 1 tick every 10ms and the
mechanism kicks in, brutally squeezing the sending window because
tp->snd_bandwidth is 0.  The problem also appears with HZ=1000, when
the RTT fluctuates in the bad spot of ~10ms.
>How-To-Repeat:
Run a kernel with HZ=100 (or HZ=1000 and a latency around ~10ms, which
is harder) and any TCP load with tcp_inflight_debug = 1.  Notice the
bandwidth reported by the log is inconsistent with the link capacity
and that the tp->snd_bwnd is incorrectly low.
>Fix:
To fix this, kickstart tp->snd_bandwidth without the weighted average
smoothing when it is 0.

Patch attached with submission follows:

Index: netinet/tcp_subr.c
===================================================================
--- netinet/tcp_subr.c	(revision 183668)
+++ netinet/tcp_subr.c	(working copy)
@@ -1783,7 +1783,9 @@
 	tp->t_bw_rtseq = ack_seq;
 	if (tp->t_bw_rtttime == 0 || (int)bw < 0)
 		return;
-	bw = ((int64_t)tp->snd_bandwidth * 15 + bw) >> 4;
+	if (tp->snd_bandwidth != 0) {
+		bw = ((int64_t)tp->snd_bandwidth * 15 + bw) >> 4;
+	}
 
 	tp->snd_bandwidth = bw;
 


>Release-Note:
>Audit-Trail:
Responsible-Changed-From-To: freebsd-bugs->freebsd-net 
Responsible-Changed-By: linimon 
Responsible-Changed-When: Tue Oct 7 20:55:46 UTC 2008 
Responsible-Changed-Why:  
Over to maintainer(s). 

http://www.freebsd.org/cgi/query-pr.cgi?pr=127928 
Responsible-Changed-From-To: freebsd-net->andre 
Responsible-Changed-By: andre 
Responsible-Changed-When: Tue Aug 10 22:25:14 UTC 2010 
Responsible-Changed-Why:  
Take over. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=127928 
State-Changed-From-To: open->patched 
State-Changed-By: andre 
State-Changed-When: Sat Aug 14 22:52:14 UTC 2010 
State-Changed-Why:  
Inflight limiter disabled and scheduled for removal. 
See SNV r211315. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=127928 
>Unformatted:
