From nobody@FreeBSD.org  Thu Dec 10 11:45:06 2009
Return-Path: <nobody@FreeBSD.org>
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id 4C8C8106566B
	for <freebsd-gnats-submit@FreeBSD.org>; Thu, 10 Dec 2009 11:45:06 +0000 (UTC)
	(envelope-from nobody@FreeBSD.org)
Received: from www.freebsd.org (www.freebsd.org [IPv6:2001:4f8:fff6::21])
	by mx1.freebsd.org (Postfix) with ESMTP id 3B90D8FC0C
	for <freebsd-gnats-submit@FreeBSD.org>; Thu, 10 Dec 2009 11:45:06 +0000 (UTC)
Received: from www.freebsd.org (localhost [127.0.0.1])
	by www.freebsd.org (8.14.3/8.14.3) with ESMTP id nBABj5r1065029
	for <freebsd-gnats-submit@FreeBSD.org>; Thu, 10 Dec 2009 11:45:05 GMT
	(envelope-from nobody@www.freebsd.org)
Received: (from nobody@localhost)
	by www.freebsd.org (8.14.3/8.14.3/Submit) id nBABj5uN065028;
	Thu, 10 Dec 2009 11:45:05 GMT
	(envelope-from nobody)
Message-Id: <200912101145.nBABj5uN065028@www.freebsd.org>
Date: Thu, 10 Dec 2009 11:45:05 GMT
From: "Dmitry S. Luhtionov" <mitya@cabletv.dp.ua>
To: freebsd-gnats-submit@FreeBSD.org
Subject: wrong netstat -w 1 output
X-Send-Pr-Version: www-3.1
X-GNATS-Notify:

>Number:         141340
>Category:       bin
>Synopsis:       netstat(1): wrong netstat -w 1 output
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    freebsd-bugs
>State:          suspended
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Thu Dec 10 11:50:03 UTC 2009
>Closed-Date:    
>Last-Modified:  Wed Jan 20 23:14:51 UTC 2010
>Originator:     Dmitry S. Luhtionov
>Release:        8.0-RELEASE
>Organization:
>Environment:
vpn8# uname -a
FreeBSD vpn8.cabletv.dp.ua 8.0-STABLE FreeBSD 8.0-STABLE #0: Wed Dec  2 10:18:16 EET 2009     mitya@vpn8.cabletv.dp.ua:/usr/src/sys/amd64/compile/vpn8  amd64
vpn8#
>Description:
On high load netstat may incorrect output
This server running mpd-5.3 as pptp server
vpn8# ifconfig -lu | wc -w
    1734
vpn8# netstat -w 1 -d -h
           input        (Total)           output
  packets  errs      bytes    packets  errs      bytes colls drops
     173K     0        89M       183K     0       129M     0     0
     170K     0        89M       181K     0       129M     0     0
     170K     0        89M       180K     0       128M     0     0
     170K     0        89M       180K     0       127M     0     0
     175K     0        91M       184K     0       130M     0     0
     173K     0        90M       183K     0       129M     0     0
     173K     0        90M       182K     0       129M     0     0
     171K     0        89M       180K     0       127M     0     0
     171K     0        89M       181K     0       129M     0     0
     172K     0        89M       181K     0       128M     0     0
     143K     0        88M       156K     0       107M     0     0
     174K     0        91M       184K     0       131M     0     0
     173K     0        90M       183K     0       130M     0     0
     175K     0        91M       186K     0       131M     0     0
     177K     0        92M       188K     0       133M     0     0
              0        71M                0                0     0
     171K     0        89M       179K     0       127M     0     0
     171K     0        89M       181K     0       128M     0     0
     175K     0        91M       186K     0       131M     0     0
      70K     0        83M        52K     0       -20M     0     0
           input        (Total)           output
  packets  errs      bytes    packets  errs      bytes colls drops
     174K     0        92M       184K     0       132M     0     0
     175K     0        93M       184K     0       132M     0     0
     174K     0        92M       184K     0       132M     0     0
     135K     0        89M       145K     0       118M     0     0
     172K     0        90M       181K     0       129M     0     0
     171K     0        89M       182K     0       129M     0     0
     172K     0        90M       183K     0       130M     0     0
     171K     0        90M       182K     0       130M     0     0
     173K     0        91M       183K     0       130M     0     0
^C
vpn8# netstat -w 1
            input        (Total)           output
   packets  errs      bytes    packets  errs      bytes colls
    145159     0   82485349     153321     0  117743533     0
    136895     0   79084654     145018     0  111877348     0
    146279     0   81710082     153952     0  116857455     0
    143634     0   80602950     152005     0  116127801     0
    144205     0   81175701     151892     0  115836499     0
    146342     0   83031415     153968     0  118093209     0
    143365     0   80564430     151508     0  115702643     0
    145253     0   82320881     153062     0  116675147     0
    146909     0   81998714     155485     0  117155601     0
     91507     0   76590439      79098     0   35481375     0
    145148     0   81429871     153728     0  116617071     0
    142632     0   80932248     150865     0  115942720     0
     21918     0   74115278 18446744073709486002     0 18446744073585465790     0
    142905     0   81095335     150861     0  115144168     0
    142216     0   80410366     150408     0  114744557     0
    142302     0   81065884     150364     0  115161521     0
    141493     0   79316895     148997     0  112673424     0
^C

>How-To-Repeat:
run netstat -w 1 again
>Fix:


>Release-Note:
>Audit-Trail:

From: Efstratios Karatzas <gpf.kira@gmail.com>
To: bug-followup@freebsd.org
Cc: mitya@cabletv.dp.ua
Subject: Re: bin/141340: netstat(1): wrong netstat -w 1 output
Date: Thu, 21 Jan 2010 00:51:33 +0200

 I believe I got this one figured out.
 
 I studied netstat's code and the actual loop is taking place inside
 function sidewaysintpr() contained in netstat/if.c
 so goto that function if you plan on reading the rest of this report.
 
 The reason you get bogus output is that the counters of various statistics
 that are kept for each interface are never again initialized to 0.
 This is the struct being used by the function to hold the various
 statistics that are printed.
 
 struct	iftot {
 	SLIST_ENTRY(iftot) chain;
 	char	ift_name[IFNAMSIZ];	/* interface name */
 	u_long	ift_ip;			/* input packets */
 	u_long	ift_ie;			/* input errors */
 	u_long	ift_id;			/* input drops */
 	u_long	ift_op;			/* output packets */
 	u_long	ift_oe;			/* output errors */
 	u_long	ift_co;			/* collisions */
 	u_int	ift_dr;			/* drops */
 	u_long	ift_ib;			/* input bytes */
 	u_long	ift_ob;			/* output bytes */
 };
 
 In each iteration we recompute (sum up) the counters of variable "sum"
 and what we actually print is the difference between current statistics
 that we gathered during the interval (defined by the -w arg) and the
 statistics of the previous loop. Take a look a this:
 
 show_stat("lu", 10, sum->ift_ib - total->ift_ib, 1);
 
 Since the counters for each statistic go only one way (up), eventually
 we are going to overflow. The counters inside "sum" are going to overflow
 slightly faster than "total's" counters so once in a while
 sum->ift_ib - total->ift_ib
 will yield a "negative number" because sum->ift_ib overflows and
 wraps near 0, becoming a small number in comparison to total->ift_ib
 In the next loop total will overflow as well and wrap around 0 itself,
 so no more crazy ultra long output.
 
 So the result of this subtraction, which is passed as a u_long, underflows
 and generates an enormously large value. Keep in mind than in 64bit
 archs a u_long is actually a really long number!
 So the max value for a u_long is around 2^64 = 1.84467441 * 10^19,
 close to what we see in the pr above. The pr originator has the amd64 version.
 
 Possible Fix:
 a) Implement an extra argument for netstat that re-initializes the
 statistics kept for each interface, of course it must be run by root.
 b) reboot the system
 
 In conclusion, I believe the state of the pr should change to "suspended" until
 someone submits a patch. I 'm sure I won't for at least 3 more weeks
 'till university exam period is over.
 
 Cheers,
 
 -- 
 
 Efstratios "GPF" Karatzas
State-Changed-From-To: open->suspended 
State-Changed-By: linimon 
State-Changed-When: Wed Jan 20 23:14:38 UTC 2010 
State-Changed-Why:  
Mark suspended awaiting a patch. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=141340 
>Unformatted:
