From nobody@FreeBSD.org  Thu Sep  2 06:07:49 2004
Return-Path: <nobody@FreeBSD.org>
Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125])
	by hub.freebsd.org (Postfix) with ESMTP id 63DE116A4CE
	for <freebsd-gnats-submit@FreeBSD.org>; Thu,  2 Sep 2004 06:07:49 +0000 (GMT)
Received: from www.freebsd.org (www.freebsd.org [216.136.204.117])
	by mx1.FreeBSD.org (Postfix) with ESMTP id 5052343D39
	for <freebsd-gnats-submit@FreeBSD.org>; Thu,  2 Sep 2004 06:07:49 +0000 (GMT)
	(envelope-from nobody@FreeBSD.org)
Received: from www.freebsd.org (localhost [127.0.0.1])
	by www.freebsd.org (8.12.11/8.12.11) with ESMTP id i8267mFW064902
	for <freebsd-gnats-submit@FreeBSD.org>; Thu, 2 Sep 2004 06:07:48 GMT
	(envelope-from nobody@www.freebsd.org)
Received: (from nobody@localhost)
	by www.freebsd.org (8.12.11/8.12.11/Submit) id i8267mld064901;
	Thu, 2 Sep 2004 06:07:48 GMT
	(envelope-from nobody)
Message-Id: <200409020607.i8267mld064901@www.freebsd.org>
Date: Thu, 2 Sep 2004 06:07:48 GMT
From: Dmitry Dvoinikov <dmitry@targeted.org>
To: freebsd-gnats-submit@FreeBSD.org
Subject: Frequent bind()/connect()'s assign same local ports to different sockets
X-Send-Pr-Version: www-2.3

>Number:         71274
>Category:       kern
>Synopsis:       Frequent bind()/connect()'s assign same local ports to different sockets
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    silby
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Thu Sep 02 06:10:20 GMT 2004
>Closed-Date:    Fri Apr 08 07:47:27 GMT 2005
>Last-Modified:  Fri Apr 08 07:47:27 GMT 2005
>Originator:     Dmitry Dvoinikov
>Release:        4.10-RELEASE
>Organization:
-
>Environment:
FreeBSD  4.10-RELEASE FreeBSD 4.10-RELEASE #0: Tue May 25 22:47:12 GMT 2004     root@perseus.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC  i386
>Description:
1. A program keeps looping forever creating TCP sockets and connecting them to some TCP server in a connect/send/recv/close fashion (sample uses HTTP server at localhost).
2. Each outgoing socket is assigned a local port number, either explicitly (with bind to zero port before connect) or implicitly (within connect).
3. After this program runs for a while, it happens that another port number assigned has just been used a few connects ago (and that used socket is probably not free, but is still alive in TIME_WAIT or some other near-terminal state). Therefore this second connect gets a port number which is not free, and therefore connect fails returning "Connection refused".
4. This is not a resource exhaustion problem, as netstat output shows only around a thousand sockets.
5. This is not a congestion problem, because the looping program is single-threaded and can't produce more than one connection at a time.
      
>How-To-Repeat:
Compile an run this program, having http server at localhost:80 (I used thttpd for it's lightweightness, but even http doesn't really matter of course, it's just any TCP server). Examine program's output for the following pattern:

bound to 2664
bound to 1267 <<<<< REPEATS
bound to 4646
bound to 3060
bound to 1374
bound to 4986
bound to 1267 <<<<< REPEATS
connect failed: Connection refused

------------------------------------------------
base64 encoded sample:
------------------------------------------------
begin-base64 644 test.c
I2luY2x1ZGUgPHN5cy90eXBlcy5oPgojaW5jbHVkZSA8c3lzL3NvY2tldC5oPgojaW5jbHVkZSA8
c3RkaW8uaD4KI2luY2x1ZGUgPHVuaXN0ZC5oPgojaW5jbHVkZSA8c3RkbGliLmg+CiNpbmNsdWRl
IDxuZXRpbmV0L2luLmg+CiNpbmNsdWRlIDxzdHJpbmcuaD4KI2luY2x1ZGUgPGVycm5vLmg+CiNp
bmNsdWRlIDxzeXMvcGFyYW0uaD4KCmNoYXIqIFJFUVVFU1QgPSAiR0VUIC8gSFRUUC8xLjBcclxu
XHJcbiI7CgppbnQgbWFpbigpCnsKCiAgICAgICAgaW50IHMsIHVudXNlZDsKICAgICAgICBzdHJ1
Y3Qgc29ja2FkZHJfaW4gYTsKICAgICAgICBjaGFyIGJ1ZlsxMDI0XTsKCiAgICAgICAgbWVtc2V0
KCZhLCAwLCBzaXplb2Yoc3RydWN0IHNvY2thZGRyX2luKSk7CiAgICAgICAgYS5zaW5fbGVuID0g
c2l6ZW9mKHN0cnVjdCBzb2NrYWRkcl9pbik7CiAgICAgICAgYS5zaW5fZmFtaWx5ID0gQUZfSU5F
VDsKICAgICAgICBhLnNpbl9hZGRyLnNfYWRkciA9IGh0b25sKDB4N2YwMDAwMDEpOwoKICAgICAg
ICB3aGlsZSAoMSkKICAgICAgICB7CgogICAgICAgICAgICAgICAgcyA9IHNvY2tldChQRl9JTkVU
LCBTT0NLX1NUUkVBTSwgMCk7CgkJaWYgKHMgPT0gLTEpCgkJewoJCQlmcHJpbnRmKHN0ZG91dCwg
InNvY2tldCBmYWlsZWQ6ICVzXG4iLCBzdHJlcnJvcihlcnJubykpOwoJCQljb250aW51ZTsKCQl9
CgogICAgICAgICAgICAgICAgYS5zaW5fcG9ydCA9IDA7CiAgICAgICAgICAgICAgICBpZiAoYmlu
ZChzLCAoc3RydWN0IHNvY2thZGRyKikmYSwgc2l6ZW9mKHN0cnVjdCBzb2NrYWRkcl9pbikpID09
IC0xKQoJCXsKCQkJY2xvc2Uocyk7CiAgICAgICAgICAgICAgICAgICAgICAgIGZwcmludGYoc3Rk
b3V0LCAiYmluZCBmYWlsZWQ6ICVzXG4iLCBzdHJlcnJvcihlcnJubykpOyAKICAgICAgICAgICAg
ICAgICAgICAgICAgY29udGludWU7CgkJfQoKICAgICAgICAgICAgICAgIHVudXNlZCA9IHNpemVv
ZihzdHJ1Y3Qgc29ja2FkZHJfaW4pOwogICAgICAgICAgICAgICAgaWYgKGdldHNvY2tuYW1lKHMs
IChzdHJ1Y3Qgc29ja2FkZHIqKSZhLCAmdW51c2VkKSA9PSAtMSkKCQl7CiAgICAgICAgICAgICAg
ICAgICAgICAgIGNsb3NlKHMpOwogICAgICAgICAgICAgICAgICAgICAgICBmcHJpbnRmKHN0ZG91
dCwgImdldHNvY2tuYW1lIGZhaWxlZDogJXNcbiIsIHN0cmVycm9yKGVycm5vKSk7ICAgICAgICAg
ICAgICAgICAgIAogICAgICAgICAgICAgICAgICAgICAgICBjb250aW51ZTsgCgkJfQoKICAgICAg
ICAgICAgICAgIGZwcmludGYoc3Rkb3V0LCAiYm91bmQgdG8gJWRcbiIsICh1bnNpZ25lZCBsb25n
KW50b2hzKGEuc2luX3BvcnQpKTsKCiAgICAgICAgICAgICAgICBhLnNpbl9wb3J0ID0gaHRvbnMo
ODApOwogICAgICAgICAgICAgICAgaWYgKGNvbm5lY3QocywgKHN0cnVjdCBzb2NrYWRkciopJmEs
IHNpemVvZihzdHJ1Y3Qgc29ja2FkZHJfaW4pKSA9PSAtMSkKICAgICAgICAgICAgICAgIHsKICAg
ICAgICAgICAgICAgICAgICAgICAgY2xvc2Uocyk7CiAgICAgICAgICAgICAgICAgICAgICAgIGZw
cmludGYoc3Rkb3V0LCAiY29ubmVjdCBmYWlsZWQ6ICVzXG4iLCBzdHJlcnJvcihlcnJubykpOwog
ICAgICAgICAgICAgICAgICAgICAgICBjb250aW51ZTsKICAgICAgICAgICAgICAgIH0KCiAgICAg
ICAgICAgICAgICBzZW5kKHMsIFJFUVVFU1QsIHN0cmxlbihSRVFVRVNUKSwgMCk7CiAgICAgICAg
ICAgICAgICB3aGlsZSAocmVjdihzLCBidWYsIHNpemVvZihidWYpLCAwKSA+IDApIHsgfQogICAg
ICAgICAgICAgICAgY2xvc2Uocyk7CgogICAgICAgIH0KCn0KCg==
====
------------------------------------------------
plain (broken ?) sample:
------------------------------------------------
#include <sys/types.h>
#include <sys/socket.h>
#include <stdio.h>
#include <unistd.h>
#include <stdlib.h>
#include <netinet/in.h>
#include <string.h>
#include <errno.h>
#include <sys/param.h>

char* REQUEST = "GET / HTTP/1.0\r\n\r\n";

int main()
{

        int s, unused;
        struct sockaddr_in a;
        char buf[1024];

        memset(&a, 0, sizeof(struct sockaddr_in));
        a.sin_len = sizeof(struct sockaddr_in);
        a.sin_family = AF_INET;
        a.sin_addr.s_addr = htonl(0x7f000001);

        while (1)
        {

                s = socket(PF_INET, SOCK_STREAM, 0);
                if (s == -1)
                {
                        fprintf(stdout, "socket failed: %s\n", strerror(errno));
                        continue;
                }

                a.sin_port = 0;
                if (bind(s, (struct sockaddr*)&a, sizeof(struct sockaddr_in)) == -1)
                {
                        close(s);
                        fprintf(stdout, "bind failed: %s\n", strerror(errno)); 
                        continue;
                }

                unused = sizeof(struct sockaddr_in);
                if (getsockname(s, (struct sockaddr*)&a, &unused) == -1)
                {
                        close(s);
                        fprintf(stdout, "getsockname failed: %s\n", strerror(errno));                   
                        continue; 
                }

                fprintf(stdout, "bound to %d\n", (unsigned long)ntohs(a.sin_port));

                a.sin_port = htons(80);
                if (connect(s, (struct sockaddr*)&a, sizeof(struct sockaddr_in)) == -1)
                {
                        close(s);
                        fprintf(stdout, "connect failed: %s\n", strerror(errno));
                        continue;
                }

                send(s, REQUEST, strlen(REQUEST), 0);
                while (recv(s, buf, sizeof(buf), 0) > 0) { }
                close(s);

        }

}
------------------------------------------------

      
>Fix:
      
>Release-Note:
>Audit-Trail:

From: Maxim Konovalov <maxim@macomnet.ru>
To: Dmitry Dvoinikov <dmitry@targeted.org>
Cc: bug-followup@freebsd.org
Subject: Re: kern/71274: Frequent bind()/connect()'s assign same local ports
 to different sockets
Date: Thu, 2 Sep 2004 10:43:56 +0400 (MSD)

 I wonder what does sysctl net.inet.ip.portrange.randomized say?
 
 -- 
 Maxim Konovalov

From: Maxim Konovalov <maxim@macomnet.ru>
To: Dmitry Dvoinikov <dmitry@targeted.org>
Cc: bug-followup@freebsd.org
Subject: Re[2]: kern/71274: Frequent bind()/connect()'s assign same local
 ports to different sockets
Date: Thu, 2 Sep 2004 11:11:36 +0400 (MSD)

 On Thu, 2 Sep 2004, 13:05+0600, Dmitry Dvoinikov wrote:
 
 > # sysctl net.inet.ip.portrange.randomized
 > net.inet.ip.portrange.randomized: 1
 
 Does switching it off help?
 
 From ip(4):
 
  Ports are allocated at random within the specified port range in order to
  increase the difficulty of random spoofing attacks.  In scenarios such as
  benchmarking, this behavior may be undesirable.  In these cases,
  net.inet.ip.portrange.randomized can be used to toggle randomization off.
 
 -- 
 Maxim Konovalov

From: Maxim Konovalov <maxim@macomnet.ru>
To: Dmitry Dvoinikov <dmitry@targeted.org>
Cc: bug-followup@freebsd.org
Subject: Re: kern/71274: Frequent bind()/connect()'s assign same local ports
 to different sockets
Date: Thu, 2 Sep 2004 11:46:22 +0400 (MSD)

 On Thu, 2 Sep 2004, 13:29+0600, Dmitry Dvoinikov wrote:
 
 > Yes, it helps, obviously, as now it has to run through
 > portrange.last ports before it gets a duplicate, and this
 > is unlikely to happen within an msl.
 >
 > Anyhow, from my point of view this is still a bug,
 > because the client code fails for no apparent reason,
 > only because the random number is so unlucky to be
 > picked up to match with the live socket. This effectively
 > means that no matter what the load is, there is a chance
 > that connection just fails randomly for no reason.
 
 Agreed.  That is why I am strongly against all those obscure defaults
 especially in -STABLE branches.
 
 -- 
 Maxim Konovalov
Responsible-Changed-From-To: freebsd-bugs->silby 
Responsible-Changed-By: maxim 
Responsible-Changed-When: Thu Sep 2 08:02:52 GMT 2004 
Responsible-Changed-Why:  
Over to the author of the random ephemeral port allocation. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=71274 
State-Changed-From-To: open->patched 
State-Changed-By: maxim 
State-Changed-When: Tue Dec 14 10:01:06 GMT 2004 
State-Changed-Why:  
Disable port randomization in RELENG_4 for upcoming 4.11-RELEASE, 
rev. 1.59.2.30 src/sys/netinet/in_pcb.c. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=71274 

From: Matteo Riondato <rionda@gufi.org>
To: bug-followup@freebsd.org
Cc: maxim@freebsd.org
Subject: Re: kern/71274: Frequent bind()/connect()'s assign same local ports
Date: Thu, 7 Apr 2005 22:24:56 +0200

 --+jhVVhN62yS6hEJ8
 Content-Type: text/plain; charset=us-ascii
 Content-Disposition: inline
 Content-Transfer-Encoding: quoted-printable
 
 I think this PR can be closed, since a patch was committed for the
 affected branch.
 Best Regards
 --=20
 Rionda aka Matteo Riondato
 Disinformato per default
 G.U.F.I. Staff Member (http://www.gufi.org)
 FreeSBIE Developer (http://www.freesbie.org)
 
 --+jhVVhN62yS6hEJ8
 Content-Type: application/pgp-signature
 Content-Disposition: inline
 
 -----BEGIN PGP SIGNATURE-----
 Version: GnuPG v1.4.0 (FreeBSD)
 
 iD8DBQFCVZcY2Mp4pR7Fa+wRAgB5AKCmzrnS5lrT0vTJtbMtWqOyWSyg/wCfbwxb
 +B3Je+t3lyq2QegDtLBuR7o=
 =dCPZ
 -----END PGP SIGNATURE-----
 
 --+jhVVhN62yS6hEJ8--
State-Changed-From-To: patched->closed 
State-Changed-By: maxim 
State-Changed-When: Fri Apr 8 07:45:09 GMT 2005 
State-Changed-Why:  
There is new random port allocation algorithm in HEAD, RELENG_5 and 
RELENG_4.  Please look at ip.4 man page and net.inet.ip.portrange 
sysctl subtree. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=71274 
>Unformatted:
