From nobody@FreeBSD.org  Sat May 31 11:03:30 2008
Return-Path: <nobody@FreeBSD.org>
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id 5C0891065678
	for <freebsd-gnats-submit@FreeBSD.org>; Sat, 31 May 2008 11:03:30 +0000 (UTC)
	(envelope-from nobody@FreeBSD.org)
Received: from www.freebsd.org (www.freebsd.org [IPv6:2001:4f8:fff6::21])
	by mx1.freebsd.org (Postfix) with ESMTP id 487558FC12
	for <freebsd-gnats-submit@FreeBSD.org>; Sat, 31 May 2008 11:03:30 +0000 (UTC)
	(envelope-from nobody@FreeBSD.org)
Received: from www.freebsd.org (localhost [127.0.0.1])
	by www.freebsd.org (8.14.2/8.14.2) with ESMTP id m4VB1jRV085072
	for <freebsd-gnats-submit@FreeBSD.org>; Sat, 31 May 2008 11:01:46 GMT
	(envelope-from nobody@www.freebsd.org)
Received: (from nobody@localhost)
	by www.freebsd.org (8.14.2/8.14.1/Submit) id m4VB1jU7085071;
	Sat, 31 May 2008 11:01:45 GMT
	(envelope-from nobody)
Message-Id: <200805311101.m4VB1jU7085071@www.freebsd.org>
Date: Sat, 31 May 2008 11:01:45 GMT
From: Alexander Pavlov <lup.mail@gmail.com>
To: freebsd-gnats-submit@FreeBSD.org
Subject: connect() function loops indefinitely
X-Send-Pr-Version: www-3.1
X-GNATS-Notify:

>Number:         124160
>Category:       kern
>Synopsis:       [libc] connect(2) function loops indefinitely
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    freebsd-net
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Sat May 31 11:10:01 UTC 2008
>Closed-Date:    
>Last-Modified:  Sun Jul 06 09:06:46 UTC 2008
>Originator:     Alexander Pavlov
>Release:        6.3-RELEASE
>Organization:
JSC Apriko
>Environment:
FreeBSD torment.skynet 6.3-RELEASE-p1 FreeBSD 6.3-RELEASE-p1 #3: Sun Mar  9 11:59:41 UTC 2008     lup@torment.skynet:/usr/obj/usr/src/sys/CLONED-CORE  i386
>Description:
I have a number of Intel platforms running FreeBSD 6.3-Release (using mpd-5.1, ipfw,ipnat).
Under some circumstances, at arbitrary time, my FreeBSD boxes stop responding to connection requests, but still remain up and running, so I can log in locally. In such state, major network functions are not working: I can't launch ngctl - it simply hangs, I can't add static arp entry - arp also hangs, and so on. I do not even see replies to ping 127.0.0.1 requests.
Trying to find clues to this problem, I wrote small program (in attach to this PR), which in normal state gives such output:
=================
inet socket testing..
1 4510 => server: creating listening socket
4510 => server: bind() ok
4510 => server: listen() ok
2 4511 => client: connecting..
4511 => client: socket() ok
4511 => client: trying to connect()..
4511 => client: connect() ok
3 4510 => server: client connected
4 4510 => server: send: "Anybody here?", res=13, errno=0
5 4511 => client: recieved: "Anybody here?", res=13, errno=0
6 4511 => client: send: "I am.", res=256, errno=0
7 4510 => server: recieved: "I am.", res=256, errno=0
=================
While in "hanged" state it gives:
=================
inet socket testing..
1 97262 => server: creating listening socket
97262 => server: bind() ok
97262 => server: listen() ok
2 97263 => client: connecting..
97263 => client: socket() ok
97263 => client: trying to connect()..
=================
and hangs until killed.
>How-To-Repeat:
It's a pity, but I don't know. In my environment it happens almost daily.
>Fix:


Patch attached with submission follows:

#include <stand.h>
#include <sys/socket.h>
#include <sys/errno.h>
#include <netinet/in.h>

#define send_str "Anybody here?"
#define	answer_str "I am."
#define server_port 631
#define listenq 1024

int main()
{ 
    int 	sockets[2];
    char 	buf[256];
    int 	pid;
    int		res;
    
    int			listenfd,connfd;
    struct sockaddr_in	clientaddr,serveraddr;

    printf("inet socket testing..\n");
    
    if (fork() != 0) {

	printf("1 %d => server: creating listening socket\n",getpid());
	listenfd = socket(AF_INET,SOCK_STREAM,0);
	if (listenfd<0) printf("%d => server: socket() error: %d\n",getpid(),errno);
	
	bzero(&serveraddr, sizeof(serveraddr));
	serveraddr.sin_family = AF_INET;
	serveraddr.sin_addr.s_addr = htonl(INADDR_ANY);
	serveraddr.sin_port = htons(server_port);

	res = bind(listenfd,(struct sockaddr *) &serveraddr,sizeof(serveraddr));
	if (res<0) {printf("%d => server: bind() error: %d\n",getpid(),errno); } else {printf("%d => server: bind() ok\n",getpid());}
	res = listen(listenfd, listenq);
	if (res<0) {printf("%d => server: listen() error: %d\n",getpid(),errno);} else {printf("%d => server: listen() ok\n",getpid());}

	while (connfd <= 0) {
	    connfd = accept(listenfd, (struct sockaddr *) NULL, NULL);
	    if (connfd > 0) printf("3 %d => server: client connected\n",getpid());
	    res = write(connfd, send_str, strlen(send_str));
	    printf("4 %d => server: send: \"%s\", res=%d, errno=%d\n",getpid(), send_str,res,errno);
	    wait(NULL);
	    res = read(connfd, buf, sizeof(buf));
	    printf("7 %d => server: recieved: \"%s\", res=%d, errno=%d\n",getpid(), buf,res,errno);
	    close(connfd);
	}

	close(listenfd);
	exit(0);

    } else {

	printf("2 %d => client: connecting..\n",getpid());
	connfd = socket(AF_INET,SOCK_STREAM,0);
	if (connfd<0) {printf("%d => client: socket() error: %d\n",getpid(),errno);} else {printf("%d => client: socket() ok\n",getpid());}

	bzero(&serveraddr, sizeof(serveraddr));
	serveraddr.sin_family = AF_INET;
	inet_pton(AF_INET, "127.0.0.1", &serveraddr.sin_addr);
	serveraddr.sin_port = htons(server_port);

	printf("%d => client: trying to connect()..\n",getpid());
	res = connect(connfd,(struct sockaddr *) &serveraddr,sizeof(serveraddr));
	if (res<0) {printf("%d => client: connect() error: %d\n",getpid(),errno);} else {printf("%d => client: connect() ok\n",getpid());}

	res = read(connfd, buf, sizeof(buf));
	printf("5 %d => client: recieved: \"%s\", res=%d, errno=%d\n",getpid(), buf,res,errno);
	res = write(connfd, answer_str, sizeof(buf));
	printf("6 %d => client: send: \"%s\", res=%d, errno=%d\n",getpid(), answer_str,res,errno);
	
	close(connfd);
	exit(0);

    }
    
    return(0);
}


>Release-Note:
>Audit-Trail:
State-Changed-From-To: open->feedback 
State-Changed-By: gavin 
State-Changed-When: Mon Jun 2 08:46:14 UTC 2008 
State-Changed-Why:  
To submitter: when the machine hangs, can you please give the 
output of "netstat -m"?  Also, are any errors logged to 
/var/log/messages or the system console?  Do you get any errors 
when you try to ping 127.0.0.1? 


Responsible-Changed-From-To: freebsd-bugs->gavin 
Responsible-Changed-By: gavin 
Responsible-Changed-When: Mon Jun 2 08:46:14 UTC 2008 
Responsible-Changed-Why:  
Track 

http://www.freebsd.org/cgi/query-pr.cgi?pr=124160 
State-Changed-From-To: feedback->open 
State-Changed-By: gavin 
State-Changed-When: Tue Jun 3 17:59:50 UTC 2008 
State-Changed-Why:  
Feedback received from submitter: 
---- 
You mean lack of mbuf's and "No buffer space availible" error during ping? 

# netstat -m 
1018/1037/2055 mbufs in use (current/cache/total) 
843/575/1418/25600 mbuf clusters in use (current/cache/total/max) 
843/565 mbuf+clusters out of packet secondary zone in use (current/cache) 
0/0/0/0 4k (page size) jumbo clusters in use (current/cache/total/max) 
0/0/0/0 9k jumbo clusters in use (current/cache/total/max) 
0/0/0/0 16k jumbo clusters in use (current/cache/total/max) 
1940K/1409K/3349K bytes allocated to network (current/cache/total) 
0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) 
0/0/0 requests for jumbo clusters denied (4k/9k/16k) 
0/5/6656 sfbufs in use (current/peak/max) 
0 requests for sfbufs denied 
0 requests for sfbufs delayed 
0 requests for I/O initiated by sendfile 
0 calls to protocol drain routines 

No errors in log files, and no errors seen during ping. But a wierd 
detail: when i do ping 127.0.0.1 from tty #1 and switch to another tty 
(#2) and also run `ping 127.0.0.1` there, I see one reply in tty #1, 
and then both ping's hang. 
---- 

http://www.freebsd.org/cgi/query-pr.cgi?pr=124160 
Responsible-Changed-From-To: gavin->freebsd-net 
Responsible-Changed-By: gavin 
Responsible-Changed-When: Mon Jun 30 11:36:12 UTC 2008 
Responsible-Changed-Why:  
Over to maintainers. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=124160 
>Unformatted:
