From jin@gracie.lbl.gov  Tue May 18 16:12:10 2004
Return-Path: <jin@gracie.lbl.gov>
Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125])
	by hub.freebsd.org (Postfix) with ESMTP id 8158B16A4F9
	for <FreeBSD-gnats-submit@freebsd.org>; Tue, 18 May 2004 16:12:09 -0700 (PDT)
Received: from gracie.lbl.gov (gracie.lbl.gov [131.243.2.175])
	by mx1.FreeBSD.org (Postfix) with ESMTP id 2722D440ED
	for <FreeBSD-gnats-submit@freebsd.org>; Tue, 18 May 2004 15:20:33 -0700 (PDT)
	(envelope-from jin@gracie.lbl.gov)
Received: from gracie.lbl.gov (localhost.lbl.gov [127.0.0.1])
	by gracie.lbl.gov (8.12.9p2/8.12.9) with ESMTP id i4IMIrf9061909
	for <FreeBSD-gnats-submit@freebsd.org>; Tue, 18 May 2004 15:18:53 -0700 (PDT)
	(envelope-from jin@gracie.lbl.gov)
Received: (from jin@localhost)
	by gracie.lbl.gov (8.12.9p2/8.12.9/Submit) id i4IMIrDZ061908;
	Tue, 18 May 2004 15:18:53 -0700 (PDT)
	(envelope-from jin)
Message-Id: <200405182218.i4IMIrDZ061908@gracie.lbl.gov>
Date: Tue, 18 May 2004 15:18:53 -0700 (PDT)
From: Jin Guojun (DSD staff) <jin@gracie.lbl.gov>
Reply-To: j_guojun@lbl.gov
To: FreeBSD-gnats-submit@freebsd.org
Cc:
Subject: rpc.lockd core dumped in questioning place
X-Send-Pr-Version: 3.113
X-GNATS-Notify:

>Number:         66837
>Category:       bin
>Synopsis:       rpc.lockd core dumped in questioning place
>Confidential:   no
>Severity:       critical
>Priority:       high
>Responsible:    mr
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Tue May 18 16:20:20 PDT 2004
>Closed-Date:    Fri Jul 16 12:54:44 GMT 2004
>Last-Modified:  Fri Jul 16 12:54:44 GMT 2004
>Originator:     Jin Guojun (DSD staff)
>Release:        FreeBSD 4.9-RELEASE i386
>Organization:
>Environment:
System: FreeBSD 4.9-RELEASE FreeBSD as well as 5.x


	FreeBSD 4.x and 5.x
>Description:
	rpc.lockd often core dump in 5.x RELEASES, now found in 4.x,
	and it causes NFS and file related service hanging.
	5.x rpc.lockd problem was caused from some Linux hosts,
	but we do not have time to catch it due to the lockd can
	be killed every minute, so we downgraded NFS server to 4.9.
	Now the problem is often caused by Mac OSX, and trace is
	appended. 
	It is not clear where is the problem. Three places:
	libc, syslog, and maybe lockd itself.
	It looks like in libc, which can be potential problem to
	cause syslog system and all programs use syslog to crash.

Core was generated by `rpc.lockd'.
Program terminated with signal 11, Segmentation fault.
Reading symbols from /usr/lib/librpcsvc.so.2...done.
Reading symbols from /usr/lib/libc.so.4...done.
Reading symbols from /usr/libexec/ld-elf.so.1...done.
#0  0x880ea31c in getenv () from /usr/lib/libc.so.4
(gdb) where
#0  0x880ea31c in getenv () from /usr/lib/libc.so.4
#1  0x880cf100 in tzset () from /usr/lib/libc.so.4
#2  0x880cf2f4 in localtime_r () from /usr/lib/libc.so.4
#3  0x880cf828 in ctime_r () from /usr/lib/libc.so.4
#4  0x880c9ce5 in vsyslog () from /usr/lib/libc.so.4
#5  0x880c9be5 in syslog () from /usr/lib/libc.so.4
#6  0x804a60a in log_from_addr (fun_name=0x804b91c "nlm_unlock_msg", 
    req=0xbfbff5c0) at procs.c:86
#7  0x804ae05 in nlmproc4_unlock_msg_4_svc (arg=0xbfbff080, rqstp=0xbfbff5c0)
    at procs.c:524
#8  0x80494ad in nlm_prog_4 (rqstp=0xbfbff5c0, transp=0x8051040)
    at nlm_prot_svc.c:389
#9  0x880aa485 in svc_getreqset2 () from /usr/lib/libc.so.4
#10 0x880888d7 in svc_run () from /usr/lib/libc.so.4
#11 0x804a567 in main (argc=2, argv=0xbfbff6b4) at lockd.c:104
(gdb) p fun_name
No symbol "fun_name" in current context.
(gdb) up 6
#6  0x804a60a in log_from_addr (fun_name=0x804b91c "nlm_unlock_msg", 
    req=0xbfbff5c0) at procs.c:86
86        syslog(LOG_DEBUG, "%s from %s", fun_name, hostname_buf);
(gdb) p fun_name
$1 = 0x804b91c "nlm_unlock_msg"
(gdb) p hostname_buf
$2 = "godel.lbl.gov", '\000' <repeats 26 times>
(gdb) l
81        else  /* No hostname available - print raw address    */
82        {
83          strcpy(hostname_buf, inet_ntoa(addr->sin_addr));
84        }
85
86        syslog(LOG_DEBUG, "%s from %s", fun_name, hostname_buf);
87      }
88
89
90      /* get_client -------------------------------------------------------------- */
(gdb) up
#7  0x804ae05 in nlmproc4_unlock_msg_4_svc (arg=0xbfbff080, rqstp=0xbfbff5c0)
    at procs.c:524
524       if (debug_level) log_from_addr("nlm_unlock_msg", rqstp);
(gdb) up
#8  0x80494ad in nlm_prog_4 (rqstp=0xbfbff5c0, transp=0x8051040)
    at nlm_prot_svc.c:389
389             result = (*local)((char *)&argument, rqstp);
(gdb) up
#9  0x880aa485 in svc_getreqset2 () from /usr/lib/libc.so.4
(gdb) up
#10 0x880888d7 in svc_run () from /usr/lib/libc.so.4


>How-To-Repeat:
	Let me know if any further trace is needed for this case.

>Fix:

	


>Release-Note:
>Audit-Trail:

From: "Jin Guojun [NCS]" <j_guojun@lbl.gov>
To: freebsd-gnats-submit@FreeBSD.org
Cc:  
Subject: Re: bin/66837: rpc.lockd core dumped in questioning place
Date: Thu, 20 May 2004 12:57:15 -0700

 More detiled trace is included.
 
     -Jin
 
 Core was generated by `rpc.lockd'.
 Program terminated with signal 11, Segmentation fault.
 Reading symbols from /usr/lib/librpcsvc.so.2...done.
 Reading symbols from /usr/lib/libc.so.4...done.
 Reading symbols from /usr/libexec/ld-elf.so.1...done.
 #0  0x880ea31c in __findenv (name=0x880f1073 "TZ",
 offset=0x805b000)
     at /usr/src/4.9-RELEASE/lib/libc/../libc/stdlib/getenv.c:91
 91      }
 (gdb) where
 #0  0x880ea31c in __findenv (name=0x880f1073 "TZ",
 offset=0x805b000)
     at /usr/src/4.9-RELEASE/lib/libc/../libc/stdlib/getenv.c:91
 #1  0x880cf100 in tzset ()
     at
 /usr/src/4.9-RELEASE/lib/libc/../libc/stdtime/localtime.c:986
 #2  0x880cf2f4 in localtime_r (timep=0xbfbfe338, p_tm=0xbfbfe2b0)
     at
 /usr/src/4.9-RELEASE/lib/libc/../libc/stdtime/localtime.c:1096
 #3  0x880cf828 in ctime_r (timep=0xbfbfe338,
     buf=0xbfbfe364 "Tue May 18 20:42:19 2004\n")
     at
 /usr/src/4.9-RELEASE/lib/libc/../libc/stdtime/localtime.c:1362
 #4  0x880c9ce5 in vsyslog (pri=7, fmt=0x804b834 "%s from %s",
     ap=0xbfbfefc0 "\034\004\b￿ \f")
     at /usr/src/4.9-RELEASE/lib/libc/../libc/gen/syslog.c:177
 #5  0x880c9be5 in syslog (pri=7, fmt=0x804b834 "%s from %s")
     at /usr/src/4.9-RELEASE/lib/libc/../libc/gen/syslog.c:107
 #6  0x804a60a in log_from_addr (fun_name=0x804b91c
 "nlm_unlock_msg",
     req=0xbfbff5a0) at procs.c:86
 #7  0x804ae05 in nlmproc4_unlock_msg_4_svc (arg=0xbfbff060,
 rqstp=0xbfbff5a0)
     at procs.c:524
 #8  0x80494ad in nlm_prog_4 (rqstp=0xbfbff5a0, transp=0x8051040)
     at nlm_prot_svc.c:389
 #9  0x880aa485 in svc_getreqset2 (readfds=0x8050080, width=5)
     at /usr/src/4.9-RELEASE/lib/libc/../libc/rpc/svc.c:461
 #10 0x880888d7 in svc_run ()
     at /usr/src/4.9-RELEASE/lib/libc/../libc/rpc/svc_run.c:78
 #11 0x804a567 in main (argc=2, argv=0xbfbff690) at lockd.c:104
 (gdb) l
 86              const char *name;
 87      {
 88              int offset;
 89
 90              return (__findenv(name, &offset));
 91      }
 (gdb) p name
 $1 = 0x880f1073 "TZ"
 (gdb) p offset
 $2 = (int *) 0x7002f383
 
 
Responsible-Changed-From-To: freebsd-bugs->cperciva 
Responsible-Changed-By: bms 
Responsible-Changed-When: Tue Jun 22 16:31:34 GMT 2004 
Responsible-Changed-Why:  
Colin has an express interest in such bugs 

http://www.freebsd.org/cgi/query-pr.cgi?pr=66837 
State-Changed-From-To: open->closed 
State-Changed-By: mr 
State-Changed-When: Fri Jul 16 12:54:05 GMT 2004 
State-Changed-Why:  
commited 


Responsible-Changed-From-To: cperciva->mr 
Responsible-Changed-By: mr 
Responsible-Changed-When: Fri Jul 16 12:54:05 GMT 2004 
Responsible-Changed-Why:  
commited 

http://www.freebsd.org/cgi/query-pr.cgi?pr=66837 
>Unformatted:
