From kris@obsecurity.org  Sat Feb 11 06:12:25 2006
Return-Path: <kris@obsecurity.org>
Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125])
	by hub.freebsd.org (Postfix) with ESMTP id 94ADD16A420
	for <FreeBSD-gnats-submit@freebsd.org>; Sat, 11 Feb 2006 06:12:25 +0000 (GMT)
	(envelope-from kris@obsecurity.org)
Received: from elvis.mu.org (elvis.mu.org [192.203.228.196])
	by mx1.FreeBSD.org (Postfix) with ESMTP id F09BD43D78
	for <FreeBSD-gnats-submit@freebsd.org>; Sat, 11 Feb 2006 06:12:19 +0000 (GMT)
	(envelope-from kris@obsecurity.org)
Received: from obsecurity.dyndns.org (elvis.mu.org [192.203.228.196])
	by elvis.mu.org (Postfix) with ESMTP id AC4ED1A3C1B
	for <FreeBSD-gnats-submit@freebsd.org>; Fri, 10 Feb 2006 22:12:19 -0800 (PST)
Received: by obsecurity.dyndns.org (Postfix, from userid 1000)
	id D34635152B; Sat, 11 Feb 2006 01:12:18 -0500 (EST)
Message-Id: <20060211061218.D34635152B@obsecurity.dyndns.org>
Date: Sat, 11 Feb 2006 01:12:18 -0500 (EST)
From: Kris Kennaway <kris@FreeBSD.org>
To: FreeBSD-gnats-submit@freebsd.org
Cc:
Subject: Changing system date causes panic in nd6_timer
X-Send-Pr-Version: 3.113
X-GNATS-Notify:

>Number:         93170
>Category:       kern
>Synopsis:       Changing system date causes panic in nd6_timer
>Confidential:   no
>Severity:       serious
>Priority:       high
>Responsible:    freebsd-bugs
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Sat Feb 11 06:20:07 GMT 2006
>Closed-Date:    Sat Mar 28 23:44:48 UTC 2009
>Last-Modified:  Sat Mar 28 23:44:48 UTC 2009
>Originator:     Kris Kennaway
>Release:        FreeBSD 7.0-CURRENT amd64
>Organization:
>Environment:
FreeBSD/amd64
>Description:

I ran ntpdate on an amd64 system with ipv6 enabled and a skewed clock
(ntpdate stepped it back by about an hour), and immediately got a
use-after-free panic in ifaddr.  When I rebooted with memguard enabled
on this malloc type and retried, I got this panic upon changing the
date forward, then back, then forward again (also note the garbage
return data from ntpdate):

# date 200606011200
Thu Jun  1 12:00:00 UTC 2006
# ntpdate ntp.apple.com
16 Jan 00:40:18 ntpdate[612]: step time server 17.254.0.28 offset -~9000pm6}9426375508.195959 sec
# date 200606011200
Thu Jun  1 12:00:00 UTC 2006

Fatal trap 12: page fault while in kernel mode
cpuid = 0; apic id = 00
fault virtual address   = 0xffffffff91bd2198
fault code              = supervisor write, protection violation
instruction pointer     = 0x8:0xffffffff80321346
stack pointer           = 0x10:0xffffffffbcfa1b60
frame pointer           = 0x10:0xffffffffbcfa1b90
code segment            = base 0x0, limit 0xfffff, type 0x1b
                        = DPL 0, pres 1, long 1, def32 0, gran 1
processor eflags        = interrupt enabled, resume, IOPL = 0
current process         = 14 (swi4: clock sio)
[thread pid 14 tid 100010 ]
Stopped at      nd6_timer+0x106:        movl    %eax,0x198(%rbx)
db> wh
Tracing pid 14 tid 100010 td 0xffffff03e15d6c30
nd6_timer() at nd6_timer+0x106
softclock() at softclock+0x279
ithread_execute_handlers() at ithread_execute_handlers+0x12f
ithread_loop() at ithread_loop+0x99
fork_exit() at fork_exit+0xdf
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffffffbcfa1d40, rbp = 0 ---

Unfortunately I can't dump on this system, but:

(kgdb) list *(nd6_timer+0x106)
0xffffffff80321346 is in nd6_timer (../../../netinet6/nd6.c:585).
580                                     goto addrloop; /* XXX: see below */
581                     }
582                     if (IFA6_IS_DEPRECATED(ia6)) {
583                             int oldflags = ia6->ia6_flags;
584
585                             ia6->ia6_flags |= IN6_IFF_DEPRECATED;
586
587                             /*
588                              * If a temporary address has just become deprecated,
589                              * regenerate a new one if possible.

>How-To-Repeat:

Run the above two commands in a loop

>Fix:



>Release-Note:
>Audit-Trail:

From: David Malone <dwmalone@maths.tcd.ie>
To: bug-followup@FreeBSD.org, kris@FreeBSD.org
Cc: rwatson@FreeBSD.org, iedowse@maths.tcd.ie, dwmalone@maths.tcd.ie
Subject: Re: kern/93170 : Changing system date causes panic in nd6_timer
Date: Sat, 11 Feb 2006 16:59:38 +0000

 I'm fairly sure I know what's going on here, as I've seen a few
 more of these panics. There seems to be a bundle of IPv6 neighbour
 discovery code that uses splnet() and hasn't had any locking added
 to it yet. One of these bits of code is walking some global lists
 and is obviously getting into trouble.
 
 I believe this same code is also causing dirhash panics people have
 reported 'cos some of the memory it is using is getting reused by
 dirhash.
 
 I sent the message below to Robert a while ago - since I haven't
 heard back from him, I'm going to have a go at adding some locking.
 
 	David.
 
 Subject: IPv6 address list locking.
 Date: Fri, 20 Jan 2006 17:48:57 +0000
 From: David Malone <dwmalone@maths.tcd.ie>
 Message-ID:  <200601201748.aa69772@salmon.maths.tcd.ie>
 
 Hi Robert,
 
 I've been trying to trace some use-after-free problems that have
 paniced the kernel in dirhash. A few weeks ago I got a panic that
 indicated that the ifnet malloc pool might be the problem. Using
 memguard on my desktop in Dublin paniced today and, judging by the
 reports of the guy who reset it for me, it paniced at nd6_timer+0xe8.
 
 This looks like a bit of nd6_timer code that accesses a global list
 of IPv6 addresses stored in a variable called in6_ifaddr. Rather
 annoyingly this variable has the same name as a structure, so it
 is difficult to grep for. However, I don't see any locking of
 accesses to this variable.
 
 Do you have a scheme for locking globals like this in the IPv6
 stack? Does it look like nd6_timeout is violaing this locking?
 
 	David.
 
 in6.c:  for (ia = in6_ifaddr; ia; ia = ia->ia_next) {
 in6.c:  for (ia = in6_ifaddr; ia; ia = ia->ia_next) {
 in6.c:  for (ia = in6_ifaddr; ia; ia = ia->ia_next) {
 in6_src.c:      for (ia = in6_ifaddr; ia; ia = ia->ia_next) {
 nd6.c:  for (ia6 = in6_ifaddr; ia6; ia6 = nia6) {
 nd6.c:                  for (ia = in6_ifaddr; ia; ia = ia_next) {
 nd6_rtr.c:      for (ifa = in6_ifaddr; ifa; ifa = ifa->ia_next) {
 nd6_rtr.c:              for (ifa = in6_ifaddr; ifa; ifa = ifa->ia_next) {
 nd6_rtr.c:              for (ifa = in6_ifaddr; ifa; ifa = ifa->ia_next) {
 nd6_rtr.c:      for (ia = in6_ifaddr; ia; ia = ia->ia_next) {
State-Changed-From-To: open->feedback 
State-Changed-By: kmacy 
State-Changed-When: Tue Nov 20 06:59:13 UTC 2007 
State-Changed-Why:  

Does this still happen? 

http://www.freebsd.org/cgi/query-pr.cgi?pr=93170 
State-Changed-From-To: feedback->suspended 
State-Changed-By: linimon 
State-Changed-When: Sun Mar 2 02:40:46 UTC 2008 
State-Changed-Why:  
Feedback was not received, but it does not sound as though this problem 
has been fixed.  Mark as suspended since no one seems to be working on it. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=93170 
State-Changed-From-To: suspended->closed  
State-Changed-By: brucec 
State-Changed-When: Sat Mar 28 23:43:52 UTC 2009 
State-Changed-Why:  
Fixed in rev 1.63 of src/sys/netinet6/nd6.c 

http://www.freebsd.org/cgi/query-pr.cgi?pr=93170 
>Unformatted:
