From nobody@FreeBSD.org  Thu Apr 11 19:16:05 2013
Return-Path: <nobody@FreeBSD.org>
Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115])
	by hub.freebsd.org (Postfix) with ESMTP id EC8DC2AC
	for <freebsd-gnats-submit@FreeBSD.org>; Thu, 11 Apr 2013 19:16:05 +0000 (UTC)
	(envelope-from nobody@FreeBSD.org)
Received: from red.freebsd.org (red.freebsd.org [IPv6:2001:4f8:fff6::22])
	by mx1.freebsd.org (Postfix) with ESMTP id DDB21149D
	for <freebsd-gnats-submit@FreeBSD.org>; Thu, 11 Apr 2013 19:16:05 +0000 (UTC)
Received: from red.freebsd.org (localhost [127.0.0.1])
	by red.freebsd.org (8.14.5/8.14.5) with ESMTP id r3BJG5BD055824
	for <freebsd-gnats-submit@FreeBSD.org>; Thu, 11 Apr 2013 19:16:05 GMT
	(envelope-from nobody@red.freebsd.org)
Received: (from nobody@localhost)
	by red.freebsd.org (8.14.5/8.14.5/Submit) id r3BJG5GQ055823;
	Thu, 11 Apr 2013 19:16:05 GMT
	(envelope-from nobody)
Message-Id: <201304111916.r3BJG5GQ055823@red.freebsd.org>
Date: Thu, 11 Apr 2013 19:16:05 GMT
From: Todd Blum <todd@toddblum.org>
To: freebsd-gnats-submit@FreeBSD.org
Subject: ipsec-tools 0.8.0 racoon tends to segfault when multiple Phase1's aren't establishing
X-Send-Pr-Version: www-3.1
X-GNATS-Notify:

>Number:         177785
>Category:       ports
>Synopsis:       security/ipsec-tools: ipsec-tools 0.8.0 racoon tends to segfault when multiple Phase1's aren't establishing
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    vanhu
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Thu Apr 11 19:20:00 UTC 2013
>Closed-Date:    
>Last-Modified:  Tue Apr 22 03:33:21 UTC 2014
>Originator:     Todd Blum
>Release:        8.1
>Organization:
>Environment:
FreeBSD mbsnet-pf1.mbspchost.com 8.1-RELEASE-p13 FreeBSD 8.1-RELEASE-p13 #1: Fri Dec  7 16:55:26 EST 2012     root@snapshots-8_1-i386.builders.pfsense.org:/usr/obj./usr/pfSensesrc/src/sys/pfSense_SMP.8  i386

>Description:
I had two Phase1's that were not coming up, and it seemed like racoon
was segfaulting once every 4 days or so.  Then when I had three Phase1's
down, it seems like racoon started to segfault once every few days.

I've been noticing this behavior for several months.

I've opened this ticket at ipsec-tools with more detailed information:

https://sourceforge.net/tracker/?func=detail&aid=3603844&group_id=74601&atid=541482

>How-To-Repeat:
Try configuring 3 or more Ipsec tunnels that will not come up, then wait
7 days.
>Fix:
My current workaround is to reboot if racoon crashes (falling back on
CARP slave while rebooting):

#!/bin/sh
#
# Find the pid of the process (PPID will be the shell that started it)
#  remember no spaces allowed between varnames, just equals sign, and the value
# Script name cannot contain the word racoon in order to to avoid self-triggering

sleep 30

FIND_PROC=`pgrep racoon`
# if FIND_PROC is empty, the process has died; restart it

if [ -z "${FIND_PROC}" ]; then
      echo racoon failed at `date`
       nohup  shutdown -r now &
fi

exit

With cron entry:

*     *     *     *     *     root     /root/ipsec-watchdog.sh >> /root/ipsec-watchdog.log

>Release-Note:
>Audit-Trail:

From: Mark Linimon <linimon@lonesome.com>
To: Todd Blum <todd@toddblum.org>
Cc: freebsd-gnats-submit@FreeBSD.org
Subject: Re: misc/177785: ipsec-tools 0.8.0 racoon tends to segfault when
 multiple Phase1's aren't establishing
Date: Fri, 12 Apr 2013 18:59:56 -0500

 Does this apply to a port, or to the base system?

From: Todd Blum <todd@toddblum.org>
To: Mark Linimon <linimon@lonesome.com>
Cc: freebsd-gnats-submit@freebsd.org
Subject: Re: misc/177785: ipsec-tools 0.8.0 racoon tends to segfault when
 multiple Phase1's aren't establishing
Date: Wed, 17 Apr 2013 18:44:45 -0400

 --e89a8f642c9c88318d04da963bd4
 Content-Type: text/plain; charset=ISO-8859-1
 
 Hi Mark,
 
    This is for ports.  I should reference previous problem report:
 
 http://www.freebsd.org/cgi/query-pr.cgi?pr=168104
 
 Thanks,
 
 Todd
 
 --e89a8f642c9c88318d04da963bd4--
Responsible-Changed-From-To: freebsd-bugs->freebsd-ports-bugs 
Responsible-Changed-By: linimon 
Responsible-Changed-When: Thu Apr 18 01:08:46 UTC 2013 
Responsible-Changed-Why:  
ports PR. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=177785 
Responsible-Changed-From-To: freebsd-ports-bugs->sumikawa 
Responsible-Changed-By: edwin 
Responsible-Changed-When: Thu Apr 18 01:09:10 UTC 2013 
Responsible-Changed-Why:  
Over to maintainer (via the GNATS Auto Assign Tool) 

http://www.freebsd.org/cgi/query-pr.cgi?pr=177785 

From: Todd Blum <todd@toddblum.org>
To: FreeBSD-gnats-submit@freebsd.org, freebsd-bugs@freebsd.org
Cc:  
Subject: Re: misc/177785: ipsec-tools 0.8.0 racoon tends to segfault when
 multiple Phase1's aren't establishing
Date: Mon, 22 Apr 2013 13:37:01 -0400 (EDT)

 The error message 'failed to get sainfo' is usually appearing in the logs
 prior to the segfaults, then not at all afterwards:
 
 Apr 16 09:59:51 192.168.116.250 racoon: [xx.xx.xxx.xx] ERROR: unknown
 Informational exchange received.
 Apr 16 09:59:55 192.168.116.250 racoon: ERROR: failed to get sainfo.
 Apr 16 09:59:57 192.168.116.250 racoon: [yy.yy.yy.yyy] ERROR: unknown
 Informational exchange received.
 Apr 16 10:00:02 192.168.116.250 racoon: ERROR: failed to get sainfo.
 Apr 16 10:00:02 192.168.116.250 racoon: ERROR: failed to get sainfo.
 Apr 16 10:00:16 192.168.116.250 racoon: ERROR: failed to get sainfo.
 Apr 16 10:00:23 192.168.116.250 racoon: ERROR: failed to get sainfo.
 Apr 16 10:00:23 192.168.116.250 racoon: ERROR: failed to get sainfo.
 Apr 16 10:00:41 192.168.116.250 racoon: ERROR: failed to get sainfo.
 Apr 16 10:00:44 192.168.116.250 racoon: ERROR: failed to get sainfo.
 Apr 16 10:00:44 192.168.116.250 racoon: ERROR: failed to get sainfo.
 Apr 16 10:01:02 192.168.116.250 racoon: ERROR: failed to get sainfo.
 Apr 16 10:01:06 192.168.116.250 racoon: ERROR: failed to get sainfo.
 Apr 16 10:01:06 192.168.116.250 racoon: ERROR: failed to get sainfo.
 Apr 16 10:01:14 192.168.116.250 racoon: INFO: respond new phase 1
 negotiation: zz.zz.zz.zz[500]<=>hh.hh.hh.hh[500]
 Apr 16 10:01:14 192.168.116.250 racoon: INFO: begin Identity Protection
 mode.
 Apr 16 10:01:17 192.168.116.250 racoon: INFO: ISAKMP-SA established
 zz.zz.zz.zz[500]-hh.hh.hh.hh[500] spi:baa4c93e8c16198c:482ba6110eeabc0
 c
 Apr 16 10:01:17 192.168.116.250 racoon: INFO: purged IPsec-SA proto_id=ESP
 spi=2201026904.
 Apr 16 10:01:17 192.168.116.250 racoon: INFO: purged IPsec-SA proto_id=ESP
 spi=3679806084.
 Apr 16 10:01:18 192.168.116.250 racoon: INFO: respond new phase 2
 negotiation: zz.zz.zz.zz[500]<=>hh.hh.hh.hh[500]
 Apr 16 10:01:18 192.168.116.250 racoon: INFO: IPsec-SA established: ESP
 zz.zz.zz.zz[500]->hh.hh.hh.hh[500] spi=119993144(0x726f338)
 Apr 16 10:01:18 192.168.116.250 racoon: INFO: IPsec-SA established: ESP
 zz.zz.zz.zz[500]->hh.hh.hh.hh[500] spi=2718404122(0xa2078e1a)
 Apr 16 10:01:19 192.168.116.250 racoon: INFO: ISAKMP-SA expired
 zz.zz.zz.zz[500]-hh.hh.hh.hh[500] spi:baa4c93e8c16198c:482ba6110eeabc0c
 Apr 16 10:01:19 192.168.116.250 racoon: INFO: ISAKMP-SA deleted
 zz.zz.zz.zz[500]-hh.hh.hh.hh[500] spi:baa4c93e8c16198c:482ba6110eeabc0c
 Apr 16 10:01:27 192.168.116.250 racoon: ERROR: failed to get sainfo.
 Apr 16 10:01:27 192.168.116.250 racoon: ERROR: failed to get sainfo.
 Apr 16 10:01:30 192.168.116.250 racoon: ERROR: failed to get sainfo.
 Apr 16 10:01:51 192.168.116.250 racoon: ERROR: failed to get sainfo.
 Apr 16 10:01:51 192.168.116.250 racoon: ERROR: failed to get sainfo.
 Apr 16 10:01:55 192.168.116.250 racoon: ERROR: failed to get sainfo.
 Apr 16 10:02:13 192.168.116.250 racoon: ERROR: failed to get sainfo.
 Apr 16 10:02:13 192.168.116.250 racoon: ERROR: failed to get sainfo.
 Apr 16 10:02:16 192.168.116.250 racoon: ERROR: failed to get sainfo.
 Apr 16 10:02:37 192.168.116.250 racoon: ERROR: failed to get sainfo.
 Apr 16 10:02:37 192.168.116.250 racoon: ERROR: failed to get sainfo.
 Apr 16 10:02:40 192.168.116.250 racoon: ERROR: failed to get sainfo.
 Apr 16 10:02:41 192.168.116.250 racoon: INFO: respond new phase 1
 negotiation: zz.zz.zz.zz[500]<=>hh.hh.hh.hh[500]
 Apr 16 10:02:41 192.168.116.250 racoon: INFO: begin Identity Protection
 mode.
 Apr 16 10:02:42 192.168.116.250 kernel: pid 45397 (racoon), uid 0: exited
 on signal 11 (core dumped)
 
 Is there any relation to this error report?
 
 https://bugs.launchpad.net/ubuntu/+source/ipsec-tools/+bug/913935

From: Todd Blum <todd@toddblum.org>
To: FreeBSD-gnats-submit@freebsd.org, freebsd-bugs@freebsd.org
Cc:  
Subject: Re: misc/177785: ipsec-tools 0.8.0 racoon tends to segfault when
 multiple Phase1's aren't establishing
Date: Tue, 23 Apr 2013 10:21:55 -0400

 --e89a8f642c9c5362fd04db07e80d
 Content-Type: text/plain; charset=ISO-8859-1
 
 Today I've found that I had duplicate IPSec tunnels configured in pfSense,
 one disabled and the other enabled.
 
 I've moved this tunnel elsewhere, and I've removed both from the pfSense
 config to see if this improves my racoon stability.
 
 --e89a8f642c9c5362fd04db07e80d--
Responsible-Changed-From-To: sumikawa->ports 
Responsible-Changed-By: sumikawa 
Responsible-Changed-When: Wed Apr 24 16:19:59 JST 2013 
Responsible-Changed-Why:  
This is a bug report for security/ipsec-tools, not security/racoon2 

http://www.freebsd.org/cgi/query-pr.cgi?pr=177785 
Responsible-Changed-From-To: ports->freebsd-ports-bugs 
Responsible-Changed-By: linimon 
Responsible-Changed-When: Wed Apr 24 09:07:15 UTC 2013 
Responsible-Changed-Why:  
Canonicalize assignment. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=177785 

From: Todd Blum <todd@toddblum.org>
To: FreeBSD-gnats-submit@freebsd.org, freebsd-bugs@freebsd.org
Cc:  
Subject: Re: misc/177785: ipsec-tools 0.8.0 racoon tends to segfault when
 multiple Phase1's aren't establishing
Date: Wed, 1 May 2013 13:04:45 -0400

 --089e01228a445a15dc04dbab1dcb
 Content-Type: text/plain; charset=ISO-8859-1
 
 racoon segfaulted again, but this time without any sainfo messages.
 
 The crash coincided with an ISP outage that affected at least 6 remote
 endpoints.  DPD was enabled on these tunnels:
 
 ...
 May  1 01:18:27 192.168.116.250 racoon: INFO: ISAKMP-SA deleted
 my.end.poi.nt[500]-x.x.x.x [500] spi:48131b4e56ac24b8:32ef67f65454935e
 May  1 01:18:28 192.168.116.250 racoon: [y.y.y.y ] INFO: DPD: remote
 (ISAKMP-SA spi=622012ee7f51261d:7e39cc0f5ee916a0) seems to be dead.
 May  1 01:18:28 192.168.116.250 racoon: INFO: purging ISAKMP-SA
 spi=622012ee7f51261d:7e39cc0f5ee916a0.
 May  1 01:18:28 192.168.116.250 racoon: INFO: purged IPsec-SA
 spi=2284023606.
 May  1 01:18:28 192.168.116.250 racoon: INFO: purged IPsec-SA spi=187964617.
 May  1 01:18:28 192.168.116.250 racoon: INFO: purged ISAKMP-SA
 spi=622012ee7f51261d:7e39cc0f5ee916a0.
 May  1 01:18:28 192.168.116.250 racoon: INFO: ISAKMP-SA deleted
 my.end.poi.nt[500]-y.y.y.y [500] spi:622012ee7f51261d:7e39cc0f5ee916a0
 May  1 01:18:29 192.168.116.250 racoon: [z.z.z.z ] INFO: DPD: remote
 (ISAKMP-SA spi=3c837090349206bf:1086e896dce5e982) seems to be dead.
 May  1 01:18:29 192.168.116.250 racoon: INFO: purging ISAKMP-SA
 spi=3c837090349206bf:1086e896dce5e982.
 May  1 01:18:29 192.168.116.250 racoon: INFO: purged IPsec-SA
 spi=3531119898.
 May  1 01:18:29 192.168.116.250 racoon: INFO: purged IPsec-SA spi=124488619.
 May  1 01:18:29 192.168.116.250 racoon: INFO: purged ISAKMP-SA
 spi=3c837090349206bf:1086e896dce5e982.
 ...
 
 --089e01228a445a15dc04dbab1dcb--
Responsible-Changed-From-To: freebsd-ports-bugs->vanhu 
Responsible-Changed-By: vanhu 
Responsible-Changed-When: Tue May 21 15:53:32 UTC 2013 
Responsible-Changed-Why:  
Hi. 

Can you provide us a backtrace of the crash ? 


http://www.freebsd.org/cgi/query-pr.cgi?pr=177785 

From: Todd Blum <todd@toddblum.org>
To: FreeBSD-gnats-submit@freebsd.org, freebsd-bugs@freebsd.org
Cc:  
Subject: Re: misc/177785: ipsec-tools 0.8.0 racoon tends to segfault when
 multiple Phase1's aren't establishing
Date: Wed, 29 May 2013 17:11:18 -0400

 --001a11c2ab98a3206d04dde1d261
 Content-Type: text/plain; charset=ISO-8859-1
 
 E-mailed a core dump privately.
 
 The problems seemed to have correlated with DSL outages of a specific ISP.  The
 ISP replaced/repaired a DSL DSLAM and possibly some core routers as well.
 
 racoon has been up stable now for several weeks since this change.
 
 --001a11c2ab98a3206d04dde1d261--
>Unformatted:
