From nobody@FreeBSD.org  Tue Oct 11 09:50:11 2011
Return-Path: <nobody@FreeBSD.org>
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id 4894E106567D
	for <freebsd-gnats-submit@FreeBSD.org>; Tue, 11 Oct 2011 09:50:11 +0000 (UTC)
	(envelope-from nobody@FreeBSD.org)
Received: from red.freebsd.org (red.freebsd.org [IPv6:2001:4f8:fff6::22])
	by mx1.freebsd.org (Postfix) with ESMTP id 2A4D28FC0C
	for <freebsd-gnats-submit@FreeBSD.org>; Tue, 11 Oct 2011 09:50:02 +0000 (UTC)
Received: from red.freebsd.org (localhost [127.0.0.1])
	by red.freebsd.org (8.14.4/8.14.4) with ESMTP id p9B9o1Te022684
	for <freebsd-gnats-submit@FreeBSD.org>; Tue, 11 Oct 2011 09:50:01 GMT
	(envelope-from nobody@red.freebsd.org)
Received: (from nobody@localhost)
	by red.freebsd.org (8.14.4/8.14.4/Submit) id p9B9o1wP022683;
	Tue, 11 Oct 2011 09:50:01 GMT
	(envelope-from nobody)
Message-Id: <201110110950.p9B9o1wP022683@red.freebsd.org>
Date: Tue, 11 Oct 2011 09:50:01 GMT
From: Damien Fleuriot <dam@my.gd>
To: freebsd-gnats-submit@FreeBSD.org
Subject: net / [carp] [patch] when preemption is enabled carp interface assumes MASTERship immediately even with higher advbase/advskew
X-Send-Pr-Version: www-3.1
X-GNATS-Notify:

>Number:         161483
>Category:       kern
>Synopsis:       [carp] [patch] when preemption is enabled carp interface assumes MASTERship immediately even with higher advbase/advskew
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    freebsd-net
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Tue Oct 11 10:00:19 UTC 2011
>Closed-Date:    Thu Oct 13 14:04:11 UTC 2011
>Last-Modified:  Thu Oct 13 14:04:11 UTC 2011
>Originator:     Damien Fleuriot
>Release:        applies to freebsd 8.x , 9.x
>Organization:
Hi-Media
>Environment:
irrelevant, problem still exists in SRC files for FreeBSD 9.0-BETA3
>Description:
Refer: my post on -stable
http://docs.freebsd.org/cgi/getmsg.cgi?fetch=368260+0+current/freebsd-stable

Refer the same problem on 8.x
http://www.freebsd.org/cgi/query-pr.cgi?pr=161123

Under normal operating circumstances, a CARP interface goes through the following states:
- INIT : when it's down
- BACKUP : immediately upon being brought up, the interface assumes a BACKUP role and starts its timer to know if it should claim mastership.
- MASTER : if the delay has expired (advbase * 3) without the interface seeing another master, it assumes mastership.


BUG: When preemption is enabled (net.inet.carp.preempt=1) , the CARP interface immediately assumes MASTERship regardless of its advbase and advskew values.

This causes CARP switchovers when a firewall from a CARP cluster is rebooted, for example.

In our case, this actually led to lost client connections, lost database sessions, developers' daemons crashes because of lost java/db connections...



This is a known problem with OpenBSD 3.8 and lower's implementation of CARP.
This has been fixed as of OpenBSD 3.9.
>How-To-Repeat:
Set up 2 boxes with a shared CARP IP.
Enable CARP preemption.

Bring down your CARP interface on the BACKUP box.
Bring it up again.
Notice how your interface assumed MASTERship for a short time.
Check with dmesg which confirms that your box actually preempted.
>Fix:
Remove all the code portion from sys/netinet/ip_carp.c that handles the INIT->MASTER transition immediately if preemption is enabled.

Patch attached.


Patch attached with submission follows:

--- sys/netinet/ip_carp.c       2011-09-24 18:10:38.000000000 +0200
+++ sys/netinet/ip_carp.c       2011-10-11 10:11:27.000000000 +0200
@@ -1424,24 +1424,10 @@
 
        switch (sc->sc_state) {
        case INIT:
-               if (carp_opts[CARPCTL_PREEMPT] && !carp_suppress_preempt) {
-                       carp_send_ad_locked(sc);
-#ifdef INET
-                       carp_send_arp(sc);
-#endif
-#ifdef INET6
-                       carp_send_na(sc);
-#endif /* INET6 */
-                       CARP_LOG("%s: INIT -> MASTER (preempting)\n",
-                           SC2IFP(sc)->if_xname);
-                       carp_set_state(sc, MASTER);
-                       carp_setroute(sc, RTM_ADD);
-               } else {
-                       CARP_LOG("%s: INIT -> BACKUP\n", SC2IFP(sc)->if_xname);
-                       carp_set_state(sc, BACKUP);
-                       carp_setroute(sc, RTM_DELETE);
-                       carp_setrun(sc, 0);
-               }
+               CARP_LOG("%s: INIT -> BACKUP\n", SC2IFP(sc)->if_xname);
+               carp_set_state(sc, BACKUP);
+               carp_setroute(sc, RTM_DELETE);
+               carp_setrun(sc, 0);
                break;
        case BACKUP:
                callout_stop(&sc->sc_ad_tmo);


>Release-Note:
>Audit-Trail:
Responsible-Changed-From-To: freebsd-bugs->freebsd-net 
Responsible-Changed-By: linimon 
Responsible-Changed-When: Tue Oct 11 17:56:56 UTC 2011 
Responsible-Changed-Why:  
Over to maintainer(s). 

http://www.freebsd.org/cgi/query-pr.cgi?pr=161483 
State-Changed-From-To: open->closed 
State-Changed-By: glebius 
State-Changed-When: Thu Oct 13 14:03:56 UTC 2011 
State-Changed-Why:  
Dup of kern/161123 

http://www.freebsd.org/cgi/query-pr.cgi?pr=161483 
>Unformatted:
