From kuznec@lg08.hcn-strela.ru  Mon Nov 30 12:02:23 2009
Return-Path: <kuznec@lg08.hcn-strela.ru>
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id 6191E1065679
	for <FreeBSD-gnats-submit@freebsd.org>; Mon, 30 Nov 2009 12:02:23 +0000 (UTC)
	(envelope-from kuznec@lg08.hcn-strela.ru)
Received: from lg08.hcn-strela.ru (lg08.hcn-strela.ru [194.87.124.19])
	by mx1.freebsd.org (Postfix) with ESMTP id 1D5CC8FC1C
	for <FreeBSD-gnats-submit@freebsd.org>; Mon, 30 Nov 2009 12:02:23 +0000 (UTC)
Received: by lg08.hcn-strela.ru (Postfix, from userid 1001)
	id E1211114036; Mon, 30 Nov 2009 14:47:14 +0300 (MSK)
Message-Id: <20091130114714.E1211114036@lg08.hcn-strela.ru>
Date: Mon, 30 Nov 2009 14:47:14 +0300 (MSK)
From: Alexey@lg08.hcn-strela.ru, V.Kouznetsov@lg08.hcn-strela.ru
Reply-To: alexey@kouznetsov.com
To: FreeBSD-gnats-submit@freebsd.org
Cc:
Subject: CARP arp replays with wrong src mac
X-Send-Pr-Version: 3.113
X-GNATS-Notify:

>Number:         141023
>Category:       kern
>Synopsis:       [carp] CARP arp replays with wrong src mac
>Confidential:   no
>Severity:       serious
>Priority:       low
>Responsible:    glebius
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Mon Nov 30 12:10:01 UTC 2009
>Closed-Date:    Fri Dec 16 13:03:51 UTC 2011
>Last-Modified:  Fri Dec 16 13:03:51 UTC 2011
>Originator:     Alexey V. Kouznetsov
>Release:        FreeBSD 7.2-STABLE amd64
>Organization:
>Environment:

System: FreeBSD xx 7.2-STABLE FreeBSD 7.2-STABLE #0: Fri Sep 11 17:52:57 MSD 2009 root@xx:/usr/obj/usr/src/sys/xxx90911 amd64

Any 6.x 7.x versions (o386 and amd64) of FreeBSD with CARP interfaces.
at list for em* based interfaces. Possible for any.

>Description:

We have vlan interface configured:

Base:
vlan254: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=3<RXCSUM,TXCSUM>
        ether 00:15:17:bb:5c:db
        inet 10.254.31.253 netmask 0xffffff00 broadcast 10.254.31.255
        media: Ethernet autoselect (1000baseTX <full-duplex>)
        status: active
        vlan: 254 parent interface: em1
carp:
carp254: flags=49<UP,LOOPBACK,RUNNING> metric 0 mtu 1500
        inet 10.254.31.254 netmask 0xffffff00
        carp: MASTER vhid 35 advbase 1 advskew 10

(it will be same result if we do not use vlan, but configure on em, w/o vlans)

remote machine sent broadcars request for 10.254.31.254:
14:26:40.792991 00:22:b0:51:23:03 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 56: arp who-has 10.254.31.254 tell 10.254.31.232

and machive answered:
14:26:40.792997 00:15:17:bb:5c:db > 00:22:b0:51:23:03, ethertype ARP (0x0806), length 42: arp reply 10.254.31.254 is-at 00:00:5e:00:01:23

As we can see this reported mac 00:00:5e:00:01:23.. but source mac for
ARP REPLAY is 00:15:17:bb:5c:db, not 00:00:5e:00:01:23.

And currently I have some devices on network, who checked the SRC macs
carefully and ignored such replays.

>How-To-Repeat:
configure carp interface and ping such interface from remove machine.
look for tcpdump (-e) for results

>Fix:

No known fixes.. sorry.




>Release-Note:
>Audit-Trail:
Responsible-Changed-From-To: freebsd-bugs->freebsd-net 
Responsible-Changed-By: linimon 
Responsible-Changed-When: Mon Nov 30 21:21:50 UTC 2009 
Responsible-Changed-Why:  
Over to maintainer(s). 

http://www.freebsd.org/cgi/query-pr.cgi?pr=141023 
State-Changed-From-To: open->analyzed 
State-Changed-By: glebius 
State-Changed-When: Sat Oct 15 09:08:48 UTC 2011 
State-Changed-Why:  
Let's look at this closer. 


Responsible-Changed-From-To: freebsd-net->glebius 
Responsible-Changed-By: glebius 
Responsible-Changed-When: Sat Oct 15 09:08:48 UTC 2011 
Responsible-Changed-Why:  
I'm now working on major rewrite of CARP for FreeBSD 10, and 
I'd like to take all related PRs.  

http://www.freebsd.org/cgi/query-pr.cgi?pr=141023 

From: Gleb Smirnoff <glebius@FreeBSD.org>
To: alexey@kouznetsov.com, Alexey@lg08.hcn-strela.ru,
        V.Kouznetsov@lg08.hcn-strela.ru
Cc: bug-followup@FreeBSD.org
Subject: Re: kern/141023: [carp] CARP arp replays with wrong src mac
Date: Sat, 15 Oct 2011 17:19:38 +0400

   Alexey,
 
   as you described in the original PR:
 
 > and machive answered:
 > 14:26:40.792997 00:15:17:bb:5c:db > 00:22:b0:51:23:03, ethertype ARP (0x0806), length 42: arp reply 10.254.31.254 is-at 00:00:5e:00:01:23
 > 
 > As we can see this reported mac 00:00:5e:00:01:23.. but source mac for
 > ARP REPLAY is 00:15:17:bb:5c:db, not 00:00:5e:00:01:23.
 > 
 > And currently I have some devices on network, who checked the SRC macs
 > carefully and ignored such replays.
 
 According to RFC826, which is current standard for ARP implementations,
 the hardware address in the transmission layer does not need to match
 the hardware addrees in the ARP reply packet itself. At least I can't
 find any evidence in the RFC that it must or even should. Moreover, it
 is mentioned in the RFC that:
 
 "Ethernet transmission layer (not necessarily accessible to the user)"
 
 So, RFC states that data in the Ethernet header isn't crucial for
 the operation of address resolution protocol.
 
 Correct me, if I am wrong above.
 
 Thus, any devices that perform such strict checking on ARP replies,
 are violating RFC. And I hope these devices are capable to turning
 this feature off. From my point of view, this strictness doesn't
 bring in any security, since with any modern network card sender
 hardware address can be faked with the same small efforts as faking
 address in the ARP packet itself.
 
 So, your devices should be fixed, not CARP in FreeBSD. However, we
 are open to bringing in compatibility with other non-RFC-compliant
 devices if it is possible to do this in non-destructive way, for
 example having this feature off by default. If you are interested in
 this feature, you can submit a patch, that implements it, it would
 be reviewed and committed to source tree.
 
 -- 
 Totus tuus, Glebius.

From: "Alexey Kouznetsov" <alexey@kouznetsov.com>
To: "'Gleb Smirnoff'" <glebius@FreeBSD.org>
Cc: <bug-followup@FreeBSD.org>
Subject: RE: kern/141023: [carp] CARP arp replays with wrong src mac
Date: Sun, 16 Oct 2011 13:04:30 +0400

 -----Original Message-----
 From: Gleb Smirnoff [mailto:glebius@FreeBSD.org] 
   Alexey,
 
   as you described in the original PR:
 
 >> and machive answered:
 >> 14:26:40.792997 00:15:17:bb:5c:db > 00:22:b0:51:23:03, ethertype ARP
 >>(0x0806), length 42: arp reply 10.254.31.254 is-at 00:00:5e:00:01:23
 >> 
 >> As we can see this reported mac 00:00:5e:00:01:23.. but source mac for
 >> ARP REPLAY is 00:15:17:bb:5c:db, not 00:00:5e:00:01:23.
 
 >> And currently I have some devices on network, who checked the SRC macs
 >> carefully and ignored such replays.
 
 > According to RFC826, which is current standard for ARP implementations,
 > the hardware address in the transmission layer does not need to match
 > the hardware addrees in the ARP reply packet itself. At least I can't
 > find any evidence in the RFC that it must or even should. Moreover, it
 > is mentioned in the RFC that:
 
 > "Ethernet transmission layer (not necessarily accessible to the user)"
 
 > So, RFC states that data in the Ethernet header isn't crucial for
 > the operation of address resolution protocol.
 
 > Correct me, if I am wrong above.
 
 > Thus, any devices that perform such strict checking on ARP replies,
 > are violating RFC. And I hope these devices are capable to turning
 > this feature off. From my point of view, this strictness doesn't
 > bring in any security, since with any modern network card sender
 > hardware address can be faked with the same small efforts as faking
 > address in the ARP packet itself.
 
 > So, your devices should be fixed, not CARP in FreeBSD. However, we
 > are open to bringing in compatibility with other non-RFC-compliant
 > devices if it is possible to do this in non-destructive way, for
 > example having this feature off by default. If you are interested in
 > this feature, you can submit a patch, that implements it, it would
 > be reviewed and committed to source tree.
 
 Yes, it looks to be true. But real world is little bit more complex. Last
 device I saw was Dlink DES 3028 with modern firmware, who also did filtering
 as described above. (devices at examples above was not our, we are ISP and
 devices was client's, not remember who was manufacturer for them. But
 sometime clients did not understand why device used to be work at other ISP
 and will not work here.)
 
 Another related problem: only traffic with L2 src of carp interface
 (00:00:5e:00:01:xx) is multicast traffic with CARP announces. So if you have
 long network and block multicast at user's end, there are possible and
 devices will not know where router live at and will flood over the subnet
 (until traffic reach core, where we have enabled such multicast).
 
 I did  some Q&D solution, I do arping from all carp interfaces with L2 SRC
 00:00:5e:00:01:xx once a minute. In general it solved all our problems ;)
 
 Btw I leave such ISP and currently I do not use carps, but I home I'll use
 it in future.
 
 With best regards
 /Alexey
 
State-Changed-From-To: analyzed->closed 
State-Changed-By: glebius 
State-Changed-When: Fri Dec 16 13:01:32 UTC 2011 
State-Changed-Why:  
We came to an agreement that CARP in FreeBSD doesn't violate ARP 
RFC, while some equipment does. 

Originator had been informed that a compatibility with weird equipment 
can be brought into FreeBSD, just send patches :) 

Since originator no longer has access to this eqiupment, close the PR. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=141023 
>Unformatted:
