From nobody@FreeBSD.org  Mon Jan 24 19:01:28 2005
Return-Path: <nobody@FreeBSD.org>
Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125])
	by hub.freebsd.org (Postfix) with ESMTP id 6F27A16A4CE
	for <freebsd-gnats-submit@FreeBSD.org>; Mon, 24 Jan 2005 19:01:28 +0000 (GMT)
Received: from www.freebsd.org (www.freebsd.org [216.136.204.117])
	by mx1.FreeBSD.org (Postfix) with ESMTP id 202EF43D3F
	for <freebsd-gnats-submit@FreeBSD.org>; Mon, 24 Jan 2005 19:01:28 +0000 (GMT)
	(envelope-from nobody@FreeBSD.org)
Received: from www.freebsd.org (localhost [127.0.0.1])
	by www.freebsd.org (8.13.1/8.13.1) with ESMTP id j0OJ1RA9003682
	for <freebsd-gnats-submit@FreeBSD.org>; Mon, 24 Jan 2005 19:01:27 GMT
	(envelope-from nobody@www.freebsd.org)
Received: (from nobody@localhost)
	by www.freebsd.org (8.13.1/8.13.1/Submit) id j0OJ1RT4003681;
	Mon, 24 Jan 2005 19:01:27 GMT
	(envelope-from nobody)
Message-Id: <200501241901.j0OJ1RT4003681@www.freebsd.org>
Date: Mon, 24 Jan 2005 19:01:27 GMT
From: Rusty Nejdl <rnejdl@ringofsaturn.com>
To: freebsd-gnats-submit@FreeBSD.org
Subject: 460.status-mail-rejects shows destination domain instead of source IP
X-Send-Pr-Version: www-2.3

>Number:         76626
>Category:       conf
>Synopsis:       [patch] 460.status-mail-rejects shows destination domain instead of source IP
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    freebsd-bugs
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Mon Jan 24 19:10:21 GMT 2005
>Closed-Date:    
>Last-Modified:  Sat Jun 16 12:40:04 GMT 2007
>Originator:     Rusty Nejdl
>Release:        5.3
>Organization:
>Environment:
[tethys]:/home/rnejdl> uname -a
FreeBSD tethys.ringofsaturn.com 5.3-STABLE FreeBSD 5.3-STABLE #0: Sat Jan 22 10:38:22 CST 2005     root@tethys.ringofsaturn.com:/usr/obj/usr/src/sys/SATURN  i386
[tethys]:/home/rnejdl>

>Description:
/usr/src/etc/periodic/daily/460.status-mail-rejects

Checking for rejected mail hosts:
  25 atshaw.com (451... resolve)
  24 EMAILHOSTER.COM (550... http://www.spamhaus.org/SBL)
  22 cohesionventures.com (550... denied)
  19 cohesionventures.com (550... server)
  18 matronics.com (550... denied)
  16 ringofsaturn.com (550... server)
  15 atshaw.com (550... denied)
  13 atshaw.com (550... server)
  12 ringofsaturn.com (550... denied)
   9 danicfinancial.com (451... resolve)
   6 cohesionventures.com (553... Corporation)
   5 ringofsaturn.com (553... Corporation)
   5 cohesionventures.com (550... http://www.spamhaus.org/SBL)
   5 atshaw.com (553... IP's)
   4 emailhoster.com (550... denied)
   4 ATSHAW.COM (550... http://www.spamhaus.org/SBL)
   3 tethys.ringofsaturn.com (550... denied)
   3 saturnconsulting.com (550... server)
   3 saturnconsulting.com (550... denied)
   3 cohesionventures.com (553... IP's)
   3 atshaw.com (553... Corporation)
   3 atshaw.com (553... Clients)
   2 tethys.ringofsaturn.com (553... IP's)
   2 ringofsaturn.com (553... IP's)
   2 ringofsaturn.com (553... Brazil)
   2 ringofsaturn.com (550... http://www.spamhaus.org/SBL)
   2 emailhoster.com (550... server)
   2 cohesionventures.com (553... #Spammer)
   2 authentickungfudallas.com (550... server)
   2 atshaw.com (553... Users)
   1 ringofsaturn.com (553... exist)
   1 ringofsaturn.com (550... 218.219.154.210)
   1 ringofsaturn.com (550... 204.9.210.123)
   1 ringofsaturn.com (451... resolve)
   1 ringo.fsbusiness.co.uk (550... [61.11.26.142])
   1 hydrolawn.com (553... IP's)
   1 hydrolawn.com (550... server)
   1 helixdfw.com (553... IP's)
   1 emailhoster.com (553... IP's)
   1 emailhoster.com (553... Brazil)
   1 emailhoster.com (550... 64.14.48.142)
   1 emailhoster.com (550... 64.14.48.133)
   1 dinhglobal.com (550... server)
   1 cohesionventures.com (553... users)
   1 cohesionventures.com (553... exist)
   1 cohesionventures.com (553... bounced.)
   1 cohesionventures.com (553... Brazil)
   1 authentickungfudallas.com (553... Spammer)
   1 authentickungfudallas.com (553... Brazil)
   1 authentickungfudallas.com (550... denied)
   1 atshawdot.ca (550... [62.14.104.36])
   1 atshawdot.ca (550... [61.11.26.142])
   1 atshaw.dotca (550... [202.54.51.5])
   1 atshaw.com (553... exist)
   1 atshaw.com (553... Spammer)
   1 atshaw.com (553... #Spammer)

This is a list of the destination domains.  I want to see instead a list of the hosts that have been rejected.  
>How-To-Repeat:
Simply execute the command with a default sendmail installation.  Here's an example of a reject line:

Jan 24 12:58:17 tethys sm-mta[79791]: j0OIviDL079791: ruleset=check_rcpt, arg1=<atshaw@atshaw.com>, relay=[210.187.94.17], reject=550 5.7.1 <atshaw@atshaw.com>... Fix reverse DNS for 210.187.94.17,or use your ISP server

The relay should be shown by periodic script, not atshaw.com.
>Fix:
I have solved the problem using gawk, which isn't acceptible for normal installs as gawk is a port.  However, perhaps this solution can be adapted to work correctly for the normal install.

[tethys]:/home/rnejdl> diff -u /etc/periodic/daily/460.status-mail-rejects /usr/src/etc/periodic/daily/460.status-mail-rejects
--- /etc/periodic/daily/460.status-mail-rejects Sun Oct 10 13:13:34 2004
+++ /usr/src/etc/periodic/daily/460.status-mail-rejects Mon Jan 24 12:55:07 2005
@@ -1,6 +1,6 @@
 #!/bin/sh
 #
-# $FreeBSD: src/etc/periodic/daily/460.status-mail-rejects,v 1.8.2.5 2002/05/13 21:36:44 brian Exp $
+# $FreeBSD: src/etc/periodic/daily/460.status-mail-rejects,v 1.16.4.1 2005/01/24 14:44:47 brian Exp $
 #

 # If there is a global system configuration file, suck it in.
@@ -32,57 +32,27 @@
            echo
            echo Checking for rejected mail hosts:

-       #    rc=$({
-       #       for f in `find /var/log -name maillog\* \
-       #               \( -mtime 1 -o -mtime 2 \) | xargs ls -tr`
-       #       do
-       #               case $f in
-       #                       *.gz)   zcat -fc $f;;
-       #                       *.bz2)  bzip2 -cd $f;;
-       #                       *)      cat $f;;
-       #               esac
-       #       done
-           start=`date -v-1d '+%b %d' | sed 's/0\(.\)$/ \1/'`
+           start=`date -v-1d '+%b %e'`
            n=$(($daily_status_mail_rejects_logs - 2))
            rc=$({
                while [ $n -ge 0 ]
-                do
-                    if [ -f /var/log/maillog.$n ]
-                    then
-                        cat /var/log/maillog.$n
-                    elif [ -f /var/log/maillog.$n.gz ]
-                    then
-                        zcat -fc /var/log/maillog.$n.gz
-                    elif [ -f /var/log/maillog.$n.bz2 ]
-                    then
-                        bzcat -fc /var/log/maillog.$n.bz2
-                    fi
-                    n=$(($n - 1))
-                done
-                cat /var/log/maillog
-           } | /usr/local/bin/gawk '
-               BEGIN {
-                   today=systime();
-                   yesterday=strftime("%b %d", today-86400);
-                   today=strftime("%b %d", today);
-                   gsub(" 0", "  ", today); gsub(" 0", "  ", yesterday);
-               }
-               {
-                   relay=gensub("^" yesterday ".*, relay=([^,]+), reject=.*",
-                               "\\1", 1);
-                   if (relay != $0)
-                       rejects[relay]++;
-                   else if (match($0, "^" today))
-                       exit;
-               }
-               END {
-                   for (relay in rejects) {
-                       printf("%4d %s\n", rejects[relay], relay);
-                       total += rejects[relay];
-                   }
-                   if (total > 0)
-                       printf("%4d TOTAL\n", total);
-               }' | sort -fnr | tee /dev/stderr | wc -l)
+               do
+                   if [ -f /var/log/maillog.$n ]
+                   then
+                       cat /var/log/maillog.$n
+                   elif [ -f /var/log/maillog.$n.gz ]
+                   then
+                       zcat -fc /var/log/maillog.$n.gz
+                   elif [ -f /var/log/maillog.$n.bz2 ]
+                   then
+                       bzcat -fc /var/log/maillog.$n.bz2
+                   fi
+                   n=$(($n - 1))
+               done
+               cat /var/log/maillog
+           } |
+               sed -n -E "s/^$start"'.*ruleset=check_[^ ]+, +arg1=<?([^@]+@)?([^>,]+).*reject=([^ ]+) .* ([^ ]+)$/\2 (\3... \4)/p' |
+               sort -f | uniq -ic | sort -fnr | tee /dev/stderr | wc -l)
            [ $rc -gt 0 ] && rc=1
        fi;;

Exit 1

>Release-Note:
>Audit-Trail:

From: Gregory Shapiro <gshapiro@freebsd.org>
To: bug-followup@freebsd.org, rnejdl@ringofsaturn.com
Cc:  
Subject: Re: conf/76626: [patch] 460.status-mail-rejects shows destination
	domain instead of source IP
Date: Thu, 14 Jun 2007 21:00:04 -0700

 Your patch assumes that only the relay= is of interest in the list.
 However, in my opinion, the address you are rejecting is more interesting
 in most cases.  For example, from my own logs:
 
 i un 14 00:01:32 gir sm-mta[9280]: l5E71S9N009280: ruleset=check_mail, arg1=<tzdelhi@netbizmoms.com>, relay=ip-51.net-82-216-27.versailles2.rev.numericable.fr [82.216.27.51], reject=451 4.1.8 Domain of sender address tzdelhi@netbizmoms.com does not resolve
 
 Jun 14 00:05:17 gir sm-mta[9349]: l5E75ErZ009349: ruleset=check_rcpt, arg1=<benco@example.com>, relay=ful.cnchost.com [297.157.49.28], reject=400 4.0.0 Temporary failure
 
 Jun 14 00:12:13 gir sm-mta[9552]: l5E7C812009552: ruleset=check_mail, arg1=<newtripod.com@wonfuproductions.com>, relay=dsl081-247-036.sfo1.dsl.speakeasy.net [64.81.247.36], reject=450 4.1.2 <newtripod.com@wonfuproductions.com>... MX lookup failure for wonfuproductions.com
 
 In all three cases, I'm more interested in the address that was rejected
 instead of the host sending that mail.

From: "Rusty Nejdl" <rnejdl@ringofsaturn.com>
To: "Gregory Shapiro" <gshapiro@freebsd.org>
Cc: bug-followup@freebsd.org, rnejdl@ringofsaturn.com
Subject: Re: conf/76626: [patch] 460.status-mail-rejects shows destination 
     domain instead of source IP
Date: Sat, 16 Jun 2007 07:01:19 -0500 (CDT)

 Gregory,
 
 Well, to me, since most of the time when I am rejecting emails, the sender
 email address is spoofed.  I am definitely more interested in knowing what
 ISP's are spamming me most.
 
 [tethys]:/home/rnejdl> /etc/periodic/daily/460.status-mail-rejects
 
 Checking for rejected mail hosts:
  289 TOTAL
    4 [208.97.234.204]
    3 thisistoyou.com [208.66.235.120]
    3 dropspecials.com [69.30.230.84]
    3 [203.156.49.110]
    2 mx1.gatetowinner.com [64.71.164.137]
    2 hn.kd.dhcp [61.52.201.38] (may be forged)
    2 chhor.brillianticon.com [70.42.184.61]
 
 So, when I blocked 208.97.234.204, I managed to block 4 spams from that IP
 in the last 24 hours.
 
 To be honest, the display as it is shown below is of no use to me.  It
 would be great if we could have a way to configure it to show the field
 that you wanted displayed.
 
 Sincerely,
 Rusty Nejdl
 
 Gregory Shapiro wrote:
 > Your patch assumes that only the relay= is of interest in the list.
 > However, in my opinion, the address you are rejecting is more interesting
 > in most cases.  For example, from my own logs:
 >
 > i un 14 00:01:32 gir sm-mta[9280]: l5E71S9N009280: ruleset=check_mail,
 > arg1=<tzdelhi@netbizmoms.com>,
 > relay=ip-51.net-82-216-27.versailles2.rev.numericable.fr [82.216.27.51],
 > reject=451 4.1.8 Domain of sender address tzdelhi@netbizmoms.com does not
 > resolve
 >
 > Jun 14 00:05:17 gir sm-mta[9349]: l5E75ErZ009349: ruleset=check_rcpt,
 > arg1=<benco@example.com>, relay=ful.cnchost.com [297.157.49.28],
 > reject=400 4.0.0 Temporary failure
 >
 > Jun 14 00:12:13 gir sm-mta[9552]: l5E7C812009552: ruleset=check_mail,
 > arg1=<newtripod.com@wonfuproductions.com>,
 > relay=dsl081-247-036.sfo1.dsl.speakeasy.net [64.81.247.36], reject=450
 > 4.1.2 <newtripod.com@wonfuproductions.com>... MX lookup failure for
 > wonfuproductions.com
 >
 > In all three cases, I'm more interested in the address that was rejected
 > instead of the host sending that mail.
 >
 
 
>Unformatted:
