Subj : dupe checking
To   : All
From : Markus Reschke
Date : Sun Feb 04 2018 14:49:26

Hi!

Currently I'm looking into a dupe checking issue when an echomail lacks the 
MSGID. From hpt/src/dupe.c I've learned that hpt creates a pseudo MSGID for 
dupe checking if the message doesn't include one:

int dupeDetection(s_area *area, const s_message msg) {
....
    str = (char*)MsgGetCtrlToken((byte*)msg.text, (byte*)"MSGID:");

    if (str == NULL) 
    {   /* Kluge MSGID is not found so make it from crc of message body */
        if (msg.text)
        {
            char *hbuf=NULL;
            xstrscat(&hbuf,msg.text,msg.fromUserName,msg.datetime,msg.toUserName,msg.subje ctLine,NULLP);
            xscatprintf (&str, "MSGID: %08lx",strcrc32(hbuf, 0xFFFFFFFFL));
            nfree(hbuf);
        }
        else 
            return 1; /*  without msg.text - message is empty, no dupeCheck */
    }

The pseudo MSGID is created by a hash of the message text, fromUserName, 
DateTime, toUserName and subject. Please correct me if I'm wrong. The message 
text includes all the kludge lines like MSGID and also SEEN-BY and PATH. When I 
receive the same echomail from two or more echo links and the mail doesn't have 
a MSGID hpt would create a pseudo MSGID for all copies of the mail. And since 
every copy of the mail has different PATH and SEEN-BY lines each pseudo MSGID 
would be different. The result is that all copies of the echomail are 
considered to be different, i.e. no dupe found. I think PATH and SEEN-BY lines 
should be removed from the message text before creating the hash. This could be 
done by temporily exchanging the last char before the first SEEN-BY with a 0. 
This way the same pseudo MSGID would be created for each mail copy and the 
copies would be detected as dupes. What do you think?

ciao,
Markus

--- 
 * Origin: *** theca tabellaria *** (2:240/1661)

.