Subj : Squish dupe detection.
To   : Dieter Ringhofer
From : mark lewis
Date : Thu Nov 28 2002 02:20 pm

 ML>> And yes, GIGO is broken in that when it receives an entire group of
 ML>> messages from one source, all at the same time and all having the
 ML>> same timestamp, it will assign them all the same MSGID, regardless of
 ML>> other header (subject:) content.

 DR> Gigo is known to be a dupe catapult since many years.

what's wrong with the KEEP_THREADS option?

 DR> Therefore it's recommended to get rid of it or to use it only
 DR> when you are not feeding any other system. When using Squish
 DR> you have some chances to catch real dupes and ignore most
 DR> false ones (AFAIR they are called nopes). Using any other
 DR> tosser you're out of luck.

 ML>> I'll try some batch file magic on the incoming .X and .D files to
 ML>> stagger each pair's timestamp and see if that helps out.  It will be
 ML>> quite tricky though.

 DR> I don't believe you will be very successfull. Instead of
 DR> wasting to much time with Gigo you might have a look at
 DR> Fidogate. You should be able to find binaries for Linux, BSD,
 DR> OS/2 and Win* (and complete source code).

the sources for GIGO are available and this stuff could be fixed... hummm... 
maybe i'll approach the author and see about him putting it on sourceforge for 
group development and refinement...

)\/(ark

 
* Origin: (1:3634/12)

.