Subj : Squish dupe detection. To : Mark Lewis From : Dieter Ringhofer Date : Fri Nov 29 2002 09:58 pm Hi Mark! DR>>>> Gigo is known to be a dupe catapult since many years. ml>>> what's wrong with the KEEP_THREADS option? DR>> Sorry, I'm not familiar with Gigo and don't know what this DR>> features offers for this. I have put it into trash after DR>> having a look at it many years ago due to it's msgid handling. ml> i don't know what version you looked at but there were specific changes ml> made to MSGID stuff in GIGO... I had a look at released binaries as well as at source code (must be somewhere on tape and harddisk). I could also watch what happens when some newbie activated a gate using Gigo. ml> FSC-0035 is supported... ml> SAFE_THREADS maintains a local database of MSGID, References and ml> Newsgroups instead of trying to store all this in the FTN control lines... ml> it allows for safer message threading and dupe checking at the cost of ml> ~150 bytes per message... there is a maint program to be run against the ml> database periodically... Here you have the reason when you don't have an "encapsulated circuit" for messages being gated with Gigo. In the very moment such a message leaves that circuit (comes to another system which carries original message) it is a dupe but cannot be identified. ml> the May 24, 1994 version contained specific work done for squish ml> problems... ml> [quote] ml> MSGID and Message-ID stuff worked on, trying to remove ml> an incompatability with Squish 1.10's dupe checker. ml> Also trying to get messages from internet->fidonet->internet ml> to re-use the original Message-ID: field, so that dupe checkers ml> on the net will catch it (Apparently, at least one country ml> has a problem with dupe loops between the gates and the net). ml> MSGID's when using the NIKE option are now "midxxxxxxxxxxxxx" ml> instead of "" .. The brackets were causing ml> the new Squish 1.10 to think everything was a dupe. Great, eh? ml> [/quote] This assumption maybe is ... hmm... not fully correct. Squish 1.10 needs some specific setup to work correct (see my previous posting about this). I fear even Scott didn't know what's possible to do with his software. :-)) I met him after release of Max v3 and Squish v1.1. He looked very astonished when I showed him setup of my system and explained what I did to achieve something. He saw it while sitting beside me some hundred kilometers away and having a beer (or two). DR>> FTN1 - Gigo - RFC - Gigo - FTN2 DR>> When having a close look at this posting you'll encounter DR>> THREE different msgids in dependancy on where you pick it: DR>> Now you can link systems FTN1 and FTN2 (they feed each other DR>> directly or via many other systems). What happens? ml> a dupe loop that would happen just like it would for echomail... With normal echomail this is no real problem because recipient system is able to identify dupes when msgid is not altered. Such a dupe will not be exported anymore. DR>> You will have a mail running in a never ending loop! ml> until someone notices it and breaks the loop just like already has to be ml> done with some echomail loops... Even with routing structure over here I never had need to interfere manually as long as msgid has been intact. DR>> Several years ago author of Gigo has been asked to fix this DR>> behaviour and he refused to do it. ml> i remember something about some of this but i never got signed up to the ml> support areas before jason left fidonet and other things that took him ml> away from us... however, i do specifically remember something about some ml> "formula" being used that would create the same msgid on all systems ml> pulling the same message... however i don't recall what ever happened with ml> that... He refused and Gigo became kind of banned in Germany because we flamed sysops using it almost to death. Those Gigo gates caused real problems due to heavy load of some servers at this time. ml> no matter... if GIGO is put on sourceforge someone can still fix this ml> problem if it is still a problem... This will be a problem as long as original msgid is not preserved and included as such in any message. DR>> Over here in Germany you'll encounter many FTN-RFC gateways DR>> but, AFAIK most of them use Fidogate or parts of it at least. DR>> Any gate will be closed very fast when first dupe is DR>> encountered. ml> the same happens (or should) when any echomail dupe loop is discovered... There's no real distribution when software being used for gating works as it should be. ml> however, we must also note that some dupe loops are by design... ml> consider a fully connected backbone network... ml> hubA=========hubB ml> | \ / | ml> | \ / | ml> | \ / | ml> | \ / | ml> | \ / | ml> | X | ml> | / \ | ml> | / \ | ml> | / \ | ml> | / \ | ml> | / \ | ml> hubC=========hubD No problem at all. We did similar over here for long time to ensure and speed up distribution of mail. Dupe prevention is done when each hub feeds specific systems which have absolutely no feed in direction to another hub. It needs some discipline at downlink side but works fine when this is given. ml> dupes are a way of life and required to ensure proper message ml> propogation... however, those dupes should not be sent to systems linked ml> off each hub... Change msgid and things will become much more worse. :-) DR>> looks like a chaotical knot. This chaotical system works fine DR>> since years and: Nobody can destroy it (we had such an DR>> exercise in 1993). ml> i heard of that incident... glad the system is robust enough to handle ml> such... People have been robust enough and kept it alive. cu, Dieter --- * Origin: Wndos s god fr cmnicaton (2:2476/14) .