Subj : Squish dupe detection. To : Dieter Ringhofer From : mark lewis Date : Fri Nov 29 2002 07:25 pm ml>> i don't know what version you looked at but there ml>> were specific changes made to MSGID stuff in GIGO... DR> I had a look at released binaries as well as at source code DR> (must be somewhere on tape and harddisk). what version of the source code? it might be something that could help unless it is after such was released to the public... DR> I could also watch what happens when some newbie activated DR> a gate using Gigo. hunh? ml>> FSC-0035 is supported... ml>> SAFE_THREADS maintains a local database of MSGID, References and ml>> Newsgroups instead of trying to store all this in the FTN control ml>> lines... it allows for safer message threading and dupe checking ml>> at the cost of ~150 bytes per message... there is a maint program ml>> to be run against thedatabase periodically... DR> Here you have the reason when you don't have an "encapsulated DR> circuit" for messages being gated with Gigo. In the very DR> moment such a message leaves that circuit (comes to another DR> system which carries original message) it is a dupe but cannot DR> be identified. ahhh... that would indicate that SAFE_THREADS is only good for local bases and not echo areas... ml>> the May 24, 1994 version contained specific work done for squish ml>> problems... ml>> [quote] ml>> MSGID and Message-ID stuff worked on, trying to remove ml>> an incompatability with Squish 1.10's dupe checker. ml>> Also trying to get messages from internet->fidonet->internet ml>> to re-use the original Message-ID: field, so that dupe checkers ml>> on the net will catch it (Apparently, at least one country ml>> has a problem with dupe loops between the gates and the net). ml>> MSGID's when using the NIKE option are now "midxxxxxxxxxxxxx" ml>> instead of "" .. The brackets were causing ml>> the new Squish 1.10 to think everything was a dupe. Great, eh? ml>> [/quote] DR> This assumption maybe is ... hmm... not fully correct. Squish DR> 1.10 needs some specific setup to work correct (see my DR> previous posting about this). yes, i'm aware that squish has numerous shortcomings... DR> I fear even Scott didn't know what's possible to do with his DR> software. :-)) I met him after release of Max v3 and Squish DR> v1.1. He looked very astonished when I showed him setup of my DR> system and explained what I did to achieve something. He saw DR> it while sitting beside me some hundred kilometers away and DR> having a beer (or two). <> DR>>> FTN1 - Gigo - RFC - Gigo - FTN2 DR>>> When having a close look at this posting you'll encounter DR>>> THREE different msgids in dependancy on where you pick it: DR>>> Now you can link systems FTN1 and FTN2 (they feed each other DR>>> directly or via many other systems). What happens? ml>> a dupe loop that would happen just like it would for echomail... DR> With normal echomail this is no real problem because recipient DR> system is able to identify dupes when msgid is not altered. DR> Such a dupe will not be exported anymore. true... that is why there was developed a method that all systems would "calculate" the same info for the MSGID... however, i seem to recall that it was a bit "short" in its working... DR>>> You will have a mail running in a never ending loop! ml>> until someone notices it and breaks the loop just like already ml>> has to be done with some echomail loops... DR> Even with routing structure over here I never had need to DR> interfere manually as long as msgid has been intact. that is a GoodThing OB-) DR>>> Several years ago author of Gigo has been asked to fix this DR>>> behaviour and he refused to do it. ml>> i remember something about some of this but i never got signed ml>> up to the support areas before jason left fidonet and other ml>> things that took him away from us... however, i do specifically ml>> remember something about some "formula" being used that would ml>> create the same msgid on all systems pulling the same ml>> message... however i don't recall what ever happened with ml>> that... DR> He refused and Gigo became kind of banned in Germany because DR> we flamed sysops using it almost to death. Those Gigo gates DR> caused real problems due to heavy load of some servers at this DR> time. understood... at that time, though, things were really in flux and there was no set standard... i remember several proposals on ways of doing MSGIDs but i don't know how well they really worked... it wasn't long after that that jason was "headed out the door" for numerous reasons... ml>> no matter... if GIGO is put on sourceforge someone can still ml>> fix this problem if it is still a problem... DR> This will be a problem as long as original msgid is not DR> preserved and included as such in any message. so you are recommending "converting" the original RFC Message-ID to the "originating system" portion of the MSGID? then what to do for the "serial number" portion? or just carry the original Message-ID in full as another control line? DR>>> Over here in Germany you'll encounter many FTN-RFC gateways DR>>> but, AFAIK most of them use Fidogate or parts of it at least. DR>>> Any gate will be closed very fast when first dupe is DR>>> encountered. ml>> the same happens (or should) when any echomail dupe loop is ml>> discovered... DR> There's no real distribution when software being used for DR> gating works as it should be. there is if an area is being passed from one system (the gateway) to numerous others... isn't there? ml>> however, we must also note that some dupe loops are by design... ml>> consider a fully connected backbone network... ml>> hubA=========hubB ml>> | \ / | ml>> | \ / | ml>> | \ / | ml>> | \ / | ml>> | \ / | ml>> | X | ml>> | / \ | ml>> | / \ | ml>> | / \ | ml>> | / \ | ml>> | / \ | ml>> hubC=========hubD DR> No problem at all. We did similar over here for long time to DR> ensure and speed up distribution of mail. Dupe prevention is DR> done when each hub feeds specific systems which have DR> absolutely no feed in direction to another hub. It needs some DR> discipline at downlink side but works fine when this is given. this is how both (Z1B and NAB) distribution systems are connectied here in Z1... only they each have more than four systems as shown above in that simple diagram... ml>> dupes are a way of life and required to ensure proper message ml>> propogation... however, those dupes should not be sent to ml>> systems linkedoff each hub... DR> Change msgid and things will become much more worse. :-) i'm sure... we have that problem from time to time with fubar software still in use... wildcat and PC Board stuff used to be the worst... wildcat especially after an upgrade to a newer version... it used to export entire message bases back into the distribution system... DR>>> looks like a chaotical knot. This chaotical system works fine DR>>> since years and: Nobody can destroy it (we had such an DR>>> exercise in 1993). ml>> i heard of that incident... glad the system is robust enough ml>> to handle such... DR> People have been robust enough and kept it alive. excellent! )\/(ark * Origin: (1:3634/12) .