Document: FSC-0039=0A= Version: 04=0A= Date: 29-Sep-90=0A= =0A= =0A= A Type-2 Packet Extension Proposal=0A= Mark A. Howard 1:260/340@FidoNet=0A= =0A= Status of this document:=0A= ------------------------=0A= This FSC suggests a proposed protocol for the FidoNet(r) community,=0A= and requests discussion and suggestions for improvements. Distribution=0A= of this document is unlimited.=0A= =0A= Fido and FidoNet are registered marks of Tom Jennings and Fido = Software.=0A= FTS-0001 is a copyrighted work of Randy Bush.=0A= =0A= Introduction=0A= ------------=0A= This document serves two major purposes. The first is an attempt to = define=0A= and document the Type-2 packet which is widely in use in FidoNet today.=0A= Although FTS-0001 defines the structure of a Type-2 packet, the natural=0A= evolution of our network, mostly with regards to addressing = methodology,=0A= has made it necessary to utilize hitherto unused portions of the header=0A= to insert Zone and Point information. Also, it has become apparent = that=0A= some of the existing fields are not large enough to accomplish their=0A= original tasks.=0A= =0A= The second is to propose a simple mechanism to allow FidoNet to begin = to=0A= utilize advanced mail packing techniques. It is quite apparent that = while=0A= Type-2 has served us faithfully for some time now, the evolution of our=0A= network in terms of technical and physical complexity has caused us to=0A= consider more efficient and functional ways to pack mail.=0A= =0A= It should be made clear that with the exception of the Capability Word,=0A= Capability Word Validation Copy, ProductCode(hi), and Revision(minor),=0A= which are proposed extensions to the Type-2 packet header, this FSC is=0A= an attempt to correctly document existing practices with regards to the=0A= insertion of zone and point info by at least three mail processors in=0A= use today.=0A= =0A= =0A= The Type-2 Header (Where's the Zone?)=0A= -------------------------------------=0A= Although FTS-0001 has recently been updated to reflect the use of some = of=0A= the areas in the packed message header for zone data, at least two = other=0A= methods for inserting the zone information have been adopted, making it=0A= necessary to provide support for both formats in all of the zone aware = mail=0A= processors. The end result is more code, and redundant information in = the=0A= packet header.=0A= =0A= This has presented a problem in logistics, as it is difficult to = consider=0A= the project of updating mail processors using one type to use the = other.=0A= As sufficient indentification is provided, in the form of the product = code,=0A= to determine the expected location of the zone information, and because=0A= code already exists in most of the mail processors to overcome this, = this=0A= proposal does not attempt to suggest that one method be used over the=0A= other, rather the intent is to attempt to document the extensions in = use,=0A= and the products involved.=0A= =0A= See the accompanying chart and cross-reference.=0A= =0A= =0A= The Product Code=0A= ----------------=0A= Based upon the current rate of requests for product codes from the = FTSC,=0A= it is probable that the Product Code byte will not be large enough to=0A= accomodate all of the codes required. While it is not reasonable to=0A= expect that all Type-2 processors will eventually check the hi-order = byte=0A= proposed here, it is likely that 'current' mail processors will. This=0A= can be nothing but benefical, as it will force users to upgrade their=0A= mail processors to a product which will as a minimum, support Type-2=0A= with Zone and Point extensions, and quite possibly, processors that = will=0A= utilize more advanced mail packing techniques, making Type-2 extinct = once=0A= and for all.=0A= =0A= =0A= The Capability Word (How do we GET there from here?)=0A= -----------------------------------------------------=0A= Everybody would like to see more efficient and functional ways to pack = and=0A= exchange mail. Several Type-3 message bundle proposals exist, but none=0A= really address a problem which must be solved first. The problem is = that=0A= since FidoNet is a hobbyist network, no demands can be placed on any = one=0A= sysop to upgrade or change their bundling software. Because of this, = it=0A= is necessary to consider strategies which allow for the existence of = Type-2=0A= bundlers in the network topology.=0A= =0A= Considerable advantages can be realized, however, between systems that=0A= consent to use advanced bundling techniques. One way to do this is to=0A= simply send netmail to all of your connecting systems, saying "Hey, = I've=0A= got a Type-3 bundler now, how about you?" This could become quite=0A= tiresome, and does not represent much of an improvement on the current=0A= situation.=0A= =0A= What would be desirable is a network that would 'upgrade itself'. = Given a=0A= situation where mail processors of various capabilities will exist in a=0A= network topology, the goal is to provide a mechanism whereby two links = can=0A= determine and utilize the most efficient bundling method to use, in a=0A= manner transparent to the sysop.=0A= =0A= For instance, let's say that the FTSC releases the Type-7 All New = Singing=0A= and Dancing bundle format. Well, your current version of SlingToss can=0A= only support Types 2, 3, and 5. One of your downlinks gets a new = version=0A= of MailMangle which can support Types 2, 3, 4, and 7. Well, it is = quite=0A= obvious that since you and he are exchanging 4 megs of mail each night,=0A= and it's an overseas call, that it would be in your interest to obtain = a=0A= new version of SlingToss which can support Type 7.=0A= =0A= Note that this is *optional*. Because both processors can support = Type-3,=0A= they will continue to exchange Type-3 mail quite happily, even though=0A= MailMangle is happily advertising the availability of Type-7. Even = your=0A= downlinks which are still using stone-age Type-2 processors will be = fine,=0A= as SlingToss will always export Type-2 bundles when no higher = capability=0A= can be determined.=0A= =0A= So, after dashing off the check to the author, your new version of=0A= SlingToss comes in the mail! You rush over to your system, and = install it.=0A= The next time SlingToss exports mail to the MailMangle system, it says=0A= "Hey! I can now support Type 2, 3, 5, and 7! So, whattya got?" This = is=0A= no skin off MailMangle's nose, he's had Type-7 for quite a while, and = he=0A= begins to export Type-7 bundles to SlingToss. "It's about time.", he = says.=0A= =0A= Now, this scenario is made possible by implementing a 'Capability = Word' in=0A= the present and future packet headers. The Capability Update mechanism=0A= depends on several assumptions:=0A= =0A= 1) Any Advanced Capability Bundler *MUST* be capable of receiving and=0A= faithfully processing Type-2 bundles. Hopefully, the inbound = packets=0A= will be in the new format proposed by this document, but then again,=0A= this is not an exact science. What this means is that it is likely=0A= that some packets may arrive with the Capability Word (CW) set to 0.=0A= In this case, Type-2 is assumed, assuring compatibility. The only=0A= caveat is that it is conceivable that some obscure mail processor=0A= uses the location proposed for the CW for other arcane purposes. = This=0A= | can detected through the CWValidation Copy, which is byte-swapped = and=0A= | compared with the CW at that time. If a mismatch is found, a CW of=0A= | type 0 is assumed, and a Type-2 bundling method is used.=0A= =0A= 2) An Advanced Capability Bundler, hereafter referred to as a Type-N=0A= Bundler, must have a method to store and maintain the node-by-node=0A= capability information. This can be done many ways, and in fact=0A= several processors already have begun to maintain node information=0A= outside of that found in AREAS.BBS, mostly to implement pre-arranged=0A= alternate compression methods. In a text configuration file, you=0A= might see the following:=0A= =0A= ; Address Comp Send LastCW ; Comments=0A= Node 1:260/340 ZIP Auto 7 ; Auto detect & upgrade=0A= Node 1:135/20 LZH 3 2,3,7 ; Always send Type-3=0A= Node 1: ARC 2 0 ; Stone-Age processor=0A= Node 1:135/4 --- Auto 7 ; Sent me netmail=0A= Node 1: --- 0 0 ; Don't send CW=0A= =0A= In this example, the fields are:=0A= =0A= Address - downlink address. Note that this is not necessarily=0A= relative to echomail, only, it is possible to append=0A= information to the node database as netmail packets are=0A= receieved from different addresses.=0A= =0A= Comp - desired mail compression method.=0A= =0A= Send - Auto - automatically determined maximum common packing=0A= method to use. Automatically update to LastCW=0A= when packing.=0A= =0A= LastCW - Last CW received from remote system.=0A= =0A= =0A= 3) A Type-N Bundle will always advertise it's capabilities in the CW=0A= regardless of the type being sent. As shown in the above example,=0A= it allows Type-N processors to automatically track the capability=0A= of your system. Again, in cases where a stone-age processor is=0A= being used, this field will be ignored, and in the unusual event=0A= that it is not ignored, and is somehow harmful to the far system,=0A= the Type-N processor can be configured to send a CW of 0.=0A= =0A= The format of the Capability Word is designed to support up to 15 = future=0A= bundle types, and is bit-mapped to facilitate the easy determination of=0A= the maximum common level supported between two nodes:=0A= =0A= msb Capability Word lsb=0A= Node Supports ------------FTSC Type Supported----------------=0A= =0A= U 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2=0A= =0A= 2, 3, and 7 0 0 0 0 0 0 0 0 0 0 1 0 0 0 1 1=0A= 2, 3, and 5 0 0 0 0 0 0 0 0 0 0 0 0 1 0 1 1=0A= 2 (this FSC) 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1=0A= Stone Age** 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0=0A= ^=0A= +--- Indicates UseNet RFC-822 capability=0A= =0A= ** - a mismatch in the CWValidation Copy also=0A= produces a CW=3D0.=0A= =0A= In this example, the Type-N bundler would first compare the remote CW=0A= | and the byte-swapped remote CWValidation Copy, and check for a = mismatch.=0A= Prior to the compare, the MSB of the CW's are masked, as this bit is=0A= reserved to indicate whether the mail processor is capable of also=0A= accepting UseNet RFC-822 bundles. Following the MSB mask, and bit=0A= comparison, if they do not match, a remote CW of 0 is assumed. If they=0A= match, the Type-N processor would AND the local and remote CW words,=0A= obtaining a CW expressing the Types which are in common to both = systems.=0A= The most significant Type will be used, by default, but note that this=0A= assumes that new bundling Types will be increasingly more efficient or=0A= in some way more beneficial.=0A= =0A= Because this may not always be the case, there should be a method = provided,=0A= as illustrated above, to override the automatic upgrade should this = become=0A= the case.=0A= =0A= The MSB of the CW is used to indicate whether the mail processor can = accept=0A= UseNet RFC-822 bundles or not. It is a separate indicator, and not = intended=0A= to be used as a part of the above comparison, however can be used to = also=0A= advertise RFC-822 capability to other systems. Since RFC-822 is 'set = in=0A= stone', there is no need to assign more than one capability bit.=0A= =0A= It might seem somewhat limiting to only consider the possibility of 15=0A= different future bundling methods, but it is my opinion that the = careful=0A= use of a 'Sub-Type' byte in the packet header can allow for the = variations=0A= on a single theme, and that proposals for new formats should be = evaluated=0A= by the FTSC to determine whether sufficient benefit can be realized in=0A= the implementation of the given format, prior to assigning a new type=0A= code.=0A= =0A= =0A= Mailers=0A= -------=0A= It is quite clear to me that should this concept take hold, that the=0A= logical place to store node capability data is in the local nodelist=0A= database, or an index-associated data file. As above, it is necessary=0A= to generate Type-2 packets for whatever purpose, unless it is known=0A= by prior association, that the far mailer can accept other types of=0A= packets. It is also quite reasonable to assume that a nodelist flag=0A= could be assigned to advertise the CW for a given node, which the=0A= native mailer nodelist compiler could then immediately determine the=0A= preferred bundling method for any given node in FidoNet.=0A= =0A= Another possibility would be to pass a capability advertisement in the=0A= extensible portion of a handshake protocol, which may or may not = already=0A= exist in FidoNet.=0A= =0A= The approach suggested previously in this document suggests the use of=0A= a text configuration file, but it is quite obvious that many benefits=0A= can be realized through the use of the nodelist, including the use of=0A= additional flags to indicate the preferred compression method, etc.=0A= =0A= =0A= Summary=0A= -------=0A= This document has been created in an attempt to define a method to = allow=0A= the future expansion and enhancement of FidoNet technology mail = processors=0A= and mailers, and in a way that is the least disruptive to existing mail=0A= operations. The intent is to provide for an environment that is as = open,=0A= and extensible as possible.=0A= =0A= The mechanism described should allow many different types of processors=0A= (FTSC-registered) to exist in the network at once, and to provide an=0A= environment which is designed to operate at it's maximum efficiency=0A= wherever possible or practical.=0A= =0A= Revision 2 of this document was produced to implement suggestions made=0A= primarily by Jan Vroonhof, who suggested the use of the CW Validation=0A= Copy. Jan presented this idea in his FSC-0048, along with other = concepts=0A= relating to the correct indentification and handling of zone and point=0A= addressing. This document sanctions the improvements to the CW as=0A= recommended, but does not address or support the other extensions=0A= recommended in FSC-0048.=0A= =0A= =0A= Thanks=0A= ------=0A= To Ward Christensen, creator of XModem and BYE.=0A= =0A= Tom Jennings, who started this whole mess.=0A= =0A= Joaquim Homrighausen, for lots of good ideas, and motivation. = Here's=0A= another Lamborghini to work on.=0A= =0A= Wynn Wagner, Oliver McDonald, Roeland Meyer, Andrew Farmer, Claude=0A= Warren, Jan Vroonhof, Bob Hartman, and Vince Perriello, who all=0A= contributed in some way to the creation of this document, mostly=0A= through their messages in NET_DEV.=0A= =0A= =0A= =0A= Type-2 Packet Format (proposed, but currently in use)=0A= -----------------------------------------------------=0A= Field Ofs Siz Type Description Expected value(s)=0A= ------- --- --- ---- -------------------------- -----------------=0A= OrgNode 0x0 2 Word Origination node address 0-65535=0A= DstNode 2 2 Word Destination node address 1-65535=0A= Year 4 2 Int Year packet generated 19??-2???=0A= Month 6 2 Int Month " " 0-11 (0=3DJan)=0A= Day 8 2 Int Day " " 1-31=0A= Hour A 2 Int Hour " " 0-23=0A= Min C 2 Int Minute " " 0-59=0A= Sec E 2 Int Second " " 0-59=0A= Baud 10 2 Int Baud Rate (not in use) ????=0A= PktVer 12 2 Int Packet Version Always 2=0A= OrgNet 14 2 Word Origination net address 1-65535=0A= DstNet 16 2 Word Destination net address 1-65535=0A= PrdCodL 18 1 Byte FTSC Product Code (lo) 1-255=0A= * PVMajor 19 1 Byte FTSC Product Rev (major) 1-255=0A= Password 1A 8 Char Packet password A-Z,0-9=0A= * QOrgZone 22 2 Int Orig Zone (ZMailQ,QMail) 1-65535=0A= * QDstZone 24 2 Int Dest Zone (ZMailQ,QMail) 1-65535=0A= Filler 26 2 Byte Spare Change ?=0A= | * CapValid 28 2 Word CW Byte-Swapped Valid Copy BitField=0A= * PrdCodH 2A 1 Byte FTSC Product Code (hi) 1-255=0A= * PVMinor 2B 1 Byte FTSC Product Rev (minor) 1-255=0A= * CapWord 2C 2 Word Capability Word BitField=0A= * OrigZone 2E 2 Int Origination Zone 1-65535=0A= * DestZone 30 2 Int Destination Zone 1-65535=0A= * OrigPoint 32 2 Int Origination Point 1-65535=0A= * DestPoint 34 2 Int Destination Point 1-65535=0A= * ProdData 36 4 Long Product-specific data Whatever=0A= PktTerm 3A 2 Word Packet terminator 0000=0A= =0A= * - extensions to FTS-0001=0A= =0A= Ofs, Siz are in hex, other values are decimal.=0A= =0A= =0A= Zone/Point Aware Mail Processors (probably a partial list)=0A= ----------------------------------------------------------=0A= Prod=0A= Code Name - Uses QOrg/QDstZone Orig/DestZone Orig/DestPoint=0A= ---- ----------- ------------- ------------- --------------=0A= 0x0C FrontDoor Reads/Updates Yes Yes=0A= 0x1A DBridge ????? Yes Yes=0A= 0x45 XRS Reads/Updates Yes Yes=0A= 0x29 QMail Yes ????? Not point-aware=0A= 0x35 ZMailQ Yes ????? Not point-aware=0A= 0x3F TosScan Reads/Updates Yes Yes=0A= =0A= =0A= =0A= =0A= =0A=