The following is an edited transcript of the special GEnie IBM RoundTable
Real Time Conference held June 15, 1986 to discuss micro communication
protocols, especially Xmodem, Ymodem, and Zmodem.  Chuck Forsberg,
communications consultant, author of Professional YAM, and originator
of the Ymodem and Zmodem protocols was the guest of honor.

-----

Welcome to the IBM PC Roundtable.
Your RoundTable Leaders are:
    Paul Homchick (GE Mail: HOMCHICK)
    David Kozinn  (GE Mail: D.KOZINN)
and Charles Strom (GE Mail: STROM)
 
**************************************
**      BIG DEAL RTC devoted to     **
**  Communications Protocols with   **
**  special guest Chuck Forsberg    **
**  author of PRO-YAM, originator   **
**  of YMODEM and ZMODEM protocols  **
**    >TONITE< at 8:30 P.M. EDT     **
**************************************
 
<[chas] STROM> Many moons ago (actually only a few, perhaps 1978 or
1979), Ward Christensen decided that there was a need to develop a file
transfer protocol usable for micro to micro communications.  He did
this in a vacuum, not knowing the way it was done on mainframes, and
xmodem (really correctly called Christensen protocol) was born.  There
have been a succession of "improvements" and frills culminating in some
real added functionality known as YMODEM and now ZMODEM.  Take it away
Paul

<[Paul] HOMCHICK> Xmodem is a receiver-driven protocol, by that I mean
that the receiver controls the handshaking, it works like this.  When
you want to start a transfer, you must start programs up on each end,
it is not automatic like kermit and some other mainframe protocols.
The sending end simply waits for the receiver to send it something it
recognizes.

The original XMODEM was very simple, it had a vocabulary of two: an ACK
(acknowledge) and a NAK (neg ack) So it started out by sending a series
of NAKs to the sender, meaning I haven't gotten anything yet, go ahead.
When the receiver saw a NAK, it sent its first block, which in XMODEM
was 128 bytes surrounded by some control bytes.  128 seemed like a good
number because it was the same as the sector size in CP/M -- the only
micro operating system around at that time -- specifically, it sent a
SOH (start of header) the block number (example: 1) the block number
compliment in 8 bit arithmetic (255) the 128 bytes of data, and an
8-bit checksum of the 128 byte data stream.  The checksum was one of
the failings of early xmodem because is was a simple modulo 255 sum of
all the bytes.  It was easy to have two different data streams come up
with the same checksum, so errors could creep in.

On the receiving end, it looks for the SOH, the sector number and the
compliment, does its own checksum on the data and compares it to the
checksum byte it got.  If all was OK, it sends an ACK, if not: a NAK,
and the sender is supposed to re-send.  Now (big breath...) When we are
all finished, the sender transmits an EOT (end of transmission), and
that would tell the receiver that was all there was.  The problems with
all of this were....

  1) the checksum was not too accurate, and

  2) noise on the line could send a false EOT to the receiver and
     prematurely cancel the transfer... usually about 250K into a
     300K xfer (heh heh).

It's interesting to note that only just now are commercial programs
beginning to offer CRC instead of checksum, which brings up the next
XMODEM improvement.

Any questions so far?

<[Mike] SCHEINBERG> If checksum was a failure, then why is checksum
used alongside CRC for XMODEM?  Also, what is the difference between
checksum and CRC?

<[Paul] HOMCHICK> Checksum is used because it was the only method
officially documented by Ward Christensen for a long time, even though
the PD programs were using CRC....

<[Mike] SCHEINBERG> So CRC is still not official?

<[Paul] HOMCHICK> Ward has said that he wishes he had taken a few days
longer to develope the protocol and not used the simple checksum, as
the differences.  CRC *IS official.  In the YMODEM doc, which has
Ward's latest thinking on the subject, he endorses CRC.

The CRC is a polynomial.  Can chas or david help with what it is
exactly?  If not, I'll try from memory.

<[David] D.KOZINN> It's a CCITT Polynomial, that's about all I
remember.

<[Peter] P.HANDSMAN> One major improvement in ANY protocol would be a
'start from any block' I don't know just how many times I lost a file
after 400 blocks, is this being considered?

<[Paul] HOMCHICK> Zmodem has that capability. but let me talk about the
CRC real quick.

<[Paul] HOMCHICK> Chuck, we were just discussing the history of micro
protocols, had got to the introduction of CRC, care to carry on with
CRC and Ymodem, then Modem?

<[Chuck] CAF> well - what is there to say about CRC -

<[Paul] HOMCHICK> I mentioned the problems with the single EOT and the
error in Checksum.

<[Chuck] CAF> Most of the errors in XMODEM don't involve the CRC, but
the weak spots in the protocol that have been discussed previously.
YMODEM isn't any better, it's faster in networked situations because of
the longer block length. Of course, on Genie having XMODEM in the nodes
helps quite a bit.  Then, earlier this year, one of the VAN packet
switched networks commissioned me to develop ZMODEM.

<[Paul] HOMCHICK> Ulp, too fast!  Please explain Ymodem.

<[Chuck] CAF> well, there are several "levels" of YMODEM, YMODEM is a
set of extensions to XMODEM.  First historically was the batch
transmission, with the file name sent in a block number 0.  Next, when
I was downloading files from Unix systems, came the 1k blocks.  The 1k
blocks are signified by starting each block with 002 instead of 001
(SOH).  Full YMODEM also sends and uses the file length and
modification date in the header packet.  This allows the receiver to
throw away the extra garbage that XMODEM based protocols otherwise add
to files at the end.  Questions??

<[Van] AVFLOWERS> Ok if you increased the size of the Xmodem blocks
from 128 to a bigger number would the transfer be any faster?, if so
why don't we do that.

<[Chuck] CAF> With Xmodem in the nodes, the speed difference isn't so
great.  Perhaps it would be better to go all the way to ZMODEM, which
gives other advantages.  Next ??

<[don] D.GOLDMAN> On Crosstalk, I have the option of selecting the
block size.  What should I use?

<[Chuck] CAF> Crosstalk's proprietary protocol is incompatible with
XMODEM or anything that is in the public domain.  Their XMODEM protocol
is rather primitive.  It doesn't even support CRC.  next??

<[charles] C.DENGLERE> my question is there not any type of automatic
file transfer protocol that will set the block size to the most
effective transfer size and use that as a means of sending the file?

<[Chuck] CAF> The most effective block size is not always something
that the programs can divine.  More comments when we get to ZMODEM.
next??

<[KEN] KWSCHILDT> I am trying to use a peek to read a memory location
to verify the carrier detect on a serial port?  any suggestions?

<[Chuck] CAF> the carrier status is not stored in any memory location.
You need to do an input instruction to sample the carrier detect bit in
the 8250 chip.  There may be other methods applicable to Basic I don't
know of.  I don't know the syntax, but do recall there was some sort of
in and out instructions in basica.  Would have to do some research and
experimentation to find out more.  next??

<[David] D.KOZINN> Ok, Chuck, please tell us about Modem.

<[Chuck] CAF> Early this year one of the packet switched networks
commissioned me to develop a successor protocol to XMODEM/YMODEM that
was suitable for operation over packet switched networks.  They have
been unhappy with the low throughput users were getting with XMODEM,
and the 1k blocks of YMODEM were not a total solution. Kermit wasn't
the greatest (Kermit Sliding Windows) because of high overhead and
inability of using sliding windows on some types of operating systems.
The first thinking was to make more changes to XMODEM, but this seemed
less and less desirable as the planning went on.  You've heard the old
saw about the pig's ear and silk purse ... So, I reset to zero, so to
speak. I wanted to refrain from new technology, so used basic routines
that would be available to one used to writing XMODEM/CRC programs.

The structure of blocks is completely different, EVERYTHING is
protected by CRC-16.  Well, almost everything, five Ctrl-X in a row is
recognized as an abort so one can keyboard a cancel sequence.  I don't
want to get in to too much detail here, but there are two types of
header packets, binary and hex.  The receiver only sends the hex type,
since flow control (X-on, X-off) may be in effect.  The sender can send
either.  A binary data packet - one or more - can follow certain binary
header packets.  Here is where things get interesting.  The sender
sends an initial hex packet that wakes up the receiver, if the receiver
isn't already running.  The receiver then sends a receive-init packet,
which tells the sender what the receiver can handle (more later).  The
sender then (in simplest form) sends a FILE header packet, followed
with a binary data packet containing the file name, length,
modification date, etc.  The length is used for display only, the
actual file length is that which is sent. The receiver responds to
this, then the sender sends a DATA header and data packets.  There are
four things that may happen at the end of each data packet (assuming no
errors):

1.  The receiver sends an ACK and waits for the next header

2.  The receiver sends an ACK, and accepts more data packets without
    pause

3.  The receiver sends nothing, and accepts further data packets

4.  The receiver sends nothing, waits for another header.

Here's where the "magic" starts..........

If the receiver told the sender it can write to disk and do serial i/o
at the same time. the sender keeps pumping data packets without waiting
for any ACK.  In event of error, the receiver interrupts the sender
with a header containing the byte count it has received OK.  The sender
seeks to that location in the file, and resumes. If the receiver
indicated inability to do serial i/o and disk i/o, the sender tells the
receiver to ACK each N bytes, where N was sent by the receiver to the
sender in the receive init header.  This mode of operation is similar
to YMODEM. Since a number of data packets may form a frame (beginning
with the DATA header) there are some possibilities for tradeoffs.

<[chas] STROM> how can N range?

<[Chuck] CAF> N is sent to the sender , 0 implies infinite buffer,
anything up to 32k gives the length before one must force an ACK
response.

Back to packets... When using 1200 baud, the default is 256 byte
packets, so an error is detected usually within 256 bytes, and
transmission resumes.  When the sender gets to the end of the file, it
sends a ZEOF header with the ending byte position.  If the receiver
agrees, it closes the file, otherwise tells the sender to go back to
the last good byte.  Any questions so far??

<[Paul] HOMCHICK> how hard is this to implement, could it be popular
with modem program writers?

<[Chuck] CAF> The goal was to make ZMODEM easy to implement, since that
was the main reason for XMODEM being so popular.  I'm not as happy with
the complexity as one could wish, there seems to be a minimum level of
complexity of protocol that will give RELIABLE transfers and also
accommodate the various environments.  I could counter that XMODEM is
very difficult to implement a robust accurate version, there are a
number of secret tricks in Pro-YAM X/YMODEM to improve the robustness
(ability to finish a transfer) and integrity/accuracy.  Most programs
are not too forgiving of poor phone lines.  Providing C code should be
a help, and I don't use any new CRC which would cause a problem.

<[don] D.GOLDMAN> is error checking the same with xmodem as modem?

<[Chuck] CAF> ZMODEM uses the same CRC-16 that XMODEM CRC uses.  It is
used to protect everything, not just the data, so ZMODEM is more robust
and accurate than XMODEM without requiring Pro-YAM's tricks. next?

<[don] D.GOLDMAN> since the block size id larger, how is error checking
affected

<[Chuck] CAF> The CRC-16 polynomial used is considered reliable for
block lengths up to about 4k.  The longest packet length in ZMODEM is
1k, and may be shortened to 64 without too much loss in efficiency.
The tradeoff here is speed of recovery from line hits versus overhead
of send CRC more often.

<[Paul] HOMCHICK> still, since there is no ack/nak turnaround delay,
the efficiency can be kept high.

<[Mike] SCHEINBERG> Would ZMODEM be able to have one computer be called
and then at one appropriate time be able to send DATA without any
problems? (Because It automatically sends packets, and automatic
transfer starts.)

<[Chuck] CAF> On automatic calling, ZMODEM is very well suited for
that, but that is also a function of the comm program that supports the
protocol also. next??

<[chas] STROM> is the modem protocol description in the public domain?
I know I have capability in Pro-Yam...  but where can I call (Modem at
other end)?

<[Chuck] CAF> ZMODEM protocol is totally PD.

<[Chuck] CAF> Telegodzilla runs ZMODEM, [and soon we may see] ...
PIBTERM and Red Ryder.  Also, the YAMDEMO.EXE program can be called
with the DOS EXEC function from a BBS program for implementing ZMODEM
transfer.

<[chas] STROM> (telegodzilla is Chuck's dial-in system)

<[Paul] HOMCHICK> Red Ryder?  have you been talking to Scott Watson?

<[Chuck] CAF> Yes, Scott gave me a call the other day.

<[Paul] HOMCHICK> (Red Ryder is the best MacIntosh comm program)

<[Chuck] CAF> Note I said possibly.

<[Paul] HOMCHICK> how about the network that commissioned it?

<[Chuck] CAF> ZMODEM is an application-to-application program, the
network is transparent to it.

<[tom] TDAOUST> Robust Schmobust. A program is more efficient, faster,
more accurate, more reliable but robust?  I had a woman that was
robust, but she was none of the above.  Hmmm

<[Chuck] CAF> Ever had a XMODEM transfer quit because of line hits?
That's what robustness (or the lack of it) is all about.

<[Al M] SERVU> is ZMODEM cp/m, dos or both

<[Chuck] CAF> It's mainly DOS and Unix, but could be implemented on
CP/M. However, since most -nearly all- CP/M systems can't do disk
writes while receiving data from the phone line, it can't do streaming.

<[Al M] SERVU> it can be used by XENEX V then?

<[Chuck] CAF> ZMODEM is actually four protocols in one, not counting
fallback to YMODEM.  The distributed rz and sz programs are for Unix
SYS III/V and 4.x BSD.

<[Al M] SERVU> is it available and if so where?

<[Chuck] CAF> The current version of these programs are in rzsz[12].shq
on TeleGodzilla, and have been out on Usenet.  Guess I need to upload
them.

<[Paul] HOMCHICK> that was my question, we will get rz and sz and put
them here if OK with CAF.

<[Paul] HOMCHICK> Are they good on VMS, too?

<[Chuck] CAF> As for VMS, I can't get any doco on the QIO calls needed
to do the sampling.

I might mention, The most popular aspects of ZMODEM are, 1) Only one
command needed to transfer file(s) or download commands 2) ability to
recover if a file transfer is interrupted, without having to start
over, 3) insensitive to network delays, 4) robust transfers and rapid
recovery from line hits. next ???

<JOHNCOOPER> CAF, I have all the docs DEC publishes on VMS's QIOs

<[Chuck] CAF> I could use info on QIO's for all the tty line magic.
cd $Y head y1.c

<[David] D.KOZINN> Well, that about wraps up the formal part of the
conference, I'd like to thank Chuck for taking the time to drop in.

<[Chuck] CAF> ok CUL.

** End **
.