			 SixPack File Transfer Protocol
			   Copyright 1996 Kaz Kylheku



Contents...

		1.	Introduction
		1.1	  SixPack Features
		2.	Installation
		2.1	  UNIX Systems
		2.2	  Customization
		3.	Using SixPack
		3.1	  Receiving a File
		3.2	  Sending a File
		3.3	  Limits
		3.4	  Tips
		3.4.1	    Killing SixPack
		3.4.2	    Adjusting the Buffer Size
		3.5	  SixPack over TCP/IP
		3.6	  SixPack with Minicom (and Other Programs)
		3.6.1	    Installation
		3.6.2	    Sending and Receiving
		4.	Bugs and Caveats
		5.	Licensing


1.	Introduction

In a nutshell, The SixPack File Transfer Protocol belongs to a family of
software whose members are implementations of Zmodem, Ymodem and the like. It
is a file transfer protocol intended primarily for moving data between two
computers over a raw serial link, and is intended to be a reliable, robust
replacement for these other protocols. SixPack consists of two distinct
programs: one for sending and one for receiving. These are executed on separate
computers and cooperate together to effect the transfer of a file from one
machine to the other.


1.1	SixPack Features

SixPack is a fairly simple sliding window protocol, implemented over a driver
which encodes all outgoing and incoming packets to and from a special encoding.
This encoding consists of only sixty-four printable ASCII characters chosen
from the set [A-Za-z0-9./] (not including the square brackets). These same
characters, incidentally, are used in the representation of encrypted passwords
in the /etc/passwd file on a UNIX system. 

SixPack can reassemble data segments received out of order, like robust
implementations of the Internet TCP protocol, and can deal with multiple
``holes'' (missing segments). It uses a time-out/retransmit discipline.

Information is exchanged in the form of variable-length packets which carry
full 32 bit checksums. Prior to transmission, packets are chopped up into six
bit units which are copied into individual bytes (hence the name SixPack), and
put through a translation table which maps six bit codes to the aforementioned
64 ASCII characters (whose encoding, ironically, requires seven bits).

SixPack currently does not have a lot of ``bells and whistles''. It cannot
transmit and reconstruct directory trees; in fact one execution of SixPack
transfers only one file. However, I don't think that you will complain when
SixPack performs well in a situation where other popular protocols completely
fail: tink of it like giving up some comfort and luxury to obtain the benefits
of a rugged ``off road'' vehicle.


2	Installation

SixPack was developed under Linux but has been ported to a few of other
UNIX systems with ease. It was written with portability in mind, and not just
portability to UNIX systems. Therefore, most it is written in very
portable ISO C language---any portability problems in its non-system-dependent
portions are to be considered bugs rather than design flaws. The external data
encoding routines even make sure that machines that use different
representations for signed integers can communicate correctly, though SixPack's
protocol doesn't take advantage of this because it doesn't use signed integers
in transmission :).  The ASCII encoding is performed in a character-set
independent way, using a table containing numeric constants, not character
literals whose execution-environment values depend on the C implementation. All
of the system-dependent details of actually setting up a communication link and
transmitting or receiving data are confined to separate translation units and
accessed through an abstract driver interface. Thus if you wish to port SixPack
to a operating system other than UNIX, your job should not be much more
difficult than writing a communication driver which obeys the abstract
interface and perhaps tweaking some of the interactive aspects of the program
(or pair of programs, rather), such as the handling of diagnostic messages and
command line arguments.


2.1	UNIX Systems

SixPack should compile ``out of the box'' under HP-UX 9 and Linux.  To build
it, simply go to the directory called ``code'' and type ``make''. The two
UNIX-specific files are ``posixtty.c'' and ``tcpdev.c''. They are the terminal
I/O driver, and TCP/IP driver respectively. These are the only files which
contain code that uses features not defined by the ISO C language: if something
goes wrong during compilation, it will probably happen in these modules. In
particular, some of the terminal driver flags in posixtty.c might not be
recognized on some older UNIX systems. If you have trouble compiling the TCP/IP
driver, you can surgically remove it by dropping all mention of it from the
makefile and editing the driver table (drivtab.c). The TTY-based driver is
essential, however.

The Makefile is set up for the GNU C compiler. You need to edit the CC and
CFLAGS definitions to suit your compiler. Make sure that you use an ANSI/ISO
C compiler or a compiler operating in ANSI/ISO mode. Older compilers that
don't support function prototypes will not work.

When the make completes, two executables should be left in the code directory:
the programs ``sps'' and ``spr'' (SixPack Send and SixPack Receive).


2.2	Customization

As of version 1.1, SixPack comes with 80386 assembly-language modules which can
directly replace the functionality of the sixbit.c and exrep.c C language
modules. They can be used on 386+ systems which have the GNU assembler and GCC
compiler, provided that the rest of the program is built with the compiler's
default settings wherever those settings affect binary compatbility. Simply
uncomment the line of the Makefile

    #MACHOBJ = exrep-i386.o sixbit-i386.o

by removing the hash sign, and comment out the line

    MACHOBJ = exrep.o sixbit.o

by placing a hash sign in front of it, then run make to rebuild the programs.

The exrep module contains routines for transforming elementary data, such
as words and strings into an external form suitable for transmission. The
sixbit module converts data consisting of eight bit bytes into six bit
bytes. The assembly language modules do these tasks somewhat faster, which
may benefit users with slower, older computers. Or they may help drive
a very high-speed link more efficiently (but note section 3.4.2 on window
sizes).


3.	Using SixPack

Assuming that you have SixPack on a UNIX system (the only supported platform,
currently), you have two new commands available: spr and sps. I will
assume that these are in your shell's command search path. I will assume that
you have also installed these commands on a remote host computer to which you
can connect via a serial link, and that this serial link is achieved through a
tty device called /dev/modem. Your actual device may have a different name (but
if you have permissions, you can always create a symbolic link from /dev/modem
to whatever device you use). SixPack is not communication software, it does not
provide a ``terminal mode''; if you have ever used the ``rz'' or ``sz''
programs which implement the Zmodem protocol, you will find SixPack very
familiar. You should have a communication program such as Minicom, Kermit or
Seyon---what have you---for interacting with the host and starting remote
commands. When this guide tells you to start a command on the remote UNIX host,
it is assumed that you are in the ``terminal mode'' of your communication
program and are able to type commands to the remote machine's shell.


3.1	Receiving a File

To receive a file, first connect to the remote host using your favorite
communication program. Your program should be one which allows you to escape
out of terminal mode and run a command on your local computer without
disconnecting from the remote host. Your communication program should also be
one which does not ``fight'' with SixPack over received data (some versions of
the ``cu'' program supposedly steal input from the tty even when another
process is supposed to be able to read from it---but then again, ``cu'' is not
likely to be your favorite communication program, is it?)

On the _remote_ host, you have installed the ``sps'' program (SixPack Send),
so execute the command

	sps <filename>

supplying, of course, the actual name of the file you wish to send in the place
of <filename>. The first time you use SixPack, send a _small_ file, of only a
dozen or so kilobytes in length, perhaps. No matter what file you choose, you
should see the following _output_:

	aaiaaclac3i/

What you are seeing is a SixPack packet being sent over the data link. It
is a special invitation to the other SixPack peer program to initiate
a connection. 

It is time to escape to the shell on your _local_ and start up the SixPack
receive program, and you have about a minute to do it. You must know the name
of your terminal device; I will assume that it's called /dev/modem. If you are
on a system where terminal devices are randomly assigned each time you run the
communication program, you might have to interrogate the program to tell you
the actual name. When in doubt, ask the local modem guru.  Anyway, enter the
command

	spr -v -t /dev/modem

The -v flag tells spr to be ``verbose''; it will display some
copyright information and, as it is receiving data, it will perform a modest
display showing the increasing sequence numbers. To see more detailed
diagnostics, type more v's, like ``spr -vvt /dev/modem''. Up to three v's
are useful in the current version.

When your file is transmitted, spr will exit. An examination of the current
directory should reveal that a file has been deposited there. The length
and name of this file should match the length and name of the corresponding
file on the remote host, and their contents should be identical.

Finally, return to your terminal program and resume your session with the
remote host.


3.2	 Sending a File

Assuming that you have successfully performed the steps described in
section 3.1, sending a file to the remote host should be easy. This is,
in effect, a slight modification of the above steps.

On the _remote_ host, type the command

	spr

This time, you do not specify a -v flag. This would be harmful, since the
verbose information would be sent to the same device where the data packets are
supposed to go (i.e. the data terminal connection) and would mess up the
communication quite badly. As a rule of thumb, the -v flag should only
be used on the local machine. You also don't have specify the -t /dev/modem
option. By default, spr and sps will use the /dev/tty device, which, on the
remote host, is equivalent to the data connection---that is, your terminal.

Escape to the shell on the local host and type:

	sps -v -t /dev/modem <filename>

Where <filename> is, of course, the name of the file you wish to send.
You should see the title of the program with some copyright notes, and
a little animated display showing increasing sequence numbers. The sps
program will quit when the transfer completes (in most cases---see Section 4!)


3.3	Limits

SixPack uses 32 bit sequence numbers internally. A sequence number is actually
a byte position in the stream of characters that are been transmitted
not a packet number.  SixPack will not handle wrapped sequence numbers
properly, so do not send files larger than 4 gigabytes (less 68 bytes---68
bytes are used to transmit the filename). I hope that this limit is not
particularly constraining. :) The 68 bytes brings to mention another limit: the
name of the file being transmitted is represented as a zero terminated string
up to 63 characters in length (and the file size is transmitted in four bytes
as a 32 bit unsigned number, hence the total of 68 bytes of overhead). The sps
program will automatically strip the leading path name from the file being sent
(e.g. a name like ``/var/tmp/foo'' will be transmitted as simply ``foo'') and
refuse to run if the remaining filename is longer than 63 characters; spr will
always write the received file to its current working directory.


3.4 	Tips

3.4.1	Killing SixPack

A characteristic of the SixPack protocol is that all packets are terminated
by a slash character (if you recall, this character belongs to the set of
64 printable characters used to represent data). This convention means
that a slash cannot appear inside a packet, and special escape conventions
are used to make sure that when a slash occurs inside a packet it
is represented in some other way when the packet is transmitted.
Now here comes the good part: both sps and spr will _quit_ when they
receive too many empty packets. An empty packet is, simply, nothing
followed by a slash. What this means: to abort sps or spr, simply type
a lot of slashes to the terminal on which they are invoked. Try this
now; type

	spr

your terminal should freeze as spr expects packets from its buddy. 
You cannot kill spr with Ctrl-C or any other UNIX terminal trick---it
disables all of these for the safety of the data transmission. Now,
hold down the backslash (or use a repeat key in conjunction with the
slash---do whatever it takes to type a lot of slashes into spr).
The slashes you type will not be echoed, so don't despair if you don't
see them. 

	//////////////////////////////////////////////////////////

Before long, spr will bail out and your shell prompt will re-appear,
probably along with a lot of slashes that go into the shell as input.

It is not probable that in the normal transmission of a file even over a noisy
or lossy data link a lot of empty packets will be systematically transmitted in
a row, hence this convention is unlikely to ever result in the spontaneous
termination of the file transfer.


3.4.2	Adjusting the Buffer Size

The default sliding window size for spr is 8192 bytes, and 4096 bytes for sps.
These sizes have been found to work well in practice, but they can be adjusted
from the command line. The only adjustment I would recommend in the current
version is a lowering of the send buffer size; for example, to decrease the
size of the send buffer to 2 kilobytes and send a file called ``foo'':

	sps -b 2048 foo

The effect of decreasing the send buffer size is to reduce the amount of data
that can be transmitted without requiring acknowledgment. This is prudent on
very poor connections, such as when you are going through terminal equipment
that has poor buffering and flow control, and cannot withstand sustained
blasting of data. If you suspect that you are getting poor performance, or if a
-vvv diagnostic shows a lot of retransmissions and the use of very small
packets, try shrinking the send window, first to 2048 bytes, and, failing that,
to perhaps as low as 1024 bytes or even lower. Note that using an unnecessarily
small window will do nothing but hurt the performance of SixPack.

You can also choose to use larger windows to try to increase the performance
of SixPack over high bandwidth or high latency networks. As noted in Section
4, there is no flow control in the form of window size advertizements in
SixPack: thus make sure that the receiver's buffer size is 1.5 to 2 times
as large as that of the sender.

Example: suppose that you want to use SixPack over a link whose 
latency is 100 milliseconds, and whose bandwidth is 128Kbps. The
bandwidth-delay product is 12.8 kilobits, or about 1.6 kilobytes. This is
how much data you can have ``up in the air'' at once, so this is
(at least) how large you would need to make your send window to keep
the pipe full. If you don't keep the pipe full, you cannot take advantage
of the full bandwidth. Obviously, a window size of 4096 is more than
enough for typical serial connections whose bandwidth is often no more than
38,400-115,200 kbps, and whose latency is quite low (i.e. point to point
connection through a pair of modems or direct RS-232 null modem cable).

Note that due to the implementation of ``regions'', there is a practical upper
limit on window size. A region is a data structure which represents a set of
discontinous byte ranges representing the portions of a file that have been
received. At present, there is a limit on how many holes there can be in the
receive window, determined by a compile-time constant REGIONS_MAX in the file
region.h which is set to 32. This means that the receive stream can be broken
into up to 32 discontinous regions at any time, which allows for 31 holes.
Suppose that there are 31 holes, and a piece arrives from the sender. Since the
region data structure cannot be extended, the piece will be dropped.  If you
want to use large windows, you should increase REGIONS_MAX and recompile.  The
smallest segment size to which SixPack ever drops is 128 bytes, so the maximum
possible number of regions that could be required for a window size of n
kilobytes is roughly 4n (if 128 byte segments are being transmitted
and every other one is lost).

The initial segment size can be adjusted in the sps program using the -s
option.  The default is 1024. The maximum size of a segment is 60,000 bytes.
Whenever SixPack times out when waiting for an acknowledgement, it cuts
its segment size in half (if the segment size is greater than 255) and
retransmits any unacknowledged bytes using the smaller segment size.

3.5	SixPack over TCP/IP

Yes, SixPack contains a driver that will enable it to work over TCP/IP
connections (big deal). Of course, the sliding window protocol and ASCII
encapsulation features are useless in that case, and when IP networking is
present, chances are that you can just use FTP. But this driver only took five
minutes to implement, and demonstrates the ability of SixPack to be compiled
with multiple drivers for different devices. I could just as well have used
UDP, which would have actually made use of SixPack's protocol capabilities.

To choose the TCP driver with either sps or spr, use the -d argument:

	spr -d tcp

You will also have specify a device name. With the TCP driver, a device
name is either a TCP port number (which specifies a passive listening
endpoint), or a combination of host name and port number (which specifies
an active connection attempt).

For example, suppose you want to send a file ``ship'' from host ``mars''
to host ``venus''. Simply do this:

	venus % spr -d tcp -t 7505
	mars % sps -d tcp -t venus:7505 ship

For a neat display, have the mars and venus shells in a adjacent windows
and use the -v argument!


3.6	SixPack with Minicom (and Other Programs)

SixPack is easiest to use when it is installed as an extension to an
interactive communication program such as the popular Minicom by Miquel van
Smoorenburg which is part of most Linux distributions. Here is
how you install SixPack for use with Minicom.


3.6.1	Installation

Let's assume that you have installed the sps and spr executable in the
directory /usr/local/bin.  Execute Minicom. Then type Ctrl-A O and choose
``File transfer protocols'' from the configuration menu using the arrow keys
and hit Enter.   The file transfer protocol configuration screen comes up.
Type the letter next to a free entry. In the Name column type ``sixpack''
then type Enter. The cursor moves to the next column where you are expected
to type the executable path name. Type ``/usr/local/bin/sps -vt %l''.
(The %l is a special marker which will be replaced by the tty device name
when Minicom executes sps). Type Enter. In the Need Name column, type 'Y',
since SixPack Send requires a filename argument. Hit Enter. In the Up/Down
column, type 'U' since sps is for uploading, not downloading. (This
is important, because it determines whether the protocol will appear in
the Send or the Receive protocol menu). Then in the final column, type 'Y'
to enable I/O redirection---this means that the output of SixPack will
be redirected into a small window that will be popped up by Minicom.
You will be able to see SixPack diagnostic messages in this window.

Now, install the spr program. Locate another free entry in the protocol
table and repeat the above procedure. Substitute ``/usr/local/bin/spr -vt %l''
in the Program column, type 'N' in the Need Name column (since, as you know,
spr can determine the filename on its own) and say 'D' in the Up/Down column,
since spr is for downloading.

Hit Esc or Enter to leave the file transfer configuration screen. Then
choose ``Save Setup as dfl'' from the main configuration menu to save
the new settings. Select ``Exit'' or hit the Esc key to return to 
Minicom's terminal mode.


3.6.2	Sending and Receiving

Now, you should be able to use SixPack. To send a file, type ``spr'' on the
remote host.  Type Ctrl-A S, and select the sixpack protocol from the small
menu which Minicom pops up. A dialog box appears into which you must type the
filename (either a full path name or a name that is relative to your configured
upload directory). If all goes well, SixPack should kick into action. If an
empty file transfer window comes up informing you that the upload is ``READY'',
you must have mis-configured the protocol---that is, you probably typed the
path name of the program incorrectly in the file transfer setup menu. On the
other hand, if the file transfer window appears with an error message from
SixPack complaining about not being able to find the filename, you have
obviously typed an incorrect filename (or perhaps you have messed up the
command line options to SixPack). The spr program on the other end is still
waiting, so type Ctrl-A S, and try again or kill spr by typing slashes in
terminal mode.

Receiving a file is similar to sending, except that Minicom will not prompt
you for a filename. Type ``sps <filename>'' on the remote host. 
Unlike Zmodem, SixPack will not spring into action automatically, because
Minicom will not recognize the packet from SixPack as the initiation of
a file transfer. You must type Ctrl-A R to bring up the list of download
protocols and choose SixPack.  If all goes well, SixPack's diagnostic
output will appear in a status window, and the file will soon be deposited
into your download directory.

Using SixPack with other communication programs should be similar, except
that you may have to edit configuration text files whose format will,
needless to say, vary from program to program. Different programs will
also have slightly different conventions for passing the terminal device
name as an argument to the -t option in SixPack's command line.


4.	Bugs and Caveats

Yes, there are bugs. Not run of the mill ``segmentation violation'' bugs
(hopefully) but subtle design flaws which translate to minor nuisance in the
use of the program. SixPack has exceeded my own expectations to the point that
I didn't care to address these problems in the current release; it has enabled
my coworkers and myself to transfer files in situtations where other protocols
completely failed us. In the failure of other protocols, I saw an opportunity
to develop a new file transfer utility and SixPack was born.

One flaw is that when either side of the transmission thinks that it has
finished, it transmits three ``kill packets'' in sequence. It does not wait for
these packets to be acknowledged.  As a result, the following situation can
arise: the sender transmits the last few portions of the file. The receiver
gets them all, and realizes that it has the entire file. So it sends the final
acknowledgment to the sender followed by the three ``kill packets''. However,
these packets all become lost or garbled in transmission. The result is that
the sender doesn't know that the file has been received by the other end, and
it finally gives up after a long timeout.

A related minor nuisance is that sps will sometimes ``spam'' the shell of a
remote host with data after the spr has finished receiving and quit. Sometimes
this can have the effect as though you typed an excruciatingly long line of
input into the shell. Just be patient, this flooding is not harmful to a
robustly written shell.

Another problem is that the program uses hard-coded, static timeout values for
retransmission. These are not dynamically adjusted to the latency of the
connection, and cannot currently be overridden on the command line. I have made
the timeout values generous enough that SixPack can be easily used over
networks where data travels through several hosts and/or terminal servers.

A notable flaw is that the retransmission strategy is far from perfect.
What happens is that the receiver acknowledges each piece of the stream that
was successfully received. If the sender detects too many duplicate
acknowledgments for the same byte position, it will retransmit a small piece
of the stream for that byte position. But if the sender times out, it will
retransmit everything it has in its window starting from the highest
acknowledged position. This is wasteful, because the receiver may have a few
pieces of the retransmitted information already and only needs to fill a few
small holes before it can jump its acknowledgment forward.  In a future
version of the protocol, the sender will learn about the ``holes'' in the
receiver's version of the stream, and will retransmit only segments to fill
these holes.

There is no flow control, unlike in the TCP protocol. The receiver
has no way to advertize how much buffer space it has available, and it
thus cannot advertize having zero buffer space. The protocol currently
assumes that the receiver consumes data as fast as possible.

There is currently no way to disable the ASCII encoding, even though SixPack
would work without it in situations where a clean end-to-end connection is
available. There should be some flexibility in the transmit/receive module to
allow for the encoding to be defeated, as well as to turn off the delimiting of
packet boundaries which is not needed with devices that delimit packets
(e.g. UDP, Ethernet)

SixPack does not treat text files specially. It will not convert files
between character sets such as EBCDIC and ASCII. Even within the ASCII domain,
it will not convert between the various ways of representing text files
(such as converting carriage return and line feed pairs to line feeds
or vice versa). Furthermore, it transmits the file name unmodified, thus
if SixPack where used to send a file from a system which uses the EBCDIC
character encoding, the filename received by the ASCII-based equipment on the
other end would not make sense. This is a matter of convention at the very
highest level protocol in SixPack, and is not a deeply rooted portability
problem. I simply got lazy and decided to transmit the name raw.

SixPack does not treat binary files specially. It assumes that all bytes
in a file have unsigned values in the range 0-255, and transmits all
information in the form of eight bit octets. On a system with 9 bit
characters, some external utility will have to be used to convert a
native nine bit file into eight bit octets so that it can be reliably
transferred: SixPack will otherwise strip away the ninth bit. (Yes, such
systems are rare, but they do exist. The regulars of the comp.lang.c Usenet
newsgroup will tell you of at least one computer on which the C implementation
has 9 bit bytes, and 36 bit long integers).


5.	Licensing

The program is freely distributed in source code form under the conditions
of the GNU Public License (GPL) found in the accompanying LICENSE file.
Please read this file.

The purpose of the GPL is to enable individuals and organizations to profit
from the use and distribution of free software without being held hostage by
greedy software developers and distributors. Free distribution and source code
availability is particularly important to a communication protocol, which,
after all, helps to bridge platforms and users, not to isolate them.

Remember, software carrying the GPL license is not public domain; it remains
the copyright of the author(s), but it permits free redistribution.

Good luck!

						Kaz Kylheku
						2869 East 14th Avenue
						Vancouver, B.C.
						CANADA
						V5M 2H8

						kaz@cafe.net
