************************************************************************
*           Myricom GM networking software and documentation           *
*                 Copyright (c) 2000 by Myricom, Inc.                  *
* All rights reserved.  See the file `COPYING' for copyright notice.   *
************************************************************************

README for GM-1.4

This directory tree includes the Myricom `GM' Myrinet messaging layer
in either source or binary format.  Documentation is in the "doc"
directory.  For updates to this software, visit
`http://www.myri.com/scs'.

************************************************************************
All problems with GM and Myrinet installation/setup should be directed
to help@myri.com.
************************************************************************


Because this release include kernel-mode code, any bugs can
potentially crash your machine, or even (possibly) cause loss or
corruption of data on disk.

********************
Status & Limitations
********************

GM-1.1 (and up) messages (on the wire) are not backwards-compatible with
GM-1.0x messages. If you use gm-1.1 (and up) on any machine in your network,
it should be running on all the machines in your network. 

GM does NOT work with threading or forking. MPICH-over-GM
will not either. "system()" calls from a GM program or
mpich-over-gm program will cause GM to fail with strange errors.

This release supports core GM functionality and TCP/IP for the
following platforms:
	Windows NT 4.0 for x86 PCs.
	Windows 2000 for x86 PCs.
	Linux 2.0 and 2.2 for x86 and alpha
	Linux 2.2 for Sparc64 and PPC
	Solaris 2.6 for x86 and Ultrasparc
	Solaris 2.7 for x86 and Ultrasparc (32-bit mode only)
	Solaris 2.7 for Ultrasparc (64-bit mode using Sun compiler)
	Tru64 (OSF1 V4.0e, V4.0f, V5.0) for Alphas
	FreeBSD v4.0 (no support yet for 3.2 or 2.2.x)
	VxWorks for ppc Motorola MTX, Force, and CSPI boards.

By default, this software only supports 8 ports, and only 6 of these
(ports 2,3,4,5,6,7) are available to non-privileged user-level software.
Port 3 is used by the IP-over-Myrinet driver, so if you have
done
	ifconfig myri0 <addr> up
port 3 will not be available to
users.

Some of the security features of GM are disabled.  In particular,
opening privileged ports is not restricted to privileged users.

************************************************************************
IF YOU DO NOT HAVE A SWITCH IN YOUR CONFIGURATION - READ THIS CAREFULLY
          ---
GM will work with a loopback cable, when directly connected to another
machine or when connected to a switch.  When two machines are directly
connected by a cable it is NOT possible for a gm process to send a message
to itself. There must be a switch in between in order for packets to be
looped back to the sender. Specifically, 
	gm_allsize -v 
will FAIL to work in this configuration.
MPICH-over-GM should work in this configuration as will IP-over-GM
since both of these protocols loop messages above the GM messaging layer.
************************************************************************


********************
Setting up a network
********************

To set up a network to use the GM software, you must install a Myrinet
interface and the GM drivers on each machine that will participate in the
network, except that on one machine you must run a mapper.  This "primary
mapper" machine must be running any time a new node is added to the
network, or after a catastrophic network failure. The mapper must run
at least once every time a node reboots or a driver is loaded.

**********************
Building from Source
**********************

For any system other than NT and vxworks you can use the commands below.
You no longer need the lanai3 and lanai7 compiler tools
since we ship the MCPs pre-compiled.

./configure 
make
cd binary
./GM_INSTALL

To build the MCP too, then
./configure --enable-mcp-build
make

If you _ONLY_ have LANai7-based boards in your cluster (PCI64 cards), then

./configure --enable-new-features [any-other-options-you-want]

This will use the optimized setting for LANai7 including
	crc32 on the links
	larger page cache on the card
	jumbo MTU for IP-over-GM

If you have older 256KB or 512KB PCI or SBus cards, please see the note
at the end of this file.

Windows NT
==========

Currently, to use GM you must do the following to set up your network:

o On each machine that will use GM:
    o Install a supported Myrinet interface and the GM driver. (see below)
    o Use a cable to connect the interfaces to a common Myrinet.
o Make sure *exactly one* machine on the network is configure as a
  "primary mapper."  This is done by running gm/binary/sbin/mapper
  with the "active.args" file. Alternatively, on NT, you can install
  the driver with an "active" option.

The GM Myrinet mapping software automatically maps and configures
the network.

Unix
====
To install GM, simply

[using a binary dist]
cd <your_gm_dir>
./GM_INSTALL

YOU MUST RUN ONE MAPPER ON YOUR NETWORK. You do this by

	cd sbin
	mapper active.args

You may edit active.args to increase/decrease the amount of
information printed during mapping, or to adjust timeouts etc.
You will find the mapper source in gm/mt.

To build the mapper tools (simulator etc):
	[in a source distribution]
	cd gm/mt
	make all gm


Building the GM MCP requires lanai3-binutils, lanai3-gcc and gendat.
These tools are available at http://www.myri.com/scs/L3. 
Just building from the released source does not require the lanai tools.


***********************************************
Special considerations for this release
***********************************************

If you run into GM bugs, you may experience the following bugs, which may
be handled as follows on Windows NT.
  () System Panics ("Blue Screen of Death" on Windows NT) and system lockups:
     If the system panics you must reboot by power-cycling the machine
     or pressing the "reset" button.  This release is not known to
     panic machines.
  () GM non-operation under windows NT:
     If GM doesn't crash the machine but stops responding, you can often
     get GM operational again without rebooting the machine by terminating
     all GM programs on the nonresponsive node and executing the following
     sequence of commands:
	net stop gm_mapper
	net stop gm
	net start gm
	net start gm_mapper

***************************
Installing a Binary Release
***************************

Unix
====

The Unix binary release can be installed as follows, but you must
have superuser privileges to do so:
	> cd <desired_installation_directory>
	> gunzip -c gm-<version>.tar.gz | tar xf -
	> cd gm-<version>
        > ./GM_INSTALL
If further installation steps are required, they will be described
by the output of GM_INSTALL.

Windows NT
==========

Install the driver and DLLs as follows.  

NOTE: You must be user "Administrator" or a member of group
"Administrators" for these procedures to succeed.

Installation From Floppy Disk:

   Go to the Control Panel, open the "Network" control, click the
   "Adapters" tab, click "Add", click "Have Disk", insert the floppy disk
   in drive A:, and click "OK".  Reboot.

Installation From Zip archive:
 
   Unzip the archive (by running it) onto a floppy disk or into any
   hard disk directory. Under the network devices control panel, select
   Have Disk when asked what you want to install.

NOTE: If you have installed GM-0.5 or earlier and installation
fails with an error stating that a Register Service entry already
exists, you must use "regedt32" to remove all "GM," "GM_Mapper," and
"GM_SNMP" subtrees from the HKEY_LOCAL_MACHINE database in the
registry.  You can find these subtrees using the "Find Key" menu item
in regedt32.  This problem has been fixed in later releases.

*******************************
Building user-level GM programs
*******************************

Building GM programs is straightforward.  Using the "gm.h" header file
and the associated library in the GM binary release, one can build GM
applications just as with any other library.  The only difference is
that before the program can be run, the GM driver and any dynamically
linked GM library must be installed.  Architecture-specific details
follow.

Unix
====

Under Unix, the header file is called "gm.h" and the library is "libgm.a."
Unix dynamically linked libraries are not yet supported.

Windows NT
==========

Under NT, the header file is "gm.h," the import library is "gm.lib,"
and the dynamically linked library is "gm.dll."

************************
Building the source tree
************************

Subject to the terms of the copyright notice included in the GM source
release, users outside Myricom may modify and recompile the GM
software.  The proceedure for doing so is outlined below.

Building for Unix
=================

The following special software is required to build GM:
	o Gnu "make"
	o An ANSI C/C++ compatible compiler.
	o The LANai compiler "lanai3-gcc" and LANai binary
	  utilities are required if you modify the Myrinet interface
	  firmware, which runs on the interface's LANai RISC processor.  This
	  software is available at http://www.myri.com/scs/L3/.

In most cases, the software can be built by typing the following,
substituting the desired installation directory for <prefix>:
        > ./configure --prefix=<prefix> --
	> make
Libraries will be installed in <prefix>/lib, header files in
<prefix>/include, etc.

A successful "make" will create a "binary" directory containing a
complete binary release, which may be installed as described in the
"Installing a Binary Release" section.

Building for Windows NT
=======================

The following special software is required to build GM:
	
	o Windows NT 4.0 Service Pack 3 or Service Pack 4
	o The cygwin environment (beta 20.1 or better) from
	  http://www.cygnus.com/win32/.
	o Microsoft Visual C++ with DevStudio Service Pack 1.
	o The Microsoft Developer Network (MSDN):  The MSDN Software
	  Development Kit (SDK) and Device Driver Kit (DDK) must be
	  installed.
	o The LANai compiler "lanai3-gcc" is required if
	  you modify the Myrinet interface firmware, which runs on
	  the interface's LANai RISC processor.

Under a cygwin32 bash prompt, extract the source archive as follows,
substituting the version number of the GM release for "<version>."
	> tar xzf gm-<version>.tar.gz
	> cd gm-<version>
Then, the source can be built as follows, assuming you have properly
installed cygwin32 and Visual C++ so that all the required programs
are in your PATH environmental variable.
	> export CC=cl CXX=cl
	> ./configure
	> make

A successful "make" will create a "binary" directory containing a
complete binary release, which may install as as described in the
"Installing a Binary Release" section.

****************
GM Test Programs
****************

See the tests.README file in the binary/bin directory.


*************************************************************************
Compiling for boards with less than 1Meg of memory (e.g. 256KB or 512KB)

If you have older 512KB cards and you are using them on
alphas or Ultrasparcs, then you MUST configure with
the option --with-max-sram=512
./configure --with-max-sram=512 [any-other-options-you-want]
512KB cards in x86 machines will work fine without the --with-max-sram=512
and you'll get better performance.



If you have older 256KB cards only, then you MUST configure with
the option --with-max-sram=256
./configure --with-max-sram=256 [any-other-options-you-want]
If you hare tyring to use these old cards on alpha systems,
please contact Myricom (help@myri.com). This "compact" MCP
will still not fit in 256KB cards on alphas.

LANai   ptr(b)  pag(KB)      config flags               size(KB)
4       32      4                                       492     Fits in  512KB board
4       32      4            --with-max-sram=256        242     Fits in  256KB board
4       32      8                                       516     Only fits in 1024KB board
4       32      8            --with-max-sram=256        250     ** OK, but close
4       32      8            --with-max-sram=512        465     Fits in  512KB board
4       64      16                                      617     Only fits in 1024KB board
4       64      8                                       545     Only fits in 1024KB board
4       64      8            --with-max-sram=256        265     ** TOO BIG
4       64      8            --with-max-sram=512        494     Fits in  512KB board
7       32      4                                       519     Only fits in 1024KB board
7       32      4          --enable-new-features        1048    Only fits in 2048KB board
7       32      8                                       543     Only fits in 1024KB board
7       32      8          --enable-new-features        1072    Only fits in 2048KB board
7       64      16                                      644     Only fits in 1024KB board
7       64      16         --enable-new-features        1670    Only fits in 2048KB board
7       64      8                                       572     Only fits in 1024KB board
7       64      8          --enable-new-features        1598    Only fits in 2048KB board

*******************
Contact Information
*******************

If you have questions or comments about this documentation, email
"help@myri.com."

