This file explains the Software Tools Group at the National Center for Supercomputing Applications. Also included are explanations of the software developed here, and available to the public through the ftp and mail archive servers. For information on how to obtain software from the servers, please read the README.FIRST document. SOFTWARE TOOLS GROUP The Software Tools Group (STG) at the National Center for Supercomputing Applications develops data analysis and visualization software that is released into the public domain, i.e., free and unrestricted. From its conception, STG has committed itself and the development of its products to the furthering of scientific discoveries that can be made by technical organizations, research scientists, and students alike. Until very recently, scientists had to search through pages and pages of numbers representing their experimental results, looking for deviations or similarities that would support their hypotheses or show them new facts. The time involved in analysis meant loss of time in further experimentation. And although data analysis software was available, it was either too expensive or too complex for scientists not knowledgeable in computer science to manipulate. However, by providing user-friendly, analytical software that runs on a variety of low-cost workstations, STG has been able to lessen time, computer expertise, and money detractions for many users, giving them the opportunity to concentrate on their science and enlarge their potential for discovery. FROM NETWORKING TO DEVELOPMENT Established in the fall of 1985, the Software Tools Group (STG) has gone through extraordinary changes since its establishment. STG's handful of members initially focused on working within NCSA, often functioning as operation administrators by developing local networks, utilities, and facilities at the center. Eventually, the group initiated its first development project, NCSA Telnet, which provided remote users with Macintosh and IBM access to NCSA across a network system. STG released the program in 1987 for public domain use. Today, NCSA Telnet for the PC and the Macintosh has an estimated 100,000 combined users. Since NCSA Telnet's release, the group has concentrated its efforts on software development. NCSA SCIENTIFIC VISUALIZATION SOFTWARE SUITE With the support of Apple Computer Inc., STG concentrated its energy in the late 1980s on creating a suite of tools which, working alone or in conjunction, would analyze and/or visualize data on the Macintosh II. With the advent of the Macintosh tool suite, researchers could enter their data results in NCSA DataScope and have the program generate a data image. By moving that image to other suite applications, such as NCSA Image and NCSA PalEdit, they could analyze their data by manipulating its color representation and displaying it using a variety of plots and data functions to bring out salient features that they might not have noticed or which might have taken years to discover otherwise. Putting the results in NCSA Layout allowed users to create presentation-quality slides which they could show to their colleagues. For example, two California scientists utilized the NCSA Macintosh tool suite to analyze stress along the San Andreas fault. One of the scientists went on to use the tools in his own research, the study of convection in the earth's mantle. He further utilized NCSA software by incorporating parts of its code in other analysis tools and also by using the suite in this teaching. The beauty of the suite is its simplicity, interactivity, and generality. Data can be entered for analysis from a wide range of disciplines; the tool suite does not discriminate. CREATING A FILE TRANSPORT SYSTEM As networking environments brought data from different people and machines into more frequent contact, the disparity between the file formats people used became frustratingly apparent as they prevented the transfer of information. STG realized the need for facilitating information and created NCSA HDF (Hierarchical Data Format), a multi-object file structure that shares data (palettes, annotations, datasets) between people, projects, and machines on a network. This program is continually updated, and it has spawned other programs to supplement its functionality. One such program is NCSA HDF Vset which stores irregularly-gridded meshes and multi-variate graphical and scientific data. One of the greatest benefits of HDF is the interoperability it provides among all NCSA software. STG makes sure that each program supports the reading and writing of HDF files and their contents. NCSA Import2HDF for the Macintosh and NCSA Reformat/XReformat translate non-HDF files into HDF files on the Macintosh and UNIX systems, respectively. The respectability and influence of the HDF file system continues to grow as more facilities adopt it. Among the national labs using HDF are the National Institutes of Health (NIH), Los Alamos National Laboratory, Naval Research Laboratory, and Space Telescope Science Institute; other supercomputer centers using HDF include San Diego, UCLA, Cornell, and North Carolina Supercomputing Centers; private companies like Thinking Machines, Inc., and NCSA's industrial partners also utilize the format. In addition, STG collaborates with places like the National Center for Atmospheric Research (NCAR), Lawrence Livermore Laboratory, and Massachusetts General Hospital on using HDF as part of their file systems. BECOMING MORE SCIENCE SPECIFIC With the release of NCSA GelReader, a visualization and analysis tool for molecular biologists, STG has begun to create more discipline-specific applications than the generic data analysis tools contained in the NCSA Scientific Visualization Software Suite. By aiding the extraction of information from electrophoretic gel images, and doing so on low-cost hardware, GelReader can assist biologists in large and small laboratories alike. One exciting application for the program is being tested by a scientist in the home offices of Britain's Forensic Sciences Service who is using GelReader in his studies on DNA fingerprinting. Programmers are also developing a chemistry tool, NCSA ChemTool, through assistance from IBM. As a 3D tool with strong graphic and transparency capabilities, scientists will easily be able to use the program to generate meaningful models of their data and expand upon the insights available in the computational chemistry field. Although similar chemistry packages exist on the market today, NCSA can provide the finished program with comprable features at little or no cost. The Software Tools Group will continue to explore new, and more specialized areas of software development, always with the stipulation that the tool will apply, if not across a general spectrum of science, then at least across a wide area of a particular field of science. LOOKING IN DIFFERENT DIRECTIONS STG continues to expand the types of software produced as well as the types of workstations that it runs on. Recently, programmers have developed X window tools, XImage and XDataSlice, that incorporate the NCSA Scientific Visualization Tool Suite as well as other features. These tools have already proved to be of practical use to a University of British Columbia scientist who used them to aid in his NASA research on the impact of cloudiness on the global climate. In addition, programmers have developed NCSA PolyView for use on the Silicon Graphics 4D Series Workstation. PolyView displays and represents 3D data as vertices and polygons and allows users to interactively change the data's viewing parameters. Users can then take a 'flight' through their data, using a fly file which contains various camera positions in 3D space. A rapidly expanding area of computational science that STG developers are incorporating into their tools is distributed computing, a feature that frees people from the limitations of their local workstation. With distributed capabilities, users in essence can distribute complex processes to different platforms (machines) which are better suited to deal with them. For example, if users needed to generate a 3D rendering of an image, and they had a Silicon Graphics workstation available that's perfectly suited for doing so, they could access the SGI to do the rendering and then have it send the results back to their local workstation. NCSA DataScope and NCSA Image are two packages that now feature distributed capabilities. As STG moves forward in its research and development of software tools, its commitment to and involvement in computational science, for both the community and the individual, will also continue. The Software Tools Group is constantly looking to the future to create products that increase scientific discovery everywhere. NCSA SOFTWARE RELEASES NCSA Distributed DataScope is an interactive data analysis tool that displays 32-bit scientific data values in spreadsheet form or as simple scaled, interpolated, or polar color raster images. The program now features new distributed capabilities across TCP/IP network connections; thus, NCSA DataScope is no longer limited to the processing power of the Macintosh alone. In addition, notebook functions can now automatically be executed on a remote compute server, significantly increasing computational speed on a supercomputer. NCSA Image is a color imaging and analysis application that permits manipulation of 2D and 3D image datasets. Distributed capabilities across TCP/IP network connections provide power for faster and more sophisticated computations, while volumetric rendering capabilities within the program allow further data interpretation on the Macintosh. Specific data manipulation features in NCSA Image include histogram equalizations, contrast enhancements, operations on the data using non-linear filters and convolution kernels, and useful utility functions. Data analysis features display and animate 8-bit binary scientific data as color raster images, and generate color contour, ordered dither, shaded data, and surface plots. NCSA Layout is a presentation tool that allows you to display and annotate 2D data images so that you can photograph your Macintosh screen display with a 35mm camera and produce presentation-quality slides. NCSA Layout now permits batch capabilities that generate animation sequences on the Cray or other UNIX systems, which you can download to the Macintosh for further manipulation. It lets you enumerate images and annotate those images with grid lines, overlays, plain or transparent text, tick marks, color bars, colored canvases, and graphics from non-NCSA software such as MacDraw, PowerPoint, and PixelPaint. NCSA Layout also prints halftones. NCSA PalEdit is an interactive palette creation tool that helps you produce a 'clut' resource tailored to your needs. It allows you to compose palettes of any combinations of components from four color models-RGB, CMY, HSV, and HSL; and to copy and save palettes, so that they can be pasted from the Clipboard or read from HDF, raw palette, Klutz DA, Canvas, or PixelPaint files into other applications. The program now features interval manipulation tools which allow range selections in the palette and component plots, so you can make palette adjustments in isolated areas. NCSA Telnet is the Macintosh's link to the TCP/IP networks. It is an implementation of DARPA standard telnet that provides interactive access from any Macintosh to telnet hosts on TCP/IP networks; lets you have simultaneous connections to numerous computers across the network, and includes a standard file transfer server (FTP) to allow you to share files with other remote machines and user's; VT 102 emulation; Tektronix 4014 emulation; interactive color raster graphic capabilities; domain name lookup; user-defined macro keys; full color, font style, and font size support; and customized windows whose contents may be scrolled, cut, copied, pasted, and printed. (See below for the corresponding program for the IBM PC.) NCSA HDF, or Hierarchical Data Format, is a multi-object file format that facilitates the transfer of graphical and floating-point data between various machines and operating systems, including the Cray, Sun, Alliant, Silicon Graphics, Macintosh, and IBM PC computers. HDF allows self-definitions of data content and easy extensibility for future enhancements or compatibility with other standard formats; includes Fortran and C calling interfaces; utilities to prepare raw image of data files or for use with other NCSA software; and interface for storing and retrieving compressed or uncompressed raster images with palettes; an interface for storing and retrieving scientific datasets of up to seven dimensions together with information about the data, such as labels, units, formats, and scales for all dimensions. NCSA HDF Vset, is the latest storage scheme in HDF. It can accommodate existing HDF data elements as well as varied, irregularly structured and non-uniform datasets of any size and dimension. Specifically, HDF Vset makes the following two features available to you: (1) All kinds of regular (array-type) and irregular datasets may now be stored in one HDF file. This includes mesh data, multi-variate datasets, sparse matrices, finite-element data, spreadsheets, splines, non-Cartesian coordinate data, etc., and (2) Related and diverse datasets can now be linked together hierarchically to form logical groupings. Full or partial access to data, along with the linking of storage elements, make logical groupings in an HDF file easy to use and retrieve. NCSA HDF UCD, is a high level user library that is built on top of HDF Vset. Focusing on creating a geometric representation for the unstructured cell data, this library allows users to process data generated by the finite element package into the format that is more efficient for visualization software. It also provides an efficient and correct algorithim for surface extraction. Using the NCSA HDF UCD library (libucd.a) will save the user a lot of post- processing time. It also produces output data compatible with NCSA Polyview. OTHER NCSA RELEASES NCSA Contours for the Macintosh NCSA Contours is a simple graphics tool that represents 2D raw raster scientific data as contour, 3D, and shaded data plots. It supports the Macintosh desk accessories and cut, copy, and paste features and allows you to print and save all or part of a plot in PICT format, which can be read and manipulated by such commercial software packages as SuperPaint, CricketDraw, and MacDraw. NCSA GelReader for the Macintosh NCSA GelReader is a color-imaging and analysis tool developed for molecular biologists or others in need of electrophoretic gel data analysis. The program extracts information from digitized electrophoretic gel images such as DNA fragments. Allowing interactive manipulation of lanes and bands, the tool helps estimate the molecular weight of bands as well as generate reports of them. NCSA GelReader was created with the assistance of UIUC's Center for Prokaryotic Genome Analysis (CPGA). NCSA HyperCard Scientific Animation Package for the Macintosh The NCSA HyperCard Scientific Animation Package allows you simply and efficiently to create a movie animation of a series of graphic images. It is an add-on XCMD for HyperCard that imports sequences of graphics images into a Hyper-Card stack of cards. Specifically, the package allows you to translate graphics images, typically created on the Cray supercomputer and translated into a series of Tektronix 4014 drawing commands, into a PICT file; read the resultant file into a HyperCard stack; and animate the images. NCSA Import2HDF for the Macintosh NCSA Import2HDF is a tool developed to enable users to convert non-HDF files, i.e., FITS, TIFF, GIF, raw raster image, and ASCII, to HDF files. Once the files are converted, users can utilize NCSA-developed software or other software that works with HDF format files. NCSA Import2HDF also allows users to change labels associated with elements of HDF files, display HDF file contents, display RISs in the form of color raster images, and display SDSs in spreadsheet fashion. NCSA Reformat/XReformat for UNIX-Based Systems NCSA Reformat/XReformat, similar to NCSA Import2HDF, enables users to convert non-HDF files, i.e., FITS, TIFF, GIF, Sun and raw raster files, and X Window dumps to HDF files. Once the files are converted, users can utilize NCSA-developed software or other software that works with HDF format files. Distribution of this product contains code for two distinct version of Reformat: Reformat, a command-line driven program, and XReformat, a graphical interface driven program. NCSA XDIFF/XFIX for the MacOS and UNIX-Based Systems NCSA XDIFF/XFIX consists of two programs designed to help you maintain identical source code files on different systems. NCSA XDIFF creates a file of just the edits you make to an ASCII text file on your local machine. After you transfer this file to the remote system, NCSA XFIX incorporates the edits into your file on the remote machine. The NCSA XDIFF/XFIX package is currently available for Macintosh, Sun, Alliant, and Cray computers. NCSA PC Show for the IBM PC, XT, AT, and PS/2 NCSA PC Show is a color-imaging application that displays 8-bit binary scientific data as color raster images on an IBM PC or compatible, with the Enhanced Graphics Adapter (EGA), Video Graphics Adapter (VGA), or Revolution 512 graphics card from Number Nine Computer (N09). NCSA PC Show can read raw raster, raw palette, or HDF files; map images according to a user-defined palette; display multiple images in sequences; animate multiple image sequences; and rotate the palette. NCSA Telnet for the IBM PC NCSA Telnet provides interactive access from any IBM PC or compatible to telnet hosts on TCP/IP networks. It is an implementation of DARPA standard telnet that maximizes the local processing power of the PC that allows you to maintain simultaneous connections to numerous computers across the network. It features a built-in standard FTP server for file transfer, full subnetting support, Tektronix 4014 graphics emulation, scrollback, domain name lookup, RARP for dynamic IP address assignment, full color support for the new Ethernet board for the PC and PS/2, and VT102 emulation in multiple, simultaneous sessions. NCSA CompositeTool for the Sun Workstation NCSA CompositeTool is a presentation tool that lets you annotate 2D images so that you can photograph your Sun screen with a 35mm camera and produce presentation quality slides. It allows you to create a display using a user-defined palette, color bars, grid lines, XY axes, tick marks, text tilting, multiple raster images, contour intervals of a 2D data image, and vectors of 2D data. Moreover, you may perform screen dumps of selected portions of your layout, and store and retrieve layouts to be photographed or edited at a later date. The program now features NCSA HDF support. NCSA ImageTool for the Sun Workstation NCSA ImageTool is a color-imaging application that displays 8-bit binary scientific data as color raster images or black-and-white contour, shaded data, ordered dither, or 3D plots. It allows you to plot selections of data in XY graphs; print contour plots, 3D plots, and XY graph; and modify and save color palettes. NCSA X DataSlice for the X Window System NCSA X DataSlice is a color-imaging application that was developed to permit analysis of 3D 32-bit floating-point scientific data stored in HDF files. It allows you to view any 2D plane, or slice, of data along the three cartesian coordinate x, y, and z axes in spreadsheet form and as a color raster image; adjust the viewing axes to suit your needs; interpolate raster images from the data slice; load user-defined palettes to enhance imaging and analysis; frame multiple data slices together on the screen; and animate raster images along any coordinate axes. NCSA X DataSlice features synchronized selection between text and image windows; dicing (which allows you to view a data slice as a plane in the data matrix, revealing the relative location of the slice in the dataset); and 3D volume rendering (which allows you to view the 3D dataset as a whole and select and view surfaces within the scientific dataset by setting transfer functions). NCSA X Image for the X Window System NCSA X Image is a color-imaging and data analysis tool that combines much of the functionality of the Macintosh tools: NCSA DataScope, NCSA Image, and NCSA PalEdit. It allows you to view and manipulate data values, including datasets calculated on the CRAY X-MP or CRAY-2 Supercomputer and digitized pictures; reads from HDF, raw raster and raw palette files; displays actual data values in spreadsheet form, as full color cartesian and polar images, and as color contour, shaded data; creates XY graphs and color histograms which display the frequency distribution data. Moreover, NCSA X Image allows you to create and customize color palettes by "mixing" color components to define individual colors or by inverting, compressing, expanding, or rotating entries within the palette. NCSA Height-Color Visualizer for SGI Systems The NCSA Height-Color Visualizer is a non-interactive batch utility that enables you to easily create 3D animations from time-dependent data. Input for the Height-Color Visualizer consists of two 2D arrays, each having a single variable at every node. Using the Height-Color visualizer, you can manipulate the first of the two arrays to create a height geometry, so that both fields assume the same input dimensions. Thus the analysis of the correlation of data in the NCSA Height-Color visualizer is as easy as associating regions of height with coincident regions of color. NCSA Isosurface Visualizer for the Silicon Graphics Workstation The NCSA Isosurface Visualizer is a non-interactive batch utility that enables you to easily create and view animations of isosurfaces derived from 3D data. Input data consists of a single 3D scalar array of floating-point values. Indices of this array form the x, y, z coordinates; the variable at each node determines where the isosurface passes through the data. You can save polygonal isosurfaces in either NCSA HDF Vset format for use and viewing in NCSA PolyView or ASCII format. NCSA PolyView for the Silicon Graphics IRIS Workstation NCSA PolyView is an interactive visualization tool for HDF Vset data. PolyView displays an HDF Vset of polygons, or points, as a 2D or 3D interactive image with optional annotation, which may be written to an HDF raster format file. The program allows you to: interactively manipulate and display objects; render images as points, lines, or polygons; choose constant or gouraud shaded polygons; load and manipulate the color map; animate time series data; view a fly-by of the data using a script file; and pick subregions and get original data values. Alvis (3D Alpha-Shape Visualizer) for the Silicon Graphics IRIS Workstation Alvis 1.0 consists of three modular programs: detri, mkalf, and alvis. "detri" computes the three-dimensional Delaunay triangulations of a given point set. It can be used for grid generation. "mkalf" computes the family structure of alpha shapes. "alvis" is the alpha shape visualizer, where the user can interactively select a desired shape. Alpha shapes address the fundamental problem of scattered data where the connectivities are unknown. Given an arbitrary point set in 3D space, alvis constructs and defines a family of shapes with varying details. NCSA MinMaxer for the Silicon Graphics IRIS Workstation and Sun Workstation MinMaxer computes the following triangulations in 2 dimensions: Dulaunay, regular, minmax angle, minmax height, minmax slope, and plane-sweep. The original purpose of minmaxer was to provide a tool which allows visualizing triangulation algorithims. However, it evolves more or less in the direction of automatic and interactive grid generation. Feedbak about the usefullness of this tool is extremely helpful for our further development of the 3D version. The program can be run in "No-Graphic Mode" (-ng) on any Unix machine and in "Graphic Mode" on a silicon graphics workstation. Trademarks and Credit Information Alliant is a registered trademark and CONCENTRIX is a trademark of Alliant Computer Systems Corporation. HyperCard, Macintosh, Macintosh II, and Macintosh IIX are trademarks of Apple Computer, Inc. MacDraw is a registered trademark of CLARIS Corporation. Cray and UNICOS are registered trademarks of Cray Research, Inc. CricketDraw is a trademark of Cricket Software. VT102 is a registered trademark of Digital Equipment Corporation. Enhanced Graphics Adapter, Video Graphics Adapter, IBM AT, IBM PC, and IBM Personal System/2 are registered trademarks and IBM XT is a trademark of International Business Machines Corporation. The X Window System is a trademark of the Massachusetts Institute of Technology. Revolution 512 is a trademark of Number Nine Computer. PixelPaint is a trademark of Pixel Resources, Inc. SuperPaint is a trademark of Silicon Beach Software, Inc. Silicon Graphics and IRIS are trademarks of Silicon Graphics Corporation. Sun, Sunview and Sun Workstation are registered trademarks of Sun Microsystems, Inc. Tektronix 4014 is a trademark of Tektronix, Inc. Ethernet is a trademark of Xerox Corporation. .