% % This is an example of the article format. % % To use a different font size: % \documentstyle[12pt]{article} 12 point % \documentstyle[11pt]{article} 11 point % \documentstyle{article} 10 point % \documentstyle[11pt]{book} % % This redefines the margins for 8.5x11 paper. % \oddsidemargin=0in \evensidemargin=0in \headheight=0pt \headsep=0.2in \textwidth=6.5in \textheight=7.0in \title{NCSA Data Transfer Mechanism \\ Programming Manual \\ \vskip 1.5in } \author{ Version 2.3 \\ \\ } \date{February 1992 \\ National Center for Supercomputing Applications \\ University of Illinois at Urbana-Champaign} \begin{document} \maketitle \tableofcontents \listoffigures \chapter{Introduction} \section{Introduction} The Data Transfer Mechanism (DTM) is a message passing facility. It is designed to simplify the task of interprocess communication and to facilitate the creation of sophisticated distributed applications in a heterogeneous environment. To accomplish this, DTM provides a method of interconnecting applications at run-time and provides reliable message passing complete with synchronization and transparent data conversion. DTM has been optimized for large messages (100 KBytes and up), but is efficient for smaller messages as well. The purpose of this manual is to provide the applications programmer and research scientist with a basic understanding of how to write or modify code to support the use of DTM. No prior knowledge of DTM is required, nor knowledge of the underlying concepts behind distributed computing. However, knowledge of the programming language "C" is necessary if the reader desires to use the many sample programs found within. All sample code is written in ANSI C. \section{Outline} This tutorial has two parts: a description of the DTM protocol and a tutorial on how to use the DTM and message class library. The first section presents the major ideas behind DTM and how it works, with suggestions on how DTM can be used to enhance applications. The tutorial presents a step by step look at how to add DTM to an application, from creating a port to sending/receiving a message to checking ports for communication capability. Hopefully plenty of source code examples are provided in this document. Each example comes from a working program which is provided with the electronic distribution of the DTM source and documentation. For a copy of these examples, or to find out where these examples exist on NCSA machines, send mail to "softdev@ncsa.uiuc.edu". To report bugs in either the example code or in the DTM library itself, send mail to "bugs@ncsa.uiuc.edu". \section{Version} This tutorial was written for DTM version 2.3. To determine which version of DTM you are currently using, check the README file included in the DTM tar file or make and run the "version" program supplied with the examples. \section{Font Conventions} Italics are used for: \begin{itemize} \item{} file names, program names and options for user commands \item{} new terms when they are defined Courier Font is used for: \item{} source code, constants, or anything that needs to be typed verbatim \item{} structure types, structure members, functions and macros Italic Courier Font is used for: \item{} arguments to functions, since they are arbitrary Boldface is used for: \item{} chapter and section headings \item{} titles of examples \end{itemize} \chapter{The Basic Concepts} This section includes an overview of the basic concepts behind DTM: messages and message passing, ports and types of communication, the DTM protocol, and guidelines for developing applications using DTM. This section presents general concepts only. For examples on how to apply these concepts, and how to use the DTM library, see Part II. \begin{itemize} \item{} DTM Messages \itemitem{} Message Headers \itemitem{} Data Section \item{} DTM Ports \itemitem{} Port Direction \itemitem{} Addresses \itemitem{} Absolute Addresses \itemitem{} Logical Port Names \itemitem{} Quality of Service \itemitem{} Synchronous Communication \itemitem{} Asynchronous Communication \item{} DTM Applications 7 \itemitem{} Message Level vs. Port Level Multiplexing \itemitem{} Specifying DTM Ports \end{itemize} \section{DTM Messages} A DTM message is simply a delimited string of bytes. Logically, the DTM message consists of two parts, a header and a data section. The header is used to describe the attributes of the information contained in the data section. The fundamental attribute of a message is the class of the message, representing the type of information contained within the message. Although message classes are user-definable, support for several common classes are available within the DTM library. Two features distinguish the message header from the data section. First, the header is sent or received in its entirety while the data section may be sent or received using as many calls as is appropriate. Second, no data conversion is provided for the header whereas DTM provides automatic type conversion for the data section when necessary and when requested. The most important aspect of DTM messages is that DTM does not provide any buffering of data. The programmer is responsible for allocating sufficient storage for both the header and the data section. The data is copied directly from the system buffers into the application's data space, minimizing the number of copies necessary and giving the programmer full control over application memory. Both the message header and data section are optional. Many control messages consist only of a message header. It is also possible to have applications that communicate using only a data section, although it is not supported by NCSA applications. \subsection{Message Headers} DTM messages should be self-describing. The header is designed to contain information about the attributes of data stored in the data section. This information should always include the class of the message. Optional information may include the type of the data (char, int, float, etc.), a title, dimensions, or any other information relevant for interpreting the data section. Although technically indistinguishable from the rest of the DTM message, the header is treated differently: the header can be written and received separately. If the programmer fails to allocate sufficient memory to store the entire header, the buffer will be filled and the remaining header discarded. On first examination this appears to place an undue burden on the programmer, but in practice it is rarely a problem. The header is built at run-time from a limited set of functions, creating a practical limit on the size of the header. Any information making the prediction of the header size difficult probably would be better placed in the data section of the message. Since data conversion is not provided for the header, the programmer must be prepared to make the header machine-independent if the message will transfer between architectures. The easiest method is to create the header as an ASCII string or as an XDR buffer. The DTM library contains a set of utility functions that aid in building and parsing ASCII headers. These functions, along with a complete description of message classes, are described in more detail in Part II: Message Classes. \subsection{Data Section} The data section is best thought of as a stream of elements. Each element is generally a primitive data type such as a character or integer although more complex types are possible. For instance, a DTM\_TRIPLET is a data type consisting of one integer and three floats. The DTM library contains functions that perform all necessary type conversions when the data is sent. Automatic type conversion is done at the sending end of a connection. Since messages are delimited, the application may receive up to the end of a message, but may not continue without receiving the end of message mark. Since the end of message mark prevents an application from receiving the beginning of the next message, the application need not know the number of elements contained in the message a priori. An application that is writing a message may send as many elements in a block as is convenient and can send as many of these blocks as needed. A receiving application may receive as many blocks with as many elements as necessary to receive the entire message. The block size Ñ the number of elements sent or received at one time Ñ may differ between applications. Each application may choose a size that is appropriate for its task. No correspondence exists between the number of blocks sent and the number of blocks received or between the block size sent and the block size received. The only equivalence is between the number of elements sent and the number received. If the receiver elects not to receive all the elements, the remaining elements are discarded. \section{DTM Ports} All messages are exchanged through DTM ports. A DTM port is an endpoint for a unidirectional communication channel through which DTM messages may be sent or received. The endpoints of DTM ports are dynamic, and the destination for messages can be modified at any time. A name service and separate connection control messages are used to manage the port endpoints. DTM ports support multiple fan-in and fan-out. An output port is capable of sending messages to multiple listening applications while the input port is capable of accepting messages from multiple sending applications. DTM handles the replication of outgoing messages and the queuing of incoming messages. DTM ports are based on a reliable communications service such as TCP/IP and have been implemented on UNIX machines using the Berkeley sockets library. While sockets is considered an API, it is really a generalization of the transport layer interfaces. The sockets library presents a very complex programming model. An important function of DTM is to hide the complexity of using sockets directly. \section{Port Direction} Because DTM ports are unidirectional, the port must be specified as either an input or an output port when it is created. This is accomplished by using the appropriate call to either DTMmakeInPort() or DTMmakeOutPort(). Both of these calls return an integer descriptor that is used to reference the port in subsequent DTM calls. Because socket connections are bi-directional, DTM ports should be bi- directional as well. Two factors influenced the decision to provide only unidirectional ports. First, the synchronization provided by DTM requires a handshaking signal be sent in the opposite direction. If data was allowed to flow in this direction as well, it would be difficult to distinguish the data from the handshaking. It would probably require that DTM provide buffering for data so that the handshaking signal could be received correctly. The second reason was more philosophical. Requiring that the input and output ports of an application be separate maximizes the flexibility of the application. If the application was built with a single bi-directional port it would not be able to receive messages from one application and send them to a second. This is often desirable, especially for debugging. The uni-directional port forces the programmer to use two ports. \subsection{Addresses: Absolute Addresses} Absolute ports have the Internet address of the port declared explicitly when created. The Internet port address is of the form "hostname:port" where the host name is either the actual host name or an IP number in dot decimal notation. The host name is optional; if missing, the local host name is assumed. The port number represents the TCP/IP port number, in the range of 1024..65535. For input ports, the address represents where the port will listen for incoming connections. A host name specified for an input port is ignored; only the port number is meaningful. For output ports, the address represents the location where the port should attempt to connect (i.e., the address of an input port). To avoid the use of an absolute port number on an input port, it is possible to specify the address as ":0". This requests that the system assign a free TCP port number for the DTM port. To determine which TCP port is being used call DTMgetPortAddr(). The use of absolute port numbers is cumbersome and unreliable, and is best suited to starting a small number of applications by hand for debugging or for use in shell scripts. To avoid the need to recompile applications to change the TCP port number, it is best to get the port information from the command line. \subsection{Logical Port Names} Because of the problems associated with the use of absolute port addresses, DTM supports the use of logical port names. The logical port address is simply an ASCII string, where by convention the string denotes the purpose of the port. When a port is created with a logical port name, the absolute address of the port is determined and is registered with a DTM connection server. It is the responsibility of the DTM connection server to connect the ports. To do this, the server is able to send commands to DTM output ports that can cause the output port to connect to one or more DTM input ports, disconnect from one or more input ports, or discard messages without actually sending them. Connection commands can be sent anytime during the execution of the application. The next time a message is sent on the output port, the command will take effect. The ability to dynamically alter the destination of output ports is one of most powerful capabilities of DTM. The DTM distribution contains a sample DTM connection server. The server reads a configuration file, starts requested applications and connects the DTM ports. The functionality of the connection server may be incorporated into other applications. \section{Quality of Service} The quality of service option is misnamed. A more appropriate name may be mode of operation. Currently, quality of service allows the programmer to choose between synchronous and asynchronous operation of output ports. Future version of DTM will allow other options. \subsection{Synchronous Communication} DTM provides for synchronous message passing. Each message sent is preceded by a RequestToSend flag and acknowledged by the receiver with a ClearToSend flag. This handshaking prevents the receiver from being overrun by messages. Handshaking is especially important with interactive applications, to minimize latency. Because this handshaking takes place at the beginning of messages, it has a minimal impact on the throughput of messages. However, this impact is noticeable when sending many small messages or when the round trip time between the sender and receiver is large. To help in these situations, DTM is optimized to send as much data with the RequestToSend as possible. In the case of DTMwriteMsg(), the entire message, including the RequestToSend, is written with one system send call. The result of this piggybacking is that the ClearToSend may actually be sent after the actual message has been received. This prevents performance degradation for small messages while providing the benefits described above. \subsection{Asynchronous Communication} The preceding section described the synchronous operations of DTM. However, an output port may be defined as asynchronous. In asynchronous mode the output port will still generate RequestToSend flags, but the ClearToSend flag will be ignored. This allows the application to send messages as fast as possible. As each message is sent, it is copied into the local system transmit buffer and eventually delivered to the remote system. If the system transmit buffer fills up, further attempts to write messages will block. Since it is difficult to detect when a buffer is full, it is best to use this mode of operation with caution. \section{DTM Applications} DTM is a flexible communications library and may be used in several different manners. This section attempts to explore some of the possible options when developing DTM applications. \subsection{Message Level vs. Port Level Multiplexing} An application may define a DTM port for each type of message it will send or receive. This is known as port level multiplexing. Port level multiplexing is most effective for output ports. For example, if an application is going to produce three types of data sets, providing a port for each type will allow each data set to be sent to separate applications. The data sets can then be operated on in parallel. This is more efficient than serializing the messages down a single port to multiple destinations. When sending to multiple destinations, a copy of the message is made for each destination and the copy is sent independently over the TCP connection. In addition, each message may not be intended for all the destination applications. This adds to the complexity of the destination application since it must determine if the message is intended for it. In contrast, input ports seem to be most effective when they are not typed according to the message they expect to receive. Rather, input ports should be treated identically and each message should be examined and handled correctly based on the message class. This is known as message level multiplexing. Message level multiplexing works well with DTM messages since only the header is returned on the call to DTMbeginRead(). The header may be examined to determine the message class and other relevant information. After the header has been decoded the appropriate routine may be called to receive and process the data section of the message. If the message is inappropriate for the application the call to DTMendRead() will discard the data in the data section. \subsection{Specifying DTM Ports} Many DTM applications receive connectivity information from the command line. With many applications developed at NCSA, the DTM port address is preceded on the command line by the flag "- DTMIN" for a single input port or "-DTMOUT" for a single output port. Applications should attempt follow this convention since it will make invocation easier for users and for usage within scripts. It should be noted that this method works with either absolute address or logical port names. It is often convenient to use the absolute address when debugging an application and switch to logical port names after the application is running. As stated above, the DTM port address should be specified at run-time and not hard-coded into the application. There is one exception to this rule: servers can be designed to listen at well-known addresses. Typically, a server of this type will register itself in the system services table. \chapter{The DTM Tutorial} The remainder of this document is organized into a step-by-step tutorial on the use of DTM. The first two sections deal with sending and receiving messages respectively. The next section deals with error detection and error handling. The final section explains the use of the various messages classes. \begin{itemize} \item{} Sending Messages \itemitem{} Creating Output Ports \itemitem{} Writing Message Headers \itemitem{} Writing Single Data Types \itemitem{} Writing Multiple Data Sets and Types \itemitem{} Determining Writing Capabilities \item{} Receiving Messages \itemitem{} Creating Input Ports \itemitem{} Reading Message Headers \itemitem{} Reading Data \itemitem{} Reading Multiple Data Sets and Types \itemitem{} Determining Reading Capabilities \item{} Error Detection \item{} Message Classes \itemitem{} A Simple Message Class: MSG \itemitem{} Scientific Data Sets: SDS \itemitem{} Raster Image Sets and Palettes: RIS \& PAL \itemitem{} Surface Description Language: SDL \end{itemize} \section{Sending Messages} The follow sections deal with the sending of a DTM message from one application to another. The basic procedure is to create an output port, create a message header, and send the message. Since, messages may contain zero or more data sets another section deals with the use of multiple data sets. The final section deals with determining if the application will block if it attempts to send a message. \subsection{Creating Output Ports} The function DTMmakeOutPort() creates an output port Ñ a port for writing data. The two arguments are the port address and the quality of service, respectively. The port address follows the port naming conventions presented in Part I: DTM Ports, Addresses. The quality of service argument is an integer that describes the quality of service desired. It can take one of three different values: DTM\_SYNC, DTM\_ASYNC, and DTM\_DEFAULT for synchronous, asynchronous and default communications. DTM\_DEFAULT is currently defined to operate in the same manner as DTM\_SYNC and should be used by most applications. Upon successful creation, DTMmakeOutPort() returns a positive port identification number, the number used by other DTM functions to reference that port. Unlike the first releases of DTM, the port ids will not necessarily be allocated in sequence when more than one output port is created. Example 1 demonstrates how an output port is created. \begin{figure} \hrule \begin{verbatim} #include ... int outport = DTMERROR; /* Output port id */ ... outport = DTMmakeOutPort("gallant.ncsa.uiuc.edu:8899", DTM\_DEFAULT); \end{verbatim} \hrule \caption{ Example 1. Hard coded host names and port numbers like "gallant.ncsa.uiuc.edu:8899" are generally not recommended. It is better to determine the port address from the command line.} \end{figure} The creation of ports can fail for a variety of reasons, and while error detection will be discussed more fully later, it is important to mention some of it now. Should the creation of a port fail, the value returned is DTMERROR, a constant value defined in the header file dtm.h. It is important that this error be caught before any attempts to write data are made. In Example 2, the output port is read from the command line and if the port id value created is negative, the program is exited. \begin{figure} \hrule \begin{verbatim} #include ... int outport = DTMERROR; /* Output port */ ... for (i = 1; i < argc; i++) { if (!strcmp(argv[i], "-DTMOUT")) outport = DTMmakeOutPort(argv[++i], DTM\_DEFAULT); } ... if (outport == DTMERROR) { printf("The outport was not specified or the argument "); printf("was in error.\n"); printf("The output port was assigned an error value. "); printf("(out = %d)\n", outport); printf("Usage: %s -DTMOUT \n \n", argv[0]); exit(0); } \end{verbatim} \hrule \caption{Example 2. Check the command line for the option "-DTMOUT" indicating the port address, then check the results of creating the output port for errors, exiting upon failure. This method also catches the error of forgetting to specify a port on the command line as well.} \end{figure} \section{Writing Message Headers} The first call to DTMbeginWrite() attempts to establish all necessary connections on the port specified by the given port id. Once the connections are made, it attempts to send the required number of bytes from the character array header. DTMbeginWrite() will block until the connections are made and the header has been sent. If DTMbeginWrite() fails for some reason, the value DTMERROR is returned. Except in the case of an error, every DTMbeginWrite() must have a corresponding DTMendWrite(), as this marks the end of a DTM message. Together, DTMbeginWrite() and DTMendWrite() send a message to the remote process. Example 3 shows the simplest possible complete program using these two functions to create an output port and send a simple message. \begin{figure} \hrule \begin{verbatim} #include #include #include main(int argc, char *argv[]) {{} char *header = "This is a header"; /* DTM header. */ int outport = DTMERROR; /* Output port */ outport = DTMmakeOutPort(":7788", DTM\_DEFAULT); DTMbeginWrite(outport, header, DTMHL(header)); DTMendWrite(outport); return(0); {}} \end{verbatim} \hrule \caption{Example 3. Create an output port and write the header on that port. The macro DTMHL is used to determine the exact length of the given header including the terminating null character. } \end{figure} \section{Writing Single Data Types} DTMwriteDataset() writes the data portion of a message via the port specified by the port id. DTMwriteDataset() can be called only after a successful call to DTMbeginWrite() and any number of calls can be made before calling DTMendWrite(). In addition to the port id, DTMwriteDataset() takes the base address of the data, the number of data elements to send, and a DTM constant indicating the type of the data. Example 4 demonstrates how to write a data set using DTMwriteDataset(). The data types available and their corresponding constants are shown in Example 5. \begin{figure} \hrule \begin{verbatim} #include ... int outport = DTMERROR; /* DTM output port */ char *header = "This is the header"; /* DTM header */ float dataset[BUFSIZE]; /* The data buffer */ ... /* Create the ports & initialize the data here. */ ... DTMbeginWrite(outport, header, DTMHL(header)); DTMwriteDataset(outport, dataset, BUFSIZE, DTM\_FLOAT); DTMendWrite(outport); \end{verbatim} \hrule \caption{Example 4. Write a simple header and a small data set consisting of five floating point numbers to a previously created output port. } \end{figure} When necessary, DTM will automatically convert data types when sending data from one architecture to another Because DTM does not buffer its data, type conversion takes place in the user's buffer making DTMwriteDataset() a destructive process. The programmer must be sure to make a copy of the buffer if the data must be re-used after a call to DTMwriteDataset(). DTM currently provides type conversion for types listed in Example 5. If type conversion is undesirable, the data can be sent as raw character data (DTM\_CHAR). DTM\_CHAR should also be used when the data has been converted using outside type conversion routines (i.e., XDR). \begin{figure} \hrule \begin{verbatim} C DATA TYPE DTM TYPE CONSTANT char DTM\_CHAR short DTM\_SHORT int DTM\_INT float DTM\_FLOAT double DTM\_DOUBLE complex DTM\_COMPLEX struct DTM\_TRIPLET DTM\_TRIPLET \end{verbatim} \hrule \caption{ Example 5. A table of the different types that DTM can pass in a message requiring type conversion. The DTM\_TRIPLET type is a composite type consisting of an int tag and three floats and one use of it will be discussed in the section dealing with the SDL message class. } \end{figure} In general, most applications will have sequences of calls like the one in Example 4 -- one call to each of the three DTM write functions and no intervening code. In cases like this, the function call overhead for two of these functions and the extra send required to transmit the header and data separately can be eliminated by using the function DTMwriteMsg(). DTMwriteMsg() compacts the functionality of these three calls into one and performs only a single system write call. Example 6 has the same functionality as the code in Example 4, yet uses only the single call to DTMwriteMsg(). \begin{figure} \hrule \begin{verbatim} #include ... int outport = DTMERROR; /* DTM output port */ char *header = "This is the header"; /* DTM header */ float dataset[BUFSIZE]; /* The data buffer */ ... /* Insert port creation and data initialization code here */ ... DTMwriteMsg(outport,header,DTMHL(header),dataset,BUFSIZE,DTM\_FLOAT); } \end{verbatim} \hrule \caption{Example 6. Write the same dataset as in Example 4 using one call to DTMwriteMsg(). } \end{figure} \section{Writing Multiple Data Sets and Types} Though it is more efficient to send large data sets with one call, it is possible to send data in multiple small buffers, as there is no restriction on the number of times DTMwriteDataset() can be called between DTMbeginWrite() and DTMendWrite(). Example 7 shows how a large array of data can be sent in smaller chunks. While the example is of limited use in the real world, it does demonstrate how it is possible to maintain a small buffer, fill it several times and send each chunk individually. \begin{figure} \hrule \begin{verbatim} #include ... int incr = 10, /* Data block size */ outport = DTMERROR; /* DTM output port */ char *header = "This is the header"; /* DTM header */ float dataset[BUFSIZE], /* Data buffer */ *dptr = dataset; /* Data pointer */ ... /* Create the port and initialize the data here */ ... DTMbeginWrite(outport, header, DTMHL(header)); for (dptr = dataset; dptr < dataset+BUFSIZE; dptr += incr) DTMwriteDataset(outport, dptr, incr, DTM\_FLOAT); DTMendWrite(outport); \end{verbatim} \hrule \caption{Example 7. Split a large data set into several small pieces by sending portions of that data through the previously created output port. As has been mentioned and will be seen in the next section (Receiving Messages), the data can be received in block sizes different from the size being used to send this data. This example assumes that BUFSIZE is a multiple of 10. } \end{figure} When more than one data type must be sent, it is usually desirable to send separate messages. However, occasions do arise when it is more convenient or sensible to send multiple data sets of differing types between single DTMbeginWrite() and DTMendWrite() calls. Example 8 demonstrates that the programmer is not restricted to sending one data type per message. \begin{figure} \hrule \begin{verbatim} DTMbeginWrite(outport, header, DTMHL(header)); DTMwriteDataset(outport, floatdata, FDIM, DTM\_FLOAT); DTMwriteDataset(outport, intdata, IDIM, DTM\_INT); DTMendWrite(outport); \end{verbatim} \hrule \caption{Example 8. Write two arrays of data Ñ one floating point of size FDIM and one integer of size IDIM Ñ to the previously created output port. All other code appears the same as in Example 7. } \end{figure} \section{Determining Writing Capability} It is possible to avoid blocked writing calls by checking a particular port ahead of time for the ability to write a message. This is helpful when a program must continue processing while waiting to send a message. DTMavailWrite() will return a true (greater than 0) value if the specified port is currently available for writing. Example 9 shows some code that checks an output port for the ability to write before calling DTMwriteMsg(). \begin{figure} \hrule \begin{verbatim} #include ... int iteration = 1, /* Main loop counter */ outport = DTMERROR; /* The output port */ char *header = "This is the header"; /* The message header */ ... /* Create the port and initialize the data here */ ... while (!DTMavailWrite(outport)) printf("Do something useful for iteration #%d.\n", iteration++); DTMwriteMsg(outport, header, DTMHL(header), NULL, 0, DTM\_CHAR); \end{verbatim} \hrule \caption{ Example 9. Until the port is ready for writing, keep doing some sort of processing, in this case a simple loop counter. If blocking is undesirable, use DTMavailWrite() to "poll" the port. } \end{figure} \section{Receiving Messages} The basic procedure for receiving messages parallels that of sending messages. The application must create an input port, receive the message header and receive the end of message. Each of these tasks are described below. There is also a discussion of receiving multiple data sets and determining when a message is ready to be read on a port. \subsection{Creating Input Ports} Ports that receive message are created in a similar manner as ports created for writing messages. DTMmakeInPort() uses the same port naming conventions and quality of service options as described for DTMmakeOutPort() with two exceptions: it makes little sense to specify a host name for an input port, and the quality of service should always be DTM\_DEFAULT. All host name specifications and the quality of service options DTM\_SYNC and DTM\_ASYNC have no effect on input ports and are ignored. As with its writing counterpart, the value returned by DTMmakeInPort() is a positive integer port id when the port is successfully created, and DTMERROR otherwise. Example 10 shows how to create an input port with some limited error checking. \begin{figure} \hrule \begin{verbatim} #include ... int i = 0, /* Argument counter */ inport = DTMERROR; /* DTM input port */ ... for (i = 1; i < argc; i++) if (!strcmp(argv[i], "-DTMIN")) inport = DTMmakeInPort(argv[++i], DTM\_DEFAULT); if (inport == DTMERROR) { fprintf(stderr, "\nUsage: %s -DTMIN \n\n", argv[0]); exit(0); } \end{verbatim} \hrule \caption{Example 10. Create the input port by parsing the command line looking for the argument "-DTMIN". Abort the program if no input port was specified or created. } \end{figure} \subsection{Reading Message Headers} The receipt of messages begins with a call to DTMbeginRead(). Like its counterpart DTMbeginWrite(), this call is a blocking call. If no message is ready to be received from the given port, DTMbeginRead() will wait until a message is available or until it times out (returning DTMERROR). DTMbeginRead(), when successful, fills the header character string buffer with the header from the sending process. If the buffer is larger than the incoming header, only the number of bytes in the message header are saved. DTMbeginRead() returns the number of bytes stored in the header. If the header buffer provided is smaller than the message header, the buffer is filled and the remaining bytes discarded. The header itself can be of any length, but because it is impossible to determine the header's length beforehand, it is recommended that headers never exceed DTM\_MAX\_HEADER characters in length. DTMendRead() signals that no more data is to be read from the current message. Any data that remains in the message is discarded. Every call to DTMbeginRead() must have a corresponding call to DTMendRead(), except in the case of an error. Example 11 shows how a header is read from a port and prints the header received. \begin{figure} \hrule \begin{verbatim} #include ... char header[DTM\_MAX\_HEADER]; /* DTM header of unknown size */ int inport = DTMERROR; /* Input port */ ... DTMbeginRead(inport, header, sizeof header); DTMendRead(inport); \end{verbatim} \hrule \caption{Example 11. Begin reading from the previously created input port. If no writer is sending data then the program will block at the DTMbeginRead() call. Make the assumption that the header being sent will not exceed DTM\_MAX\_HEADER characters in length. } \end{figure} \subsection{Reading Data} DTMreadDataset() reads the data portion of a message from a specified port. There is no limit to the number times DTMreadDataset() can be called and no limit on the number of differing data types that can be received between calls to DTMbeginRead() and DTMendRead(). Upon receipt of data from the specified port, DTMreadDataset() returns the number of elements of the specified type received. If no data was read (a signal that no more data can be received from the given message), the value returned is 0 (zero). Should the buffer be too small to contain all of the data waiting, DTMreadDataset() will leave the remainder on the input stream. Subsequent calls to DTMreadDataset() can be made to retrieve the remaining data. In Example 12, data is read into an arbitrarily large buffer. \begin{figure} \hrule \begin{verbatim} #include ... int nelem = 0, /* Number of elements read */ inport = DTMERROR; /* DTM input port */ char header[DTM\_MAX\_HEADER]; /* DTM header */ float dataset[MAXBUFSIZE]; /* Data buffer */ ... DTMbeginRead(inport, header, sizeof header); nelem = DTMreadDataset(inport, dataset, MAXBUFSIZE, DTM\_FLOAT); DTMendRead(inport); \end{verbatim} \hrule \caption{ Example 12. Read the data from the previously created port using a single call to DTMreadDataset() using an arbitrarily large buffer. Also assume that the data being received is of floating point type [DTM\_FLOAT]. Note: This code does not check for errors and allows DTMbeginRead() to block until a writer is ready to send the message. } \end{figure} DTMreadMsg() is a call analogous to DTMwriteMsg(). In situations where there is no more than one DTMreadDataset() per DTMbeginRead()ÐDTMendRead() pair, DTMreadMsg() combines the functionality of the trio of functions into one call. However, DTMreadMsg() is not as useful as DTMwriteMsg(). Neither the exact number of elements to be read nor the data type is usually known in advance, and DTMreadMsg() provides no way of basing memory allocation on information contained in the header. Additionally, the function call overhead saved by using DTMreadMsg() is nonexistent: internally the code consists of calls to those three functions. \subsection{Reading Multiple Data Set and Types} Often it is not possible to determine ahead of time the exact amount of data to be read. (This is especially true for messages of the SDL class to be discussed later.) A flexible way to handle reading when dimensions are unknown is to make multiple calls to DTMreadDataset(), filling a buffer with contiguous pieces of the data set, processing the buffer and requesting more. Example 13 shows how this can be accomplished. \begin{figure} \hrule \begin{verbatim} #include ... int nelem = 0, /* Number of elements read */ datanum = 0, /* Data block size */ chunk = 1; /* Block # of data read */ char header[DTM\_MAX\_HEADER]; /* DTM header */ float dataset[MAXBUFSIZE]; /* Data set buffer */ ... DTMbeginRead(inport, header, sizeof header); while ((nelem=DTMreadDataset(inport,dataset,MAXBUFSIZE,DTM\_FLOAT)) > 0) { printf("\nRead chunk #%d of size %d\n", chunk++, MAXBUFSIZE); for (i = 0; i < nelem; i++, datanum++) printf("Data value %d = %f\n", datanum, dataset[i]); } DTMendRead(inport); \end{verbatim} \hrule \caption{ Example 13. Read the data using multiple DTMreadDataset() calls. The loop that reads the data is designed to read the data in MAXBUFSIZE sized chunks, independent of the block size used to write the data, and will print out the values received to demonstrate that the data was properly received. } \end{figure} Reading multiple data types can be handled in a similar fashion, however it is important to ensure that the types are being read in the proper order. Reading data as one type when it was sent as another produces undefined results. Example 14 shows how to read multiple data types using multiple calls to DTMreadDataset(). \begin{figure} \hrule \begin{verbatim} #include ... int dataval = 1, /* The current data value*/ nelem = 0, /* # of elements read */ fdim = 0, /* Floating point dim */ idim = 0, /* Integer buffer dim */ intdata[MAXBUFSIZE]; /* Integer buffer */ char header[DTM\_MAX\_HEADER]; /* DTM header */ float floatdata[MAXBUFSIZE]; /* Floating point buffer */ ... DTMbeginRead(inport, header, sizeof header); sscanf(header, "This is the header. Fdim: %d Idim: %d", &fdim, &idim); DTMreadDataset(inport, floatdata, fdim, DTM\_FLOAT); DTMreadDataset(inport, intdata, idim, DTM\_INT); DTMendRead(inport); \end{verbatim} \hrule \caption{ Example 14. Read the two datasets using two DTMreadDataset() calls. Remember that since each call to DTMreadDataset() has no way of determining the type of data ahead of time, the dimensions of the data must be parsed ahead of time from the header, and exactly that many elements of each type must be read. In this example, it is assumed that the header contains this information. } \end{figure} \subsection{Determining Reading Capability} Two functions return the status of an input port. They vary in complexity and their use should be dependent upon the type of behavior desired. The simplest function, DTMavailRead(), polls the input port for availability and returns a boolean value depending on its status. The return of a true value indicates that a message is waiting to be received on the given port. Example 15 demonstrates how to use DTMavailRead() to continue processing information while waiting for input. \begin{figure} \hrule \begin{verbatim} #include ... int iteration = 1, /* Main loop counter */ inport = DTMERROR; /* The input port */ char header[DTM\_MAX\_HEADER]; /* The message header */ ... while(!DTMavailRead(inport)) printf("Do something useful for iteration #%d.\n", iteration++); DTMbeginRead(inport, header, sizeof header); DTMendRead(inport); \end{verbatim} \hrule \caption{ Example 15. Poll a port for waiting messages. The loop will continuously execute until the message has been successfully read from the input port. } \end{figure} A second function, DTMselectRead()1, performs a system select call on a list of DTM ports and a list of file or socket descriptors. DTMselectRead() allows the application to poll multiple ports and file descriptors simultaneously. This function will return when one of two conditions has been met: a DTM port or file descriptor has become ready for reading or the time-out value supplied to DTMselectRead() has expired. The time-out value given to DTMselectRead() is set in milliseconds. A value of -1 indicates that DTMselectRead() should block indefinitely waiting for a port or file descriptor to become ready. Two data structures exist for determining whether a port or a descriptor is ready for reading: Sock\_set and Dtm\_set. The structure of Dtm\_set is shown in Example 16. When DTMselectRead() returns, all of the ports that have messages pending will have the status flag element set to the value DTM\_PORT\_READY; all others have the value DTM\_PORT\_NOT\_READY. DTMselectRead() resets the status flag of each structure on each invocation. Therefore an application need only check the flag - it does not need to reset the flag between calls to DTMselectRead(). \begin{figure} \hrule \begin{verbatim} typedef struct Dtm\_set { int port; /* An input port id */ int status; /* The status of the port */ }Dtm\_set ; \end{verbatim} \hrule \caption{ Example 16. The data structure, Dtm\_set, used by DTMselectRead() for determining the input availability of a list of ports. The first element of the structure is the port id created by a call to DTMmakeInPort(). The second element is the flag indicating the status of the port. If the port has a message waiting to be received, status is set to DTM\_PORT\_READY, otherwise DTM\_PORT\_NOT\_READY. } \end{figure} Example 17 shows how DTMselectRead() can be used to determine which input ports in a list are available for reading. More complex examples of how DTMselectRead() can be used to identify a file descriptor's status are provided with the sample programs in the distribution package and are outside of the scope of this tutorial. \begin{figure} \hrule \begin{verbatim} #include ... int i = 0, /* Argument counter */ error = DTM\_OK; /* DTMselectRead error code */ Dtm\_set portset[2]; /* Port connection information */ ... /* Initialize the Dtm\_set structure to default values here. */ ... for (i = 1; i < argc; i++) { if (!strcmp(argv[i], "-port1")) portset[0].port = DTMmakeInPort(argv[++i], DTM\_DEFAULT); if (!strcmp(argv[i], "-port2")) portset[1].port = DTMmakeInPort(argv[++i], DTM\_DEFAULT); } ... /* Check for errors in creating the ports */ ... while (1) { if (error = DTMselectRead(portset, 2, NULL, 0, -1)) { if (portset[0].status == DTM\_PORT\_READY) readMSG(portset[0].port); else if (portset[1].status == DTM\_PORT\_READY) readMSG(portset[1].port); } ... } \end{verbatim} \hrule \caption{ Example 17. Determine which port is ready for reading data by calling DTMselectRead() with the given port list. Read the message from the ports whose status flags indicate that they are ready. } \end{figure} \section{Error Detection} Success or failure of any DTM function can be determined by comparing the function's returned value against the predefined constant DTMERROR. Code fragments presented so far have ignored this value making the unsafe assumption that their operations have always succeeded. Unfortunately, there are a variety of causes of communications failure, most of which are beyond the programmer's control. In the event of an error, the global variable DTMerrno is set to indicate the type of error that occurred. Each error number has a specific definition which can be determined via the function DTMerrmsg(). Example 18 shows how Example 4 would look when designed to handle all possible DTM failures. \begin{figure} \hrule \begin{verbatim} #include ... int outport = DTMERROR; /* DTM output port */ char *header = "This is the header"; /* DTM header */ float dataset[BUFSIZE]; /* The data buffer */ ... /* Create the ports & initialize the data here. */ ... if (DTMbeginWrite(outport, header, DTMHL(header)) == DTMERROR) {{} printf("An error was encountered by DTMbeginWrite(){\}n"); printf("\t%s{\}n", DTMerrmsg(1)); exit(0); } if (DTMwriteDataset(outport,dataset,BUFSIZE,DTM\_FLOAT) == DTMERROR) { printf("An error was encountered by DTMwriteDataset()\n"); printf("\t%s\n", DTMerrmsg(1)); exit(0); } if (DTMendWrite(outport) == DTMERROR) { printf("An error was encountered by DTMendWrite()\n"); printf("\t%s\n", DTMerrmsg(1)); exit(0); } \end{verbatim} \hrule \caption{ Example 18. In this example, each of the DTM writing functions are checked for the return of an error. If an error has occurred (the function has returned a value of DTMERROR), print a warning message tailored for that particular function. Also print out the standard DTM error message using DTMerrmsg(), and then exit the program. None of these errors are fatal, so the use of exit() is not required. } \end{figure} Flexible modules are designed to handle error conditions gracefully but the level of flexibility depends on the severity of the error and upon the developer's needs. The programmer must decide how to handle error conditions. A complete list of possible DTMerrno values and the corresponding definitions appears in Appendix A. \section{Message Classes} Each DTM message used in NCSA applications is considered an instance of a given message class and the DTM library contains a set of methods for building and parsing the message header. The class information contained in the message header provides information on the type of the data contained in each message. While DTM message classes are useful, they are not part of DTM proper. Rather, they represent the protocol NCSA applications use to exchange messages. To allow for interoperation between NCSA applications and other applications, users are encouraged to use the message classes provided. If the message classes provided are insufficient, the user has two other options: develop a new message class or a new message system. Define a new message class allows some interoperability, although NCSA applications will not be able to make use of these messages. If the message class system is inappropriate, the user can develop a new message system, remembering that both the header and data section of a message are optional. In the preceding sections, the code presented was designed for simplicity - both the data dimensions and the data type were assumed and the header contained no useful information. Essentially each message was classless. This is usually not the case. Normally, predefined class methods are inherited from a parent class to create the header for transmission and complementary methods are used to parse the header during reception. DTM comes with several of these predefined message classes. These classes are designed to cover the needs of the majority of the work done at NCSA: scientific visualization. Classes exist for passing scientific data sets, raster images and palettes, and surface descriptions. New user-defined classes should be subclassed from another class or the root DTM class. Any new data values or tags added to a header must have macros and functions for constructing and parsing the header. Since character string headers should not exceed DTM\_MAX\_HEADER characters in length (including the null character), large amounts of data should be sent in the data section of the message. DTM root class functions and macros that may be inherited all start with the letters "dtm\_" and perform operations solely on the header. Their definitions are provided in an appendix. Example 19 shows how the SDS class has implemented two of its macros, SDSsetClass and SDScompareClass. \begin{figure} \hrule \begin{verbatim} #define SDSclass "SDS" #define SDSsetClass(header) dtm\_set\_class(header, SDSclass) #define SDScompareClass(header) dtm\_compare\_class(header, SDSclass) \end{verbatim} \hrule \caption{ Example 19. An SDS class macro's behavior is actually defined by using DTM primitive functions. } \end{figure} \subsection{A Simple Message Class: MSG} The minimum amount of information a header should contain is a class identifier, usually an acronym or mnemonic that describes how to interpret the data. This constant should be unique and should not conflict with any previous existing classes. Example 20 shows a header created by the simplest message class MSG and how it is dissected. To use the MSG class macros and functions, include the file dtm.h. \begin{figure} \hrule \begin{verbatim} MSG STR 'This is a MSGclass message.' Message class: "MSG" String tag: "STR" String data: "This is a MSGclass message." \end{verbatim} \hrule \caption{ Example 20. A simple MSG class message. Note that the tag "STR" is used to indicate that the delimited string is the subject of this message. } \end{figure} MSG class messages contain only three pieces of information: the class identifier "MSG" and a tag "STR" where the characters following "STR" (delimited by single quotes) form the text string that is the subject of this message. This header was created using macros provided in the header file dtm.h. Macros to parse the header Ñ macros that produced the dissection found in Example 20 Ñ are also present in dtm.h. Example 21 shows how these macros are used to create and parse the header. \begin{figure} \hrule \begin{verbatim} #include #include ... char *header[DTM\_MAX\_HEADER]; /* The MSG class header */ ... /* Constructing the header */ MSGsetClass(header); MSGsetString(header, "This is a MSGclass message."); ... /* Parsing the header */ if (MSGcompareClass(header)) { MSGgetString(header, string); printf("The string is: '%s'\n", string); } \end{verbatim} \hrule \caption{ Example 21. MSG class construction and parsing are presented together in this example. The size of the header is limited to be DTM\_MAX\_HEADER bytes in length - a value defined in the header file dtm.h. The macro MSGcompareClass() causes all non-MSG class messages to be ignored. } \end{figure} \subsection{Scientific Data Sets: SDS} The scientific data set class (SDS) is a message class designed specifically for passing multidimensional arrays of data. The message header contains information about the array being sent such as the dimensions and data type of the array while the data portion of the message consists of the actual elements of the array. To use the SDS class macros and functions, include the file sds.h. An SDS header can contain information about the following items: the rank and dimensions of the data set, the minimum and maximum values contained in the data, the type of the data, and a character string title. The rank and dimension information is required, the other attributes are optional. If no data type is specified, the type is assumed to be floating point. Example 22 shows a sample piece of code for creating an SDS class message and writing it to a given port. \begin{figure} \hrule \begin{verbatim} #include #include ... int rank, /* Rank of the dataset (# of dims) */ size, /* Size of the floating pt. buffer */ dims[3]; /* Dimensions for this example */ char header[SDSsize]; /* SDS header */ float *array = NULL; /* Data array */ ... /* Port creation and data creation code here */ ... SDSsetClass(header); SDSsetDimensions(header, rank, dims); SDSsetTitle(header, "This is an SDS array"); SDSsetType(header, DTM\_FLOAT); DTMwriteMsg(outport,header,SDSHL(header),array,size,DTM\_FLOAT); \end{verbatim} \hrule \caption{ Example 22. Create an SDS header using the SDS class header creation functions. Attempt to write it and a floating point array of size size out the output port using DTMwriteMsg(). } \end{figure} Reading scientific data sets can be tricky if maximum flexibility is desired. The dimensions of the array can be determined from the header and memory for the data allocated automatically. The data type can also be retrieved should the type differ from the default floating point. Example 23 gives a moderately flexible piece of code for reading a multidimensional array from a given DTM port. \begin{figure} \hrule \begin{verbatim} #include #include ... int rank = 0, /* Rank of the dataset (# of dims) */ size = 0, /* Size of the array in floats */ dims[10]; /* Max # of dims for this example */ float minimum = 0.0, /* Minimum value */ maximum = 0.0, /* Maximum value */ *array = NULL; /* The array of data */ char header[SDSsize], /* SDS header */ title[80]; /* Title of this message */ DTMTYPE type = DTM\_FLOAT; /* Default type is DTM\_FLOAT */ ... DTMbeginRead(inport, header, sizeof header); if (SDScompareClass(header)) { SDSgetMinMax(header, &minimum, &maximum); SDSgetType(header, &type); SDSgetTitle(header, title, sizeof title); SDSgetDimensions(header, &rank, dims, 10); size = SDSnumElements(rank, dims); array = (float *)malloc(size * sizeof(float)); DTMreadDataset(inport, array, size, DTM\_FLOAT); } DTMendRead(inport); \end{verbatim} \hrule \caption{ Example 23. Receive and parse an SDS message. Assume that there are no errors in any of the DTM functions but check to be certain that the message class received is indeed an SDS class message. If the message is an SDS class message, parse the various pieces of information, but don't do anything with them. Allocate space based on the dimensions, read the data and then end reading from that port. } \end{figure} \subsection{Raster Image Sets and Palettes: RIS \& PAL} Raster images and palettes are commonly used in scientific visualization. The raster image set (RIS) class of messages allows for two types of images to be sent, eight bit and twenty-four bit. In the case of eight bit images, a separate message class has been created for handling the palette. The palette (PAL) message class makes no restrictions on the size of the palettes, though the default is set to be 256 individual colors. To use the RIS class and PAL class macros and functions, include the header file ris.h. Example 24 shows how a previously created eight bit image is sent along with a palette. Code to receive this information is given in Example 25. \begin{figure} \hrule \begin{verbatim} /*****************************************************************/ /* Create an RIS message. Assume that the image already exists. */ /*****************************************************************/ int xdim = 256, /* X dimension of the image */ ydim = 256, /* Y dimension of the image */ size = xdim * ydim; /* Size of the array */ char header[RISsize], /* SDS header */ *image = NULL; /* Raster image */ ... RISsetClass(header); RISsetDimensions(header, xdim, ydim); RISsetTitle(header, "This is an RIS image"); RISsetType(header, RIS8BIT); DTMwriteMsg(outport, header, RISHL(header), image, size, DTM\_CHAR); ... /******************************************************************/ /* Create an PAL message. Assume that the palette exists. */ /******************************************************************/ int elements = 768; /* Number of elements in palette */ char header[PALsize], /* SDS header */ *palette = NULL; /* The palette */ ... PALsetClass(header); PALsetSize(header, elements/3); PALsetTitle(header, "This is a grayscale palette"); DTMwriteMsg(outport,header,PALHL(header),palette,elements,DTM\_CHAR); \end{verbatim} \hrule \caption{ Example 24. This example shows how to create RIS class and PAL class messages using the header constructors provided with each class. Even though the default type of image is RIS8BIT, and the default number of elements in a palette is 256, it is best to set them explicitly. } \end{figure} \begin{figure} \hrule \begin{verbatim} int size = 0, /* Size of the array in floats */ xdim = 0, /* X dimension of the image */ ydim = 0; /* Y dimension of the image */ char title[80], /* Title of this message */ *image = NULL, /* Image received */ *palette = NULL; /* The color palette */ RISTYPE type = RIS8BIT; /* Default type is RIS8BIT */ /*****************************************************************/ /* Check to see if the message class is RIS */ /*****************************************************************/ if (RIScompareClass(header)) { ... RISgetDimensions(header, &xdim, &ydim); RISgetTitle(header, title, sizeof title); RISgetType(header, &type); size = xdim * ydim * ((type == RIS24BIT) ? 3 : 1); image = (char *)malloc(size * sizeof(char)); DTMreadDataset(inport, image, size, DTM\_CHAR); } ... /*****************************************************************/ /* Check to see if the message class is PAL */ /*****************************************************************/ else if (PALcompareClass(header)) { ... PALgetTitle(header, title, sizeof title); PALgetSize(header, &elements); palette = (char *)malloc(3 * elements * sizeof(char)); DTMreadDataset(inport, palette, 3*elements, DTM\_CHAR)) } \end{verbatim} \hrule \caption{ Example 25. Compare the message class of an already received header to see if it is an RIS class message. If it isn't an image, check to see if the message is a PAL class message, creating the appropriate data structures based on the message class and the image type. No error checking is performed to ensure that the number of pixels read equals the number indicated by the dimensions, nor if the DTM functions return error flags. } \end{figure} \subsection{Surface Description Language: SDL} The surface description language (SDL) is a simple way of representing three dimensional information. The header supplies information about how the DTM\_TRIPLET values are to be organized when finally displayed. The data gives the coordinate information as well as information on surface color and surface normals. To use the SDL class macros and functions, include the header file sdl.h. DTM\_TRIPLETS contain three floating point values and an integer tag describing how those values are to be interpreted. When triplets are used in conjunction with the SDL message class, those tags are predefined with particular meanings as shown in Example 26. \begin{figure} \hrule \begin{verbatim} TAG VALUE INTERPRETATION SDLposition The coordinates in three-space SDLcolor The color in space with a where 0 ² r,g,b ² 1.0 SDLnormal A normal vector in three-space SDLtranslate A translation vector SDLrotate Rotation about the x, y, and z axes. SDLscale A scaling transform along the x, y and z axes. SDLmaxTriplet The upper limit on reserved values (= 1024). \end{verbatim} \hrule \caption{ Example 26. The tag value of a DTM\_TRIPLET as it is used in conjunction with the SDL message class. DTM has reserved the values from SDLmaxTriplet down for further development with all other values available for independent use. } \end{figure} In general, SDLcolor and SDLnormal triplets apply to all SDLposition values that follow until superseded by another SDLcolor or SDLnormal respectively. SDLtranslate, SDLrotate and SDLscale all apply the viewing transform to the current transform matrix (or however the transforms are being represented). How these data representations are to be organized depends on the value of the "primitive" type specified. Again there are predefined SDL constants with particular meanings, as shown in Example 27. \begin{figure} \hrule \begin{verbatim} PRIMITIVE TYPE INTERPRETATION SDLpoint Each SDLposition is a separate point. SDLlineseg Pairs of SDLposition values form a line segment. SDLtriangle Trios of SDLposition values form a triangle. SDLquad Groups of four SDLposition values form a quadrilateral. SDLsphere Each value is the center of a sphere of unspecified radius. SDLmaxPrim\_t Maximum number of reserved primitive types (= 1024). \end{verbatim} \hrule \caption{ Example 27. The different primitive types defined for the SDL message class. DTM has reserved the values from SDLmaxPrim\_t down for further development with all other values available for independent use. } \end{figure} Example 28 contains a code fragments for creating and parsing an SDL class message. \begin{figure} \hrule \begin{verbatim} #include #include ... /*******************************************************************/ /* Read an SDL class message, parsing the header. */ /*******************************************************************/ char header[DTM\_MAX\_HEADER]; /* Input header */ struct DTM\_TRIPLET triplets[TBUF]; /* Triplet data buffer */ ... DTMbeginRead(inport, header, sizeof header); if (SDLcompareClass(header)) { SDLgetTitle(header, title, sizeof title); SDLgetPrimitive(header, &primitive); while ((nelem = DTMreadDataset(inport,triplets,TBUF,DTM\_TRIPLET)) > 0) printTriplets(triplets, nelem); } DTMendRead(inport); ... /*******************************************************************/ /* Write an SDL class message. */ /*******************************************************************/ char header[SDLsize]; /* The output header */ struct DTM\_TRIPLET triangle[NUMTRIS]; /* Triangle data */ ... SDLsetClass(header); SDLsetTitle(header, "Example 7.7: A Triangle"); SDLsetPrimitive(header, SDLtriangle); DTMwriteMsg(outport,header,SDLHL(header),triangle,NTRIS,DTM\_TRIPLET) \end{verbatim} \hrule \caption{ Example 28. This example demonstrates how to parse and how to construct an SDL class message. When reading, if the message class is SDL, get the title and primitive type then read and interpret the data. When writing, create a header with all of the pertinent information. In both cases, the SDL class constructor functions are used to build the header. In both cases, errors are ignored. 1DTMselectRead() is being replace by a new routine, DTMselect(). This routine provides a more consistent interface and allows applications to check a port for writing as well as reading. The routine will be available in DTM2.4. Although DTMselectRead() will be available for backward compatibility, users should be prepared to use DTMselect(). } \end{figure} \appendix \chapter{Appendices} The appendices present a collection of information that couldn't be presented in the regular text: the possible error codes and their meanings, prototypes for the DTM functions (both C and Fortran), and a listing of the commonly used DTM data structures. For more information about the functions listed, consult each function's UNIX manpage. \begin{itemize} \item{B.} Error Codes and Their Meanings \item{C.} Function Prototypes \item{D.} Structure Definitions \item{E.} Fortran Specifications \end{itemize} \chapter{Error Codes and Their Meaning} \section{Overview} This appendix lists all of the possible values that the variable DTMerrno can take and the meaning of each of those values as would be printed by a call to DTMerrmsg(0). The definitions in parenthesis are provided for when further clarification is necessary. \begin{verbatim} Values Meanings DTM_OK: DTMNOERR: No error. DTMMEM Out of memory - can not create port. (Out of memory.) DTMHUH Invalid port name - should be 'hostname:tcp port'. DTMNOPORT Out of DTM ports - 256 ports max. (No DTM ports available.) DTMPORTINIT Couldn't initialize port. (DTM port not initialized.) DTMCALL DTM routines called in wrong order. DTMEOF Encountered EOF. (EOF error.) DTMSOCK Error creating socket. (Socket error.) DTMHOST Bad hostname. (That hostname is not found/bad.) DTMTIMEOUT Timeout waiting for connection. DTMCCONN Couldn't connect. (DTM cannot connect (network down?).) DTMREAD DTM read error. (Error returned from system read.) DTMWRITE DTM write error. (Error returned from system write(v).) DTMHEADER DTM header to long for buffer. DTMSDS SDS error. DTMSELECT Select call error. DTMENV Environment not setup. DTMBUFOVR User buffer overflow. DTMCORPT Port table corrupted. DTMBADPORT Bad port supplied to library. (Port identifier is bad/corrupt/stale.) DTMBADACK Bad ack to internal flow control. DTMADDR Bad address. DTMSERVER Problem communicating with server. \end{verbatim} \chapter{ASCII File Format} \section{Overview} This appendix lists the available DTM functions and macros. These routines are presented in alphabetical order starting with the functions beginning with "DTM" and ending with those beginning with "dtm\_". Those functions beginning with "DTM" are the functions that deal with creating ports, altering their behavior, and delivering messages, whereas the "dtm\_" set of primitive functions usually deal with the construction of messages. This appendix is limited to the functions that make up the DTM message passing system and the primitive functions for header manipulation. For information on individual message class functions and macros, consult the UNIX man page for the particular class desired. Each function below is presented using the non-ANSI, Kernighan \& Ritchie method of declaring functions. Each of these functions has a prototype that is included only if the compiler is ANSI compliant. The K\&R method of function declaration is used here to make argument comments easier to comment. For more information about the behavior of these functions, consult the individual UNIX man pages. \section{Functions} DTMaddInPortSocket Takes a file descriptor and adds it as a connection to a DTM port. Used by applications that are started by inetd to remap stdin into a DTM port. \begin{verbatim} int DTMaddInPortSocket(port, socket) int port; /* Add the connection to this port. */ int socket; /* The file descriptor to add. */ \end{verbatim} \subsection{DTMaddInput} Add a callback procedure to the given port based on the given input condition. The port must be an input port. Note: if X is not included, it is not likely that this function will be used. However, support for it is given in the case when the typedef for XtInputCallbackProc does not exist. In this case, the argument proc should be of type DTMfuncPtr. \begin{verbatim} int DTMaddInput(port, condition, proc, client\_data) int port; /* The input port. */ long condition; /* The condition mask. */ caddr\_t client\_data; /* Optional client data */ XtInputCallbackProc proc; /* The callback procedure */ (DTMfuncPtr proc;) \end{verbatim} \subsection{DTMavailRead} Poll a port to determine if it has a message pending. \begin{verbatim} int DTMavailRead(port) int port; /* The input port id to poll */ \end{verbatim} \subsection{DTMavailWrite} Poll a port to determine if it is capable of sending a message. \begin{verbatim} int DTMavailWrite(port) int port; /* The output port id to poll */ \end{verbatim} \subsection{DTMbeginRead} Begin to read a message from the given port, blocking until a message is ready to be received. Store the maximum number of bytes of the message header in the given header buffer, discarding the remainder. Return the number of bytes read into the header or DTMERROR upon an error. \begin{verbatim} int DTMbeginRead(port, header, num\_bytes) int port; /* The input port */ void *header; /* The buffer to store the header */ int num\_bytes; /* The length of the header buffer */ \end{verbatim} \subsection{DTMbeginWrite} Begin to write a message header from the given port, blocking until an input port is ready to receive the message. Send the given number of bytes of the header buffer. Return the value DTM\_OK upon having written the header, DTMERROR otherwise. \begin{verbatim} int DTMbeginWrite(port, header, num\_bytes) int port; /* The output port */ void *header; /* The message header buffer */ int num\_bytes; /* The length of the header */ \end{verbatim} \subsection{DTMcheckRoute} Checks for new routing information. Applies only to output ports (as routing information concerns only output ports). \begin{verbatim} int DTMcheckRoute(port) int port; /* The output port to check. */ \end{verbatim} \subsection{DTMdestroyPort} Close all connections associated with the given port and remove the port id from the port table. \begin{verbatim} int DTMdestroyPort(port) int port; /* The port id to destroy */ \end{verbatim} \subsection{DTMendRead} Mark the end of the current message, discarding any data that remains, and prepare to receive the next message. \begin{verbatim} int DTMendRead(port) int port; /* The input port */ \end{verbatim} \subsection{DTMendWrite} Mark the end of the current message, signaling that no more data currently is to be transmitted across the given port. \begin{verbatim} int DTMendWrite(port) int port; /* The output port */ \end{verbatim} \subsection{DTMerrmsg} Print or return an error message corresponding to the current status of the DTMerrno value. \begin{verbatim} char *DTMerrmsg(quiet) int quiet; /* Print/return the error string? */ \end{verbatim} \subsection{DTMgetConnectionCount} Returns the number of ports connected to a given port id. The port id can represent either an input or an output port. The second argument is currently unused. \begin{verbatim} int DTMgetConnectionCount(port, num) int port; /* The port id to check */ int *num; /* Unused at this time */ \end{verbatim} \subsection{DTMgetPortAddr} Get the character string address of the supplied port id, and store the first length characters of that address into the buffer address. The address is returned in form "000.000.000.000:0000." \begin{verbatim} int DTMgetPortAddr(port, address, length) int port; /* The port id */ char *address; /* The IP address returned */ int length; /* The length of the address buffer */ \end{verbatim} \subsection{DTMgetRemotePortAddr} Returns the list of remote addresses that an output port is connected to. \begin{verbatim} int DTMgetRemotePortAddr(port, addresses, num\_addrs) int port; /* The output port id. */ char ***addresses; /* The returned list of addrs. */ int *num\_addrs; /* The number of returned addrs */ \end{verbatim} \subsection{DTMmakeInPort} Create an input port id from the given port name using the specified quality of service. Only a quality of service value of DTM\_DEFAULT has any meaning. The value returned is either the new port id or the error flag DTMERROR. \begin{verbatim} int DTMmakeInPort(portname, qservice) char *portname; /* The port number and address */ int qservice; /* The quality of service */ \end{verbatim} \subsection{DTMmakeOutPort} Create an output port id from the given port name using the specified quality of service. The value returned is either the new port id or the error flag DTMERROR. \begin{verbatim} int DTMmakeOutPort(portname, qservice) char *portname; /* The port number and address */ int qservice; /* The quality of service */ \end{verbatim} \subsection{DTMreadDataset} Read the number of elements of data of the specified data type from the specified port and store them in the dataset buffer. The value returned is the number of elements read or DTMERROR if an error during reading was encountered. \begin{verbatim} int DTMreadDataset(port, dataset, elements, datatype) int port; /* The input port to read from */ void *dataset; /* The data set buffer */ int elements; /* Available # of buffer elements */ DTMTYPE datatype; /* The data type to read */ \end{verbatim} \subsection{DTMreadMsg} Read from the supplied input port the maximum number bytes of the header and the given number of elements of the specified data type into separate buffers. Return the number of data elements read or the value DTMERROR if an error was encountered during the read. \begin{verbatim} int DTMreadMsg(port, header, num\_bytes, dataset, elements, datatype) int port; /* The input port to read from */ char *header; /* The header buffer */ int num\_bytes; /* The # of bytes in the header */ void *dataset; /* The data set buffer */ int elements; /* Available # of buffer elements */ DTMTYPE datatype; /* The data type to read */ \end{verbatim} \subsection{DTMselectRead} Perform a system select on the given port list and the list of socket or file descriptors. Time-out after the given number of milliseconds or do not block if the number specified is less than 0. Set the status flags for each of the ports according to their readiness for reading. \begin{verbatim} int DTMselectRead(dtmset, num\_ports, sockset, num\_desc, period) Dtm\_set *dtmset; /* The list of ports & status flags */ int num\_ports; /* The number of ports in the list */ Sock\_set *sockset; /* The list of descriptors */ int num\_desc; /* The number of descriptors */ int period; /* The time-out in milliseconds */ \end{verbatim} \subsection{DTMsendRoute} Send a message to an output port instructing it to add or delete connections. \begin{verbatim} int DTMsendRoute(fd, sendto, addcount, addlist, rmcount, rmlist) int fd; /* The output socket. */ char *sendto; /* The destination address:port #. */ int addcount; /* The number of addresses to add. */ char **addlist; /* The list of addresses to add. */ int rmcount; /* Remove this many addresses. */ char **rmlist; /* The list of addresses to remove. */ \end{verbatim} \subsection{DTMwriteDataset} Write the given number of data elements of the specified data type to the specified port and store them in the dataset buffer. The data type is converted automatically if needed. The value returned is DTM\_OK upon successful writing of the data or DTMERROR if an error was encountered. \begin{verbatim} int DTMwriteDataset(port, dataset, elements, datatype) int port; /* The output port */ void *dataset; /* The data buffer */ int elements; /* Number of data elements to send */ DTMTYPE type; /* The data type */ \end{verbatim} \subsection{DTMwriteMsg} Write the specified number of bytes of the header and the number of elements of a particular type of the dataset to the given output port. All of the header and data buffer are written using one system send. The data type is converted automatically if needed. The value returned is DTM\_OK upon successful writing of the data or DTMERROR if an error was encountered. \begin{verbatim} int DTMwriteMsg(port, header, num\_bytes, dataset, elements, datatype) int port; /* The output port */ void *header; /* The message header buffer */ int num\_bytes; /* The length of the header */ void *dataset; /* The data buffer */ int elements; /* Number of data elements to send */ DTMTYPE type; /* The data type */ \end{verbatim} \subsection{dtm\_find\_tag} Return the position of the given tag within the given header. If the tag is not found within the tag, a NULL value is returned. \begin{verbatim} char *dtm\_find\_tag(header, tag) char *header; /* The header */ char *tag; /* The tag to search for */ \end{verbatim} dtm\_header\_length Determine the number of bytes in the given header up to (and counting) the terminating null character ('$\backslash$0'). \begin{verbatim} dtm\_header\_length(header) char *header; /* Determine this header's length */ \end{verbatim} \subsection{dtm\_get\_char} Get up to the given length characters of the delimited string following the specified tag and store the result in the destination string. Return 0 with success. If no delimited text string exists in the header, or if the tag value is not found, return the value DTMERROR. \begin{verbatim} int dtm\_get\_char(header, tag, dest, length) char *header; /* The header */ char *tag; /* The tag to search for */ char *dest; /* The string destination buffer */ int length; /* Number of destination bytes */ \end{verbatim} \subsection{dtm\_get\_float} Search the character string header for the given tag value and assign the floating point value that follows that tag to the floating point argument. Conversion of the string real value is accomplished through the standard C function sscanf. If the tag value is not found within the header, or if no value follows the tag, return a value of DTMERROR, otherwise return 0. \begin{verbatim} int dtm\_get\_float(header, tag, realval) char *header; /* The header */ char *tag; /* The tag to search for */ float *realval; /* The value found */ \end{verbatim} \subsection{dtm\_get\_int} Search the character string header for the given tag value and assign the integer value that follows that tag to the integer argument. Conversion of the string integer value is accomplished through the standard C function sscanf. If the tag value is not found within the header, or if no value follows the tag, return a value of DTMERROR, otherwise return 0. \begin{verbatim} int dtm\_get\_int(header, tag, integer) char *header; /* The header */ char *tag; /* The tag to search for */ int *integer; /* The value found */ \end{verbatim} \subsection{dtm\_set\_char} Add the character string tag to the header immediately followed by the source string. The character string source is delimited by single quote marks so that the contents of the source do not adversely affect interpretation of the header. \begin{verbatim} void dtm\_set\_char(header, tag, source) char *header; /* The header */ char *tag; /* The tag to insert */ char *source; /* The string to insert */ \end{verbatim} \subsection{dtm\_set\_float} Add the character string tag to the header immediately followed by the given floating point value written as a string. The floating point argument is converted to a string using the standard C function sprintf. \begin{verbatim} void dtm\_set\_float(header, tag, realval) char *header; /* The header */ char *tag; /* The tag to insert */ float realval; /* The value to insert */ \end{verbatim} \subsection{dtm\_set\_int} Add the character string tag to the header immediately followed by the given integer written as a string. The integer is converted to a string using the standard C function sprintf. \begin{verbatim} void dtm\_set\_int(header, tag, integer) char *header; /* The header */ char *tag; /* The tag to insert */ int integer; /* The value to insert */ \end{verbatim} \subsection{dtm\_version} Print the current version of the DTM library to stderr. void dtm\_version() \chapter{Structure Definitions} \section{Overview} This list is a compilation of the individual data structure definitions created by DTM and used by the functions presented in Appendix B. It is provided for reference only. \section{Functions} THIS SECTION DESCRIPTION(for latex purposes only) \subsection{DTMqserv} The quality of service options for ports. \begin{verbatim} #include #define DTM\_DEFAULT DTM\_SYNC typedef enum { DTM\_SYNC = 0, /* Synchronous communication */ DTM\_ASYNC /* Asynchronous communication */ }DTMqserv ; \end{verbatim} \subsection{Dtm\_set} DTMselectRead port and status information. \begin{verbatim} #include typedef struct Dtm\_set { int port; /* DTMmakeInPort port id */ int status; /* DTM\_PORT\_NOT\_READY */ }Dtm\_set; \end{verbatim} \subsection{DTM\_TRIPLET} Three dimensional data structure type used primarily by the SDL class. \begin{verbatim} #include struct DTM\_TRIPLET { int tag; /* Triplet interpretation tag */ float x, y, z; /* Three-space coordinate */ }; \end{verbatim} \subsection{DTMTYPE} Standard data types for which type converters exist and are supported by DTM. This list of types is subject to future revisions. \begin{verbatim} #include typedef enum { DTM\_CHAR = 0, /* Character data types */ DTM\_SHORT, /* Short ints (2 bytes) */ DTM\_INT, /* Integers (4 bytes) */ DTM\_FLOAT, /* IEEE floating point format */ DTM\_DOUBLE, /* IEEE double precision */ DTM\_COMPLEX, /* FORTRAN complex (two floats) */ DTM\_TRIPLET /* DTM triplets: one int, two f.p. */ }DTMTYPE; \end{verbatim} \subsection{Sock\_set} Socket and file descriptor status information. \begin{verbatim} #include typedef struct Sock\_set { int sockfd; /* Socket or file descriptor */ int status; /* Status of the descriptor */ }Sock\_set; \end{verbatim} \chapter{Fortran Specifications} \section{Overview} Currently, only a limited number of the DTM functions are supplied with Fortran equivalents. These functions provide only the basic functionality of DTM Ñ port creation, and reading and writing data. Below is a listing of the available Fortran functions with their corresponding C equivalent. Unless otherwise commented, each Fortran function behaves exactly as its C counterpart. \begin{verbatim} DTMAR (DTMavailRead) integer function DTMAR(port) integer port DTMAW (DTMavailWrite) integer function DTMAW(port) integer port DTMBR (DTMbeginRead) integer function DTMBR(port, header, hdrlen) integer port character header*(*) integer hdrlen DTMBW (DTMbeginWrite) integer function DTMBW(port, header, hdrlen) integer port character header*(*) integer hdrlen DTMER (DTMendRead) integer function DTMER(port) integer port DTMEW (DTMendWrite) integer function DTMEW(port) integer port DTMGPA (DTMgetPortAddr) integer function DTMGPA(port, portnm, pnmlen) integer port character portnm*(*) integer pnmlen DTMMIP (DTMmakeInPort) Equivalent to DTMmakeInPort() with the quality of service set to always be DTM\_DEFAULT. integer function DTMMIP(portnm) character portnm*(*) DTMMOP (DTMmakeOutPort) Equivalent to DTMmakeOutPort() with the quality of service predefined to always be DTM\_DEFAULT. integer function DTMMOP(portnm) character portnm*(*) DTMRD (DTMreadDataset) Even though the second argument "datset" is declared here to be of type real, any type of data can be passed to this function. integer function DTMRD(port, datset, size, type) integer port real datset integer size integer type DTMWD (DTMwriteDataset) Even though the second argument "datset" is declared here to be of type real, any type of data can be passed to this function. integer function DTMWD(port, datset, size, type) integer port real datset integer size integer type DTMWM (DTMwriteMsg) Even though the second argument "datset" is declared here to be of type real, any type of data can be passed to this function. integer function DTMWM (port, header, hdrlen, datset, dsize, type) integer port character header*(*) integer hdrlen real datset integer dsize integer type \end{verbatim} \end{document} .