.if n .pH port.chap07 @(#)chap07	40.4
.\" Porting manual: I/O Multiplexer Ch. 7
.\"
.\" Ps = POSTSCRIPT
.\" Sd = NeWS
.\" Cs = CSCRIPT
.ds Ps P\s-2OST\s+2S\s-2CRIPT\s+2
.ds Sd \s-2Ne\h'-0.2n'WS\s+2
.ds Cs \CS\s-2CRIPT\s+2
.BK "Porting the X11/NeWS Server"
.CH "I/O Multiplexing"  7
.H 1 "I/O Multiplexing" 
This section describes how to port the I/O Multiplexing 
function to the server.  
Although most of the coding excerpts and descriptions refer to the SunOS
implementation, the actual subroutine functionality described in this 
chapter is
implemented for the System V product.  Thus, the documentation can be
used as a reference for developers porting to either System V or BSD
derived systems.
.H 2 "NPSI"
.P
Before continuing, we should discuss NPSI, the non-portable systems
interface.  NPSI consists of a set of functions and routines that act
as an interface layer between the server and the new hardware platform
and Operating System.  The NPSI is provided with the source,
however, it may be necessary to port some of the NPSI to certain
systems.  Where special NPSI porting is required, it will be noted the
text.
.H 2 "I/O Multiplexing to the Server"
Once the \*(Ps interpreter is capable of reading a
file containing some basic \*(Ps code,
you are ready to proceed to the next step,  
which is to develop code to handle I/O multiplexing.
.P
I/O multiplexing is the method 
the server uses to read from, and write to, the various clients to 
which it is simultaneously connected.
.P
In order to get the I/O multiplexing part of the server working,
you will need to implement the following five new functions:
.BL
.LI
.UI WaitForInput()
.LI
.UI run_io_waiters()
.LI
.UI add_selectable_file()
.LI
.UI remove_selectable_file()
.LI
.UI make_async()
.LE
.P
Before we describe how each of these routines is implemented,
let us first discuss how they provide the server with I/O
multiplexing capabilities.
.H 3 "Preliminary Information for I/O Multiplexing"
When the server simply reads from a file, as it does in a \*(Ps 
Interpreter-only implementation, 
it will never encounter a situation where there will
be no data to read. However, in a complete implementation of the server,
this situation happens all the time.
.P
At any given time, the server will have many active connections to 
different clients.  The server will continually encounter situations 
where it is trying to read, but data will be unavailable on a given 
connection. This is also a possibility when the server is trying to write
to a connection. Flow and congestion control implemented in modern 
protocol stacks will cause situations where writes cannot be serviced 
immediately.
.P
The server handles flow and congestion control
by performing all reads and writes in a non-blocking fashion.
This means that the operating system will return control to the
program if the I/O operation can not be performed in a timely
fashion (i.e., if it would block).
If an operation would block, the O/S returns an indication,
usually in the form of an error code. When this happens,
the server can suspend the currently active
process thread and activate another process thread
if one is available.
.P
Note the proviso that another process thread be available.
This is not always the case.
All threads may be blocked waiting for I/O.
When this happens,
the server must call a routine called
.UI WaitForInput().
.P
The name of this subroutine is a bit of a misnomer, as
.br 
.UI WaitForInput() 
may also be called when the server is waiting to output.
Nevertheless, the purpose of this subroutine
is to block execution of the server until I/O is possible on one or
more of the input or output streams associated with a process thread.
.P
.UI WaitForInput()
is generally implemented using some facility of
the operating system provided for this purpose.
For example, on BSD derived UNIX systems
(including SunOS)
the 
.UI select(2) 
system call provides the needed functionality.
On System V derived systems,
.UI poll(2) 
provides this functionality.
.P
Both of these system calls present a list of file descriptors
in which a process is interested, and whether reading and
writing is possible. The system call blocks 
until one or more of the file descriptors
is available for I/O. 
In order for this to work, of course,
we have to have the list of file descriptors in
which we're interested. 
(One important consideration has to do with networking.  
The I/O multiplexing system call, 
.UI "poll
or
.UI "select" ,
must indicate that a listening socket is ready when a connection comes in.
See the section on 
.UI accept_connect()
interface in the Chapter on Network I/O.)
File descriptor I/O availability is constructed in a complex manner 
and is described below.
.H 3 "Struct Execution Environment"
The execution environment structure is used to describe a single
lightweight process thread.  It contains many fields, including the
the state in which the interpreter should restart the suspended thread 
.UI (restart_state)
,  a forward
pointer for constructing linked lists 
.UI (next) 
and others.

.H 3 "Constructing the I/O File Descriptor List"
.P
When an I/O attempt by a thread fails, the code sets the 
.UI ee\(hy>event 
field to the file descriptor for which I/O failed, plus one.
One is added because 0 indicates that the thread is runnable.
Thus we distinguish between a process blocked on 
.UI stdin
(file descriptor 0) and a process ready to run by adding one
to the file descriptor.  For example, here is the code for 
.UI pf_getc_nb 
in 
.UI parsefile.h .
.SS
#define pf_getc_nb(e,c) \\
	psio_pgetc_nb(f, c, \\
		ee->restart_state = e; \\
		ee->event = psio_fileno(f) + 1; \\
		goto suspend_process; \\
		CPPCONCAT(read_,e): \\
		f = es->env.file.file);
.SE
.P
The code in 
.UI run_process() 
checks 
.UI ee\(hy>event 
when the interpreter it invoked to run the thread returns
(i.e., when the interpreter has suspended the thread).
If the value is greater than zero,
a routine called
.UI add_selectable_file
is called along with the file descriptor which caused the
thread to block ( 
.UI ee\(hy>event\(hy1) .
.UI add_selectable_file
is also given an indication of whether the thread
was blocked due to reading or writing.
If due to writing,
.UI ee\(hy>event 
has the value
.UI event_write_flag
(
.UI 0x4000 )
added to it.
.P
.UI add_selectable_file() 
is then the mechanism by which
a list of file descriptors on which to multiplex 
is accumulated.
.P
A companion routine to
.UI add_selectable_file()
is
.UI remove_selectable_file().
This is used to remove a file from the
list of files on which to multiplex.
It is called when a file is closed, and at other 
times to clean up.
.P
When the O/S returns an indication that I/O is possible on a file
descriptor,
.UI WaitForInput() 
must run each process thread which was blocked waiting
for I/O.
Running each process is done by calling a routine called 
.UI run_io_waiters().
.P
Eventually, control is returned to the top of the 
.UI PostScript()
routine.  This routine beings with a switch statement which switches
on the restart state field and branches to the appropriate label.  
Note that the 
.UI pf_getc_nb 
macro causes a label to be placed in line
.UI (CPPCONCAT(read_,e)) .
For example, if the code contained 
.UI pf_getc_rb(10,c) ,
the label 
.UI read_10:
would be placed in line.  This is how process threads are suspended
and resumed.

An additional function is 
.UI make_async().
This is used by the server to make a file descriptor non\(hyblocking.
It is called by 
.UI open_file 
in 
.UI objects.c.

.H 3  \f5WaitForInput\fP
.P
.UI WaitForInput() 
has no entry parameters.
It does use the global list of files on which to multiplex, 
which is set up by 
.UI add_selectable_file() .
Two examples of 
.UI WaitForInput() 
are provided.
One is for Sun systems,
and follows the BSD style interface.
It uses 
.UI select(2) 
to multiplex on the list of file descriptors.
The other version is for System V systems, which 
uses 
.UI poll(2) 
to multiplex.  These are located in the following 
directory:
.P
Sun Systems:
.UI usr.bin/xnews/xnews/server/NPSI/os/sun/muxio.c
.P
SystemV Systems:
.UI usr.bin/xnews/xnews/server/NPSI/os/sysV/muxio.c
.P
If you're porting to a UNIX environment,
chances are one or the other of these can
be used as is or with little modification.
Later, we will modify 
.UI WaitForInput() 
to handle mouse and keyboard events, but for now 
handling just files will suffice.
.P
The basic job of 
.UI WaitForInput() 
is to block execution of the server until
one or more of the file descriptors, which a process thread is blocked on,
becomes available for I/O.
.P
.UI WaitForInput()
then calls 
.UI run_io_waiters() 
for all ready read file descriptors
and all ready write file descriptors.
.UI run_io_waiters() 
determines which threads can run and runs
each by calling 
.UI run_process.
.H 3  "\f5run_io_waiters()\fP"
.UI run_io_waiters() 
is a bit different from the other subroutines,
as it does not need to present a standard interface to the 
system\(hyindependent
part of the server: it is called only by 
.UI WaitForInput() .
In fact,
.UI run_io_waiters() 
has a different interface in the Sun and System V versions.
.P
On System V systems,
.UI WaitForInput()
passes 
.UI run_io_waiters() 
the number of the file descriptor which is ready for I/O.
On Sun systems, 
.UI WaitForInput()
passes to 
.UI run_io_waiters() 
the bit mask array returned by 
.UI select(2) .
Thus, for Sun systems,
.UI run_io_waiters() 
has to do a bit more work than for System V systems
in that it has to pick apart the bit masks,
in which each bit corresponds to a file descriptor.
The System V interface is simpler because the O/S returns the information
about what file descriptors are ready in a more straightforward form, in 
.UI pollfd 
structures 
(which is a System V data structure located in 
.UI "/usr/include/sys/pollfd.h" ),
each entry of which corresponds to a file descriptor.  
.P
While the input interface to 
.UI run_io_waiters() 
is insignificant, its functionality is not.
The server maintains a linked list of threads for each file descriptor
on which a thread may block.
So, when a file descriptor becomes available for I/O,
each thread which was blocked must be awakened.
The linked list is rooted in the array 
.UI process_io_waiters ,
which has one entry for each file descriptor.
The 
.UI next 
field of 
.UI "struct execution_environment" 
is a forward pointer to the next thread in the list.
.P
Both the System V and Sun versions of 
.UI run_io_waiters() 
have the following
code in common.  This code follows the linked list of blocked threads and
collects them into a list of process threads to awaken:
.SS
register struct execution_environment *ee;
register struct execution_environment *awakeq = 0;
register struct execution_environment **eep;
register desired_event;

remove_selectable_file(fd, read_flag);
desired_event = fd + (read_flag ? 1 : event_write_flag + 1);

/* make a process queue of all awakened processes */
eep = &process_io_waiters[fd];
while (ee = *eep) {
    if (ee->event == desired_event) {
        *eep = ee->next;
        if (awakeq) {
            ee->next = awakeq->next;
            awakeq->next = ee;
        } else
            ee->next = ee;
        awakeq = ee;
    } else
        eep = &(ee->next);
}
.SE
Note that this code also calls 
.UI remove_selectable_file() 
(described below) to
remove the file descriptor from the list of file descriptors 
on which to multiplex.
This is done since once the thread associated with the 
file descriptor is awakened,
we no longer need to select on it unless the thread becomes
blocked again. But if the thread becomes blocked,
it is up to the main interpreter code to
add the file descriptor to the list to select on.
If we left it on the list here,
we might needlessly select on it and hang the server.
.P
Also, note the use of the variable 
.UI read_flag .
In both the Sun and System V versions of 
.UI run_io_waiters() ,
.UI read_flag 
is passed in as a parameter by 
.UI WaitForInput() .
If it is non\(hyzero, we are looking for threads blocked
when reading the file descriptor. 
Otherwise, we are looking for threads blocked when writing.
.P
The variable 
.UI desired_event 
is set to correspond to what
the 
.UI event 
field of the 
.UI "struct execution_environment" 
would be set to if it was blocked on that file descriptor and
for that reason (read or write). In other words, 
.UI desired_event
is set to the 
.UI fd+1 
if
.UI read_flag
is non-zero, otherwise,
.UI fd+event_write_flag+l ,
the difference being
.UI event_write_flag 
(0x4000).
.P
At the end of the 
.UI while 
(
.UI "ee = *eep" ) 
loop, 
.UI awakeq 
will consist of the queue of threads to awaken, threaded through the
.UI next
field.
.P
The following code, also in common in both the Sun and System V
versions of
.UI run_io_waiters() , 
runs through this queue and calls 
.UI run_process 
for each thread in the awakened queue.
.SS
        /* run the processes in the awakened queue */
        while (awakeq) {
            ee = awakeq->next;
            if (ee == awakeq)
                awakeq = 0;
            else
                awakeq->next = ee->next;
            ee->next = 0;
            run_process(ee);
        }
.SE
.P
.H 3 \f4add_selectable_file\fP
.P
.UI add_selectable_file() 
is used to add file description to the list on which to multiplex.
.UI add_selectable_file() 
takes two parameters.
The first is the file descriptor on which we will multiplex.
The second is an indication of whether we are multiplexing on reading
or writing that file descriptor.
.P
If you examine the Sun and System V versions of this code,
you will see how the lists of file descriptors on which to multiplex
are handled. On Sun systems, two arrays of bit masks are used,
one for file descriptors on which to multiplex reads and one for 
file descriptors on which to multiplex writes.
.UI read_flag 
is thus used to decide in which array to set the bit corresponding
to the file descriptor.
.P
In contrast, on System V systems, one array of 
.UI "struct pollfd"
is maintained. The event field of that struct is a bit mask.
For the purposes of the server, only the 
.UI POLLIN 
and 
.UI POLLOUT 
flags are used. Thus,
.UI read_flag 
is used to determine whether to set the 
.UI POLLIN 
or 
.UI POLLOUT
flag in the event field of the array entry corresponding to the
file descriptor on which to multiplex.
It is important to keep in mind that the code must be able to multiplex
on the same file descriptor for both reading and writing separately.
This is necessary for situations in which two threads are both
using the same file descriptor,
with one reading and one writing.
.P
As with the interface between 
.UI WaitForInput() 
and 
.UI run_io_waiters(),
the manner in which information is passed between 
.UI add_selectable_file()
and 
.UI WaitForInput() 
concerning what file descriptors 
on which to multiplex and for what (read or write)
is purely an internal matter.
The nature of the interface should be driven by operating system
considerations, such as the interface to the
I/O multiplexing system call.
.H 3 \f4remove_selectable_file()\fP
This routine is a companion routine to 
.UI add_selectable_file() .
It is used to remove file descriptors from the list to multiplex on.
It is called principally from 
.UI run_io_waiters() .
The routine 
.UI remove_selectable_file() 
also takes two parameters,
the file descriptor to remove from the multiplex list,
and the ``reason'' (read or write).
As with 
.UI add_selectable_file() ,
what we do with the 
.UI reason 
flag depends on O/S considerations and the interface between 
.UI add_selectable_file 
and 
.UI WaitForInput() .
.P
.H 3 \f4make_async()\fP
.P
This subroutine is called for each file descriptor when opened in 
.UI open_file() .
This subroutine takes one parameter, which is 
the file descriptor to set to non\(hyblocking.
Exactly how this routine functions is determined by
the operating system interface.  On most modern UNIX implementations,
there is an option to the 
.UI fcntl(2) 
system call which tells the O/S
to turn on the 
.UI O_NDELAY 
or 
.UI FNDELAY
(no delay on I/O) option for a file descriptor.
.P
The Sun version of 
.UI make_async() 
looks like this:
.SS
void
make_async(fd)
    int             fd;
{
    if (fcntl_OR_flags(fd, FASYNC | FNDELAY))
        perror("make_async");
    /*
     * we want to ignore the errors on this fcntl.  It doesn't
     * make sense to do this to files that exist on the disk, 
     * but we do want to do it devices.  And we can't tell
     * the difference between the two...
     */
    fcntl(fd, F_SETOWN, news_pid);
}
.SE
.UI fcntl_OR_flags() 
is a subroutine which sets a new bit in the flags
associated with a file descriptor.
It may or may not make sense to implement this function for
your O/S depending on how the operating system works.
The important thing is that 
.UI make_async() 
must inform the O/S that it
should not block the process which does I/O on the associated 
file descriptor
(that process is, of course, the server)
if the I/O operation can't be completed right away.
.P
Both 
.UI make_async() 
and 
.UI fcntl_OR_flags 
appear in the file
.UI usr.bin/xnews/xnews/server/npsi/os/sysV/SunOSI.c.
.P
.H 1 "Testing the Server's I/O Multiplexing"
Once you have a set of functions written,
you must then debug them. 
The basic strategy for debugging is to set up a number of process threads
that accept input from a source which may block, and
see that the server multiplexes between the processes as input
becomes available.
.P
Assuming your system has multiple tty ports,
you can run the server on one tty and then use two others to
provide input.
After you log in on the two terminals which will provide input,
determine the tty ports by running the tty command.
The tty command should output a string of the following form:
.SS
    /dev/ttyxxx
.ft1
.SE
Note these strings as they will be used momentarily.
Next, put the shell into an infinite loop so that it won't read from the
tty. This can be done with the following 
.UI ksh 
script:
.SS
        while true ; do
        sleep 65536
        done
.ft1
.SE
A similar 
.UI csh 
script can be written.
Once this is done,
all input will accumulate until another process reads it.
.P
Next,
run the server and provide it with the following initialization
string as input, either in your 
.UI init.ps 
or on the command line:
.SS
\fH{ (/dev/ttyp1) (r) file cvx exec } fork\fP
\fH(/dev/ttyp2) (r) file cvx exec\fP
.ft1
.SE
.P
You will, of course, want to change the two 
.UI /dev/tty... 
strings to match
the output from the tty command on either terminal.
.P
This \*(Ps code creates a new process thread
to open one tty, and then opens it and starts executing the input.
It also simultaneously opens the second tty and starts executing the
input from it.
.P
To test,
type in \*(Ps code
from either tty alternating between the two.
If your I/O multiplexing function is working,
output on the terminal running the server should correspond to the input
from both of the input streams.
For example, if you enter 
.SS
(Terminal 1) print
.ft1
.SE
on one tty, and then
.SS
(Terminal 2) print
.ft1
.SE
.P
on the other, the output on the server terminal should be
.SS
Terminal 1
Terminal 2
.ft1
.SE
.P
If this is not the case,
you will need to debug the multiplexing I/O.
.P
One thing to keep in mind is that a process thread should run as soon
as input or output is available.
For example,
the threads set up in the above example should run as soon as
the return key is pressed on the terminal serving as an input
stream.
.P
Another thing to note is that a thread is likely to block on write
only if its output sink is a network connection or a pipe.
Files and ttys don't normally block,
but a pipe can become full and output on a network connection
can block if the congestion control mechanism acts to stop output.
For example,
on a TCP connection,
the window can be closed prohibiting transmission.
.P
.H 1 "Debugging the Server's I/O Multiplexing"
.P
The best way to debug the I/O multiplexing function 
is to run the server under
a debugger and interrupt the debugger when the server hangs in
an unexpected place.
Recall that good behavior on the part of the server is
when it accepts input from either input stream as
it becomes available.
So, if it is accepting input from only one stream,
or neither stream, let the server block
(that is, don't give it any input).
Then interrupt the debugger.
The debugger will then tell you where in the server code it is.
.P
Chances are, it will have blocked either on a 
.UI read 
system call or on the I/O multiplexing (
.UI select 
or 
.UI poll ) 
system call.
If the server has blocked on the
.UI read 
system call,
it is likely that your 
.UI make_async()
function is incorrect and is
not setting the input stream to be non\(hyblocking.
If the server is blocking on the 
.UI select/poll 
call,
it is probably due to 
.UI add_selectable_file 
or 
.UI WaitForInput()
, and it is acceptable to block in this case.
.P
So that you may understand the code and the flow of execution through
the server, the following text describes
what the server does when reading input.
.P
.H 3 "Server Execution While Reading Input"
.P
Since input from ttys is line at a time,
the server is likely to block only when reading at the beginning
of a new line.
This is done in 
.UI parse_file.h 
at the line
.UI "pf_getc_nb(1, c)"
right after the label 
.UI restart_parse .
.P
This macro, which appears at the beginning of 
.UI parse_file.h ,
attempts to read a character from the input stream.
It looks like this:
.SS
#define pf_getc_nb(e,c) \\
        psio_pgetc_nb(f, c, \\
                ee->restart_state = e; \\
                ee->event = psio_fileno(f) + 1; \\
                goto suspend_process; \\
                CPPCONCAT(read_,e): \\
                f = es->env.file.file);
.SE
As you can see, this is simply an invocation of the macro
.UI psio_pgetc_nb ,
which appears in 
.UI usr.lib/libcps/psio.h .
The listing for 
.UI psio_pgetc_nb 
is as follows:
.SS
#define psio_pgetc_nb(p, dest, pausecode) { \\
    if (--(p)->cnt >= 0) \\
        (dest) = *(p)->ptr++ & 0377; \\
    else \\
        while (1) { \\
            psio_clearblockok(p); \\
            dest = psio_filbuf(p); \\
            if ((int)dest >= 0 || (!psio_error(p) && \\
                                !psio_blocked(p))) \\
                break; \\
            (p)->flag &= ~PSERR; \\
            pausecode; \\
        } \\
}
.SE
>From this, we can infer that the following code from the invocation of
.UI pf_getc_nb 
is the 
.UI pausecode 
mentioned in 
.UI psio_pgetc_nb :
.SS
                ee->restart_state = e; \e
                ee->event = psio_fileno(f) + 1; \e 
                goto suspend_process; \e
                CPPCONCAT(read_,e): \e
                f = es->env.file.file); \e
.SE
This is what will be executed if the 
.UI read 
on the input stream would
block.
As you can see,
the event field of 
.UI "struct execution_environment"
is set to the file descriptor on which we are reading (
.UI "psio_fileno(f)) + 1" 
).
The 
.UI restart_state 
field is set to the number used in the invocation of 
.UI pf_getc_nb() ,
in this case, 1.
.P
The 
.UI restart_state 
field is used at the beginning of
.UI PostScript() 
in 
.UI PostScript.c 
in a 
.UI switch 
statement.
This is done so
that the interpreter will
continue from where it left off when it
suspended itself.
.P
The suspension is, of course, done within the 
.UI pausecode 
where
.UI "goto suspend_process" 
is coded.
.P
The next line after that,
.SS
        CPPCONCAT(read_,e):
.ft1
.SE
does what one might expect,
it tells the C pre\(hyprocessor to concatenate the string 
.UI read_
and the value of the macro parameter 
.UI e
(which is passed as a constant such as ``1'', ``2'', etc...)
in the invocation of 
.UI pf_getc_nb .
.P
In this case,
the label 
.UI read_1:
will be generated.
If you examine the 
.UI switch(ee\(hy>restart_state) 
statement mentioned above,
you will see that 
.UI read_1 
is the label to which the interpreter will
branch if 
.UI restart_state 
is 1.
.P
Now, let's look at 
.UI psio_pgetc_nb 
in detail.
.SS
#define psio_pgetc_nb(p, dest, pausecode) { \\
    if (--(p)->cnt >= 0) \\
        (dest) = *(p)->ptr++ & 0377; \\
    else \\
        while (1) { \\
            psio_clearblockok(p); \\
            dest = psio_filbuf(p); \\
            if ((int)dest >= 0 || (!psio_error(p) \\
                                && !psio_blocked(p))) \\
                break; \\
            (p)->flag &= ~PSERR; \\
            pausecode; \\
        } \\
}
.SE
This macro consists of an 
.UI if 
statement.
The purpose of this 
.UI if 
statement is to see if any characters are buffered from the
last 
.UI read 
system call on the input stream.
If any are, the next one is 'returned' in 
.UI dest .
If not,
.UI psio_filbuf()
is called to fill the input stream's buffer.
If it succeeds in filling the buffer
(i.e., reading), it will break out of the 
.UI "while (1)" 
loop and the 
.UI pf_getc_nb 
macro will be complete.
If not,
the 
.UI pausecode 
will be executed.
In the case of 
.UI pf_getc_nb ,
the behavior will be as described above: the interpreter will set the
.UI restart_state 
field of 
.UI ee
to 1 and the 
.UI event 
field of
.UI ee
to the file descriptor being read, and will jump to 
.UI suspend_process.
.P
Now, let's look at 
.UI psio_filbuf .
This is where the 
.UI read 
on the input stream will actually be performed.
.UI psio_filbuf() 
appears in 
.UI usr.lib/libcps/psio.c :
.SS
psio_filbuf(p)
    register PSFILE *p;
{
    register    avail;
    psio_nbdeclare;

    psio_clearblocked(p);
    if (p->flag & (PSSTRG | PSEOF | PSERR | PSWRITE)) {
        p->flag |= PSEOF;
        return -1;
    }
    psio_nbsetup(p);
    INITBUF(p);
    while (1) {
        avail = p->bufsiz - p->protected;
        if (avail <= 0)
            return -1;
        p->ptr = p->base + p->protected;
        if (p->assoc.outputside && 
                psio_needsflush(p->assoc.outputside)) {
                psio_flush(p->assoc.outputside);
                psio_clearoutblocked(p->assoc.outputside);
        }
        psio_nbexec(p->cnt, read, p->file, p->ptr, avail);
        if (p->cnt > 0)
            return psio_getc(p);
        if (p->cnt == 0) {
            p->flag |= PSEOF;
            return -1;
        }
        if (errno == EINTR)
            continue;
        if (errno == EWOULDBLOCK) {
            if (psio_blockok(p)) {
                psio_wait(p->file, 1);
                continue;
            } else {
                psio_setblocked(p);
                return -1;
            }
        }
        p->flag |= PSERR;
        return -1;
    }
}
.SE
The important part is the invocation of the macro
.UI psio_nbexec , 
which also appears in 
.UI psio.c :
.SS
#define psio_nbexec(ret, type, fd, buf, siz) { \\
    if (psio_is_nb && !(psio_old_flags & O_NDELAY)) { \\
        /* Set to be non-blocking */        
        if (fcntl(fd, F_SETFL, psio_old_flags | O_NDELAY) < 0)\\
            return -1; \\
    } \\
    ret = type(fd, buf, siz); \\
    if (psio_is_nb && !(psio_old_flags & O_NDELAY)) { \\
        register type_errno; \\
        /* Restore original flags */
        type_errno = errno; \\
        if (fcntl(fd, F_SETFL, psio_old_flags) < 0) \\
            return -1; \\
	errno = type_errno; \\
    } \\
}
.SE
In this case, the test of 
.UI psio_is_nb 
will fail.
.UI Psio_is_nb 
is declared in 
.UI psio_filbuf()
using the macro 
.UI psio_nbdeclare.
It is set using the macro
.UI psio_nbsetup:
.SS
#define psio_nbsetup(p)	\e
    if ((psio_is_nb = psio_nonblock(p)) && \e
      (psio_old_flags = fcntl(p->file, F_GETFL, 0))< 0) return -1 
.SE
The 
.UI PSNONBLOCK 
bit flag (what 
.UI psio_nonblock() 
in 
.UI psio.h 
checks)
is not likely to be set,
so 
.UI psio_is_nb 
will be zero.
.P
Execution in 
.UI psio_nbexec
will then fall to the next line, which reads
.SS
    ret = type(fd, buf, siz);
.SE
This doesn't seem to make much sense until you look at
the declaration of 
.UI psio_nbexec :
.SS
#define psio_nbexec(ret, type, fd, buf, siz)
.SE
and how it is used in 
.UI psio_filbuf() :
.SS
psio_nbexec(p->cnt, read, p->file, p->ptr, avail);
.SE
For 
.UI psio_filbuf(),
this turns into
.SS
p->cnt = read(p->file, p->ptr, avail);
.SE
This then is where the actual 
.UI read 
system call on the input file
descriptor is performed. This is also where the server would block if 
.UI make_async() 
was malfunctioning.
.P
After the 
.UI read , 
the test on 
.UI psio_is_nb 
will again fail, so execution will continue in 
.UI psio_filbuf() .
.P
If all is well, and the read succeeded,
.UI psio_filbuf() 
returns the next character in the input stream.
Otherwise,
.UI psio_filbuf() 
will return 
.UI -1 .
This will cause 
.UI psio_getc_nb 
to invoke the 
.UI pausecode 
which will suspend the thread.
.P
When the thread is suspended, execution transfers to 
.UI suspend_process 
in 
.UI PostScript() .
This does some clean up, and then the routine
.UI PostScript() 
returns.
.P
Recall from our previous discussion on Debugging the PostScript
Interpreter, that when 
.UI PostScript() 
returns, it returns to 
.UI run_process() in
.UI nucleus/sched.c .
.P
One important task 
.UI run_process 
performs is to determine whether the event field of the 
.UI "struct execution_environment" 
is greater than zero.  If it is,
.UI run_process
assumes that the value in that field is a file descriptor on which
I/O would have blocked, and calls 
.UI add_selectable_file() 
to add it to the list of file descriptors on which to multiplex.
It also invokes the macro 
.UI enqueue_io_waiter 
in 
.UI server/include/PostScript.h 
to add the thread to the 
.UI process_io_waiters 
list used by
.UI run_io_waiters() .
.P
This is the code in 
.UI run_process() 
just described:
.SS
    else if (ee->event > 0) {   /* blocked on io */
	register fd = (ee->event & ~event_write_flag) - 1;
        if (fd > max_fds) {
            abort();
        }
        add_selectable_file(fd, !(ee->event & event_write_flag));
        enqueue_io_waiter(ee, fd);
    }
.SE
.UI run_process() 
will eventually return to
.UI RunScheduler() , 
which continues to cycle through the list
of processes to run in the following loop:
.SS
        while (runq) {		/* and run all these */
            cs_cursorin();
            cs_overlaysup();
            ee = runq->next;
            if (ee == runq)
                runq = 0;
            else
                runq->next = ee->next;
            ee->next = 0;
            run_process(ee);
        }			/* then accumulate new prospects: */
.SE
When the 
.UI runq 
is exhausted,
.UI RunScheduler() 
returns to the top of its 
.UI while(1) 
loop.
.P
Eventually, it gets to a line of code which says:
.SS
    if (io_ready
     || process_set == 0
     || (corpus_of(eventq)!=0 && 
        corpus_of(eventq)->corpus.event.time>now)) {
        WaitForInput(now);
    }
.SE
This is where 
.UI WaitForInput() 
is called if no other processes are ready to run.
.UI WaitForInput() 
will multiplex on the various file descriptors on which
threads are blocked.
Threads which become unblocked are actually run before 
.UI WaitForInput()
returns, since 
.UI WaitForInput() 
calls 
.UI run_io_waiters() ,
which in turn calls 
.UI run_process().
.P
.H 3 "General Debugging Hints"
.P
In the procedure for getting the interpreter working,
we recommended literally including 
.UI parse_file.h 
and 
.UI operators.h
in 
.UI PostScript.c 
to ease debugging.
We also suggest that you replace
some of the macros used with their literal expansions.
Probably the most important is 
.UI psio_nbexec 
in 
.UI psio_filbuf ,
since this is where the actual 
.UI read 
system call is done on the
input stream,
and it might block there improperly.
.P
It might also be advisable to expand 
.UI pf_getc_nb 
or 
.UI psio_pgetc_nb
if it seems they might be causing problems.

.H 3 "Where to Set Breakpoints"
.P
We recommend that you set breakpoints in the following places
when debugging:
.BL
.LI
.UI psio_filbuf()
.LI
The 
.UI read(2) 
call in 
.UI psio_filbuf 
(really 
.UI psio_nbexec)
.LI
.UI WaitForInput()
.LI
.UI pf_getc_nb 
invocation in 
.UI parse_file.h
.LE
.P
When the read in 
.UI psio_filbuf 
(really 
.UI psio_nbexec
) fails,
you should step through the subsequent code to see that the
process suspends itself properly and that 
.UI add_selectable_file()
is called properly.
You may also want to check that 
.UI add_selectable_file() 
is updating the list of file descriptors on which to multiplex properly.
.P
You should then run until 
.UI WaitForInput() 
is entered and
see that the list of file descriptors on which to multiplex is correct, 
and is being passed to the I/O multiplexing system call properly.
.P
When the I/O multiplexing system call returns, check to see that 
.UI run_io_waiters() 
is being called with
the proper file descriptor or mask, and that it is correctly
running the associated thread.



