8.1. Scheduled Coroutines

Because they share data structures and resources, scheduled coroutines can communicate among themselves much more efficiently than can separate processes. Unlike programs that run as independent processes, however, PROCEDUREs that are run as scheduled coroutines generally must be written with some awareness of other scheduled coroutines that may be running and sharing the data structures and resources in the same process.

8.2. Using Scheduled Coroutines for Stream I/O

When a stream input or output PROCEDURE is called by more than one coroutine, the STREAMS package schedules the next runnable coroutine, i.e., the next coroutine that can perform its operation without blocking. The Scheduler, a special STREAMS system coroutine, makes a single operating system call that waits until the first of the pending I/O operations can be performed, and it then resumes the coroutine for which the operation is complete. Coroutines manipulated by the scheduler are called “scheduled coroutines”; scheduled coroutines are distinguished from other coroutines only in that they are manipulated by the Scheduler. Scheduled coroutines may be created with $createCoroutine like any other coroutine.

Rescheduling can occur whenever a coroutine performs I/O to a stream or when it explicitly calls a Scheduler PROCEDURE (see Figures 8–1, 8–2, 8–3, 8–4, and 8–5). The calling coroutine is not resumed until the reason for scheduling has been satisfied, at which point it becomes eligible to be resumed. If more than one coroutine is eligible to run, the Scheduler maintains a FIFO queue of ready coroutines and uses a round robin approach to reschedule the ready coroutines in the order they become ready.

The STREAMS Scheduler performs functions similar to a multiprocess operating system's process scheduler. It schedules stream I/O operations to achieve the appearance of asynchronously running processes. Unlike an operating system, however, it cannot interrupt running coroutines based on an allotted time quantum alone. Thus, if one coroutine gains control and then begins a CPU-intensive computation that involves no I/O, that coroutine continues to run until it performs stream I/O or voluntarily makes an explicit call to the Scheduler to allow other coroutines to run.

When a coroutine is stopped waiting for the Scheduler to resume it, the effects are undefined if any coroutine other than the Scheduler resumes it.

8.3. Cautions About Shared Data Access

Another source of potential trouble is PROCEDURE re-entrancy. The same PROCEDURE may be in the course of executing in several different coroutine execution threads. Own variables in such PROCEDUREs must be considered a shared global resource if there is any possibility that the PROCEDURE might be re-entered by another coroutine before the current coroutine has exited it, due to the current coroutine being rescheduled.

To avoid the PROCEDURE re-entrancy problem, it is best not to execute PROCEDUREs in a given data section in more than one coroutine at a time. That is, for each coroutine in which a MODULE is to run, it is easiest to create a separate data section for that MODULE (where the program logic permits it).

Many conditions can cause rescheduling, including calls to errMsg and all I/O calls (even ordinary disk and terminal I/O). Since error conditions are difficult to predict, the use of semaphores for shared data access of questionable safety is recommended.

8.4. Scheduling Coroutines for Stream I/O: $queueCoroutine

Coroutines that perform scheduled I/O (“scheduled coroutines”) are created using $createCoroutine just like any other coroutine. Instead of using $resume to resume them, the Scheduler PROCEDURE $queueCoroutine is used to put a coroutine on the ready list. When the next rescheduling takes place, a coroutine is chosen from the ready list to run (it is not specified which coroutine is chosen). $queueCoroutine itself does not resume the coroutine c; it just puts c on the ready list and returns.

Coroutines that do scheduled I/O are killed using $killCoroutine, just like any other coroutine.

8.5. Voluntary Rescheduling: $msTimeout and $reschedule

$msTimeout blocks the calling coroutine until the specified number of milliseconds has elapsed. The timeout value must be positive.

$reschedule places the currently running coroutine at the end of the ready queue. A coroutine may wish to do this to timeshare itself when it performs long computations that do not involve stream I/O or any other calls that enter the Scheduler.

If the delete bit is specified in ctrlBits, the currently running coroutine is killed and the Scheduler is resumed so that other eligible coroutines may be run.

8.6. Waiting for Descendant Coroutines: $waitForDescendants

The first form (the OPTIONAL ARRAY form) of $waitForDescendants blocks until the coroutines in the children ARRAY have died. ARRAY elements that are NULLPOINTER are ignored. If the ARRAY is omitted or NULLARRAY is given, it waits until all the children in existence at the time of the call to $waitForDescendants have died.

Since new children can come into existence during the call for $waitForDescendants, in order to wait for all children of the calling coroutine to die, even those created during a call to $waitForDescendants, it is necessary to do:

WHILE $thisCoroutine.$down DO $waitForDescendants;

In the form of $waitForDescendants that takes a single coroutine POINTER, the PROCEDURE waits until the specified child coroutine has died.

The third form (the non-OPTIONAL ARRAY form) of $waitForDescendants returns when any one of the descendant coroutines in the children ARRAY has died; the deceased coroutine is produced in childThatDied, and the corresponding element of the children ARRAY is set to NULLPOINTER. Since NULLPOINTER ARRAY entries are ignored, this makes it convenient to call this form of $waitForDescendants in a loop:

new(ary,1,n);
        .
        .
        .
FOR i := 1 UPTO n DOB
    $waitForDescendants(ary,childThatDied);
    ... code to handle death of childThatDied ... END;

It is an error to wait for coroutines that are not descendants of the calling coroutine.

It is undefined what happens if, during a call in a coroutine A to $waitForDescendants, a coroutine B that A is waiting for is moved (e.g., with $moveCoroutine) so that B is no longer a descendant of A.

The possible return values from $waitForDescendants are described in Section 10.1.2.

8.7. Semaphores (Locks)

Semaphores provide a mechanism through which a set of cooperative coroutines can share common data structures and perform manipulations on them without risking an invalid simultaneous update. The Scheduler delays the execution of a coroutine that requests a semaphore until the semaphore is available.

Using semaphores, an application can build higher-level locking mechanisms. For example, the MEMSTR stream MODULE is implemented using semaphores, and XIDAK database software uses semaphores to implement read and write locks on database records with deadlock detection.

A semaphore is created by calling $newSemaphore. Its initial state is unlocked. A semaphore name is supplied when creating the semaphore. The name helps identify the semaphore in error messages, during debugging, and in the scheduled coroutine map. It need not be unique.

$semaphore may contain additional, undocumented fields besides $name.

The PROCEDUREs $lock and $unlock lock and unlock a semaphore record. Once the semaphore is locked, any coroutine that attempts to lock it blocks until some other coroutine unlocks it. An OPTIONAL timeout allows the lock request to time out if it cannot be performed within the given timeout period. Unlocking is always performed immediately. If errorOK is given in ctrlBits, error messages are suppressed for $lock and $unlock (unless a $lock or $unlock of NULLPOINTER is attempted); otherwise, fatal error messages are issued.

The possible return values from $lock and $unlock are described in Section 10.1.2.

When $lock(sem) is called, if sem is not currently locked, the lock is obtained without rescheduling the calling coroutine. If sem is locked, the current coroutine is rescheduled until sem is unlocked (or $disposeSemaphore(sem) is called) by another coroutine.

When a coroutine A is blocked waiting to lock a semaphore sem that has been locked, and another coroutine B unlocks it, the semaphore remains unlocked for an indeterminate period until a coroutine (not necessarily A) locks sem. Any other coroutine C that gets to run before A will find (using $isLocked) that the semaphore is unlocked, and could do the $lock ahead of coroutine A.

When $unlock(sem) is called and sem is locked, the first coroutine that is waiting for sem is put on the ready queue to run next, but the coroutine that called $unlock continues to run. Thus, $unlock does not reschedule the calling coroutine. If you wish to reschedule the calling coroutine and allow the coroutine to which the lock was granted to run immediately after calling $unlock, you must explicitly call $reschedule.

If $unlock(sem) is called when sem is not currently locked, an error condition occurs. If errorOK is set in ctrlBits, $unlock just returns the value $error; otherwise, errMsg is called. If no error occurs, $unlock returns a positive value.

The macro $isLocked may be used to query the state of a semaphore without rescheduling the current coroutine. It evaluates to TRUE if the semaphore is locked and FALSE if not.

A semaphore may be disposed by calling $disposeSemaphore. Any coroutines waiting for the semaphore get an error return from their $lock calls.

It is the responsibility of the programmer using semaphores to ensure that they are unlocked and/or disposed of properly when coroutines that need them are killed. Through injudicious use, it is possible to create deadlocks.

8.8. Scheduled Coroutine Map

$scheduledCoroutineMap returns a STRING showing the currently queued I/O requests by coroutines, the streams currently blocked for I/O, and the state of the coroutines and streams in the map.

This PROCEDURE may be invoked interactively by invoking the MODULE SCOMAP. SCOMAP initially prompts:

Turn on $schedInfo?  (Yes or No):

If you answer yes, information about blocked $lock calls is also included in the output.

8.9. Example Multitasking Programs

• Read characters from the keyboard as they are typed, and send them to the remote system through a serial line without echoing them on the local screen.

• Read characters from the remote serial line and display them on the screen as they are received.

These functions are shown schematically in Figure 8–6.

Example 8–7 shows how these two independent functions might be implemented as two coroutines that perform stream I/O. The INITIAL PROCEDURE creates and queues a coroutine to perform each of the functions, and then waits for both coroutines to complete. Each coroutine keeps copying until its read fails, and then the coroutine kills itself. The actual termination sequence is not shown in this simple example.

The variables $tty and serialLine refer to the controlling terminal (keyboard and screen) and an independent serial line to the remote computer, respectively.

Each coroutine is written as if it were doing normal blocking I/O. However, the Scheduler and the stream I/O system make sure that neither coroutine causes the program to go into a blocked state, thus preventing the other coroutine from running when input is available to it.

In general, a MAINSAIL program may contain an unlimited number of such coroutines. For example, to implement a windowing environment, a pair of coroutines similar to the ones above might be used for each simultaneously active window on the screen.

8.9.2. Parallel Processing Using RPC Calls

For example, the INITIAL PROCEDURE in Example 8–8 starts up remote PROCEDURE calls to several different instances of the same remote MODULE REMMOD in parallel and returns when all of them are done. If the different instances run on different CPUs, true parallel processing occurs.

The program uses the ARRAY task to hold one data section for callMod for each parallel call to the remote MODULE REMMOD. REMMOD computes some result based on a task number (the computation is presumably lengthy, or it would not have been worth the trouble to set up a separate process to do it).

The program sets up the ARRAY host to contain host names on which to execute each instance of REMMOD, and the ARRAY result to hold the answers.

The INITIAL PROCEDURE of the program starts task.ub1 coroutines executing the doCall PROCEDURE, one for each argument. It then waits for all of the coroutines to finish (die).

Each instance of doCall starts a child process running a unique instance of REMMOD, passes it the argument, and receives back the result. The doCall coroutine kills itself when it has the answer.

The example above has been kept fairly simple for clarity. It could be enhanced in several ways:

• doCall could call more than one remote PROCEDURE in REMMOD.

• The program could handle unexpected termination of child processes and restart them, possibly on a different node.

• A dynamic mechanism could be used to select hosts on which to execute each subtask, and the program could be modified to work with fewer hosts than there are subtasks.

Note that each coroutine creates a different instance of the remote MODULE. Calling the same instance of a remote MODULE in different coroutines does not speed up the program, because all of the calls to a single instance of a remote PROCEDURE are processed by the same process and go through the same communication channel.