19.1. Introduction

19.1.1. Likely Changes

LIB uses the MAINSAIL Structure Blaster. The Structure Blaster must be installed for LIB to function. Since LIB uses the Structure Blaster, the same compatibility restrictions apply to libraries with respect to MAINSAIL versions as apply to structure images stored in data files (see the MAINSAIL Structure Blaster User's Guide for details).

[While XIDAK will strive to make the format of libraries forward compatible, XIDAK reserves the right to change this format if XIDAK deems it necessary.]

19.2. General Concepts

• A file is “accessed” when it is opened. The name of the file is interpreted either by the host operating system or by LIB.

• Each file must be “stored”; disk pages are required to store the information associated with each file. These pages may be either in a separate host file or in a subset of the pages of the base file of the library that contains the file.

• A “host file” is a file stored in, and accessed through, the file system of the underlying operating system.

• The “base file” of a library is a host file that, at a minimum, contains the directory structure for the library. The base file may also be used to store library files.

• A “library file” is a file that is accessed through a library and may be stored either as a host file or within the base file of the library.

• A library file is said to be “contained” in a library if it is accessed through the library, regardless of whether it is stored as a host file or stored within the library's base file.

Each library is defined by a single host file, the base file for the library. This file always contains and stores the directory structure for the library and can optionally store files contained in the library.

Each base file contains a structure that implements a tree-structured name space for accessing objects, much like the tree-structured name space implemented by the UNIX file system for accessing files.

[The directory structure for a library is loaded from the library's base file into memory when the first library file contained in the library is opened, and kept in memory as long as any library file contained in the library is open. When all library files contained in the library are closed, the directory structure is rewritten in the base file if any changes have been made to the library.]

The base file for a library is kept open as long as any library file contained in the library is open and is closed when no library files contained in the library are open. Thus, the base file consumes only a single file handle.

If the library files are stored within the base file, then no additional file handles are required to access these files. One file handle is required to access each library file that is stored in a host file.

19.2.3. The Directory Structure

A directory consists of a set of objects. Each object has a name and is either a library file or another directory. Each object may have multiple versions, but all versions of the same object must be of the same type. The top-level directory is known as the “root directory” for the library.

Directories form a tree-structured name space. Objects are accessed by means of path names similar to those used in the UNIX operating system. A LIB path name consists of a sequence of directory names followed by an element name. The path name is interpreted by starting in the root directory and successively locating the directories named in the path.

19.2.4. Library File Name Syntax

Library files are accessed by means of a “fully qualified name”. The components of a fully qualified name for an example file name of (foo.lib)>/bar;3/baz;5/sample.msl;7 are:

The base file name must be enclosed in matching parentheses. It must be a MAINSAIL file name, i.e., a STRING that could be passed to the system PROCEDURE open.

The closing parenthesis following the base file name must be immediately followed by the $devModBrk character (usually >; see Section 22.11 of the MAINSAIL Language Manual) and a path name.

A path name always starts with a / and ends with a /. A path name is composed of a sequence of elements, each of which names a directory. Each directory name may optionally be followed by a ; followed by a version number. Directories are separated by /.

A fully qualified name may contain a (leaf) file name. The file name may optionally be followed by a version number separated by a ;.

Directory names and file names are case-insensitive, and are composed of any alphanumeric character followed by a sequence consisting of zero or more alphanumeric or punctuation characters (the allowed punctuation characters are “.”, “+”, and “-”).

The special name “..” names the parent directory of the currently connected directory and “/” names the root directory of the currently connected library (see Section 19.5.3).

Some commands also accept * to mean all objects in a specified directory, although * has not been uniformly implemented in this version of LIB.

19.2.5. Versions

Each object in a library, including directories, can have multiple versions, just as files can have multiple versions under the VAX/VMS operating system. A version number is a positive integer that specifies a distinct copy of the object (different versions do not share the same contents in any way).

A new version of a library file is created whenever the library file already exists and is opened for create, or is the destination for a ADD, COPY, or RENAME command, or whenever a file is opened with the LIB device MODULE and the create open bit is specified.

A new version of a directory is created whenever the directory already exists and is the destination of a MAKE, COPY, or RENAME command.

The number of old versions to keep may be specified on a per-directory basis by means of the CREATE, MAKE, or KEEP commands.

For objects open for input, if the version number is omitted, the highest existing version number is used.

19.2.6. Hard Delete Attribute

Associated with each directory is the “hard delete attribute”. If this attribute is set, deletions occur immediately, as under the UNIX operating system. If this attribute is clear, then deletions are deferred until the object is expunged.

19.2.7. Storage of Objects

[Currently, library files stored as separate host files are stored in the host directory using a host file name of the form zxxx.lfi, where xxx is a unique pattern generated by LIB. Host files with names of this form must not be independently manipulated or library corruption will result. The format of the host file name is subject to change.]

19.2.8. The lparms File

LIB maintains a parameters file, called lparms. This file stores definitions and other information from run to run.

If the host directory to which you are connected when running LIB contains no lparms file, then LIB looks for lparms on your home directory (as returned by the system PROCEDURE $homeDirectory). Then, if no lparms file exists on either of these directories, LIB creates one on the host directory to which you are connected when running LIB.

LIB maintains definitions made with the DEFINE command in lparms. These definitions are maintained in the form of DEFINE commands.

LIB updates the lparms file whenever the SAVE, EXPUNGE, or QUIT commands are executed, or whenever LIB is disposed, or MAINSAIL exited.

You may explicitly edit lparms to add LIB commands to be executed whenever LIB is initialized.

19.3. Device MODULE Interface

[LIB does not currently support automatic creation of directories within a library; only the root directory is automatically created when the library's base file is created. All other directories must be explicitly created with the MAKE command. A future version of LIB may provide for automatic creation of directories.]

To open a library file from a program, use the fully qualified name of the library file prefixed by LIB, as shown in Example 19–1.

To specify that an existing application, e.g., the MAINSAIL compiler, use a library file, use the standard device MODULE syntax as shown in Example 19–2.

For example, Example 19–2 shows a library file used as input to the MAINSAIL compiler. There is no need to EXTRACT the source file from the library before compiling it. The output of the compiler could also be directed back into the library. This same syntax may be used to execute MODULEs contained in a library.

19.4. Program Interface

19.4.1. doCommandsInFile and doCommandsInString

[A MODULE that calls doCommandsInFile or doCommandsInString must include their declarations with the MAINSAIL directive shown in Figure 19–4. This directive may change or be eliminated in subsequent versions of LIB.]

Except for device MODULE functions, all LIB functions are invoked by executing the commands specified in Section 19.5. Commands can be executed by calling either doCommandsInFile or doCommandsInString.

Normally doCommandsInFile and doCommandsInString print LIB's herald each time they are called. If the suppressHerald bit is set in ctrlBits, the printing of LIB's herald is suppressed.

19.4.1.1. doCommandsInFile

19.4.1.2. doCommandsInString

If useMemFiles is set in the ctrlBits parameter, the userCmdString is copied to a MEM file (see Chapter 22) and then doCommandsInFile is called with this MEM file as userCmdFile and another MEM file as userLogFile. This provides a way for doCommandsInString to provide input to MODULEs other than LIB that read input from cmdFile and write output to logFile. Upon return from doCommandsInFile, the contents of the userLogFile MEM file are converted to a STRING and produced as userLogString.

doCommandsInString normally fails if userLogString approaches 32K characters in length. If windowUserLogString is set in the ctrlBits parameter, the userLogString is “windowed”; a message is prepended, then a sliding window approximately 32K characters long is maintained that shows the most recent characters written to userLogString (older characters are discarded).

19.4.2. Exiting MAINSAIL and Disposing of LIB

In addition to the processing it does during exit, LIB must force close all library files that are open whenever LIB is disposed. The actual information written to the disk for such files is undefined. This also means that application programs that use library files may attempt to do I/O on a file that was forced close by LIB. In general, disposing of LIB should be avoided.

19.4.3. Library Integrity

This means that if a library is modified, but MAINSAIL is aborted before the directory is written to disk, the modifications are lost. The prior contents of the library should be preserved, however, because LIB is careful about how it updates the disk. It writes file pages and new directories only to disk pages that contained no active information in the prior directory, and only after writing the new directory updates the library's header page to point to the new directory disk image.

19.5. LIB Commands

Whenever a library file is specified in a command, it is specified by means of either a fully qualified name or a relative name that produces a fully qualified name when combined with the current directory connections. In general you should not include the LIB device MODULE prefix when specifying library files in commands.

In the following description of command syntax, the notation {s} means that s may either be omitted or specified exactly once. The notation {s}+ means that s must be specified one or more times.

19.5.1. Command Lines

Certain commands, e.g., LS, accept “switches”, which are modifiers to govern the detailed operation of the command. Switch values are specified as single characters, or pairs of characters, within a word that must start with the character -. Case is ignored when interpreting switches. Switch words may appear as any word but the first on a command line and there may be multiple switch words on the same command line. [Switch syntax may change in future versions.]

19.5.2.1. Mode Switches

The switches shown in Table 19–5 may be applied to any command.

The C switch sets confirmation mode for the duration of the command regardless of the current confirmation mode setting. The setting of the current confirmation mode reverts to its previous value after the command terminates.

The NC switch clears confirmation mode for the duration of the command regardless of the current confirmation mode setting. The setting of the current confirmation mode reverts to its previous value after the command terminates.

The V switch sets verbose mode for the duration of the command regardless of the current verbose mode setting. The setting of the current verbose mode reverts to its previous value after the command terminates.

The NV switch clears verbose mode for the duration of the command regardless of the current verbose mode setting. The setting of the current verbose mode reverts to its previous value after the command terminates.

19.5.3. Connection Commands

LIB supports the notion of “connecting” to a directory. Connecting to a directory eliminates the need to supply the full path name to access objects contained in the directory. LIB supports the idea of separately connecting the source and destination of operations, so that, for example, you can copy files from one directory (the source) to a different directory (the destination) without having to type the full path name of either directory.

Whenever you connect to a directory in a particular library, that library's base file is opened. You cannot delete a library that contains a directory to which you are currently connected.

The CONNECT and CD commands perform the same function. They connect both the source and destination directories to the directory s. If the connection succeeds, then the named directory is known as the “connected directory” and the base file containing such directory is known as the “connected base file”.

If s is omitted, the current connections of both the source and destination directories are disconnected. If s is specified, then it can be either a fully qualified name, or a relative name, of a directory.

Any directory to which a connection is to be made must already exist, and be accessible, or the connection fails. Once connected, the associated library base file is left open until the connection is broken.

The SRCCONNECT command operates the same way as the CONNECT command, but affects only the source connection.

The DSTCONNECT command operates the same way as the CONNECT command, but affects only the destination connection.

The PWD command prints the current source and destination connections.

19.5.4. Definition Commands

Because fully qualified names can become quite long, LIB supports the notion of a “logical name”; a (typically) short string that can be used in place of another (typically) longer string.

The DEFINE command defines a logical name. When two strings are specified, the first string is defined to be a logical name with the value of the second string. Whenever LIB is parsing a library file name and finds that the initial portion of such a name is a logical name, followed by the character “:”, the logical name and the “:” are stripped off and replaced by the logical name's definition.

If only one string is supplied, LIB prints the current definition (if any) of the logical name specified by the string. If no strings are supplied, LIB prints all logical name definitions.

For example, if the command DEFINE fl (foo.lib)>/ were given, then the name fl:baz would reference (foo.lib)>/baz. Such definitions are saved in the lparms file.

The UNDEFINE command undefines the logical name specified by the argument string(s).

19.5.5. Directory Information Commands

[The format and contents of the directory listing will likely change in future versions.]

The commands DIRECTORY and LS operate identically. They list directory information about the file or directory s.

19.5.5.1. Directory Information Command Switches

Normally DIRECTORY and LS produce a listing that contains only non-deleted objects. The D switch instead produces a listing that contains only deleted objects that have not yet been expunged.

Normally DIRECTORY and LS produce a listing, that contains a single line of information per object. For example:

ROOT;1           5:44:10 19-Oct-1986 greg    DSL      3
  bar;1          5:00:59 19-Oct-1986 greg    DSL      1
  sample.msl;1   4:32:30 19-Oct-1986 greg    FTL   3072

The S switch produces a short listing that gives only the name and version of each object. The L switch produces a long listing that contains almost all of the information LIB has about each object.

[All of the formats and information content of LS commands are subject to change.]

The normal listing produced by DIRECTORY and LS contains for each object:

• The name and version of the object.

• The date, time, and user of the last modification of the object.

• Attributes of the object. Tables 19–7 and 19–8 show how to interpret the object attributes listed.

The short listing produced by DIRECTORY and LS, shown in Example 19–9, lists the name and version of each selected object.

The long listing shown in Example 19–10 produced by DIRECTORY or LS includes the line produced by a normal DIRECTORY or LS plus a line showing the time, date, and user when the object was created, plus either a host directory name, for a directory object, or a host file name, for a file object. For directories, INF is printed if the directory will keep infinite versions of objects, or a number indicating how many versions of objects are kept.

For directories, the host directory name tells where host files are created whenever a library file is created in the library directory. If omitted, library files created within the directory are stored as part of the library's base file.

For files, the host file name specifies where the library file is stored. If omitted, the library file is stored as part of the library's base file.

The CREATE command creates a new library with host file name s. By default the root directory of the newly created library has the hard delete attribute cleared and will keep infinite versions.

If t is not specified, files created in the root directory of s are stored within the library's base file. If t is specified, then it must be the full path name of a host directory in which LIB is to store files created in the root directory of s.

CREATE fails if the library is already open, e.g., if you are connected to a directory contained within it.

On operating systems that do not support version numbers, you are queried for confirmation to overwrite s if s currently exists and confirmation mode is set (see Section 19.5.15.1). If the host file contains a valid object library, all objects contained in the library are expunged before the base file is overwritten.

In addition to the mode switches specified in Table 19–5, the CREATE command accepts the switches shown in Table 19–11.

The H switch specifies that the root directory of the newly created library should have the hard delete attribute set.

The S switch specifies that the root directory of the newly created library should have the soft delete attribute set. Since soft delete is the default, this switch is unnecessary, but is included for completeness and so that CREATE is parallel to MAKE.

A numeric switch value is treated as the number of of versions to keep in the newly created directory, while the I switch value specifies that an infinite number of versions should be kept in the newly created directory. If neither a numeric switch nor an I switch is specified, the newly created directory inherits the number of versions to keep from its parent.

Different versions of directories are completely separate objects; the sets of files in them are disjoint.

19.5.7. Directory Creation Commands

The MAKE and MKDIR commands operate identically. They create a new directory by the name of s. s may be either the full path name of a directory or a directory name relative to the current destination connection.

If t is specified, then it must be the full path name of a host directory in which LIB is to store files created in s. If t is not specified, then the directory s inherits from its parent where to store files created in s.

In addition to the mode switches specified in Table 19–5, the MAKE command accepts the switches shown in Table 19–12.

The H switch specifies that the newly created directory should have the hard delete attribute set instead of inheriting this attribute from its parent directory.

The S switch specifies that the newly created directory should have the hard delete attribute cleared instead of inheriting this attribute from its parent directory.

A numeric switch value is treated as the number of of versions to keep in the newly created directory, while the I switch value specifies that an infinite number of versions should be kept in the newly created directory. If neither a numeric switch or an I switch is specified, then the newly created directory inherits the number of versions to keep from its parent.

19.5.8. Host File Manipulation Commands

The ADDTEXT command adds the host text file s to a library under the name t by copying the host file to the library file.

The ADDDATA command adds the host data file s to a library under the name t by copying the host file to the library file.

The EXTRACT command extracts the library file s into the host file t. On operating systems that do not support version numbers, you are queried for confirmation to overwrite t if t currently exists and the confirmation option is set (see Section 19.5.15.1).

19.5.9. Object Copying Commands

The COPY and CP commands operate identically. They create a copy of object s with the name t. If s is a directory all objects in s are recursively copied. If an object named t of the same type already exists, a new version is created.

Objects can be copied across directories and libraries. Objects cannot be copied to a host file.

The modify date, time, and user of the copied object are identical to that of the original object, but that the create date, time, and user are those in effect when the copy operation is performed (this means the modify date and time may precede the create date and time).

19.5.10. Object Renaming Commands

The RENAME and MV commands operate identically. They change the name of object s to be object t. If an object name t of the same type already exists, a new version is created.

Objects can be renamed across directories and libraries. An object cannot be renamed to a host file. When an object is renamed across libraries it is equivalent to a copy followed by a delete. When an object is renamed within the same library, only the directory data structure is modified.

The modify date, time, and user of the renamed object are identical to that of the original object, but that the create date, time, and user are those in effect when the rename operation is performed.

19.5.11. Object Removal Commands

These commands remove objects from a library.

The DELETE and RM commands operate identically. They mark the object s for deletion, and if the directory containing s has the hard delete attribute set, expunge it as well.

The DROP command deletes all versions of the object s except the highest-numbered version. [Future enhancements to LIB may make it necessary to retain certain older versions.] DROP does not expunge the deleted objects unless they are contained in a directory with the hard delete attribute.

If a DROP or DELETE is attempted on a non-empty directory, and the confirmation option is set, then you are queried for confirmation before dropping or deleting the directory.

The UNDELETE command unmarks the object s for deletion. This has an effect only if the directory containing s has the hard delete attribute cleared and if the object has not been expunged.

The EXPUNGE command expunges the object s from the library. Expunging an object removes it completely from the library so that it can never be recovered. When a library file stored as a host file is expunged, the host file storing the library file is deleted. Objects are automatically expunged by the DELETE, RM, and DROP commands for directories which have the hard delete attribute set.

The DROP, UNDELETE, and EXPUNGE commands accept * as an element name indicating “all”. For example, the command DROP foo/* drops old versions of all objects contained in the directory foo, whereas the command DROP foo drops old versions of the object foo. [In a future release more general wildcards may be supported by these commands.]

19.5.12. Deletion Control Commands

These commands manipulate the directory attributes to control soft and hard delete.

The HARDDELETE command sets the hard delete attribute for the directory s and expunges s. Subsequent deletions from s immediately cause an expunge on the deleted object.

The SOFTDELETE command clears the hard delete attribute for the directory s. Subsequent deletions from s mark the object for deletion, but not expunge it. An explicit EXPUNGE command must be given before the object is actually expunged from the library. Objects marked for delete may be recovered with an UNDELETE command, if it is given before the object is expunged.

19.5.13. Version Control Command

The KEEP command modifies how many versions are to be kept in directory s. n must be either a number, in which case n versions are kept, or an initial substring of INFINITE, in which case an infinite number of versions are kept. If a number is specified, KEEP drops any versions in excess of this number.

19.5.14. SAVE Command

The SAVE command updates library base files with the current state of their memory-resident directories and updates the lparms file. If s is specified, it must be the name of a library to be saved. If s is not specified, then all open modified libraries are saved.

19.5.15. Mode Commands

These commands affect the global operation of LIB; they control its “modes”.

19.5.15.1. Confirmation Commands

The CONFIRM command sets confirmation mode. This is the default.

The NOCONFIRM command clears confirmation mode.

19.5.15.2. Verbosity Commands

The NOVERBOSE command clears verbose mode.

19.5.16. Termination Commands

The QUIT command quits LIBEX, returning to whichever MODULE originally invoked LIBEX, e.g., MAINEX.

The EXIT command quits LIBEX and exits to the operating system through the MAINSAIL system PROCEDURE exit.

19.5.17. Input Redirection Commands

The READ command tells LIBEX to read its commands from the file s.

The EXECUTE command executes an arbitrary MAINSAIL MODULE using the system PROCEDURE $invokeModule. Accordingly, s can be either the name of a MODULE or the name of a file containing the MODULE, and “,” can be appended to the name to enter MAINEX subcommand mode before executing the MODULE.

If MAINEX subcommand mode is entered, or if the executed MODULE uses cmdFile and logFile, output is directed to logFile and input obtained from cmdFile, even if the EXECUTE command is issued from doCmdsInString. If doCmdsInString needs to supply information to be read by cmdFile, or record information written to logFile, then the useMemFiles bit must be set in the call to doCmdsInString.

19.5.19. Maintenance and Diagnostic Commands

These commands are primarily used by maintenance personnel and may change or cease to be supported in future versions without notice.

The STATUS command prints a summary of LIB's status. Currently this information includes a listing of all open libraries and the number of references to each. STATUS also prints the current connections and current mode settings.

The PAGEMAP command prints out a page map of the library s. The page map shows free pages, the pages occupied by the directory data structure, and the pages used for objects stored within the library's base file.

The PAGESUMMARY command prints a short summary of page usage in the base file of the library s, including the total number of pages, total number of free pages, number of trailing free pages, and number of pages occupied by the directory structure. The library s must not be open at the time the PAGESUMMARY command is given.

The HEADER command prints the library header page from the host file s. This information contains the consistency checking information LIB uses to make sure it has a base file as well as the major and minor MAINSAIL and LIB version numbers.

19.6. LIBEX

19.6.1. Sample LIBEX Execution

Table 19–13 shows a sample execution of LIBEX, followed by access to files within a library from the compiler.