MAINSAIL Structure Blaster User's Guide, Chapter 2

previous   next   top   complete contents   complete index   framed top   this page unframed


2. Structure Blaster Function, Formats, and Performance

The Structure Blaster has been designed to read and re-create structures from data images as fast as possible. The Structure Blaster can read a data image and re-create the structure significantly faster than if each object in the structure were allocated with the PROCEDURE new and the field values and elements individually read from a file. The data image is read into contiguous memory in one I/O operation if possible (although if insufficient contiguous memory is available, the image can be broken into pieces). It contains an exact image of the final structure, along with the STRING text. The image is quickly processed to make relative POINTERs and STRING descriptors absolute, and to create a “STRING space” from the stored STRING text (if the text is small, it is copied to an existing STRING space). On output the data image is written out as it is created rather than created in memory and then written out, so that there need not be space in memory for both the structure and the data image.

PDF images are processed somewhat more slowly than data images, but faster than text images. They are portable among the different operating systems on which MAINSAIL runs and among different versions of MAINSAIL; i.e., they can be written on one machine or release of MAINSAIL and read on another, unlike data images.

Text forms are processed more slowly than data or PDF images. Text forms, like PDF images, are portable among operating systems and different versions of MAINSAIL. The noncompressed text form is a human-readable format that can be used to view or modify a structure with an ordinary text editor.

When input/output performance is important, structures should be stored as data or PDF images rather than text forms; where compactness and portability among systems are required, PDF images are preferred. Table 2–1 ranks the different structure formats based on requirements for speed, portability, image size, and human readability or editability.

Table 2–1. Relative Advantages and Disadvantages of Different Structure Image Formats
  PDF data text compressed text
portability 1† 4 1† 3
speed 2 1 4 3
image size 1* 2 4 3
readability/editability N/A N/A 1 2
1 = best; 4 = worst

* Under “image size”, 1 means smallest

† PDF images and text forms are both highly portable, although treatment of out-of-guaranteed-range values and STRING characters is different between the two.

Compared to reading a host data image, our benchmarks show that reading a PDF structure ranges from as fast or faster to as much as 4 times slower, depending on the structure and on the machine. Typically, reading a PDF structure is less than a factor of 2 times slower on machines where the host format matches PDF, and less than 3.5 times slower on machines where the host format does not match PDF.

Our benchmarks also show that a PDF image is typically 50% to 75% the size of a data image, depending on the structure and on the machine.

2.1. Alignment and Positioning of Structure Images in Files

A data image is stored by default as an integral number of pages in a file to facilitate fast I/O; however, the user may specify non-page-alignment, which saves space if a file contains numerous structures smaller than a page. Input and output of page-aligned structures are usually faster than of non-page-aligned structures.

The user can specify the starting page (for page-aligned structures) or starting position (for non-page-aligned structures) of a structure image in a file, so that more than one structure image can reside in the same file.

PDF images and text forms are never specially page-aligned.

2.2. Portability Considerations in PDF Images

Although the PDF image format is portable, the actual values of the data must also be portable if the image is to be shared among different kinds of machines. PDF images can be written for a data structure containing only portable values to a file on one machine and then read into memory on a different kind of machine.

A portable MAINSAIL data structure has the following characteristics:

No check for any of the above conditions is actually made when a PDF image is written.

All STRINGs in the structure are translated between host characters and PDF (ASCII) characters when the structure is read or written as a PDF image. Hence, STRINGs in a structure to be written in the PDF representation cannot be used to encode arbitrary binary values that should not be treated as translatable characters.

2.3. Opening Structure Blaster Files

Structure Blaster data files are usually opened with the $unBuffered bit set in the call to open. This results in a substantial performance increase on many systems. Consult the description of the PROCEDURE open in Section 43.5 of the MAINSAIL Language Manual for details on the use of the $unBuffered bit. The $unBuffered bit must not be set for text files to which a structure is written. Structure Blaster data files must be opened with the random bit set in the call to open; however, the random bit need not be set for text files.

A file may be opened for PDF I/O with the $pdf openBits bit. All structures written to such a file are PDF images. Alternatively, $pdf may be specified to $structureWrite to cause a PDF image to be written to a file not opened with the $pdf bit (on input, $structureRead automatically figures out whether an image is a PDF image rather than a data image based on information stored in the image).

If a structure is to be written to a file (data or text), the output bit must be set when the file is opened; if a structure is to be read from the file, the input bit must be set.

2.4. Portability of Data Images and Text Forms

Data images are not directly portable from one operating system to another. To move a data image from one operating system to another, it may be translated to a PDF or text form (or compressed text form for increased efficiency) on the host operating system, shipped to the target system, and translated on the target system back into a data image.

Data images may not be portable from one version of MAINSAIL to another, or from one version of the Structure Blaster to another. Compressed text forms are not guaranteed to be portable from one version of MAINSAIL to another (although as of Version 16.30 of MAINSAIL, all old text forms are readable by the current version of MAINSAIL), but are portable among systems running the same version of MAINSAIL and the Structure Blaster. If a data image created by an old version of the Structure Blaster must be used by a new version of the Structure Blaster, use the old version to translate the structure into a text form or PDF image, then use the new version to translate the text form or PDF image into a data image. The PROCEDURE $structureInfo may be used to determine the Structure Blaster version number of a structure.

Text forms that do not contain data sections are portable from one operating system to another and from one version of the Structure Blaster to another, provided that no data value in the structure exceeds the range of its data type on the target system.

Data images may change in size when translated from one operating system or version to another. Therefore, if a data image is to contain data representing or depending on file positions or structure sizes (e.g., the file position of some other data image), the user must write a utility program to adjust such positions and sizes in data images as they are translated. It is recommended that information dependent on data image sizes not be stored in data images; if a single data file is to contain multiple images, an alternative is to maintain in each file a directory (which is not itself a data image) of data images in the file.

It is the user's responsibility to write any utility program necessary to translate files containing data images from one operating system or version to another. $structureDataToText and $structureTextToData are primitives from which such a utility may be constructed.

Data sections (in data or PDF images or text forms) are not portable from one operating system to another or from one version of the corresponding object module to another; i.e., a data section should not be assumed to remain valid if the corresponding module is recompiled. This is because changes in other parts of MAINSAIL can affect what is stored in a data section.

The Structure Blaster attempts to detect data sections inconsistent with the corresponding object modules. However, this consistency check may fail. If an inconsistent data section is not detected, the effect is undefined.

In addition to the constraints imposed by the Structure Blaster itself, structures ported from one machine or version of MAINSAIL to another should not depend on values that are not meaningful on both machines or versions. In particular, a structure or the arrangement of structures in a file should not depend on values that may vary from machine to machine or version to version of MAINSAIL, including data type sizes, file name formats, and such system-dependent values as $pageSize, $charSet, and $maxInteger.


previous   next   top   complete contents   complete index   framed top   this page unframed

MAINSAIL Structure Blaster User's Guide, Chapter 2