

             Design of 3-D table insertion task
             ----------------------------------


 Author: I. Busko


 Revision history:
 12/16/96  -  First version.
 01/15/97  -  Revised after design review.
 01/20/97  -  Matches version 1.0 of code.

 Revision content:
 01/20/97: (i) internal flow control of TICOPY routine; (ii) the two main 
           data structures now store "regular" column pointers (in the 
           tbtable sense) instead of column pointers in the tcs_ sense;
           (iii) inclusion of a template table.



1. Specifications / requirements:

This task will perform the inverse operation performed by task txtable.
It will insert (in the tainsert task sense) one or more 2-D tables into 
rows of a 3-D table. Alternatively, it will create a 3-D table from the
2-D input tables. Each column in the input 2-D table is inserted as an 
array into a single cell at the specified row in the output table. 
Additional scalar columns, stored in the headers of the input table by 
txtable, will also be processed. Row/column selectors in the input table
names will be supported.

This design proposes a first, non-sophisticated version of the task. The
emphasis is on simplicity rather than providing support for all possible
situations. For instance, what to do if the size of a given column in one 
of the input tables is larger than the corresponding array size in an 
existing output table ? Throw away the extra elements ? Resize output table ? 
But what rules to follow in order to fill back the now resized arrays ? This 
design will solve problems like these by resorting to the simplest (from the 
code viewpoint) solution (in this case, just ignore the extra elements).

If the output table does not exist, the first input table in the list
will define both the column information for the output table, as well as
its maximum array size. Columns in the input and output table will be
matched by column name. If a given column in an input table does not exist 
in a previously existing output table, it will be ignored.

From design review: an existing table can be used as a template when creating 
a new 3-D output table. If no template is supplied, the first table in the
input list becomes the template.

Because the selector mechanism does not work with scalars stored in the
input tables' headers by task txtable, these scalars, if existent, will
be always inserted in the output table, provided column names match.

An error results when:
- an array is found in a cell in any of the input tables,
- datatypes in input and output columns do not agree.

The task will be named "titable" following a former proposal for naming
the 3-D table utilities.



2. Language:

SPP, to allow the use of the generic datatype compiling facility, and to
reuse significant amounts of code already developed for txtable, tximage
and tainsert.



3. Task parameters:

Name     Type                  What

input    file list/template    list of 2D table names with optional 
                               row/column bracket selectors.
output   file name             3-D table name with no row/column selectors
                               (modified in place or created from scratch).
template file name             3-D template table with column selectors
row      int                   row in output table where to begin insertion.



4. Data structures:

The main data structures are two pointer-type column descriptor arrays.
One array is associated with the output table, and the other array is 
associated with the current input table. 

The output table array is sized to store column information from both the 
actual columnar data in the input tables, as well as any scalar data 
stored in the input table headers by the txtable task. If the output table 
already exists, it will define the array size and contents completely. 
If it is being created by the task, the first input table in the list will 
define the size and content of the output descriptor array. Thus if other 
tables in the input list have additional columns (both physical or in the 
form of header-stored scalars), these additional columns will be ignored.



5. Code structure:

The listing below shows only the most important subroutines; lower-level 
functions such as decoding header keywords are not explicited.

The first section deals with creating the main column descriptor array
for the output table. If the table does not exist, column information
must be read from the input table columns themselves AND from eventual
scalar columns stored in the header by txtable.

The second section scans the input list and performs the actual insertion
operation. Again, a separate piece of code exists for the cases where a
physical column exists in the input table, or an header-stored scalar
instead. The innermost loop takes care of reading only the selected rows 
from the input table.

- Read task parameters (clget).
- Alloc work memory (malloc, calloc).
- Strip output name from eventual bracket selectors (rdselect).
- Open input list (imtopen).
- If output table already exists (access).
   **> Procedure TIUPDATE:
     - Open output table (tbtopn).
     - Create array of column pointers from output table (malloc, tcs_open).
     - Get column pointers from tcs structure.
   **> End TIUPDATE.
- Else
   **> Procedure TINEW:
      - If template table is not valid:
         - Get first table name from input list (imtgetim).
      - End if.
      - Check if it is a table (whatfile). Exit if not.
      - Break name into bracketed selectors (rdselect).
      - Open template table (tbtopn).
      - Get its total (selected and unselected) number of rows (tbpsta).
      - Scalars in input table are signaled by TCTOTAL keyword in input table 
        header. Look for it (tbhfkr) and increase number of output columns.
      - Create array of column pointers from input column selector and TCTOTAL
        info (malloc, tcs_open).
      - If no columns were matched, and no TCTOTAL keyword was found, exit.
      - Open output table (no STDOUT allowed) (tbtopn).
      **> Procedure TISETC:
         - Loop over input column pointer array, if it exists.
            - Copy column information from input to output (tbcinf, tbcdef), 
               setting the output array size to be the input number of rows
               in the case of a 2-D template, or keeping it the same size
               in the case of a 3-D template.
         - End loop.
         - Loop over all possible keyword sets (from 1 to TCTOTAL).
            - Look for TCD_xxx keyword (tbhfkr).
            - If found: 
               - Decode TCD keyword into column data (name, datatype, format)
               - Create scalar column in output table's column array (tbcdef).
            - End if.
         - End loop.
      **> End TISETC:
      - Create output table (tbtcre).
      - Close input table (tbtclo).
      - Rewind input list (imtrew).
   **> End TINEW:
- End if.
- Initialize row counter with "row" parameter value (clgeti). Set flag if row
  parameter is negative or INDEF.
**> Procedure TINSERT:
   - Loop over input list (imtlen).
      - Get table name (imtgetim).
      - Check if it is a table (whatfile). Skip and warn user if not.
      - Break name into bracketed selectors (rdselect).
      - Open input table (tbtopn).
      - Look for ORIG_ROW keyword (tbhfkr). If found, and if "row" parameter
        is negative or INDEF, supersede row counter with keyword value.
      - Find how many rows were requested by row selector (trsopen, trseval,
        trsclose).
      - Create array of column pointers from column selector (malloc,tcs_open).
      **> Procedure TICOPY:
         - Loop over output table column pointers.
            - Get column info from output table (tbcinf)
            - Choose the minimum in between this array size and the number 
              of rows selected from input table (tbalen).
            - If there are matched columns, loop over input table column 
              pointers.
               - Get column info from input table (tcs_column, tbcinf)
               - If column names match:
                  - If data types do not match, abort.
                  **> Procedure TICC:
                     - If character-type, get string size.
                       given by the minimum size computed above (malloc).
                     **> Procedure TIROWS (generic data type):
                       - Alloc buffer of appropriate type and with size 
                       - Copy selected rows from input table into buffer 
                          (trsopen, trseval, tbcgt, trsclose).
                        - Copy buffer into designated row/column (tbapt).
                        - If output exists and array is larger than buffer:
                           - Set remaining elements to INDEF.
                        - End if.
                     **> End TIROWS.
                  **> End TICC.
               - Else (no match), look for scalar data in input header:
                     - Look for TCTOTAL keyword (tbhfkr). If found:
                        - Loop over all possible keyword sets (from 1 to 
                          TCTOTAL).
                           - Look for TCD_xxx keyword (tbhfkr).
                           - Decode TCD keyword to extract column name.
                           - If column name from header matches with output 
                             column name:
                              **> Procedure TICH (generic datatype):
                                 - Look for TCV_xxx keyword (tbhfkr). If found:
                                    - Read value. 
                                    - Write scalar data (tbcpt).
                                 - Else
                                    - Warn user that input table header is 
                                      corrupted.
                                 - End if.
                              **> End TICH.
                           - End if.
                        - End loop.
                     - End if.
               - End if (Notice that a no-match case, both from columns and
                         header scalars, is not an error since the output or
                         template table may have columns that do not exist
                         among the selected columns in the input table).
            - End loop.
         - End loop.
      **> End TICOPY.
      - Free input table's array of column pointers (tcs_close, mfree).
      - Close input table (tbtclo).
      - Bump output row counter.
   - End loop.
**> End TINSERT:
- Free output table's array of column pointers (tcs_close, mfree).
- Close output table (tbtclo).
- Close input list (imtclose).
- Free work memory (mfree).








