Getref / Upref Redesign
GETREF OPEN ISSUES
The most significant output of getref is the delta file.
The delta file will contain five columns:
- Root name
- Keyword name
- Keyword value
- File name, including extension specifier
- Option flag (Insert, Update, or Delete)
- Change needed (Yes or No)
The main point to be noted is that all information needed to
specifically identify the extension containg the keyword to be
modified will be stored in a single column in the format the iraf
iamge i/o library uses when opening an image extension. Also, the
output will be sorted so that keywords in the same extension are
contiguous.
Getref can be run either from the command line, or from a CGI script.
When run from the command line, it can be run either interactively or
from a script. To disguise the difference between command line and CGI
mode, task arguments are parsed and saved in a common data structure
used by both modes. Likewise, task output is saved in an in-memory
table and converted to the proper format when output.
Error handling remains a problem, since it must be handled differently
in each of the three modes. Some interface will need to be defined
between getref and its calling script when run in script mode. The
interface for interactive and cgi mode is already implemented, though
there will need to be some improvement for cgi mode, so that warning
messages are not lost.
The new version of getref will read some of its configuration
information from hard wired arrays and other information from the
database. The most sensisble way of making the distinction is to place
data where there is only one item of information per instrument (such
as the best_ref database table name) in hard wired arrays and place
data where there are multiple items of information for each instrument
in the database. Four types of information are required for the
best_ref tables.
- The header keyword name
- The header keyword sequence number
- The database column containing the best reference file for that
keyword
- The database column containing the original reference file for that
keyword
Four similar columns are required for the best_switch table. In
addition, the database table will need an instrument column to indicate
which keywords are associated with each instrument.
Also there will need to be a database table with the extensions of the
files that contain the best_ref and best_switch keywords.
GETREF PSEUDOCODE
START MAIN:
Read command line arguments
GOSUB FORM_INPUT to read cgi form parameters
Initialize error handling
GOSUB WHICH_QRY to determine type of query
SWITCH on query type
INITIAL:
Set html template to initial page
SUMMARY:
GOSUB GETREF_ALL to do summary query and save results in table
Set html template to summary page
DETAIL:
GOSUB GETREF_ONE to do detail query and save results in table
Set html template to detail page
END SWITCH:
IF output type is html THEN
Insert table in html template and print
ELSE
Print table as ascii file
END IF
END MAIN
START GETREF_ALL:
Create new empty table
GOSUB EXPAND_ASSOC to expand association names into rootnames
LOOP over each rootname
SQL query: "BEGIN TRANSACTION rootname"
GOSUB CHECK_LOCK to see if dataset is locked
IF dataset is not locked THEN
GOSUB SET_CALIBRATE to see if calibration switch is set
GOSUB FILE_TYPES to see which files contain the best info
GOSUB BEST_FILES to determine best reference files
GOSUB BEST_SWITCHES to determine best switches
GOSUB KEY_CHANGES to get other keyword changes
ELSE
Print warning message
END IF
SQL query: "END TRANSACTION rootname"
END LOOP
EXIT with error if no rootnames processed
GOSUB SORT_TABLE to sort table by filename
END GETREF_ALL
START EXPAND_ASSOC:
LOOP over each name
SQL query: "SELECT ASM_MEMBER_NAME FROM ASSOC_MEMBER WHERE
ASM_DATA_SET_NAME = name"
IF no rows returned by query THEN
Copy name to output array
ELSE
Copy sql results to output array
END IF
END LOOP
END EXPAND_ASSOC
START CHECK_LOCK:
Determine instrument from root name
Get lock_col, dataset_name_col, and bestref_table, which
are indexed on instrument
IF retry equal no THEN
Set maxtry to 1
ELSE
Set maxtry to default value
END IF
LOOP on try from 1 to maxtry
SQL query: "SELECT lock_col FROM bestref_table WHERE
dataset_name_col = rootname WITH HOLDLOCK"
IF lock equals "N" THEN
RETURN success
ELSE
Wait for a couple seconds
END IF
END LOOP
RETURN failure
END CHECK_LOCK
START SET_CALIBRATE:
SQL query: "SELECT ODE_STATUS FROM OTFC_DATA_SET_EXCEPTIONS WHERE
ODE_DATA_SET_NAME = rootname AND ODE_STATUS = 'A'"
IF no rows returned THEN
RETURN
END IF
Determine instrument from rootname
Get reference_file containing calibrat keyword, which is indexed
on instrument
Write (rootname, "CALIBRAT", "N", reference_file, "I", "Y") to table
END SET_CALIBRATE
START FILE_TYPES:
Get instrument from rootname
Do sql query to get list of extensions of reference files
containing reference file keywords and calibration switches
(This query is TBD)
END FILE_TYPES
START BEST_FILES:
Get instrument from rootname
Do SQL query to get keyword names (This query is TBD)
Save results in keywords array
SQL query: "SELECT ORF_OLD_FIELD, ORF_BEST_FIELD FROM
OTFC_REF_FILE_FIELDS WHERE ORF_INSTRUMENT = instrument ORDER BY
ORF_SEQUENCE_NUMBER"
Save query results in old_fields and best_fields arrays
SQL query: "SELECT best_fields, old_fields FROM bestref_table
WHERE dataset_name_col = rootname"
LOOP over each keyword in keyword array
Get best reference file name associated with keyword
IF best reference file name is blank or null THEN
Get original reference file name associated with his name
END IF
IF reference file name is a valid filename THEN
Set option flag to "I"
ELSE
Set option flage to "U"
END IF
LOOP over each file_tye
Build observation file name from rootname and extension
Write (rootname, keyword, reference file name, observation
file name, option flag, change flag) to table
END LOOP
END LOOP
END BEST_FILES
START BEST_SWITCHES:
(The code for best switches is very similar to best files,
so it will not be repeated here)
SQL query: "SELECT OSF_OLD_FIELD, OSF_BEST_FIELD FROM
OTFC_REF_FILE_FIELDS WHERE OSF_INSTRUMENT = instrument ORDER BY
OSF_SEQUENCE_NUMBER"
END BEST_SWITCHES
START KEY_CHANGES:
SQL query: "SELECT OKE_FILE_EXTENSION, OKE_SUBSET, OKE_EXTVER,
OKE_IMSET_NAME, OKE_IMSET_VALUE, OKE_KEYWORD_NAME, OKE_NEW_KEYWORD_VALUE,
OKE_OLD_KEYWORD_VALUE, OKE_OPERATION FROM OKE_KEYWORD_EXCEPTIONS WHERE
OKE_STATUS = 'A' AND OKE_DATA_SET_NAME = rootname"
LOOP over each row returned by query
Build observation file name from rootname, file_extension,
extver, and extname
IF old keyword value equals new keyword value THEN
Set change flag to "N"
ELSE
Set change flag to "Y"
END IF
Write (rootname, keyword_name, new_keyword_value, observation
file name, operation, change flag) to table
END LOOP
END KEY_CHANGES
START SORT_TABLE:
Create new empty table
LOOP
Set smallest reference file name to sentinel value
LOOP over each row of original table
IF reference file name in current row is less than
minimum value THEN
Update smallest reference file name
Save row number of row with smallest value
END IF
END LOOP
IF smallest reference file name equals sentinel value THEN
BREAK LOOP
END IF
IF smallest reference file name and its associated keyword
differs from the row previously stored in the new table THEN
Copy row to new table
ELSE
Print warning message
END IF
Delete row from old table
END LOOP
END SORT_TABLE
UPREF PSEUDOCODE
START MAIN
Set old filename to ""
LOOP over each line in delta file
Get keyword name, keyword value, filename, change option
IF filename does not match old filename THEN
Close old file
Open new file
Set old filename to filename
ENDIF
GOSUB UPHEADER to update the header of a single image
extension
END LOOP
Close new file
END MAIN
START UPHEADER:
IF keyword value is a filename THEN
Add reference directory name to file
END IF
Set change to no
Read old keyword value from header
IF found THEN
IF old keyword value does not equal keyword value THEN
Set change to yes
END IF
ELSE
IF change option equals insert THEN
Set change to yes
END IF
END IF
IF change equals yes and verify equals yes THEN
Prompt user to see if change wished and set change accordingly
END IF
IF change equals yes THEN
IF change option equals delete THEN
Remove keyword from header
ELSE
Add keyword to header
END IF
END IF
END UPHEADER