kwset
Summary
This tool recomputes a set of keyword values for an observation by applying
a set of keyword rules.
Description
The keyword values will be determined by examining keyword header input
values from either a FITS file, or from fields in specific DADS database
relations, and applying a set of rules to these input values. This
results in a set of output keyword values that can either be written out
in ASCII form, performed as updates to the FITS file, or written
out in SQL format for later application to the DADS database.
One main use of this tool is to provide the BESTSWITCH function for
the On-the-fly calibration system. In this scenario, kwset is used
to recompute the calibration switch settings for an observation.
The keyword inputs in this case come from the DADS catalog. A number
of default behaviors in the tool are tailored to its use as BESTSWITCH,
but these defaults can be overriden on the command line.
Usage
$ kwset [-d {dataset_name}] [-k {keyword_list_filename}]
[-i db|fits] [-o sql|fits|ascii]
[-of {output_filename}][-r
{alt_rules_filename}]
[-f {datasets_filename}][-c
{rule_classes}]
[-ks {keyword_db_server}]
[-kd {keyword_db_name}]
[-ds {DADS_db_server}]
[-dd {DADS_db_name}]
Input
The input consists of
-
the name of a dataset (nine-character IPPPSSOOT) or a file listing the
names of datasets (REQUIRED)
-
a file listing the keyword names that will be recomputed (one keyword name
per line) (OPTIONAL)
-
a file of keyword rules in the OPUS rules format (OPTIONAL)
-
names of rule classes to load from the keyword rules file (OPTIONAL)
-
keyword values from the headers of FITS files OR
-
keyword values from specific tables in the DADS database
-
the name of the server and database to access for keyword information,
if either the -i db or -o sql arguments are used
-
the name of the server and database to access for DADS information, if
either the -i db or -o sql arguments are used
Output
The output consists of
-
updated keyword values to existing FITS header string keywords OR
-
keyword=value lines written to STDOUT or a file OR
-
SQL update statements written to STDOUT or a file
Constraints
If database input is used, the following databases and tables must be readable:
database:keyword
tables: hda_fields, dads_keywords
database:dadsotfc tables:
specified by entries in hda_fields and dads_keywords
(e.g. stis_ref_data)
If SQL output is used, the following databases and tables must be readable:
database:keyword
tables: hda_fields
database:dadsotfc tables:
otfc_calibration_fields,
tables specified by hda_fields entries (e.g. stis_ref_data)
If FITS file input is used
-
keyword values are read from the primary header only.
If FITS file output is used
-
only keyword updates are performed. New keyword values will
not be inserted into a header where they are not already present.
-
only string keyword values will be updated. Numeric keyword
updates are not currently supported.
The environment variable OPUS_DEFINITIONS_DIR must be defined
to point to the default location of the keyword rules and keyword list
files.
If you do not specify the database servers and database names using
the command line parameters (-ks, -kd, -ds, -dd), then environment
variables DSQUERY and OPUS_DB must be defined to point
to the database server and database name when database input or SQL output
is requested. The database name provided in OPUS_DB is used for the
DADS database. The default database name "keyword" will be used to
access the keyword database. Both database are assumed to be on the
DSQUERY server if you do not specify the servers explicitly.
Command Line Arguments
-d[ataset]
The 9-character name (in
IPPPSSOOT format) of a dataset. If this option is provided, the -f
option is not used.
-f[ile_of_dataset_names]
A file containing a list
(one per line) of dataset names to process. If this option is provided,
the -d option is not used.
-k[eyword_list_file]
A file listing the keyword
names to be recomputed, one per line. Defaults to files that list
the calibration switch keyword names for each instrument:
-
STIS: OPUS_DEFINITIONS_DIR:stis.switches
-
NICMOS: OPUS_DEFINITIONS_DIR:nic.switches
-
ACS: OPUS_DEFINITIONS_DIR:acs.switches
-
WFPC-2: OPUS_DEFINITIONS_DIR:wf2.switches
-
FOC: OPUS_DEFINITIONS_DIR:foc.switches
-r[ules_file_alternate]
The name of the keyword
rules file to use. The default is OPUS_DEFINITIONS_DIR:keyword_rules.lis
-c[lasses_of_rules_to_load]
A comma-delimited list of
rule classes to load from the keyword rules file (e.g. STS,STI would load
all STIS rules). Defaults are instrument-specific, depending on the
dataset name being processed:
-
STIS : STS
-
NICMOS: NCS
-
ACS: ASW
-
WFPC-2: WF2
-
FOC: FCS
-of[ile]
File to use for writing
output to. If the file exists, it will be appended to, otherwise
it is created as new. Defaults to STDOUT.
-ks
The name of the database
server to access for keyword database information. Defaults
to DSQUERY.
-kd
The name of the keyword
database to access for keyword information. Defaults to "keyword".
-ds
The name of the database
server to access for reading the DADS catalog. Defaults to
DSQUERY.
-dd
The name of the DADS database
to access for reading the DADS catalog. Defaults to OPUS_DB.
-i[nput_mode]
The source of the keyword
values:
-
db - DADS database tables (DEFAULT)
-
fits - Primary header of all FITS files matching dataset*.fits in
the current directory
-o[utput_mode]
The destination of the keyword
values:
-
sql - SQL update statements for instrument-specific DADS database tables
will be written to STDOUT or the output file (DEFAULT)
-
fits - Primary header keywords of an instrument-specific FITS data file
will be updated. For STIS, NICMOS, and ACS the dataset_raw.fits
file
is updated. For WFPC-2 and FOC, the dataset_d0f.fits
file
is updated.
-
ascii - Lines of keyword = value written to STDOUT or the output
file
Examples
$ kwset -d u2440101t
Obtains the current keyword values for dataset u2440101t from
DADS database tables (-i default), applies the keyword rules from OPUS_DEFINITIONS_DIR:keyword_rules.lis
(-r default), and writes SQL update statements (-o default) for the WFPC-2
calibration switch settings in OPUS_DEFINITIONS_DIR:wf2.switches
(-k default) to STDOUT (-of default).
$ kwset -d o46h04f5q -i fits -o fits
Obtains the current keyword values for dataset o46h04f5q from
all FITS datasets matching that dataset name in the current directory.
Applies the keyword rules from OPUS_DEFINITIONS_DIR:keyword_rules.lis
and writes updated STIS calibration switch settings listed in OPUS_DEFINITIONS_DIR:stis.switches
back
into the primary header of FITS file o46h04f5q_raw.fits.
Detailed Description
The KWSET tool follows this basic flow:
-
obtain the keyword rules that will be used during this run (the "rules
of interest")
-
obtain keyword input values from a source
-
apply the rules of interest to those values
-
write keyword output values to a destination
The command line options and defaults control which keyword source, rules,
and destination are used.
Rules of Interest
The KWSET tool is designed to be able to run on a subset
of an entire keyword rules file. The portion of the rules file that
is used is narrowed down first using the -c rule_classes command
line argument. Rule classes are currently three-character strings
that identify lines in the keyword rules file. For example, STS is
the rule class that defines rules used to set STIS calibration switch settings.
After the requested rule classes are loaded from the rules file, each
keyword whose value is to be recomputed (the keywords listed in the -k
filename) is traced through the rules. Any additional keywords
upon which the needed keyword depends must also be loaded from the keyword
source, so space is allocated for them. Each of these added keywords
is then also traced, so that when the tracing is complete, a self-contained
list of rules is identified that will be used to set the values for the
requested keywords. All other rules that happen to be loaded
will at that point be pruned off, since they are not needed. The
set of resulting rules is driven by the list of keyword values that the
user wants to recompute. All other extranneous rules are removed.
Database Input
The DADS database provides the source of keyword values
when this mode (-i db) is used. Each keyword used as a source
in the rules of interest is searched for in the keyword database
tables: hda_fields and dads_keywords. These tables
indicate the location(s) in the DADS database where the value for this
keyword can be found. Many keywords can be found in multiple locations,
so the possible sources go through a reduction scheme which must reduce
to only a single possible keyword source. If the reduction fails,
an error message is printed and the program halts. This is
considered an ambiguous situation requiring developer attention.
The reductions occur in this order:
-
if the DADS table name contains _ref_data, then the table is used
(e.g. stis_ref_data)
-
if the DADS table name contains science, then the table is used
(e.g. stis_science)
Once the database source is found for every keyword needed to evaluate
the rules, database queries are constructed to retrieve the keyword values.
The process is optimized to only query each DADS table one time,
so all fields needed from a table are retrieved in a single query.
The process is mostly generic except for the database table/field combination
stis_a_data.sclamp.
Because of the way STIS Wavecals are handled in the DADS database, there
are multiple entries in the
stis_a_data relation for each dataset,
and these entries can have different sclamp values which can result
in different rule evaluations. A special case query is performed
for stis_a_data.sclamp that returns only the science entries for
this field (a join is made to the
assoc_member table where AUTO-WAVECAL
entries can be filtered out). The stis_a_data.sclamp query
is currently the only query allowed to return multiple database records.
If this occurs for any other table, an error message is printed and the
program halts.
FITS Input
If input from a FITS file is requested, all files matching
the filespec dataset_name*.fits in the current directory are read
as possible sources for keyword input values. It is assumed
that any keyword names needed by the rules that are found in multiple FITS
files have the same value in all FITS files, so in this case the value
is obtained from any one of the files. Keyword values are only
read from the primary header.
Rule Evaluation
Once all keyword values are retrieved from the keyword
source, the keyword rules are evaluated, which normally results in some
changes to the keyword values. The changes are not yet written out
to the desired keyword destination.
SQL Output
One possible source for keyword value output is in the
form of SQL update statements. These can be used to perform updates
to the BESTSWITCH values in the DADS database. This will be the normal
operating mode when kwset is run as the BESTSWITCH tool.
In this case, the DADS table otfc_calibration_fields is queried
for each requested keyword. This query returns the fieldname in the
DADS database where the BESTSWITCH value of this keyword should be stored.
Another query is made against the keyword database to determine
that database table that corresponds to this field. Once all
requested keywords (i.e. the calibration switch names) are found in DADS,
a single SQL update statement is written for each DADS database table containing
BESTSWITCH keywords. These SQL update statements
are not applied
to the database! They are written to the specified output location
so that the user can later apply them to the database (likely wrapped in
a transaction).
FITS Output
If output to a FITS file is requested, keyword updates
are performed on a specific FITS file for each instrument.
The listing of which FITS file is updated for each instrument is given
below:
-
STIS, NICMOS, ACS: dataset_raw.fits
-
WF2, FOC:
dataset_d0f.fits
Only keyword updates are performed. Keywords that are not already
in the FITS header will currently not be added to the FITS header.