catalog_hst_non_cal.pl

This utility enables the cataloging, without a pipeline, of HST data that do not fit into the CAL or OMS archive classes, e.g. DIA, EDT, PDQ, etc.

Usage

>catalog_hst_non_cal.pl -d <data_id> [-i <input_dir>]

Context

Inputs

-d {data_id}

The <data_id> is the OSF data_id used in normal processing. It is generally not equal to the archive class. If it is set to an invalid value, such as "xxx", then a list of valid values with the name of the related archive class is printed. This tool supports all HST data_ids except for classes CAL and OMS that have associations.

-i {input_dir}

This is the optional location of the data. If not provided, current working directory is assumed.

Data

OPUS 2010.4 changes should allow the data to be ingested in either the name that comes out of the originating OPUS pipeline, or the one that DADS delivers. The data format matters. In other words, SMS data can be in either yd5a11550.pod format, or yt5813360_pod.fits format. PDQ data that is a text file called o6n901fbq.pdq should work, and PDQ data that has already been converted to fits and is called o6n901fbq_pdq.fits will also work. OMS FITS data called o6n901fbq.jif will work just as well as OMS FITS data called o6n901fbq_jif.fits.

Outputs

No real output, but runs catalog and housekeeping software for the input data. Dumps messages to the screen which could be redirected into a file.

Verbosity

For verbose messaging, especially for debugging, use the standard OPUS environment variable MSG_REPORT_LEVEL. There is no more -v option. So to turn messaging up for a run of the command-line tools, set the MSG_REPORT_LEVEL on the command line before you run the tools. This will only affect processes run in the same window thereafter, not the entire account you are running in.
E.g. setenv MSG_REPORT_LEVEL MSG_ALL

Note that MSG_REPORT_LEVEL and STDB_REPORT_LEVEL settings are changed right before calls to genreq, InteractiveIngResponse, and update_db_tool. In the case of genreq and ingrsp, this was done to make debugging these comand-line tools easier. It was thought we did not need to debug those processes since they were not changing. However, we may wish to reconsider this change in the future.

In the case of update_db_tool, no extra reporting is necessary as this tool already does a wonderful job of making itself clear. Any extra reporting is redundant and makes it more difficult to see what the tool is doing.

See_also

Examples

#:194:sherbert@tufnel> unsetenv MSG_REPORT_LEVEL
#:195:sherbert@tufnel> unsetenv STDB_REPORT_LEVEL
#:196:sherbert@tufnel> catalog_hst_non_cal.pl
2010280135825-E-ERROR-Too few arguments
Usage: 
>catalog_hst_non_cal.pl -i <input_dir> -d <data_id>  
   
   All parameters are required but the order of [-flag value] pairs does
   not matter.  

   <input_dir>     is the location of the data to be archived.  
                   Only data from one type of archive_class should be here.
                   This is the optional parameter.  Defaults to current dir,
                   if omitted.
   
   <data_id>       is the OSF data_id used in normal processing.  It is
                   generally not equal to the archive class.  If it is
                   set to an invalid value, such as "xxx", then a list
                   of valid values with the name of the related archive
                   class is printed.  This tool supports all HST
                   data_ids except for classes CAL and OMS which have
                   associations.


#:197:sherbert@tufnel> catalog_hst_non_cal.pl -i dmp -d dmp > & keepMeCat.out

#:198:sherbert@tufnel> cat keepMeCat.out 
2010280140219-I-INFO-Starting... catalog_hst_non_cal.pl -i dmp -d dmp
2010280140219-I-INFO-get_regr_path-Obtained null path name: /info/tufnel/lisa/test_sherbert_20100902_1/regr_00202/defs/null.path
2010280140219-I-INFO-get_regr_path-Found INGEST_PATH_NAME: regr
2010280140219-I-INFO-check_pipeline-Pipeline test: osf_test -p regr -pr dataset
2010280140222-I-INFO-check_pipeline-regr is idle.
2010280140222-I-INFO-catalog_hst_non_cal.pl-chdir'd to /info/tufnel/lisa/test_sherbert_20100902_1/regr_00202/dmp 
2010280140222-I-INFO-get_datasets-Adding gtja92336 to dataset list.
2010280140222-I-INFO-get_cat_resource-data_id dmp SKIPS cataloging 
2010280140222-I-INFO-catalog_hst_non_cal.pl-No catalog updates required for data_id dmp
2010280140222-I-INFO-get_hkp_resource-The grep file mask is /info/tufnel/lisa/test_sherbert_20100902_1/regr_00202/defs/hkp*.resource.
2010280140223-I-INFO-get_hkp_resource-The grep file mask is /info/tufnel/lisa/test_sherbert_20100902_1/install/opus_ops/definitions/unix/hkp*.resource.
2010280140223-I-INFO-catalog_hst_non_cal.pl-hkp_resource is hkp_misc 
2010280140223-I-INFO-Processing with data_id dmp with resource hkp_misc.
2010280140223-I-INFO-hkp_datasets-Starting housekeeping data update for dataset gtja92336. 
2010280140223-I-INFO-Using command: update_db_tool -r hkp_misc -t dmp gtja92336 2>&1
2010280140226-I-INFO Starting processing. (1)
2010280140226-I-INFO Initialized OPUS environment. (1)
2010280140226-I-INFO Loading queries from /info/tufnel/lisa/test_sherbert_20100902_1/regr_00202/defs//hkp_hst_query.lis (1)
2010280140226-I-INFO ***** loading queries from /info/tufnel/lisa/test_sherbert_20100902_1/regr_00202/defs//hkp_hst_query.lis (1)
2010280140226-I-INFO ***** query loading completed. (1)
2010280140226-I-INFO Registered callbacks. (1)
2010280140226-I-INFO ***** begin update_db_tool for gtja92336 data_id dmp ***** (1)
2010280140226-I-INFO Trying query named SET_CAT_COMPLETE_DATE (1)
2010280140226-I-INFO Title: Upate dads_archive.catalog_date to signal successful cataloging (1)
2010280140226-I-INFO 0 records updated for 
UPDATE opusauto1..dads_archive 
   SET catalog_date = getdate()
 WHERE dataset_name = 'GTJA92336'
   AND archclass = 'DMP'
   AND archdate = ( SELECT MAX(archdate) 
                    FROM opusauto1..dads_archive 
                    WHERE dataset_name = 'GTJA92336'
                      AND archclass = 'DMP'
                  )
2010280140226-I-INFO *****  end  update_db_tool for gtja92336 data_id dmp ***** (1)
2010280140226-I-INFO-hkp_datasets-Completed housekeeping data update for dataset gtja92336.
2010280140226-I-INFO-Completed...catalog_hst_non_cal.pl -i /info/tufnel/lisa/test_sherbert_20100902_1/regr_00202/dmp -d dmp

Last Updated: OPUS 2010.4