8.7 On-The-Fly Reprocessing Systems

The On-The-Fly Calibration (OTFC) system, publicly released in Dec. 1999, calibrated data at the time a user requested data from the archive. The advantages to using OTFC included the automatic application of improved calibration files and switches, use of most recent calibration software (allowing for rapid access to improved algorithms, new capabilities, and software fixes), and correction of header keywords if needed. An additional benefit is that only the uncalibrated data needs to be stored in the archive.

The On-The-Fly Reprocessing (OTFR) system replaced OTFC on May 16, 2001. The change is transparent to most HST archive users. Requests for data are submitted as usual via StarView or WWW; raw and freshly-calibrated data will be delivered. There is no need to explicitly ask for OTFR: all requests for WFPC2 data are handled by the OTFR system.

The primary difference between the two systems is that OTFR begins earlier in the data path. It uses the original telemetry files ("POD" files) received from Goddard Space Flight Center and performs all pipeline processing steps; OTFC performed only the last pipeline processing step (calibration), on raw files retrieved from the archive. An overview of the data flow for both systems is summarized in the table below. The benefits of the OTFR system encompass the benefits in the OTFC system; in addition, OTFR data needs fewer header corrections (most problems are fixed as part of the pre-calibration pipeline processing) and the system as a whole requires significantly less maintenance effort than OTFC.

UCHCOORD and OTFR.

Improved knowledge of the detector plate scales and chip rotations, as well as changes in reference pixel locations, have resulted in periodic changes to the pointing parameters, especially early in the instrument's lifetime. These header parameters, which define the mapping between the pixel and world coordinate systems, can be updated using the STSDAS task UCHCOORD. The keywords affected include the reference pixel locations (crpix*), the values of the world coordinate system at the reference location (crval*), the partial derivatives of the world coordinate system with respect to the pixel coordinates (cd*), and the orientation of the chip (orientat).

Prior to OTFR (released to the public on May 16, 2001), observers requiring the most up-to-date pointing information in their science image headers ran UCHCOORD on their calibrated images. Since the implementation of OTFR in May 2001, all WFPC2 data retrieved from the archive, regardless of its observation date, has had these corrections applied automatically before being delivered, thus we have discouraged running the UCHCOORD task on OTFR data since it is no longer needed (as described in WFPC2 STAN 45, March 2001 by Baggett, Hsu & Gonzaga). In fact, running UCHCOORD (versions prior to September 2003) on OTFR data would apply unnecessary corrections and corrupt the astrometry (for example, Section 4.3.3 in WFPC2 Data Handbook, Version 4.0, 2002, S. Baggett et al.).

The new version of UCHCOORD in the September 2003 STSDAS release will correctly check whether or not the images have been processed through OTFR, and will no longer modify the header astrometric keywords in such cases. We remind users that it is no longer necessary to run the UCHCOORD task on any WFPC2 data that is retrieved via OTFR, and we recommend that any old WFPC2 data should rather be re-retrieved via OTFR since many other calibrations are also improved.