Space Telescope Imaging Spectrograph Instrument Handbook for Cycle 14

15.1 Pipeline Processing Overview

---

Science data taken by STIS are received from the Space Telescope Data Capture Facility and sent to the STScI pipeline, where the data are unpacked, keywords extracted from the telemetry stream, and the science data reformatted and repackaged into raw (uncalibrated) FITS¹ files by the generic conversion process. All STIS data products are FITS files. The vast majority of the STIS data products are two-dimensional images that are stored in FITS image extension files as triplets of science, error, and data quality arrays. FITS image extensions offer great flexibility in data storage and allow us to package related science data that are processed through calibration as a single unit together into one file. The uncalibrated FITS files are passed through calstis, the software code that calibrates the data, producing calibrated FITS files. For more details on STIS data structure and the naming conventions for the uncalibrated and calibrated data products, see Chapter 2 in Part II of the STIS Data Handbook.

Calstis performs the following basic science data calibrations:

• Basic, two-dimensional image reduction producing a flat-fielded output image (rootname_flt.fits), which, depending on whether the data are from the MAMA or the CCD and whether they are imaging or spectroscopic data, includes the following: data quality initialization, dark subtraction, bias subtraction, non-linearity flagging, flat fielding, and photometric calibration.
• Two-dimensional spectral extraction producing a flux-calibrated, rectified spectroscopic image (usually rootname_x2d.fits for MAMA data, _sx2.fits for CCD) with distance along the slit running linearly along the Y axis and wavelength running linearly along the X axis, for spectroscopic first-order mode data.
• One-dimensional spectral extraction producing a one-dimensional spectrum of flux versus wavelength (usually rootname_x1d.fits for MAMA data, _sx1.fits for CCD), uninterpolated in wavelength space, but integrated across an extraction aperture in the spatial direction, for first-order and echelle spectroscopic data.
• Data taken in TIME-TAG mode are corrected for the Doppler shift from the spacecraft motion and output as an uncalibrated event stream by the pipeline in a FITS binary table (rootname_tag.fits). The time-tag data stream is also integrated in time to produce an uncalibrated ACCUM mode image (rootname_raw.fits) which is then passed through standard calibration. The odelaytime task in STSDAS can be used off-line to correct the TIME-TAG spacecraft times to heliocentric times.

In addition, calstis performs two types of contemporaneous calibrations:

• For CCD exposures which have been CR-split or when multiple exposures have been taken, calstis combines the exposures, producing a cosmic-ray rejected image (rootname_crj.fits) which is then passed through subsequent calibration (e.g., spectral extraction).
• For spectroscopic exposures, calstis processes the associated wavecal exposure (see Routine Wavecals) to determine the zero point offset of the wavelength and spatial scales in the science image, thereby correcting for thermal drifts and the lack of repeatability of the mode select mechanism. Whereas the uncalibrated science data are stored in the rootname_raw.fits file, the accompanying wavecal observations are stored in the file rootname_wav.fits.²

Figure 15.1 through Figure 15.3 show example output from the calstis pipeline. The calstis program propagates statistical errors and tracks data quality flags through the calibration process. calstis is available to users in STSDAS, so they can recalibrate their data as needed.³ Recalibration may be performed in its entirety in a manner identical to the pipeline calibration by using calstis, or modular components of calstis, such as basic two-dimensional image reduction (basic2d), two-dimensional spectral extraction (x2d), one-dimensional spectral extraction (x1d), or cosmic ray rejection (ocrreject). The calibration step that calstis performs on a given set of science observations depends on the nature of those observations.⁴

Since the spring of 2001, calibrated data products for STIS have been available through On-the-fly-reprocessing (OTFR), which replaces On-the-fly-calibration (OTFC). The OTFR systems starts with raw telemetry products and converts these to FITS files and adds the latest instrument calibrations. This change is transparent to the user.

Some changes have been made to calstis, and propagated to OTFR, to handle temporal changes in instrument performance, more changes will be implemented during Cycle 12.

• In July 2001, STIS operation was resumed with side 2 electronics after failure of the side 1 electronics in May. A wider range in the operating temperature of the CCD detector with the side 2 electronics has made it necessary to include a temperature-dependent scaling factor when producing and applying side 2 CCD dark reference files. This is discussed more fully in STIS ISR 2001-03.
• Since September 2002, calstis has used epoch-selected darks for the NUV-MAMA, which are scaled by a factor depending on the time and detector temperature of the science data.
• Gradual changes in spectroscopic and imaging sensitivity have been measured over the lifetime of the STIS detectors. See STIS ISR 2001-01 for full details. Currently, 1-D spectral fluxes (FLUX column in rootname_sx1.fits and rootname_x1d.fits files) extracted from data taken with the first-order low and medium resolution CCD and MAMA gratings are calibrated with a time-dependent sensitivity (TDS) factor to correct for these changes. Temporal changes in CCD spectroscopic count rates are dominated by the steadily increasing charge transfer inefficiency (CTI), whose dependence on the structure and count level of the source and background is complex. See Section 7.2.6 and Section 7.2.7 and STIS ISR 2003-03 for additional discussion. Correction of extracted CCD spectral fluxes for these CTI effects is also implemented in the standard pipeline software. TDS corrections were implemented in the pipeline for first-order MAMA spectra in September 2002 (calstis 2.13b). TDS and CTE corrections for CCD spectra were implemented in December 2003 (calstis 2.15b). Users who have data calibrated with earlier versions of the calstis pipeline software may wish to obtain new copies from the MAST archive. Corrections for time-dependent changes of extracted echelle fluxes and for the PHOTFLAM keyword values in imaging data are currently being implemented.

Figure 15.1: Two-Dimensional Rectification


 

Observers can retrieve HST data by using StarView (http://starview.stsci.edu/) or the HST archive WWW interface (http://archive.stsci.edu/hst/) to select specific datasets. One can choose where and how the data are delivered: on the archive computer staging disk for copy using anonymous FTP, directly sent to a home computer via FTP or SFTP, or for very large requests, sent on a medium of your choice (CDs, DVDs or tapes). One must be a registered archive user to be able to retrieve HST data (see http://archive.stsci.edu/hst/getting_started.html).

Figure 15.2: Cosmic Ray Rejection


 
Figure 15.3: One-Dimensional Spectral Extraction


 
¹Flexible Image Transport System.
²Unassociated wavecal observations (e.g., GO added wavecals) are stored in rootname_raw.fits, where rootname is different for the science and wavecal exposures.
³The calstis software is written in C and uses open IRAF in conjunction with a specially written I/O interface to the FITS data file.
⁴Available-but-unsupported-mode data are calibrated only through flat-fielding in the pipeline.

	Space Telescope Science Institute http://www.stsci.edu Voice: (410) 338-1082 help@stsci.edu