The tool is used to examine FITS files to determine how well they conform to the FITS standard. As the tools reads a FITS file, it probes for deviations from the FITS standard and reports on the deviations via warning and error messages. A final message is provided to indicate whether a FITS file conforms to the FITS standard sufficiently enough to be considered "acceptable" FITS; i.e., whether the file passes or fails FITS verification.
Options which provide for some leniancy in looking for non-standard FITS format conditions are built into the tool. These "leniancy" options allow certain conditions which do not strictly conform to the FITS standard to be considered non-severe deviations from the standard. This enables FITS files containing these conditions to be consider acceptable. The as-delivered HEASARC fitsverify tool however does not provide the software hooks to enable the leniancy options.
Since the leniancy options were desired to be used in the STScI OPUS based Data Processing Pipelines, the as-delivered HEASARC fitsverify tool was modified to make them available and hence the opus_fitsverify tool was created.
ERROR CONDITIONS
- Mandatory keyword not present or out of order
- Mandatory keyword has wrong datatype or illegal value
- END header keyword is not present
- Sum of table column widths is inconsistent with NAXIS1 value
- BLANK keyword present in image with floating-point datatype
- TNULLn keyword present for floating-point binary table column
- Bit column has non-zero fill bits or is not left adjusted
- ASCII TABLE column contains illegal value inconsistent with TFORMn
- Address to a variable length array not within the data heap
- Extraneous bytes in the FITS file following the last HDU
- Mandatory keyword values not expressed in fixed format
- Mandatory keyword duplicated elsewhere in the header
- Header contains illegal ASCII character (not ASCII 32 - 126)
- Keyword name contains illegal character
- Keyword value field has illegal format
- Value and comment fields not separated by a slash character
- END keyword not filled with blanks in columns 9 - 80
- Reserved keyword with wrong datatype or illegal value
- EXTEND not present in the primary array if there are extensions
- BLOCKED, if present, not among the first 36 keywords
- XTENSION keyword in the primary array
- Column related keyword (TFIELDS, TTYPEn,TFORMn, etc.) in an image
- SIMPLE, EXTEND, or BLOCKED keyword in any extension
- BSCALE, BZERO, BUNIT, BLANK, DATAMAX, DATAMIN keywords in a table
- Image WCS keywords (CTYPEn, CRPIXn, CRVALn, etc.) in a table
- Table WCS keywords (TCTYPn, TCRPXn, TCRVLn, etc.) in an image
- TDIMn or THEAP keyword in an ASCII table
- TBCOLn keyword in a Binary table
- THEAP keyword in a binary table that has PCOUNT = 0
- XTENSION, TFORMn, TDISPn or TDIMn value contains leading space(s)
- Index of any WCS keyword (CRPIXn, CRVALn, etc.) greater than
value of WCSNAXES
- Index of any table column descriptor keyword (TTYPEn, TFORMn,
etc.) greater than value of TFIELDS
- TSCALn or TZEROn present for an ASCII, logical, or Bit column
- TDISPn value is inconsistent with the column datatype
- Length of a variable length array greater than the maximum
length as given by the TFORMn keyword
- Logical column contains illegal value not equal to 'T', 'F', or 0
- Character string column contains non-ASCII text character
- Header fill bytes not all blanks
- Data fill bytes not all blanks in ASCII tables or all zeros
in any other type of HDU
- Gaps between defined ASCII table columns contain characters with
ASCII value > 127
WARNING CONDITIONS
- SIMPLE = F
- Presence of deprecated keywords BLOCKED or EPOCH
- 2 HDUs have identical EXTNAME, EXTVER, and EXTLEVEL values
- BSCALE or TSCALn value = 0.
- BLANK OR TNULLn value exceeds the legal range
- TFORMn has 'rAw' format and r is not a multiple of w
- DATE = 'dd/mm/yy' and yy is less than 10 (Y2K problem??)
- Index of any WCS keyword (CRPIXn, CRVALn, etc.) greater than
value of NAXIS, if the WCSAXES keyword is not present
- Duplicated keyword (except COMMENT, HISTORY, blank, etc.)
- Column name (TTYPEn) does not exist or contains characters
other than letter, digit and underscore
- Calculated checksum inconsistent with CHECKSUM or DATASUM keyword
With the opus_fitsverify tool, the error conditions recognized in the as-delivered HEASARC version of the tool are partitioned into two groups; i.e., regular errors and severe errors, and an additional option of only looking for and reporting on severe error conditions is provided. Thus the opus_fitsverify tool provides 3 options to be used in determining how well a FITS file conforms to the FITS standard. Option 1, the default option, is to look for and report on all warning and all error conditions. Option 2 is to only look for and report on all error conditions; i.e., all warning conditions are ignored. Option 3 is to look for and report on only severe error conditions; i.e., all regular errors and warnings will be ignored. The distinction between "severe" and "regular" errors is somewhat arbitrary as noted by Bill Pence at HEASARC. This list of conditions is as follows:
SEVERE ERROR CONDITIONS
- Mandatory keyword not present or out of order
- Mandatory keyword has wrong datatype or illegal value
- END header keyword is not present
- Sum of table column widths is inconsistent with NAXIS1 value
- BLANK keyword present in image with floating-point datatype
- TNULLn keyword present for floating-point binary table column
- Bit column has non-zero fill bits or is not left adjusted
- ASCII TABLE column contains illegal value inconsistent with TFORMn
- Address to a variable length array not within the data heap
- Extraneous bytes in the FITS file following the last HDU
REGULAR ERROR CONDITIONS
- Mandatory keyword values not expressed in fixed format
- Mandatory keyword duplicated elsewhere in the header
- Header contains illegal ASCII character (not ASCII 32 - 126)
- Keyword name contains illegal character
- Keyword value field has illegal format
- Value and comment fields not separated by a slash character
- END keyword not filled with blanks in columns 9 - 80
- Reserved keyword with wrong datatype or illegal value
- EXTEND not present in the primary array if there are extensions
- BLOCKED, if present, not among the first 36 keywords
- XTENSION keyword in the primary array
- Column related keyword (TFIELDS, TTYPEn,TFORMn, etc.) in an image
- SIMPLE, EXTEND, or BLOCKED keyword in any extension
- BSCALE, BZERO, BUNIT, BLANK, DATAMAX, DATAMIN keywords in a table
- Image WCS keywords (CTYPEn, CRPIXn, CRVALn, etc.) in a table
- Table WCS keywords (TCTYPn, TCRPXn, TCRVLn, etc.) in an image
- TDIMn or THEAP keyword in an ASCII table
- TBCOLn keyword in a Binary table
- THEAP keyword in a binary table that has PCOUNT = 0
- XTENSION, TFORMn, TDISPn or TDIMn value contains leading space(s)
- Index of any WCS keyword (CRPIXn, CRVALn, etc.) greater than
value of WCSNAXES
- Index of any table column descriptor keyword (TTYPEn, TFORMn,
etc.) greater than value of TFIELDS
- TSCALn or TZEROn present for an ASCII, logical, or Bit column
- TDISPn value is inconsistent with the column datatype
- Length of a variable length array greater than the maximum
length as given by the TFORMn keyword
- Logical column contains illegal value not equal to 'T', 'F', or 0
- Character string column contains non-ASCII text character
- Header fill bytes not all blanks
- Data fill bytes not all blanks in ASCII tables or all zeros
in any other type of HDU
- Gaps between defined ASCII table columns contain characters with
ASCII value > 127
WARNING CONDITIONS
- SIMPLE = F
- Presence of deprecated keywords BLOCKED or EPOCH
- 2 HDUs have identical EXTNAME, EXTVER, and EXTLEVEL values
- BSCALE or TSCALn value = 0.
- BLANK OR TNULLn value exceeds the legal range
- TFORMn has 'rAw' format and r is not a multiple of w
- DATE = 'dd/mm/yy' and yy is less than 10 (Y2K problem??)
- Index of any WCS keyword (CRPIXn, CRVALn, etc.) greater than
value of NAXIS, if the WCSAXES keyword is not present
- Duplicated keyword (except COMMENT, HISTORY, blank, etc.)
- Column name (TTYPEn) does not exist or contains characters
other than letter, digit and underscore
- Calculated checksum inconsistent with CHECKSUM or DATASUM keyword
opus_fitsverify [optional flags] <filename> or <@filelist.txt>
where:
optional flags are: -l list all header keywords
-q quiet; print one-line pass/fail summary per file
-e only test for error conditions; don't issue warnings
-s only test for severe error conditions
Valid combinations of the optional flags are: (-l -e) (-l -s) (-q -e) (-q -s).
If no optional flags are specified, the tool defaults to looking for all warning
and all error conditions.
'filename' is a filename template (with optional wildcards)
'filelist.txt' is an ASCII text file with a list of FITS file names, one per line
To get this information in a USAGE message, type opus_fitsverify
To get a detailed help description, type opus_fitsverify -h
The detailed help description provides the following information:
USAGE: opus_fitsverify filename ... - verify one or more FITS files
(may use wildcard characters)
or opus_fitsverify @filelist.txt - verify a list of FITS files
Optional flags:
-l list all header keywords
-q quiet; print one-line pass/fail summary per file
-e test for any (regular or severe) error conditions (ignore warnings)
-s only test for severe error conditions (ignore warnings/regular errors)
valid combinations of the optional flags are: (-l -e) (-l -s) (-q -e) (-q -s)
opus_fitsverify exits with a status equal to the number of errors + warnings.
EXAMPLES:
opus_fitsverify -l m101.fits - produce detailed verificaton report of a
single file, including a keyword listing,
and report any warnings and any errors
opus_fitsverify -q *.fits *.fit - verify all files with .fits or .fit
extensions, writing a 1-line pass/fail
message for each file
opus_fitsverify -e abc.fits - produce verificaton report for one file
and report any errors (ignore warnings)
opus_fitsverify -s abc.fits - produce a verificaton report for one file
and report only severe errors
(ignore warnings and regular errors)
DESCRIPTION:
This task reads one or more input FITS files and verifies that the
files conform to the specifications of the FITS Standard document
(known as the NASA/Science Office of Standards and Technology 'Definition
of the Current FITS Standard', document number, NOST 100-2.0, available
online at http://fits.gsfc.nasa.gov/). The input filename template may
contain wildcard characters, in which case all matching files will be
tested. Alternatively, the name of an ASCII text file containing a list
of file names, one per line, may be entered preceded by an '@' character.
The following error or warning conditions will be reported:
SEVERE ERROR CONDITIONS
- Mandatory keyword not present or out of order
- Mandatory keyword has wrong datatype or illegal value
- END header keyword is not present
- Sum of table column widths is inconsistent with NAXIS1 value
- BLANK keyword present in image with floating-point datatype
- TNULLn keyword present for floating-point binary table column
- Bit column has non-zero fill bits or is not left adjusted
- ASCII TABLE column contains illegal value inconsistent with TFORMn
- Address to a variable length array not within the data heap
- Extraneous bytes in the FITS file following the last HDU
REGULAR ERROR CONDITIONS
- Mandatory keyword values not expressed in fixed format
- Mandatory keyword duplicated elsewhere in the header
- Header contains illegal ASCII character (not ASCII 32 - 126)
- Keyword name contains illegal character
- Keyword value field has illegal format
- Value and comment fields not separated by a slash character
- END keyword not filled with blanks in columns 9 - 80
- Reserved keyword with wrong datatype or illegal value
- EXTEND not present in the primary array if there are extensions
- BLOCKED, if present, not among the first 36 keywords
- XTENSION keyword in the primary array
- Column related keyword (TFIELDS, TTYPEn,TFORMn, etc.) in an image
- SIMPLE, EXTEND, or BLOCKED keyword in any extension
- BSCALE, BZERO, BUNIT, BLANK, DATAMAX, DATAMIN keywords in a table
- Image WCS keywords (CTYPEn, CRPIXn, CRVALn, etc.) in a table
- Table WCS keywords (TCTYPn, TCRPXn, TCRVLn, etc.) in an image
- TDIMn or THEAP keyword in an ASCII table
- TBCOLn keyword in a Binary table
- THEAP keyword in a binary table that has PCOUNT = 0
- XTENSION, TFORMn, TDISPn or TDIMn value contains leading space(s)
- Index of any WCS keyword (CRPIXn, CRVALn, etc.) greater than
value of WCSNAXES
- Index of any table column descriptor keyword (TTYPEn, TFORMn,
etc.) greater than value of TFIELDS
- TSCALn or TZEROn present for an ASCII, logical, or Bit column
- TDISPn value is inconsistent with the column datatype
- Length of a variable length array greater than the maximum
length as given by the TFORMn keyword
- Logical column contains illegal value not equal to 'T', 'F', or 0
- Character string column contains non-ASCII text character
- Header fill bytes not all blanks
- Data fill bytes not all blanks in ASCII tables or all zeros
in any other type of HDU
- Gaps between defined ASCII table columns contain characters with
ASCII value > 127
WARNING CONDITIONS
- SIMPLE = F
- Presence of deprecated keywords BLOCKED or EPOCH
- 2 HDUs have identical EXTNAME, EXTVER, and EXTLEVEL values
- BSCALE or TSCALn value = 0.
- BLANK OR TNULLn value exceeds the legal range
- TFORMn has 'rAw' format and r is not a multiple of w
- DATE = 'dd/mm/yy' and yy is less than 10 (Y2K problem??)
- Index of any WCS keyword (CRPIXn, CRVALn, etc.) greater than
value of NAXIS, if the WCSAXES keyword is not present
- Duplicated keyword (except COMMENT, HISTORY, blank, etc.)
- Column name (TTYPEn) does not exist or contains characters
other than letter, digit and underscore
- Calculated checksum inconsistent with CHECKSUM or DATASUM keyword