Start with the steps in the blog: https://blogs.stsci.edu/crds/2011/01/14/delivery-procedures/
- Double-check to make sure all header keywords that are required as shown in JWST Calibration Reference Files: File Formats for the Build 6 Pipeline#Level2B are present and populated. If any are missing, update them.
- Use fitsverify to make sure there are no problems with the headers or files.
- Use certify (in either ssbx or ssbdev). Check latest pmap file at: https://jwst-crds.stsci.edu/
setenv CRDS_PATH /grp/crds/jwst
% setenv CRDS_SERVER_URL
% python -m crds.certify --comparison-context jwst_0175.pmap ./new_reference.fits
- Fix any problems shown by certify.
- Verify the scientific validity of the file.
- Test the files against the pipeline.
- Fill out delivery template: https://blogs.stsci.edu/crds/2012/01/25/step10-fill-out-and-e-mail-the-delivery-template/
- Email form to firstname.lastname@example.org
(More details to follow as they're worked out on steps 5 and 6.)
All files delivered to CRDS are now expected to have a USEAFTER date. Here are some guidelines to follow to make sure the dates are used properly. The Useafter date indicates what data the reference file should be used for.
If the reference file should be used for all data, going back to early test runs, I have been setting the USEAFTER='2000-01-01T00:00:00'. This date is before we had good test data, and indicates the reference file should be used for any and all valid science data.
The Subarrays are a little tricky. The definitions changed a couple of times along the way, necessitating that subarray reference files only be used for certain data sets, or risk crashing the pipeline.
If the subarray reference file is made with FM data, set the USEAFTER ='2000-01-01T00:00:00'. If the subarray file is made with data from CV testing at Goddard, set USEAFTER=' 2013-09-26T05:00:00'. The date is more specific here, because CV1 started a test with the old subarray definition, and then switched almost immediately to the new definitions. There is a caveat to this. If the reference file was made from BURST mode BRIGHTSKY or SUB256 data from CV3, then USEAFTER='2015-08-01T00:00:00', a date before the start of CV3.
Details by Type of reference file:
As of May 2016, we should only deliver files for Level 2A and Level 2B pipelines. Level 3 and higher reference files are not yet defined.
- DISTORTION files are delivered to us as fits files, but must be converted to a series of .asdf files using a script created by Nadia Dencheva.
- PHOTOM files for the Imager, LRS and LRS Slitless must be combined into one file before delivery, using Howard's script (unless the pipeline can use the three fits files as delivered).
- FLAT files: The EC delivers PIXELFLAT, SKYFLAT and FLAT files. The FLAT files are the combination of PIXELFLAT and SKYFLAT files. Only deliver the FLAT files, not the intermediate files. Also, whether the FLAT is an IFU file or an Imager file, both BAND and FILTER are needed. Set the one that doesn't apply to 'N/A'.
- COLCORRPL should not be delivered. This is included with CALIMAGE2 absolute flux cal, but is not applied directly in the pipeline. It should be made available to users (power law SED color corrections).
AREA file needs an 'EXP_TYPE' keyword in the header (missing from CDP-5 delivery). EXP_TYPE is one of the following:
EXP_TYPE values MIR_IMAGE MIR_TACQ MIR_LYOT MIR_4QPM MIR_LRS-FIXEDSLIT MIR_LRS-SLITLESS MIR_MRS MIR_DARK MIR_FLAT
Level 3 files that cannot be delivered yet:
PSF, PSF-OOF, PSF-MONOCHROM, APERCORR, FRINGEFREQ. RSCD is a Level 2A file, but has not yet been defined in the pipeline and also can't be delivered yet.
The procedures to follow in order to deliver new reference file types are on the following page.