Appendix: WFC3 TestingΒΆ

Author:Howard Bushouse

The test cases that were used to compare pysynphot calculations with SYNPHOT were based on a large set of WFC3 ETC regression tests originally created a few years ago by Tom Brown, which were used to validate the original implementation of the WFC3 ETC. These tests spanned a very large, multi-dimensional parameter space, covering the three WFC3 detetors (UVIS channel CCD chips 1 and 2 and the IR channel FPA), all of the UVIS and IR channel filters and grisms. The detector+filter combinations were in turn paired with a variety of input spectra, including many from the different STScI libraries of stellar spectra (e.g. k93models, Bruzual atlas), HST standard stars, and non-stellar sources (e.g. elliptical and spiral galaxies), as well as blackbody models of various temperatures, power-law spectra with various spectral indexes, and simple flat continuum (“unit”) spectra. In addition to these spectra of astronomical targets, excersizing the ETC’s various options for specifying background signal due to Zodiacal light and earthshine, and air glow necessitated the inclusion of the spectra used to model these sources. All WFC3 observing mode plus spectrum combinations were further modified using various combinations of renormalization options and extinction (“ebmv” function) settings.

The use of all of the above combinations of WFC3 observing modes and source spectra resulted in a very robust set of thousands of individual SYNPHOT test cases, each with their own unique ability to test different aspects of the synthetic photometry calculations.

As described above in section (“Test Sets”), the comparison of SYNPHOT and pysynphot results for each of these thousands of test cases were done in several ways. First, a comparison was made of the total integrated throughput computed for the WFC3 observing mode used in each test (i.e. a comparison of the “bare” throughput of the WFC3 passband without convolving it with a source spectrum). Second, a comparison was made of the total integrated response of the WFC3 observing mode convolved with the source spectrum, which we refer to as the “scalar” countrate result. Finally, a comparison was also made of the countrate spectrum that results from convolving the WFC3 observing mode with the source spectrum by taking the ratio of the SYNPHOT and pysynphot-produced spectra.

While the thousands of WFC3 test cases resulted in hundreds of reported discrepancies in the comparisons of the SYNPHOT and pysynphot results, the overall results can be summarized as follows.

1) All comparisons of the integrated throughput of the “bare” WFC3 observing mode showed that the pysynphot results matched those from SYNPHOT to less than 6e-8, with a mean absolute difference of 6e-9, and the comparison at an individual wavelength bin within the passband never exceeding 7e-7. This is well within the limits of the single-precision calculations performed within SYNPHOT and demonstrates the ability of pysynphot to produce accurate calculations of the fundamental bandpass parameters used to calibrate the photometry of WFC3 images.

2) Out of the thousands of comparisons of the integrated countrate produced by convolving a WFC3 passband with a source spectrum, only 4 individual cases yielded discrepancies between SYNPHOT and pysynphot of more than 1%. These 4 cases involved the use of two of WFC3’s narrowest filters - the FQ672N and FQ674N - and produced discrepancies of 1.3 and 1.4%, respectively. These particular discrepancies arise due to the ability of pysynphot to better represent the true shape of these narrow filter passbands when convolved with source spectra that are sparsely sampled over the width of the passbands. The mean absolute discrepancy of all of the integrated countrate comparisons for WFC3 is 3.5e-4 (0.035%), with a standard deviation of 0.0013 (0.13%). This demonstrates the ability of pysynphot to produce integrated counrate results - which are the primary results of all ETC imaging calculations - that not only match those of SYNPHOT, but in some cases are actually more accurate than SYNPHOT.

3) There were many test cases that resulted in discrepancies in individual wavelength bins when ratioing or differencing the SYNPHOT and pysynphot countrate spectra that result from the convolution of the WFC3 bandpass with a source spectrum of some type. The various discrepant results have different “cosmetic” features, depending on the particular source spectrum or WFC3 bandpass in use, but we have determined that they all result from differences in interpolation techniques used in syphot and pysynphot, which usually causes discrepancies in and around any high-frequency features in either the source spectrum or the instrument bandpass.

Specifically, failures consistently resulted from the following types of tests:

a) Some of the WFC3 narrow-band filters, like F657N and F673N, have throughput data that extends to extreme blue and red wavelengths that are well outside of the nominal passband of the filter. The throughput values at these wavelengths are naturally very small, ususally in the 1e-6 or smaller range. A few particular filters, such as those mentioned above, have throughput values at these extreme wavlengths that jump around between values of 1e-10 and 1e-6 from one wavelength bin to the next. These sharp jumps (factors of 10-10000) are handled differently between SYNPHOT and pysynphot, not only because of differences in interpolation techniques, but also because of the single-precision arithmetic used by SYNPHOT. While large discrepancies can result in individual wavelength bins of such modes, they have little to no effect on the overall photometric calculations because they contribute an insignificant amount to the total signal.

b) Nearly all test cases involving source spectra that have high-frequency features, such as those from the k93models, Bruzual, and HST standard star libraries, produced discrepancies in individual wavelength bins above the 1% threshold level, which are again due to the different ways in which the high-frequency spectral features are interpolated over by SYNPHOT and pysynphot. Also included in this category are the various types of background spectra utilized within the ETC to represent Zodiacal light, earthshine, and air glow. All of these spectra have single or multiple very bright and very narrow emission lines, which yield different results when interpolating onto a particular WFC3 wavelength set used for a given simulation. Again, while the SYNPHOT vs. pysynphot ratios or differences can be quite large (10-20%) within one or a few wavelength bins of such spectra, the overall effect on the integrated countrate for the whole simulation is insignificant.

All test cases involving smooth source spectra, such as blackbodies, power-laws, and flat continua (“unit” spectra) are not affected by the different interpolation techniques and never show any significant discrepancies within any wavelength bin of the entire spectrum.

c) Discrepancies were often found in the ratios of spectra produced for IR imaging and spectral modes that used source spectra that are not well sampled, or have abrupt changes in sampling, at IR wavelengths, such as Bruzual library. The different interpolation techniques used in SYNPHOT and pysynphot again result in different handling of these special cases, but always in the sense that pysynphot gives a more accurate or appropriate result. In test runs that took place later in the pysynphot commissioning the Bruzual spectra were replaced with equivalent Pickles library spectra, which have much better sampling in the WFC3/IR wavelength range.

4) All test cases involving the calculation of IR thermal background produced discrepancies that varied from about 1 to 5% over wavelength. The discrepancies were identical for all test cases. This was traced to a previously unknown error in the graph table used by SYNPHOT and pysynphot, which affected the two systems differently (as reported earlier). It was confirmed that pysynphot was producing the correct result.

When comparing the results of ETC calculations using SYNPHOT and pysynphot, the results were consistent with what was expected based on the differences between SYNPHOT and pysynphot outlined above. The integrated source rates for all UVIS imaging and spectral modes agreed to better than 0.5%. The only differences of note occured in the integrated thermal background signal for all IR test cases, due to the problem noted above. In all of these cases the thermal background signal constitutes a small enough fraction of the total source+background signal that the difference in the total integrated signal was less 0.5%.

Previous topic

Appendix: STIS testing

This Page