Deep Impact - Limitations of the HRI-IR Instrument Calibration By O. Groussin and K. Klaasen (April 7, 2006) 2006-11-30 DI:McLaughlin Revised for version 2.0 of the reduced HRI-IR PDS data set The calibration of the Deep Impact HRI-IR instrument data currently in the Planetary Data System (PDS) has several limitations that must be taken into account when analyzing the data. The current DI data sets included here represent the best understanding and implementation of the calibrations as of August 2006 Version 1.0 of the data were processed with earlier calibrations in November 2005 and submitted to the PDS in fulfillment of the requirement of having the data included in a permanent archive within 6 months of the encounter. Calibrations were improved during 2006 and version 2.0 of the reduced HRI-IR data set was delivered to PDS in December 2006. However, there are known shortcomings in the calibration process. The calibration applied in versions 1.0 and 2.0 of the PDS deliveries includes decompression (if necessary), linearization, dark current subtraction and absolute spectral-radiometric calibration. All of these aspects of the calibration have some limitations. In addition, there are some aspects of the data for which calibrations have not yet been adequately derived (flat field, scattered light, correction for bad pixels, and cosmic ray removal). We summarize here those limitations, and one should refer to Klaasen et al. 'Deep Impact Instrument Calibration' (2007) for a full discussion on the calibration process. Compression of the data typically decreases the pixel-to-pixel variations because one compressed DN (8 bits) usually represents a larger signal range than one un- or de-compressed DN (14 bits). This effect becomes important for cases where it is of interest to measure low levels of contrast at low signal levels. For example, if an uncompressed image of the coma has a mean signal of 100 DN (14-bit) above background and a pixel-to-pixel variation of 4 DN (14-bit), a compressed image of the same scene with LUT 0 or 1 will have a pixel-to-pixel variation of 0 DN after decompression - all contrast will be lost. However, when we have high signal like on the nucleus itself, this problem is less important since the signal is large, >1000 DN (14-bit), and so is its contrast, for example >40 DN (14-bit), compared to the size of a compressed DN (~20 DN14). In conclusion, one should be careful when interpreting low-signal-level data, <100 DN (14-bit), that have been compressed. Correction of the non-linear response function of HRI-IR is quite accurate and reliable in most cases. However, the correction is based on quadrant average response functions. A given pixel might have a somewhat different non-linearity; we have insufficient data to really know if this is true or not. We also have seen some variation in the quadrant average non-linearity functions over the span of the ground and inflight calibrations, although changes during the inflight cruise and encounter periods appear to have been minimal. We also see effects of using subframe modes on the response nonlinearity. While these effects have been taken into account to first order in our calibrations, not all of the second-order effects may be understood or characterized. The dark frame background level is strongly dependent on the exposure time, temperature of the instrument (bench, electronics, and detector), mode, and the recent history of detector resets and readouts. A careful analysis of the dark level was performed to correct for these effects, but the result is still only good to within +/- 10 DN in the best case. Correction may be significantly poorer than this, especially for the first frame in any set of HRI-IR readouts. The first frame has an elevated background level compared to subsequent frames, and the degree of elevation is a complex function of many variables. No correction for this is attempted in the calibration pipeline. While the residual dark level error is not normally an issue in the 2.0-4.8 micron wavelength range where there is a high sensitivity of the detector, it becomes a real issue at short (<2.0 microns) and long (>4.8 microns) wavelengths where the sensitivity of the detector drops rapidly. As a consequence, one should be extremely cautious when interpreting any data below 2.0 microns or above 4.8 microns. The absolute and relative spectral sensitivity calibrations are limited to the ~10% level. This uncertainty is even more pronounced below 1.5 micron, where there is clear evidence for beamsplitter effects that are less accurately corrected in the current version of the calibration. In addition, the effect of the anti-saturation filter introduces uncertainties in the radiometric calibration. This filter reduces the signal at longer wavelengths so that the thermal contribution of the nucleus does not saturate the detector. As a result, the sensitivity of the detector above 4.3 microns in the anti-saturation zone drops very rapidly to zero at 4.6 microns, and one should be very cautious when interpreting data in this region. Around 2.7 microns the transmission of the anti-saturation filter changes rapidly from ~90% to ~40%. Small residual uncertainties in the exact wavelength (and column) of this sharp transition sometimes lead to calibration anomalies (artificially high or low computed radiance), and one should be careful in any interpretation of the data in this particular wavelength region that fall behind the anti-saturation filter. Flat-field correction of HRI-IR data is not yet reliable. The bad-pixel criteria allow pixels having response rates up to 2x different than the mean response rates in their column to be considered 'good'. Without a flat-field correction, these variations remain uncalibrated. Work is proceeding on deriving flat-field corrections for pixel-to-pixel response rate differences, and a preliminary correction has been applied to the unbinned data only in the December 2005 PDS delivery. However, correcting for any spatial variations in optical throughput across the detector appears impossible with the calibration data we have. Such optical throughput variations appear to be no greater than about 5%. The HRI-IR exhibits scattered light effects that are not corrected in the calibration pipeline. Response at the 1-2% level was seen 10 pixels off the bright limb of the Moon. And a ghost image at the 3-4% level is seen about 35 slit widths away from the primary image in the cross-slit direction. Our bad pixel map criteria allow pixels to pass that have residuals to a linear fit of up to 2% rms and up to 15% maximum. We have observed some variation in the locations of bad pixels during flight; as of February 2006 (version 2.0), we are now using 4 different bad pixel maps for four flight phases. For the December 2005 PDS delivery (version 1.0), a time-independent bad pixel map with fewer bad pixels was used. There may be some uncharacterized bad pixels that have changed between calibrations. No bad-pixel reclamation has been performed on pipeline calibrated data to date. Detection and correction of cosmic ray signatures in HRI-IR frames is not yet reliable. Such signatures are best detected by differencing pairs of successive frames. But no pipeline process is currently being applied to detect or correct cosmic rays in individual frames.