ROSETTA VIRTIS-M CALIBRATION RELEASE 3.1 (31 January 2020) AUTHORS: G. FILACCHIONE (INAF) F. CAPACCIONI (INAF) M. C. DE SANCTIS (INAF) G. PICCIONI (INAF) R. POLITI (INAF) E. D'Aversa (INAF) APPROVED BY: A. CORADINI (INAF) 1. INTRODUCTION --------------- This document summarises the setups, methods and results of the ground calibration used to derive VIRTIS-M instrumental response. On this basis a calibration pipeline has been produced. The last chapter of this document describes all the necessary steps required to calibrate VIRTIS-M data. The VIRTIS instrument is composed of two independent channels which covers different scientific objectives: VIRTIS-M is a hyperspectral imager used to generate composition maps of the surface of the target, and VIRTIS-H a punctual high spectral resolution spectrometer used to derive detailed information on the coma composition. Furthermore, VIRTIS-M contains two detectors, a CCD to cover the VIS-NIR spectral range (denominated VIS channel) and a HgCdTe FPA to cover the 1-5 um range (denominated IR channel). See Coradini et al, 2007 for a full description of the instrument and its scientific objectives. A calibration campaign was performed in two steps: first at channel level (VIRTIS-M calibrated in the Italian premises of Galileo Avionica, GA, Florence, while VIRTIS-H calibrated at the Observatoire de Paris, Meudon) and later on, after assembly as the VIRTIS Flight Model including also the Digital Electronics, a further calibration was performed at instrument level, in the Jupiter thermo-vacuum chamber in IAS, Orsay. The VIRTIS-M calibration methods and results are described in Ammannito et al. (2006) Filacchione et al. (2006), Filacchione (2006), Filacchione et al (2009). The description of the Orsay experimental setups used to characterize the integrated model are discussed in Bonello et al. (2005). The objective of performing separate calibrations was to fully characterize, through specific setups, the performances of the single channels, while the final activity was devoted essentially to the optical co-alignments, radiometric calibration, verification of data handling and thermal stability. 2. The Calibration set-up in Galileo Avionica --------------------------------------------- During GA tests were realized customized opto-mechanical setups necessary to evaluate the geometrical, spectral, radiometric and internal performances. The basic setup for the calibrations consists of an optical bench over which are housed a collimator, a reference target placed at its focal plane and a folding mirror used to move the collimated beam in the instrumental FOV along the azimuthal and zenithal directions. As imaging spectrometers look at infinite distance is necessary to use a collimator to have a collimated reference beam. The GA-developed collimator uses an off-axis parabola (D=250 mm, F=1020 mm, off axis angle=8 deg) which guarantees an unobstructed beam, reduced aberrations and a high spatial scale. For VIRTIS-M the magnification ratio is equal to: MR F_spc/F_col * 152/1020 * 0.152 that is 1 mm on the collimator's focal plane corresponds to 0.152 mm, 4 pixels, on the spectrometer's detector. The collimator's focal plane is equipped with an holder able to sustain several interchangeable targets (pinholes, test slits, MTF masks, matrix of 5x5 microlamps); these elements are used for different kinds of calibrations. The collimated beam is folded towards the instrument thanks to a folding mirror placed over two OCS computer controlled, micrometric mounts able to aim it at steps of 1 mrad along the azimuthal (scan parallel to VIRTIS-M slit, along sample direction) and zenithal (scan perpendicular to the slit, along lines direction) angles. In order to reproduce the operative conditions of the satellite, the instrument is housed into a thermo-vacuum chamber. In these conditions, thanks to the internal cryo-cooler (operating on a Stirling cycle), it's possible to cold down the IR focal plane up to the operative temperature of about 70 K and the CCD at about 230 K. The collimator optical beam reaches the spectrometer's pupil thanks a CaF2 window housed in the front of the thermovacuum chamber. This window is characterized by an elevated optical transmittance in the 250-5100 nm spectral range. All opto-mechanical devices placed on the optical bench are controlled thanks to a dedicated software (OCS, Optical Control System), while VIRTIS-M is controlled thanks to a separate setup, consisting in the UT (Unit Tester) connected to the experiment through the Proximity Electronics Module (PEM). This system allows to send commands to the instrument, to start acquisitions only when all optical elements commanded by OCS are in the correct configuration and to receive back and record telemetries and scientific data. 3.SPECTRAL CALIBRATION ---------------------- The spectral calibration concerns a fundamental aspect of the functional requirements of an hyperspectral imaging spectrometer: the conversion of bands positions along the spectral axis of the detectors in wavelength units. These measurements are realized through the following steps: - characterization of the spectral performances of the monochromator to be used as a calibrated reference source; this preliminary check was performed on the emission features of a standard Hg pencil lamp; - use of the monochromator to scan in detail some spectral ranges and measure the corresponding instrumental spectral response; - fit of these spectral responses with gaussian curves to retrieve the band's parameters; - extension of these values to the remaining bands with a linear fit. From the spectral calibrations the following instrumental parameters must be deduced: - the sample central wavelength, C (nm), is the wavelength of the centroid of its spectral response function for the generic band; - the spectral range is the wavelength interval between the minimum and the maximum wavelength at which the instrument is sensitive; - the spectral sampling interval, SSI (nm), is the difference between the sample central wavelengths of two adjacent bands. As VIRTIS-M uses an holographic grating to disperse the light diffracted by the slit along the spectral direction we should expect a linear relation between the spectral positions (b, band) and the corresponding wavelengths C (nm): l_c(b)=l_0+SSI(b) - b Taking into account the general properties of the optical design we can assume that SSI(b)=SSI and that it is constant on the entire spectral range and it does not depend on the spectral position. In order to retrieve these parameters, the following measurement strategy was applied: the rough spectral range is divided in three regions over which evaluate the spectral response; for each region VIRTIS-M acquires the signal coming from the monochromator. For the VIS regions near 400, 550 and 1000 nm are chosen; for the IR 1010, 3000 and 5000 nm. These scans are realized at spectral steps less than the instrumental spectral resolution in order to sample the signal with the maximum resolution (0.4 nm for the VIS and 1 nm for the IR). In the following prospect are reported the experimental parameters of the six acquisitions made to evaluate the visible and infrared spectral calibrations. - Spectral scan range (VIS) 394.0-406.4 nm Monochromator's source: QTH lamp Number of steps: 31 Step: 0.4 nm Useful range along samples: 100 < s < 154 Useful range along bands: 93 < b < 96 - Spectral scan range (VIS) 544.0-556.4 nm Monochromator's source: QTH lamp Number of steps: 31 Step: 0.4 nm Useful range along samples: 100 < s < 154 Useful range along bands: 172 < b < 176 - Spectral scan range (VIS) 994.0-1006.4 nm Monochromator's source: QTH lamp Number of steps: 31 Step: 0.4 nm Useful range along samples: 100 < s < 154 Useful range along bands: 412 < b < 415 - Spectral scan range (IR) 970.0-1032.0 nm Monochromator's source: QTH lamp Number of steps: 31 Step: 2.0 nm Useful range along samples: 108 < s < 166 Useful range along bands: 1 < b < 3 - Spectral scan range (IR) 2970.0-3032.0 nm Monochromator's source: SiC element Number of steps: 31 Step: 2.0 nm Useful range along samples: 108 < s < 166 Useful range along bands: 211 < b < 215 - Spectral scan range (IR) 4920.0-5050.0 nm Monochromator's source: SiC element Number of steps: 31 Step: 2.0 nm Useful range along samples: 108 < s < 164 Useful range along bands: 418 < b < 428 Over the pixel illuminated during the scan we observe a signal that is equal to the convolution of the pixel and monochromator-setup spectral responses. In general the shape of this signal can be modeled with a gaussian curve. The signal corresponding at each sample illuminated by the monochromator scan (samples 100-154 in the VIS, 108-166 in the IR) is spectrally modelled thanks to a six-component gaussian fit from which we estimate the spectral baricenter of the pixel. As the monochromator's slit is not uniformly illuminated we use the mean signal along samples; to deduce a spectral calibration table reliable on the whole focal plane (independently from the sample position along the spatial axis) the results of the independent fits along the slit, e.g. at a fixed sample, are mean together. The slopes of these lines correspond to the instrumental spectral sampling interval SSI while the intercept to the starting wavelength, . During the on-ground calibrations the entire focal planes are acquired (438 bands x 270 samples) while in onflight conditions the useful region is reduced to a 432 bands x 256 samples "window". After taking into account this reduction we obtain the final spectral relationship between wavelengths and bands for the two channels (in nm) in flight conditions: Visible channel: l_c(b)=231.296+1.884-b Infrared channel: lc(b)=999.498+9.448-b Where b is the band number. From the above formulas it results that the SSI of the VIS channel is equal to 1.884 nm/band while for the IR channel is 9.448 nm/band. The VIS spectral range runs from 231.296 to 1043.09 nm while for the IR is comprised between 999.498 and 5071.5 nm. In the 999.498 - 1043.09 nm region the two channels are spectrally overlapping. These results are evaluated in high-resolution mode (no binning); for different instrumental modes is necessary to interpolate these values according to the binning value along the spectral direction. Little variations of the spectrometer's temperature can introduce shifts in the spectral response; however the calibration pipeline described in chapter 7 does not take into account this effect for the time being. The spectral calibration files generated are contained in the files - VIRTIS_M_HRES_SPECAL_10_V1.TAB - VIRTIS_M_NRES_SPECAL_10_V1.TAB located in the Calibration Directory. 3.1IR Grating contamination effect on VIS channel -------------------------------------------------- The grating is the heart of the VIRTIS instrument; on one hand its clever design allowed us to design a relatively compact instrument, more than using dichroic filters for instance; on the other hand the same clever design should have imposed additional constraints and more stringent tolerances, which were not accurately taken into account in the early phases of development of the VIRTIS instrument. One issue is related to the signal impinging on the VIS detector generated by visible light diffracted by the IR grating. The pupil of the telescope is projected on the grating, which is then fully illuminated by all the light entering the spectrometer through the slit. The central part of the grating contains grooves to diffract the VIS-NIR range while the external ring has blazed grooves optimised for the IR range. After diffraction the negative first order of the VIS portion of the grating is focussed on the CCD, while the positive first order of the IR grating is focussed on the MCT IR detector. Need not to say that the positive orders of the VIS portion of the grating are focussed on the IR detector and viceversa. But the CCD is not sensitive to the wavelengths of the IR channel and the MCT detector is not sensitive to the wavelengths of the VIS channel. So this effect creates no harm. However, VIS light impinging on the IR portion of the grating is, indeed, diffracted by the grating and consequently the negative orders are then focussed on the VIS detector. The net result is that the VIS signal is contaminated by another spectral source, which uses the same VIS signal but dispersed differently over the CCD detector. The effect has been calibrated on ground and we are able to quantify the percentage amount of contamination at each wavelength. The IR grating, depending on grooves density and diffraction order number, disperses visible-wavelength light differently than the VIS grating. This yields a tangled contamination dependent on the intensity of light at wavelength different by the nominal ones. Without IR grating contamination, the visible signal is only obtained from diffraction order -1 of VIS grating, and can be written as: DN(b)=S(b)G_(-1)^(VIS)(b) (x) ILD_b where S(b) represents the signal impinging on the grating at a wavelength nominally assigned to the pixel b of the VIS detector, G_(-1)^VIS represents the VIS grating efficiency at order -1 and ILS_b is the instrumental line shape of pixel b. The total signal measured in the presence of IR grating contamination becomes: DN_TOT (b)=DN(b)+H(b) where H is the sum of signals coming from different IR diffraction orders: H(b)=SUM_(i=-2,-8)S(b+e_i ) G_i^IR (b+e_i)(x)ILS_b Here e_i represents an order-dependent spectral shift due to the changing dispersive rate and G_i^IR the efficiencies of IR grating at order i, which have been measured during ground calibration. In a small perturbation assumption, we can recover the "true" signal by using the same measured VIS spectrum, writing at the first order: DN(b)=DN_TOT (b)-SUM_(i=-2,-8)DN_TOT (b+e_i ) G_i^IR (b+e_i )(x)ILS_b This equation can be applied independently to each measured spectrum. We must point out that due the dispersion properties of the two gratings the -1 VIS and the -5IR have the same dispersion and in this case the order-dependent spectral shift is equal to 0 (the two orders are effectively superimposed on the VIS detector). This makes it impossible to remove the contribution of the -5IR. Since the correction is additive, its relevance is not uniform over the dataset (it is larger for low signal levels). For this reason, we cannot provide simple correction factors effective for the whole dataset. However, we includes instead the IR order efficiencies G_i^IR and the associated shifts (the reported e_i) as a further calibration files located in the calibration directory. The files are: - VIRTIS_M_VIS_DIFF_OFF.TAB: Diffraction offset correction for VIRTIS-M visible channel; - VIRTIS_M_VIS_SL_DIFR_V1.DAT: Stray light diffraction correction for VIRTIS-M visible channel - VIRTIS_M_VIS_DIFF_OFF.TAB: Diffraction offset correction for VIRTIS-M visible channel; - VIRTIS_M_VIS_SL_DIFR_V1.DAT: Stray light diffraction correction for VIRTIS-M visible channel 4. GEOMETRIC CALIBRATION ------------------------ The geometric calibrations were realized through different spatially definite targets placed at the collimator focal plane; several scans over slits (oriented parallel or orthogonal to the VIRTIS-M slit) or fixed patterns of micro lamps were acquired to evaluate the instrumental geometrical performances. 4.1 Pixel and slit functions ---------------------------- The Pixel Function, PIXELF(s), is given by the convolution () of a unitary step function, V(s), representing the real pixel, and the Instrument response along the sample s direction, INST(s): PIXELF (s)=V(s) (x) INST(s) The Slit Function, SLITF(l), is given by the convolution (x) of a unitary step function, U(l), representing the real slit, and the Telescope response function along the line l direction, TEL(l): SLITF(l)=U(l) (x) TEL(l) The Spatial Width along slit, , is the full width at half maximum of the Pixel Function while the Spatial Width across slit, , corresponds to the full width at half maximum of the slit function. According to the instrumental requirements the IFOV must be equal to 250 rad while the two spatial widths must be less than 375 rad. These quantities are measured in three positions: boresight (s=l=128), position F (s=38, l=218) and position G (s=218, l=38). The experimental setup realized to made these measurements consists in a test slit (0.1 3 mm) placed at the collimator focal plane and illuminated by a Hg pencil lamp. The test slit is oriented orthogonal respect to VIRTIS's slit for the Pixel Function measurement and parallel for the Slit Function. In the first case the test slit image is translated along the s direction thanks to the azimuthal movement of the folding mirror; in the second case is translated along the l direction with the zenithal movement. Both scans are repeated and centered over boresight and at F-G positions; each scan consisted in 80 steps with a step of 32 rad. For each step VIRTIS-M acquires the signal coming from the collimator. The Hg lamp emission features are used as references to estimate the Pixel and Slit functions of the VIS (at bands 76, 77, 97, 98, 113, 114, 172, 173, 188, 189, 190) and IR (at bands 2, 14, 38, 49, 56, 74) channels. At these wavelengths the Slit and Pixel Functions, together with the corresponding spatial widths at half maximum and , are estimated with a gaussian fit. In general, the VIS channel seems to be correctly aligned and focused (the spatial widths are always near to 375 rad; the IR channel shows higher spatial widths up to 897 rad. This effect is probably caused by residual astigmatism which was not completely compensated during the spectrometer alignment. These measurements are intensively repeated during the instrument's assembling to align the telescope to the spectrometer and to reach the position of best focusing. This activity is complicated by the fact that the instrument is assembled and adjusted at ambient temperature while measurements are made in cryogenic conditions. The optical system's aberrations introduce some deviations in the almost-gaussian spatial response: especially at extreme IR wavelengths the pixel function can assume a double peak. In these cases, the monochromatic images are affected by a reduced spatial quality. 4.2 Spatial misregistration --------------------------- An overall view of the instrumental imaging capabilities is given by the scan over a pattern of equidistanced micro lamps (tungsten filament) spaced over a 5 x 5 matrix with inter-distances of 1.5 cm along rows and columns. The 25 tungsten micro lamps are placed at different quotes to reach individually the focus of the off-axis parabolic mirror of the collimator: this configuration was customized by GA to reduce optical aberrations induced by the collimator. This target, with the micro lamps switched on, is placed at the collimator focal plane and acquired by moving the VIRTIS-M scan mirror. The resulting standard data cube (s=l=256), corrected for background and dark current allows to study the overall geometrical performances and image quality over a representative number of points of the FOV. From these measurements, it was possible to identify the presence of a spectral tilt affecting the performances of the VIS channel: RGB spot images of the lamps are displaced at different positions along the sample (slit) direction. This effect is introduced by a not perfect parallelism between VIS and IR grooves directions on the diffraction grating. According to the instrument optical design the two patterns on the grating should be parallel between them and aligned with the slit's axis and focal planes sides. Unfortunately, when the tolerance levels of the grating precision were defined, this effect was not fully taken into account. During the assembly of the instrument was impossible to reduce the effect and was decided to minimize it on the IR channel and consequently to maximize it on the VIS. Despite this effect seems to be particularly deleterious it is possible to model it and then remove it during post-processing. The strategy adopted consisted in the evaluation of the displacement of each monochromatic image (at band b) respect to the first (b=0) chosen as reference. After the subtraction of the background, for each monochromatic image were evaluated the 25 lamps' spatial barycenters (at positions: sample, line) through a bi-dimensional gaussian fit. The procedure, repeated for each spectral band, allows to estimate the spectral variation of the spatial positions of each lamp. Thanks to a linear fit applied on the regular micro lamps positions (where the previous fit gives consistent results) it's possible to calculate the spectral tilt effect. In the IR some difficulties arose for the barycenter calculus: the bi-dimensional gaussian fit in fact is unable to correctly estimate the micro lamps positions for b>350; this happens because at these wavelengths the signal is strongly affected by the thermal emission of the target plate. In the next relations are indicated the linear fit results over the 25 lamps positions: s_0, l_0 are the spatial baricentres at band=0, alpha_s and alpha_l are the angular coefficients of the fitted line and Ds, Dl the maximum shifts between the spots positions over the two extreme bands (band=431 respect to band=0). For the VIS channel these shifts reach a value of: Ds=432 tan(a_s)=8.01 +- 0.17 Dl=432 tan(a_l)=1.01 +- 0.41 pixels along the whole spectral range. For the IR channel we obtain: Ds=432 tan(a_s)=0.72 +- 0.23 Dl=432 tan(a_l)=1.32 +- 0.48 While the tilt effect is not noticeable on the IR channel, it assumes a value of about 8 pixels in the VIS range along the slit direction. The cause of the tilt has been traced back to the rotation of the grating around the optical axis of the instrument, which determines the single monochromatic images of the slit to be shifted along the samples direction. This is called a parallelogram distortion, where the single monochromatic images of the slit are kept aligned with the CCD columns. This is usually seen in any monochromator when the grooves of the grating are not aligned with the entrance slit. In this case the spatial information is not mixed with the spectral one and moreover the vertical shift is linear with the wavelength. This implies that once the tilt effect is removed VIRTIS-M does not have any residual spectral contamination between two adjacent bands. The root cause of the tilt is related to the instrument design, which uses a single grating with two concentric regions; the inner one devoted to the VIS channel while the grooves of the external ring are optimized for the IR channel. Unfortunately, when the tolerance levels of the grating design were defined, this effect was not fully taken into account, and the parallelism between the grooves of the IR and VIS gratings was not properly constrained. During the assembly of the instrument was impossible to reduce the effect and was decided to minimize it on the IR channel and consequently any residual tilt was allocated to the VIS. As said above, this instrumental effect can be corrected fairly easily as the effect is linear along the bands, being null on band=0 and equal to 8 pixels on band b=431. Incidentally this corresponds to a rotation of less than 2% over the full range and the effect is negligible over a limited spectral range, making spectral comparisons on a local scale very reliable, although from an image point of view this can be annoying when using RGB with highly separated spectral components. The correction is done in the calibration pipeline by applying the detilt algorithm described below. 4.2.1 Detilt algorithm for the VIS channel ------------------------------------------ The detilt algorithm is based on the following operations: 1) resampling of the raw signal on the frame from the original (bands, sample)=(432,256) format to a "superframe" of (bands, sample)=(432, 20736) pixels. Along the spatial axis an oversampling by a factor 80 is applied, corresponding to 10 times the maximum measured tilt of 8 pixels. 2) for all columns placed at band=b the signal is shifted by a fraction of pixel Ds multiplied by a factor 80. 3) resampling of the frame to the original (bands, sample)=(432,256) format. Those operations are performed by the following IDL code (Filacchione, 2006), which transform the original datacube in a detilted cube: superframe=fltarr(432,20736) superframe_detilt=fltarr(432,20480) detilt_cube=fltarr(432,256,datalines) for l=0,datalines-1 do begin for s=0,255 do begin for ss=0,79 do begin superframe(*,s*80+ss)=reform(cubo(0:431,s,l)) endfor endfor for s=0,255 do begin for b=0,431 do begin bsh=b for ss=0,79 do begin superframe_detilt(b,s*79+ss)=superframe(b,s*79+ss+bsh) endfor endfor endfor for s=0,255 do begin for b=0,431 do begin tot=fltarr(432) for ss=0,79 do begin tot(b)=superframe_detilt(b,s*80+ss)+tot(b) endfor detilt_cube(b,s,l)=tot(b)/80.0 endfor endfor endfor where datalines are the total number of lines of the datacube. 5. SPATIAL CALIBRATION: FLAT-FIELD ---------------------------------- Flat-field matrices are commonly used to normalize the response of the pixels over the focal plane. This technique, first used to improve cameras' images, is applied with some peculiarities to imaging spectrometers too. In this case, in fact, we need to calculate for each spectral band the relative variations in response of the pixels placed along the slit's axis. Traditionally this can be obtained by observing a spatially homogeneous, or flat, source as a lambertian diffuser illuminated by a diffuse light. For VIRTIS-M we decided to use a more reliable system in order to completely eliminate possible spatial disuniformities due to the reference target surface or to the illuminating system. As there were some differences in the procedures used for the VIS and IR channel we prefer to separate these treatments in the two following paragraphs. 5.1 Flat-field: visible channel ------------------------------- The experimental setup used to evaluate the VIS flat-field consisted in a lambertian target painted with BaSO coating, placed at the collimator focal plane and illuminated by a photometric stabilized QTH lamp. Lamp's light is defocused on the target by a system of lenses to destroy the image of the filament. In this way the target shows a spatially uniform illumination. By using the azimuthal movement of the folding mirror at step of 250 um (one pixel) it was possible to move the image of the target along the instrumental slit's direction. In this way each spatial pixel acquire the signal coming from exactly the same portion of the target. The stability of the incoming flux is guaranteed by the photometrically stabilized lamp's alimentation. As the whole slit is illuminated by the target, this method can be applied at each point of it; in other words, we have 256 independent flat-field matrices acquired on consecutive points on the target. The mean of these flats is used as the flat-field matrix after having normalized the signal of each band respect to the slit's center (sample s*=127): FLAT_Field(s,b)=DN(b,s)/DN(bs^*) The VIRTIS-M-VIS flat-field is equal to 1 along the slit's center direction: at each band the matrix contains the relative variation of the pixel at sample s respect to s^*=127. Flat's useful values ranges between 0 and 1.122 with a mean value 0.947 and a standard deviation 0.109. Negative values are generally obtained on defective pixels. The pattern of vertical features (around b=160, 270, 300, 370) symmetric respect to the slit's center are characteristic of the Shafer-Offner optical design: a similar effect in fact is visible on the VIMS-V flat-field (Coradini et al. 2004). Defective pixels (dark pixels), as well donuts, like the circular structure near (b=80, s=220), can be identified on the flat matrix. As the slit is uniformly illuminated by a spatially homogeneous target the flat field matrix is not influenced by the spectral tilt effect. 5.2 Flat-field: infrared channel -------------------------------- Despite the IR flat field evaluation follows the same strategy used for the VIS, its retrieval is not so immediate: in this case, in fact, the QTH lamp guarantees a sufficient signal only up to b=179 (=2690.66 nm). In this first spectral range is therefore used the same method discussed for the VIS channel. The flat matrix in the 180330 is in saturation. The signal near b=180 is always very low and for this reason the resulting IR flat is affected here by greater uncertainties. Like the VIS, also the IR flat shows the vertical structures; defective pixels assume generally negative values while several donuts are distributed across the focal plane. 6. RADIOMETRIC CALIBRATION -------------------------- 6.1 Visible channel responsivity -------------------------------- The evaluation of the responsivity of the VIS channel opens a series of still not completely resolved problems discovered during the calibration campaign. The basic strategy adopted to estimate the radiometric calibration is to make a spectral scan from 360 to 1060 nm at steps of 2 nm by using a monochromator; a 150W QTH lamp is used as source at the monochromator input slit. VIRTIS-M acquires the signal coming from a diffusive target placed at the collimator focal plane and illuminated by the monochromator's output slit. The target is uniformly illuminated thanks to a system of mirrors, acting as a collimator, placed between it and the monochromator. VIRTIS-M collect a frame at each spectral step of the monochromator. Every 80 spectral steps of the monochromator (40 nm) the folding mirror is oriented from VIRTIS-M boresight position to a calibrated radiometer boresight to measure the input radiance. This reference radiometer has a photodiode as detector; the signal is acquired through a lock-in amplifier by choppering the input radiance to reduce noise contribution. In principle, we should expect from these measurements a peaked signal (about 2 nm wide and covering the whole slit) placed on the CCD and moving from 360 to 1060 nm during the monochromator's scan. In reality, the measured signal is more complex, characterized by several peaks placed at different spectral positions respect to the input signal. These features are generated by the IR portion of the diffraction grating: this effect completely invalidate the measurement strategy because the incoming flux is dispersed in several orders and the measurement made with the radiometer don't corresponds with the radiance measured on the order -1 VIS with VIRTIS-M. For this reason, we prefer to retrieve a more effective and reliable responsivity by using a completely different approach in which the instrument is used with a panchromatic source: the retrieval of the VIS responsivity is made through the flat field data. By this way the instrument collects in the same time the convolution of all orders, reproducing a behavior more similar to the operative functioning. Unfortunately, this method allows only a relative estimate of the responsivity function: in fact during the calibration campaign it was not possible to measure the effective input radiance during these acquisitions: these measurements were made successively thanks to a Field-Spec spectrometer used as spectral radiometer with a 1 deg FOV foreoptics. As radiometer and VIRTIS have different FOVs, the resulting input radiance FS(b) is affected by great uncertainty introduced by the solid angles subtended by the two optics. On the other hand, this uncertainty correspond to a fixed multiplicative value to be applied on the whole spectral range. Another problem encountered with these data was the low signal below 450 nm. In this spectral range in fact the radiance emitted by the QTH lamp illuminating the BaSO target was not sufficiently high to guarantee a good SNR. Below 450 nm the instrumental response can be estimated only through mathematical models. Despite these limitations the resulting responsivity, evaluated at slit's center (sample =127), seems to produce good quality, relative spectra. The responsivity R, expressed in is evaluated as: R_vis(b,s^*) = DN(b,s^*)/(FS(b) t_exp) Where DN(b,s^*) are the raw DN, FS(b) is the input spectral radiance, in W/(m^2 um sterad) measured thanks to the Field-Spec spectrometer and the exposure time (in s). The conversion in absolute or physical units () of this function was realized by using the observations of the Moon (15-16NOV2005). A photometric model of the Moon is used to estimate the spectral reflectance of a well-defined region of the Moon (Kepler crater) observed by VIRTIS-M. This signal is at the moment used to retrieve the in-flight updated responsivity R. The extension of the responsivity to the whole focal plane is possible thanks to the method explained in (Coradini et al., 2004). It basically uses the assumption that the flat-field frame is normalized respect to the same sample ( =127) used to estimate the responsivity: ITF_vis=Flat_Field_VIS(b,s) R_vis(b,s^*) 6.2 Infrared channel responsivity --------------------------------- The infrared channel responsivity is evaluated by using reference cold blackbodies at different temperatures (350 to 690 K) as radiometric sources. These measurements were realized during the calibration campaign at IAS, Orsay, with different exposure times . Planck's law obliged us to use a limited spectral region of each acquisition, limited on the short wavelengths side by dark while on the long wavelengths side by saturation. Experimental constraints don't allow to have enough signal for b < 28 (lambda < 1264 nm): in this spectral range the responsivity is however linear and is estimated thanks to a linear fit. The blackbody radiance, BB(b) (in W m^-2 um^-1 sterad ^-1) falling on a spot of about 20 samples at the slit's center, is used to retrieve the instrumental responsivity (DN m^2 um sterad)/(W s): R_IR(b,s^*)=DN(b,s^*)/(BB(b) T_ext) The mean value is considered on the spectral regions were results coming from two or more measurements are available. As discussed for the VIS channel, the is calculated for the whole frame thanks to the following expression: ITF_IR(b,s) t(b) Flat_field_IR(b,s) R_IR(b,s^*) where t(b) is the optical bench transmission, measured thanks to a specific experimental setup. 7. INTERNAL CALIBRATION ----------------------- Instrumental performances can be checked during in-flight conditions thanks to the internal calibration sequence. VIRTIS-M, in fact, can acquire reference signals thanks to the combined use of cover, shutter and VIS and IR lamps (Melchiorri et al., 2003). These lamps, housed on the side of the telescope illuminate the internal side of the external cover. The cover is placed near the entrance pupil of the instrument to minimize optical aberrations. The window of each lamp contains a transparent filter (holmium for the VIS, polystyrene for the IR) to introduce some well-shaped spectral absorption features on the overall spectrum. The signal coming from the two lamps can be used to: - check the in-flight stability of the instrumental spectral response; - check the in-flight stability of the flat-field; - monitor the evolution of defective pixels (number and distribution); - perform a check on the relative radiometric response of the instrument. The internal calibration mode, implemented in the VIRTIS-M on-board software, consist in the acquisition of a sequence of 35 frames: 5 electronic offsets, 5 backgrounds, 5 dark currents, 5 acquisitions of the IR lamp, 5 acquisitions of the VIS lamp, 5 dark currents and 5 backgrounds. The repetition of this sequence at each switch-on is fundamental to follow the instrumental temporal evolution and to monitor the overall performances in operative conditions. 8. HOW TO CALIBRATE VIRTIS-M IN-FLIGHT DATA ------------------------------------------- From the RSOC (Rosetta Science Operation Center) the VIRTIS team receives data and telemetry packets from the satellite. These packets are processed in the PI institution (INAF-IAPS, Rome, Italy) with a proprietary GSE (Ground Support Equipment), converted in standard PDS (Planetary Data System) format and RAW data, expressed in Digital Number (DN), are generated. The actual calibration pipeline consists of a number of IDL routines, developed at IAPS that, starting from the RAW data, and independently for the VIS and IR channels, accomplish the tasks of - removing all the known instrumental effects and - converting the DN in physical units. The pipeline thus generates CAL data expressed in Spectral Radiance. The steps are: 1. Remove Negative and Saturated pixels 2. Dark Subtraction 3. Detilt (only for VIS channel) 4. IR Grating contamination (only for VIS channel) 5. Removal of the Odd-Even effect on IR FPA (only for IR channel) 6. Removal of Artefacts 7. Flat-Field and Radiometric Calibration And will be described briefly hereafter. 8.1 Saturated Pixels ------------------- The first step of the data calibration is the check for the saturated pixel. We mark as saturated pixel each pixel resulting true to the following condition for IR: DN_Pixel_IR+DARK_pixel_IR >18000 DN And for VIS: DN_Pixel_VIS+DARK_pixel_VIS >32000 DN The saturated pixel is flagged by the value -1000. 8.2 Dark Subtraction -------------------- The Dark current represents a measure of the photoelectrons generated in the detector when no external photon enters the instrument. Dark current is measured using the shutter located behind the spectrometer slit, which, when closed prevents any external light to enter the spectrometer. While for the VIS channel, this is strictly true and the dark signal in DN remain fairly constant for each data cube, for the IR channel, the spectrometer thermal background constitute a source of signal which is summed to the pure dark signal. In reality then the IR Dark acquisition is the measurement of Dark (photoelectrons generated within the detector) plus Background (photoelectrons produced by the thermal photons emitted by the spectrometer envelope within the FOV of the IR detector). The background signal can change during an acquisition according to the spectrometer temperature variations (we remind that a 256 lines cube is acquired in a minimum time of 21 min). This requires different strategies for the removal of the Dark acquisition on the VIS and IR channels. For the VIS channel this is done in a straightforward manner. Dark is acquired every n science frames where n is given by the parameter DARK_ACQUISITION_RATE. The i-acquired dark is subtracted to each science frame between the i and the i+1 dark acquisition. On the contrary, for the IR channels all the m (Dark+Background) acquisitions in a cube (their number being determined by the frequency of their acquisitions and the commanded number of lines) are linearly interpolated in time; this allows to use the correct (Dark+Background) calculated for each frame acquisition and properly take into account the temporal variability of the Background contribution. 8.3 Detilt for the VIS channel ------------------------------ As said above this step in the pipeline applies solely to the VIS channel; the algorithm presented in chapter 4.2.1 is used to remove the instrumental effect. 8.4 IR grating contamination on VIS Channel ------------------------------------------- Also this step is applied only to the VIS channel. The algorithm follows what is described in chapter 3.1; the algorithm has been defined and applied to the data in the framework of the Data Archive Enhancement activity (ESA contract No. 4000118962/16/ES/JD). 8.5 Artefact Removal -------------------- The VIRTIS raw spectra show a consistent pattern independently of the object observed. This is an indication that the measured spectrum is affected by systematic artifacts superimposed on the real spectral features, preventing their clear detection. In particular, two types of artifacts can be distinguished: 1) artifacts linearly dependent on the signal; 2) Indentation between odd and even spectral bands, not correlated with the signal. The Odd-even removal is described in chapter 8.6, We developed an in-flight calibration refinement in order to minimize the effects of the linear artefacts, as follows: We consider an average spectrum of the comet nucleus signal obtained from ~2.7 million spectra acquired during the first mapping phase of the Rosetta mission in August-September 2014, from the northern hemisphere and equatorial regions of the comet, and an average spectrum of the Lutetia asteroid, for each spatial sample (s) of the detector. We excluded from the average the spatial pixels corresponding to shadowed surface parts. Spectra are processed sample by sample to trace the variability of the artifacts across the focal plane. The ratio between 67P and Lutetia spectra allows removal of spectral artifacts while keeping information of the real features. Assuming the spectrum of Lutetia is devoid of small real features (namely involving few spectral bands), we model it with a polynomial interpolation representing the absolute reference, which is then used to produce an artifact-removed (AR) spectrum of 67P (see Eq. 1): (1) (I/F (l,s)_67P)/(I/F (l,s)_Lutetia ) I/F (l,s)_(interp@Lutetia)= I/F (l,s)_(AR_67P) Here I/F is the radiance factor (pi x bidirectional reflectance) Then we calculated the average I/F and the standard deviation of the normalized I/F along the samples (s) to obtain respectively the artifact-removed (AR) spectrum and the uncertainties for each band (l). As a further check we performed the ratio between two spectra of the internal calibration lamp: one acquired during the 67P mapping phase and the other during the Lutetia flyby. Since no significant variations were found between the two acquisitions, we conclude that the detector has not experienced any aging or contamination effects between Lutetia observations (used as a reference for the calibration) and the 67P encounter (for more information see Raponi et al, 2020). In practice the removal of these linear artifacts is performed by simply multiplying the image cube obtained after the previous steps by the matrices named: - virtis_m_ir_median_artifact_v2.dat - virtis_m_vis_median_artifact_v1.dat 8.6 Odd-Even Effect (Only IR Channel) ------------------------------------- For the IR Channel after dark removal the subsequent step of the calibration process consists in correcting the odd-even effect due to different responses of even and odd pixels of the detector. We separate odd and even pixels: we interpolate the 216 even pixels responses over a 432 points vector. The same it is done for the odd channels. The corrected DN adopted is the mean of these two vectors. 8.7 Flat Field and Radiometric Calibration ------------------------------------------ When the pipeline has reached this level of elaboration both datasets, IR and VIS, are corrected for any known or assessable instrumental effects (see conclusion chapter). The counts stored in the PDS cube can be converted in physical units of spectral radiance Rad (Wm^(-2) um^(-1) sterad^(-1)) by using the following formulas: Rad(l(b),s,l)_VIS=(DN(l(b),s,l)_VIS)/(t_(exp_VIS ) ITF(l(b),s)_VIS ) Rad(l(b),s,l)_IR=(DN(l(b),s,l)_IR)/(t_(exp_IR ) ITF(l(b),s)_IR ) where: - Rad(l(b),s,l) is the cube calibrated in spectral radiance which have the same dimensions (l(b),s,l) of the raw cube; - l(b) is the wavelength associated to band b according to spectral calibration tables (files VIRTIS_M_VIS_RESP_10_V1.DAT and VIRTIS_M_IR_RESP_10_V1.DAT) of VIS and IR channels; - s,l corresponds to sample and line location of the pixel in the original cube; - t_exp is the integration time of the observations (in seconds) as indicated in the PDS header of the file for VIS and IR channels; - ITF(l(b),s) are the responsivity matrix for VIS and IR channels (files VIRTIS_M_VIS_RESP_10_V1.DAT and VIRTIS_M_IR_RESP_10_V1.DAT). This calculus can be applied to high resolution acquisitions (432 bands times 256 samples); in nominal modes, where spatial and/or spectral resolutions are reduced, it is necessary to interpolate both spectral tables and responsivity matrices according to binning values. In the CALIB directory are saved the following calibration files: - VIRTIS_M_VIS_RESP_10_V1.DAT: 432x256 double precision matrix (binary) containing the VIRTIS-M-VIS Instrumental Transfer Function, including the VIS flat-Field. - VIRTIS_M_IR_RESP_10_V1.DAT: 432x256 double precision matrix (binary) containing the VIRTIS-M-IR Instrumental Transfer Function, including the IR flat-Field. - VIRTIS_M_HRES_SPECAL_10_V1.TAB: 432 row ASCII table containing the wavelengths of the VIS and IR channels in High Resolution Mode. These files must be used for cubes collected in High Resolution Mode. Cubes in Nominal Mode (x3 binning along bands) can be calibrated by using the following spectral calibration table: - VIRTIS_M_NRES_SPECAL_10_V1.TAB: 144 row ASCII table containing the wavelengths of the VIS and IR channels in Nominal Resolution Mode. 9. Conclusion and future improvements ------------------------------------- VIRTIS-M data included in this release can be calibrated by using the described pipeline. We are aware that further improvements can be implemented in particular for: - Spectral Range variability. Thermal gradients within the instruments introduce shifts of the spectral range. This has been verified in both the VIS and IR ranges, where shift of 5 to 10 nm have been measured. However, the performances during the primary mission at the comets were considerably uniform. For this reason this effect has not been calibrated out yet. - Residual contamination IR Vs VIS. As described in chapter 3.1, the IR grating diffracts VIS light on the CCD detector and this causes a source of additional unwanted signal in the VIS range. This has been removed using the approach described, however a second order effect is due to the non-parallelism between the VIS and IR grating grooves. In other words, when projected on the CCD detector, the diffracted light from the VIS grating is tilted (as we have seen), but the IR diffracted VIS light is not. When applying the detilting algorithm we then introduce a tilt in the IR diffracted image. As the tilt correspond to 8 bands over 432 bands, the effect is negligible over a limited spectral range, and apparent only when observing areas with sudden variation of the illumination conditions (e.g., light and dark interfaces). For the time being this is considered a second order effect and not taken into account. 10. REFERENCES -------------- - Coradini, A. et al, 2007. Virtis: An Imaging Spectrometer for the Rosetta Mission. Spa. Sci. Rev., 128, pp.529-559. - Filacchione G. et al., 2006. On-ground characterization of Rosetta/VIRTIS-M. II. Spatial and radiometric calibrations. Rev.Sci.Instr., 77, pp.103106-103106-9. DOI 10.1063/1.2360786 - Ammannito E. et al., 2006, On-ground characterization of Rosetta/VIRTIS-M. I. Spectral and geometrical calibrations. Rev.Sci.Instr., 77, pp.103109-10319-10. DOI 10. 10.1063/1.2349308 - Bonello, G. et al., 2005. The ground calibration setup of OMEGA and VIRTIS experiments: description and performances. Pla. Spa. Sci., 53, pp. 711-728. DOI 10.1016/j.pss.2005.02.002. - Filacchione, G. et al., 2009. Calibration pipeline of VIS-NIR imaging spectrometers for planetary exploration: the Rosetta VIRTIS-M case. Published in "WHISPERS '09: First Workshop on Hyperspectral Image and Signal Processing: Evolution in Remote Sensing". DOI:10.1109/WHISPERS.2009.5289050 - Melchiorri, R. et al., 2003. VIRTIS-M flight lamps. Rev.Sci.Instr., 74, number 8, pp.3796-3801. - Filacchione, G., 2006. Calibrazioni a terra e prestazioni in volo di spettrometri ad immagine nel visibile e nel vicino infrarosso per l'esplorazione planetaria. Phd dissertation, Universita degli studi di Napoli "Federico II".Available from: http://www.fedoa.unina.it/1462/1/Filacchione_Ingegneria_Aerospaziale_ Navale_e_della_Qualita.pdf - Raponi et al, 2020. Infrared Detection of Aliphatic Organics on a cometary Nucleus. Nature Astronomy, https://doi.org/10.1038/s41550-019-0992-8.