European Space Agency
Research and Science Support Department
Planetary Missions Division
ROSETTA-RPC-ICA
To Planetary Science Archive Interface Control Document
RO-RPC-ICA-EAICD
Draft
30 April 2010
|
_____________________________ ____________________________
Prepared by: Hans Nilsson Approved by: Rickard Lundin
Distribution List
Recipient |
Organisation |
|
Rickard Lundin |
Swedish Institute of Space Physics, Kiruna, Sweden |
Rickard.Lundin@irf.se |
Jan Karlsson |
Swedish Institute of Space Physics, Umeå, Sweden |
jk@irf.se |
Anders Eriksson |
Swedish Institute of Space Physics, Uppsala, Sweden |
|
J. Burch |
SwRI, San Antonio, TX., United States |
|
K-H Glassmeier |
TU Braunschweig, Germany |
|
Chris Carr |
Imperial College London |
|
Jean-Gabriel Trotignon |
LPCE/CNRS, Orleans, France |
|
G. Schwehm |
SCI-SB |
|
K.R. Wirth |
SCI-SB |
|
J. Zender |
SCI-SD |
|
D. Koschny |
SCI-SB |
|
M. A’Hearn |
PDS-SBN |
|
T. Farnham |
PDS-SBN |
Change Log
Date |
Sections Changed |
Reasons for Change |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
TBD ITEMS
Section |
Description |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Table Of Contents
1.1.1 Contents 3
1.2.1 ESA’s Planetary Science Archive (PSA) 3
2.4.1 Instrument Calibrations 11
2.4.2 In-Flight Data Products 13
2.4.3 Software 14
2.4.4 Documentation 14
2.4.6 Derived and other Data Products 14
2.4.7 Ancillary Data Usage 15
3.1.1 Deliveries and Archive Volume Format 15
3.1.2 Data Set ID Formation 15
3.1.3 Directory Naming Convention 16
3.1.4 File naming Convention 16
3.2.1 PDS Standards 17
3.2.2 Time Standards 17
3.2.3 Reference Systems 17
3.2.4 Other Applicable Standards 18
3.4.1 Volume Set 18
3.4.2 Data Set 19
3.4.3 Directories 19
3.4.4.1 Root Directory 19
3.4.4.2 Calibration Directory 20
3.4.4.3 Catalog Directory 20
3.4.4.4 Index Directory 21
3.4.4.5 Browse Directory and Browse Files 21
3.4.4.6 Geometry Directory 22
3.4.4.7 Software Directory 22
3.4.4.8 Label Directory 22
3.4.4.10 Document Directory 23
3.4.4.11 Data Directory 23
4.5.1 Data Product “Uncalibrated data” Design 25
4.5.1.1 File Characteristics Data Elements 32
4.5.1.2 Data Object Pointers Identification Data Elements 32
4.5.1.3 Instrument and Detector Descriptive Data Elements 32
4.5.1.4 Structure Definition of Instrument Parameter Objects 32
4.5.1.5 Data Object Definition 32
4.5.1.6 Description of Instrument 33
4.5.1.7 Parameters Index File Definition 33
4.5.1.8 Mission Specific Keywords 33
4.5.2 Data Product “Calibrated data” Design 33
The purpose of this EAICD (Experimenter to (Science) Archive Interface Control Document) is two fold. First it provides users of the Rosetta Plasma Consortium Ion Composition Analyzer (RPC-ICA) instrument with detailed description of the product and a description of how it was generated, including data sources and destinations. Secondly, it is the official interface between your instrument team and your archiving authority.
This document describes the data flow of the RPC-ICA instrument on Rosetta from the s/c until the insertion into the PSA for ESA. It includes information on how data were processed, formatted, labeled and uniquely identified. The document discusses general naming schemes for data volumes, data sets, data and label files. Standards used to generate the product are explained. Software that may be used to access the product is explained further on.
The design of the data set structure and the data product is given. Examples of these are given in the appendix.
The Planetary Data System Standard is used as archiving standard by
NASA for U.S. planetary missions, implemented by PDS
ESA for European planetary missions, implemented by the Research and Scientific Support Department (RSSD) of ESA
ESA implements an online science archive, the PSA,
to support and ease data ingestion
to offer additional services to the scientific user community and science operations teams as e.g.
search queries that allow searches across instruments, missions and scientific disciplines
several data delivery options as
direct download of data products, linked files and data sets
ftp download of data products, linked files and data sets
The staff of the archiving authority (Planetary Science Archive, ESA, RSSD, design team) and any potential user of the RPC-ICA data.
Planetary Data System Data Preparation Workbook, February 1, 1995, Version 3.1, JPL, D-7669, Part1
Planetary Data System Standards Reference, August 1, 2003, Version 3.6, JPL, D-7669, Part 2
[Mission] Archive Generation, Validation and Transfer Plan, [month day, year], [doc number]
This EAICD describes the overall RPC-ICA archiving details. If there will be changes in the archive format or labels this will affect both this document and the software which produces the PDS format data.
PSA Planetary Science Archive
ICA Ion Composition Analyzer
IES Ion and Electron Spectrometer, part of RPC instrument suite on Rosetta
PDS Planetary Data System
RPC Rosetta Plasma Consortium
MCP Micro Channel Plate
IMA Ion Mass Analyzer, twin instrument of ICA on Mars Express
CESR Centre d’Etudes Spatiales et de Rayonnements
IRF Institutet för rymdfysik (Swedish Institute of Space Physics)
FGM Flux Gate Magnetometer
ES Electrostatic in ES analyzer
TC Telecommand
TM Telemetry
GSE Geo Solar Ecliptic coordinate system
UTC Universal time
FOV Field-of-view
HV High Voltage
PROM Programmable Read Only Memory
NRM Normal instrument model
HAR High Angular Resolution model
EXM Energy-mass matrix high mass resolution instrument model
Hans Nilsson, project manager
Swedish Institute of Space Physics
Box 812
981 28 Kiruna
Sweden
Telephone +46 980 79127 Fax +46 980 79050
The Ion Composition Analyzer is an ion spectrometer designed to study ion dynamics with mass resolution. The design of the instrument is shown in figure 1.
Figure 1 Overview of the ICA instrument
Particles enter the analyzer through an outer grounded grid.. Behind the grid is a deflection system whose purpose is to deflect particles coming from angles between 45° and 135° with respect to the instrument symmetry axis (spacecraft y-axis) into the electrostatic analyser. This deflection system is named Deflector in figure 1 and the curved path of ions affected by the deflection electric field are shown as a red and a blue line. Ions within a swept energy pass band will pass the electrostatic analyzer (ES Analyzer, ESA in figure above). This is achieved by applying a high-voltage electric field between the inner and outer plate of the ES Analyzer. Only particles within the desired energy pass band will get just the right curvature of their path from this electric field so that they can pass the ES Analyzer. The geometrical (particle-optical) properties of the ES Analyzer determines the energy resolution which is 7% for ICA The ions are then deflected in a cylindrical magnetic field set up by permanent magnets, labeled Magnets in figure 1. The field deflects lighter ions more than heavy ions. The blue line in figure 1 depicts the path of a heavy and energetic particle which deviates only a little from a straight path. Heavy particles with therefore hit close to the center of the detection area. The red line depicts the pass of an ion with lower mass and/or energy. The ions finally hit a micro-channel plate (MCP) which amplifies the signal from the particle so that it can be detected by charge sensitive amplifiers. The charges from the MCP are detected by an anode system with 16 anodes for detecting angle of arrival and 32 anodes to determine radial position and thus mass (radial position is affected by both mass and energy, but energy is known because of the ES Analyzer). The different angles of arrivals that can be detected are termed sectors, and the radial anodes are termed mass channels. Ions are analyzed in both direction and mass per charge simultaneously. The magnet assembly can be biased with respect to the electrostatic analyzer to post accelerate ions; this post acceleration enables a selection of both mass range and mass resolution. The energy range of the instrument is 25 eV/e to 40 keV/e, covered in 96 steps. The full mass matrix has 32 channels, which are grouped according to calibration data. Resolution and mass coverage will depend on ion energy and post acceleration setting. The spatial resolution of the instrument is 22.5° x 4.5°. The 16 anode sectors of the MCP constitutes a 360° field-of-view in the x-z plane of the spacecraft with 22.5° resolution. This angle will be referred to as the azimuthal angle of the instrument field-of-view. A field-of-view of 51° to 131° with respect to the spacecraft y-axis is in practice achieved through the deflection system, with an intrinsic angular resolution of the instrument of 4.5°. The electric field applied over the deflection plates bends the path of incoming particles so that those coming from some angle away from 90° with respect to spacecraft y-axis will be able to pass the electrostatic analyzer . This angle will be referred to as the elevation angle. The resolution in elevation angle is determined by the geometrical (particle-optical) properties of the instrument. The elevation angle resolution has been determined through calibration and the number of elevation steps has been tailored to provide an almost continuous coverage of the elevation field-of-view. . The steps are nominally 5.625° apart to cover as close as possible the nominal angular range of ±45° from the instrument measurement symmetry plane. Elevation angle index numbering is shown in figure 2, produced by A. Fedorov of CESR Toulouse. Figure 2 is the official reference to sector and elevation index numbering and will be included with the archived data in the GEOMETRY directory.
The spatial resolution transmitted to ground is dependent on available telemetry, as is the mass resolution. Therefore the data files from ICA contain a variable amount of mass channels, deflection angles and sectors. Which angles of arrivals (sector and elevation angle) and mass channels which have been added together must be known by the end user and this information must therefore be contained in the archive data. Data is further compressed using lossless compression, though the archived data will be uncompressed. One complete measurement cycle of all measured energies and 16 elevation angles always takes 192 s. for the standard 96 energy level mode.
The location of ICA on Rosetta is shown schematically in figure 3. The spacecraft coordinate system is shown so that sector and elevation angles (through figure 2) can be related to the spacecraft coordinate system.
The field-of-view of the ICA can also be shown using the latitude and longitude definitions shown in figure 3, which is done in figure 4.
Note that –90 and 270 degrees in figure 4 are the same. In nominal comet-pointing position the comet will be at 90° longitude, 0° latitude. The approximate angles which can be affected by the solar panels can be seen. The region between –10° and 180° is the free field-of-view of ICA. Longitudes below –10° and above 180° are totally blocked by the spacecraft for negative latitude. Figure 3 and 4 will be stored in the GEOMETRY directory of the archive.
ICA supports the RPC science goals by measurements of three-dimensional ion velocity distributions with mass resolution and derived quantities such as density, flow velocity and temperature for different ion species. Because Rosetta is not a spinning spacecraft ICA cannot achieve full angular coverage. This will limit the ability to derive 3D moments and probably make it unfeasible to automatically generate moments for the database. Cometary origin ions will often form beam distributions which when within the instrument field-of-view can be well characterized. Other common distributions will be torii and shell distributions in which case a fitting procedure can be used on a case basis, this is also true for hot enough populations when the measured part of velocity space represents a significant part of the total particle population. The most important aspect of ICA is the mass resolution which distinguishes it from the Ion Electron Spectrometer (IES) of the Rosetta Plasma Consortium (RPC). The cometary origin ions are to a large extent O+ and water group ions. These are significantly heavier than the major solar wind ions (H+ and He++). Thus the important pick-up of heavy cometary origin ions by the solar wind, and subsequent decelaration of the solar wind can be studied. As soon as the comet gets active enough to release significant amounts of ionized gas to the solar wind ICA will be able to detect increased counts of heavier ions. The gradual increase of the cometary activity, as well as fine structure related to structure in the outflow from the comet can be studied (i.e. surface jets). Boundaries between the solar wind and the cometary ionosphere can be studied, including transient plasma transfer phenomena
Data is unpacked into raw data files at IRF in Kiruna. After that data, housekeeping and geometry data are transferred directly to PDS format ASCII tables for raw data and transferred to the RPC server at Imperial College. Data is transferred to the PSA from Imperial College.
Higher level processing products will be produced in Kiruna. First calibration and geometry considerations are applied to achieve fluxes. Note that instrument background noise will have to be subtracted manually. Therefore the raw data PDS format with attached calibration tables will be the most useful format for users. Then background subtraction, which must be performed case-to-case, can be made by the users themselves, and calibration tables applied after that.. If high background levels are present the QUALITY_ID will be set to 0, otherwise to 1. The background subtraction must be performed by the ICA team in order to deliver calibrated data. This is delivered to the common RPC database. Browse graphics file of all – direction averaged spectra are produced for two mass-bins (high and low) and delivered to the RPC browse data server and to the PSA.
A possible strategy to automatically remove the background has been tried for Mars Express IMA data and can be used by the end user on high resolution ICA data. One uses the fact that for significant counts of a real signal, one will always get signal from several contiguous mass channels. Usually the signal will also be seen in several adjacent energy channels. Therefore one can remove all occurrences of counts in a single mass and energy channel, where there are no counts in neighboring channels. This gives a much lower background around significant real peaks in the data. It does lower the sensitivity of the instrument. For weak signals one needs to first integrate for several full measurement periods,. In such a case the noise will also add up and may show up as larger groups of several adjacent mass and energy channels with non-zero counts.
The raw data products are generated upon unpacking of the telemetry data. Certain information not contained in the telemetry must be added, such as currently valid calibration files. This is straightforward and can be done as the data arrive, but further validation through manual inspection will take place before ingestion into the archive. Calibrated data requires more, possibly manual work, and will be delivered later than the raw data.
All data is unpacked and processed at IRF and then delivered to the RPC server at Imperial College from where it is ingested to the ESA server. Data is delivered to the RPC server immediately upon unpacking. Further transmission to the ESA archive takes place once the data has been validated and the quality level has been checked so that the quality parameter can be determined.
The initial data validation consists of manual inspection. Attempts to cross-compare different azimuthal sectors of the instrument can be made once a statistically significant data set exists and thus determine the relative sensitivity of different azimuth sectors.
The ICA data products consists of the raw counts produced by the in-flight data (section 2.4.4, which are interpreted and turnied into calibrated data using tables containing the calibration results (section 2.4.3). Derived products (moments), section 2.4.7, are not expected to be delivered as these require manual inspection and choices which must be done on a case by case basis. There are no pre-flight data products nor any sub-system tests.
Calibration tables from the calibration of the flight model will be stored. These include energy tables, deflection angle tables, mass channel to ion mass tables for different post-acceleration levels, and counts to flux conversion tables for a given post-acceleration and MCP bias. With time the MCP ages and the bias voltage applied must be increased. This will affect the count to flux calibration and new calibration tables will have to be made based on suitable estimates of the new efficiency of the MCP. Therefore the data labels will contain information about the currently valid calibration table. The calibration tables are given in PDS format with detached labels. These are for the energy resolution:
ICAENTABLEVNN.TAB and ICAENTABLEVNN.LBL where NN is version number.
which contains 96 rows with 2 columns. The columns are
Index number of the energy level (i.e. just a counter which corresponds to the data points in the PDS data file),and energy per charge [eV] of detected ion. The beginning of table ICAEnTableV001.TAB is shown below.
0,39998.4
1,37007.3
2,34245.6
3,31683.3
4,29320.4
Note that high energies come first. This reflects the actual measurement sequence, the analyzer voltage is stepped from high to low.
For the entrance deflection the table is
ICA_EL_TABLE_VNN.TAB and ICA_EL_TABLE_VNN.LBL where NN is version number.
Which contains 16 +1 columns and 96 rows, where 16 corresponds to the amount of elevation angles and 96 to the amount of energy levels. The energy index number is given in the first column. A sample close to nominal value obtained at intermediate energy index 42 is shown below:
42,-39.2,-33.6,-28.0,-22.7,-17.5,-12.3, -7.2, -1.4, 3.7, 8.8, 14.1, 19.4, 24.9, 30.6, 35.7, 40.9
At low (below 300 eV) and high (above 15 keV) energy the angular resolution and coverage deviates significantly from nominal values. At low energy this is because of insufficient resolution of the digital to analog conversion. Therefore no entrance deflection is used below 300 eV. The elevation angle is always close to 0. At high energies the discrepancy is due to insufficient high voltage to deviate the flight path of the more energetic ions. This table can be used by the end user to get accurate conversion of elevation index into actual physical angle from the spacecraft X-Z plane. Otherwise nominal values can be used (for approximate visualization purposes) with the angle given by:
-42.1875+elev*90/16 where elev is the elevation index number (0-15)
The angle within the X-Z plane, from X towards Z, is given by the sector (or azimuth angle index).as
-168.75+azim*360/16 where azim is the azimuth index (0-15, same as sector number).
The ICA has a command, enable bad-HV masking. This sets all deflection angle and energy combinations which deviates substantially from nominal values to zero. However, experiences from Mars Express shows clearly that it is better to set the elevation angle setting to zero for the low energy case and keep the data. Therefore a new table with deflection angle as close to zero as possible is now used for energies below 300 eV where the limited elevation angle accuracy problem may occur. However, the command for bad-HV masking exists and may be used again in the future. For high energies the deflection voltage is set to as high as possible and this data will not be masked either, but will not have the deflection angle nominally specified. Therefore the end-user must have access to this table
Finally, mass look-up and mass calibration tables must be used. The mass look-up tables are used on-board to reduce the telemetry. As described in section 2 ICA has 32 mass channels. Ions of a certain mass will hit the detector surface (MCP) at some range of mass channels, and this range will vary with the energy. Therefore the mass look-up table must have one entry for each energy, telling what range of mass channels to put together. Furthermore the post-acceleration setting will also affect where the particles hit the detector surface. Therefore ICA has three mass look-up tables for three different post-acceleration settings (labeled 0, 1 and 2 in the calibration document). For high mass-resolution modes the actual mass-channel data will be transferred to ground, possibly binned (32, 16 or 8 bins). For lower resolution than that the data is instead binned into mass-groups according to the mass look-up tables. These tables are named
ICALookUpNVNN.LBL and ICALookUpNVNN.TAB
Where N is 1,2 or 3 depending on the post-acceleration setting. Note that the actual post-acceleration setting is from 0-2 according to the calibration document, but later mass look-up table 0 will be used to specify that no mass look-up table was used. The file contains 12+2 column elements and 96 rows. Each row contains post-acceleration level it is valid for, the energy index and 6 different ion mass ranges (2 values, start and stop), corresponding to H+, >O+, O+, He+, He++, O++
The column elements specifying mass range are given as start mass channel, end mass channel. A value of –1 indicates that the indicated ion mass range cannot be seen by the instrument for that particular energy and post-acceleration setting.
A sample line for energy index 42 and no post-acceleration:
0,42,-1,-1, 0,14,15,20,27,31,-1,-1,21,26
Meaning that for no post-acceleration (0) and energy level 42, H+ cannot be measured (-1), >O+ ions are found between mass channels 0 to 14, O+ between 15 to 20, He+ between 27-31, He++ cannot be measured and O++ is found between mass channels 21 and 26. The reason for the strange order is that for severe telemetry restrictions not all mass bins are transmitted, and then Larger than O+ (a large group of everything above O+), O+ and H+ are the most important to transmit.
The reason for the larger than O+ mass group (>O+) is that if only two mass ranges are allowed due to telemetry constraints, then H+ (solar wind) and heavy cometary ions can be distinguished.
A table relating the post-acceleration setting to post-acceleration high voltage index (the actual high voltage reference value set in the instrument) and the actual high voltage as given by the calibration is also provided as POST_ACC_INTERVALS_VNN.LBL (.TAB). Further details can be found in ICATABLES.PDF by A. Fedorov which is also stored in the CALIB directory. The counts to flux tables has at the time of writing not been delivered from CESR and we cannot provide a format for them. The name will be COUNT_TO_FLUX_VNN.LBL (.TAB) for the label. They are stored in the CALIB directory.
PDF graphics files defining the geometry of measurements are included in the GEOMETRY directory. One is the Sectors.pdf shown in figure 2. Another one is the ICA field-of-view (ICAlocation.jpg and ICA_FOV.JPG), also shown as figure 3 and 4 in this document.
Other Files written during Calibration
Data file on the ion source and experimental setup will be provided. (TBC)
The ICA instrument gives essentially only one raw data (Level 2) output, counts. The instrument obtains counts for each energy level, sector, elevation angle and mass channel (as described in section 2). One full distribution is obtained in 192 s for all modes except the solar wind mode which takes 64 s. The instrument make measurements in the same way regardless of available telemetry rates. To reduce the data rate the data is reduced in resolution. Elevation angles, sectors and mass channels are added together into larger bins. Data is never binned over energy and time in this way.
The raw data is thus in principle 4-dimensional, but must be stored as a two-dimensional table. The size of the sector, elevation angle and mass channel bins will vary continuously depending on available telemetry and loss-less compression ratio. It is thus not feasible to vary the tables after the current angular and mass resolution. Therefore start and stop values for each bin (sector, deflection angle and mass) must be included in the data. Start and stop values for elevation angle, sector and mass bins add 6 fields to the table. For the mass bins an on-board table-look up which is energy dependent may have been used onboard. Therefore one field is used to describe which table was used (and a keyword in the label gives which revision of the table), where table 0 indicate that no mass-lookup table was used.
Because of high voltage limitations, spacecraft body and solar panel shadow, and potentially other reasons, some angles and mass channels may not contribute counts to certain energy steps. Therefore the number of sectors, mass channels and deflection angles actually contributing to the counts for each energy step must also be known. This is however normally static and it is very inefficient to store for each energy spectrum. Instead status information taken from the header of the science data packet is included. This header information contains information about whether bad-high voltage masking and shadow masking was enabled. Bad high-voltage masking was described in section 2.4.3, and means that certain elevation angle – energy combinations are masked to zero. This masking possibility will likely not be used for the data, and the corresponding status bit will be zero. Shadow masking implements the same for certain combinations of sector and elevation angle. Figure 3 shows which sectors and elevation angles will be shadowed by the spacecraft. These will not provide useful counts whether masking was on or not so it is important for the end-user to know about figure 3, but the value of the status bit is likely not important. However, its value should be kept because otherwise the archive data is not complete and does not allow future users full control of the data set. The science header also contains information about which onboard eeprom the software and tables was taken from. This is normally not important for the end user but the PDS data set will be used as the long time archive format by the ICA team and must therefore contain all the information from the science packets.
House keeping parameters are stored in a separate file and with a separate label in the same directory as the data. When the calibration data is applied the raw counts can be translated into flux for different ion species or ion mass ranges (Level 3 data). When magnetic field and spacecraft attitude data is added the instrument coordinates can be transformed into magnetic field or geographic (likely comet-sun-orbit plane coordinates, must be common at least for RPC, sensibly for all of Rosetta). Housekeeping data will be unpacked and stored in human and PDS readable ASCII files.
One day of full resolution data will be about 1 Gbyte. The file may be difficult to handle and will therefore be subdivided into 1 hour intervals.
The unpacking software is based on the software used for Mars Express data from Aspera 3-IMA.
Currently two softwares exist for analysis of Mars Express IMA Aspera-3 IMA data, both from CESR Toulouse. These can with small modifications be used for ICA. In the long run we intend to use the raw data archive format described in this document, which will be very easy to import into any analysis software. Read and plot routines in the format we use ourselves will be made publicly available through the archive system (currently we use Matlab). A Matlab line to read an entire data file into the Matlab workspace is shown below:
indata=fscanf(fid,['%4d %*c %2d %*c %2d %*c %2d %*c %2d %*c %f,%f,%f,%f,%f,%f,%f,%f,%f,%f,%f' ... ',%f,%f,%f,%f,%f,%f,%f,%f,%f,%f,%f,%f,%f,%f,%f,%f,%f,%f,%f,%f,%f,%f,%f,%f,%f,%f,%f,%f,%f,%f' ... ',%f,%f,%f,%f,%f,%f,%f,%f,%f,%f,%f,%f,%f,%f,%f,%f,%f,%f,%f,%f,%f,%f,%f,%f,%f,%f,%f,%f,%f,%f' ... ',%f,%f,%f,%f,%f,%f,%f,%f,%f,%f,%f,%f,%f,%f,%f,%f,%f,%f,%f,%f,%f,%f,%f,%f,%f,%f,%f,%f,%f,%f' ...
',%f,%f,%f,%f,%f,%f,%f'],[113,inf]); where fid is a file pointer variable pointing to an archive table file.
The same formatting is essentially used in C. Sample Matlab files are provided in the SOFTWARE directory.
“The ICA-IMA TC/TM data formats and related software aspects” by Hans Borg
“ICA Command Description” by Hans Borg
“ICA calibration report” by Andrei Fedorov
ICATables.pdf by A. Fedorov
All stored in Portable Document Format (pdf) in the DOCUMENTS directory.
The L3 (calibrated) data can be used to calculate higher order data (moments), but as the ICA field-of-view is not complete the feasibility of such calculations must be individually judged for each event, and this is thus not foreseen to be included in the PSA. Furthermore one must for each event judge if the moments should be fitted functions (Maxwellians) to take into account truncation due to the limited field-of-view or a numerical integration (which is used when the entire particle population is within the field-of-view of the instrument).
Space craft attitude and orbit data are necessary. In particular it is necessary to transform between the spacecraft coordinate system and some common system such as GSE and any mission specific comet centered coordinate system. The RPC fluxgate magnetometer (FGM) data is necessary to transform into magnetic coordinates. A common defintion must be made for all of RPC but pitch-angle (angle between magnetic field and viewing direction) must at least be calculated. The plane perpendicular to the magnetic field may be described by another angle with zero towards the sun or towards the comet.
PDS compliant data will be delivered to ESA at DATA SET Level. One data set corresponds to one volume. Data of different processing levels will be archived in different data sets.
The complete RPCICA data set will be delivered to the Imperial College server. From there all RPC data will be delivered to ESA by the RPC archive manager.
RO-E/M/A/C-RPCICA-x-description-Vn.m
E/M/A/C = Earth/Mars/Asteroid/Comet
RO = INSTRUMENT_HOST_ID
RPCICA = INSTRUMENT_ID
x = {2,3} Codmac Data processing level numbers.
description = {UNCALIBRATED, CALIBRATED}
Vn.m = Version number
Within each data set TARGET_NAME and TARGET_TYPE will then be used to identify the current target.
(Thus they will not stay the same within one data set, but data set id will.)
DATA_SET_ID | +----+-----------+--- ... | | | DATA DOCUMENT CATALOG ... | CALIBRATED | +------+----- 2009 2010
The top level directory of the data set is named by DATA_SET_ID according to PDS specifications. This is found under a top level RPC directory structure as shown above at the Imperial College server. Under DATA a directory specifying the type of data (EDITED, CALIBRATED) is used, but this is in practice redundant as a new DATA_SET must be used for each type of data. The DATA is further subdivided as:
2009 | +------------+------------+------------... | | | JAN FEB MAR ... | +----+----+ ... | | | D01 D02 D03
ICA will produce two types of science data files for uncalibrated data and two types for calibrated data. These are data for 96 energy step modes and data for the 32 energy step mode.Furthermore an edited housekeeping data file and geometry data file will be produced.
POSITION: 012345678901234567890123.012
FILENAME: RPCICAYYMMDDTHH_xxx_nnLL.EXT
YY = Year
MM = Month
DD =Day
T = Time separator
HH=hour
xxx = file running number to allow for several files in one time period
nn = 32 or 96 depending on energy step mode. Not used for HK and GEOM files
LL = L2 for raw counts, L3 for calibrated data, HK for housekeeping and GEOM for geometry
EXT = LBL or TAB
ICA complies to PDS version 3, and we use version 3.6 of the PDS standard reference.
All references to time in the ICA PDS archive will be to UTC. Then time is displayed using the PDS standard CCYY-MM-DDThh:mm:ss.sss we will convert space craft time into UTC in this format (not GMT as the original PDS definition). The conversion will be done from the epoch 2000.
Housekeeping files will also contain the spacecraft clock time (i.e. unconverted and uncorrected ticks.)
The spacecraft Clock (OBT) is given in the label files as “1/nnnnnnnnnn.nnnnn”
It is the SPACECRAFT_CLOCK_START/STOP COUNT which is given as OBT. The OBT is described in Rosetta Time Handling, RO-EST-TN-3165, sect. 4.2.
The general principle is as follows:
SPACECRAFT_CLOCK_START/STOP_COUNT = "<reset number>/<unit seconds>.<fractional seconds>"
Where the unit seconds and the fractional seconds are separated by the full stop character (“.”). Note that this is not a decimal point. The fractional seconds are expressed as multiples of 2^(-16) = 1.53E-5 seconds and count from 0 to 2^16 -1= 65535.
E.g. in SPACECRAFT_CLOCK_START_COUNT = "1/21983325.392" the 392 fractional seconds correspond to 392 * 2^(-16) = 0.00598 decimal seconds.
The spacecraft clock could be reset during the mission (although this is not planned). This would imply a change of the zero point. The zero point of the OBT will be indicated by pre-pending the reset number (integer starting at 1) and a slash to the unit seconds, i.e. "1/" means OBT = 0 at 2003-01-01T00:00:00
UTC.
Reference systems used in this document and the ICA PDS data:
Reference system |
Description |
Instrument coordinates |
THE ICA instrument coordinate system. Retained for consistency with calibration report. Aligned with spacecraft Cartesian coordinate system, but different definition of the axes. |
Spacecraft coordinate system |
Orientation: x: pointing from the LANDER to the s/c center, perpendicular to solar array axes; y :parallel to solar array axis; pointing to the left, when standing in front of the Lander, z: pointing up |
N/A
Data will be scanned for internal consistency when decommutating to edited raw format. Derived data will when possible be compared to independent measurements by other instruments. The solar wind is particularly useful as it is a narrow beam which, when at all within the field-of-view, is typically fully measured by ICA. When changes are made to one of the RPC ion spectrometers (ICA and IES), such as changing the MCP bias, this can be made when both instruments have the sun (solar wind) in the field-of-view. That way the relative change of the sensitivity can be estimated.
Before archiving a data set from some mission phase, this set will have been used internally by RPC scientists. It is planned to base all scientific analysis on the data products formatted. To actually have the data used by scientists before delivery to archive is considered the best way of revealing problems, and this is the approach taken by RPC.
In the label a QUALITY_ID keyword is used to indicate the quality of the data. QUALITY_ID –1 means invalid data even though it is not clear under what circumstances invalid data would be delivered to the PSA. QUALITY_ID 0 will be used when the background levels are an obvious problem. For such data the user must individually check the data and try to subtract background. QUALITY_ID 1 is expected to be the most common QUALITY_ID, in which case background levels are low. For Level 3 data background should in principle always be subtracted which gives a further QUALITY_ID of 2. If Calibration tables are applied to RAW data without subtraction of background (which likely must be manual) then all QUALITY_ID will be valid for the resulting level 3 data.
The one data set on one volume basic volume organization is used. Thus the volume directory structure looks like the following:
ROOT |--- AAREADME.TXT |--- ERRATA.TXT |--- VOLDESC.TXT +--------------------------------------------------------------------------... | | | | | | | BROWSE CATALOG SOFTWARE CALIB GEOMETRY INDEX DATA
Our naming convention for the data set will follow the same principles as the DATA_SET_ID thus.
DATA_SET_NAME=“ROSETTA-ORBITER E/M/A/C RPCICA 2 DDDDDDD V1.0”
ROSETTA-ORBITER = INSTRUMENT_HOST_NAME
E/M/A/C = EARTH MARS ASTEROID COMET
RPC-ION-MASS-ANALYZER = INSTRUMENT_NAME
2 = Data processing level numbers
Data set will include Edited data or Calibrated data
DDDDDDDDDDDDD = Processing level in text, i.e. EDITED or CALIBRATED
One data set will be used for each processing level. Multiple targets will be used for each data set and within each data set TARGET_NAME and TARGET_TYPE will be used to identify the current target (Thus they will not stay the same within one data set, but data set id will). The data set name fits in the full length thus 60 characters.
Table 1: Root Directory Contents |
|
File Name |
File Contents |
AAREADME.TXT |
This file completely describes the Volume organization and contents |
AAREADME.HTM |
Hypertext version of AAREADME.TXT |
AAREADME.LBL |
A PDS detached label that describes AAREADME |
ERRATA.TXT |
A cumulative listing of comments and updates concerning all ICA standard data products on all ICA Volumes in the Volume set published to data |
VOLDESC.CAT |
A description of the contents of this Volume in a PDS format readable by both humans and computers |
CALIB/ |
Calibration directory |
CATALOG/ |
Catalogue directory |
DOCUMENT/ |
Document directory |
INDEX/ |
Index directory |
DATA/ |
Data directory |
BROWSE/ |
Browse directory |
GEOMETRY/ |
Geometry directory |
LABEL/ |
Label directory |
SOFTWARE/ |
Software directory |
Table 1: Calibration Directory Contents
|
|
File Name |
File Contents |
CATINFO.TXT |
A description of the contents of this directory |
ICA_EL_TABLE_V01.LBL, .TAB |
Calibration table describing the elevation angle corresponding to each elevation index, as a function of particle energy |
ICA_ENERGY_TABLE_V01.LBL, .TAB |
Calibration table describing the particle energy corresponding to each energy index level |
ICA_MASS_LOOK_UP_TABLEN_V01.LBL, .TAB |
Calibration tables describing the mass channels corresponding to 6 different ion mass intervals as a function of energy. N is 1,2 or 3 where table 1 is used in the case of no post-acceleration, table 2 when post-acceleration reference level is 1-4 and table3 when post-acceleration level is 5-7. |
Table 3: Catalog Directory Contents
|
|
File Name |
File Contents |
CATINFO.TXT |
A description of the contents of this directory |
UNCALIBRATED_DS.CAT |
PDS Data Set catalog description of all the ICA un-calibrated level 2 data files |
CALIBRATED_DS.CAT |
PDS Data Set catalog description of all the ICA calibrated level 3 data files |
DERIVED_DS.CAT |
PDS Data Set catalog description of all the ICA derived level 5 data files, if this will exist |
INSTHOST.CAT |
PDS instrument host (spacecraft) catalog description of the Rosetta orbiter spacecraft |
INST.CAT |
PDS instrument catalog description of the ICA instrument |
MISSION.CAT |
PDS mission catalog description of the Rosetta mission |
PERSON.CAT |
PDS personnel catalog description of ICA Team members and other persons involved with generation of ICA Data Products |
REF.CAT |
ICA-related references mentioned in other *.CAT files |
SOFTWARE.CAT |
PDS software catalog description of ICA software |
Table 4:Index Directory Contents
|
|
File Name |
File Contents |
INDXINFO.TXT |
Dataset Index File |
|
Browse index file |
|
Data index file |
|
No other index files will be used |
Contains png files of all-direction raw spectrograms binned into two mass bins (low and high) for a fixed time interval of 1 hour. The data is organized in the same structure as the data, i.e. year, month and day are directories which contain png and corresponding label (.LBL) files. . Description in BROWSEINFO.TXT.
Table 5: Browse Directory Contents
|
|
File Name |
File Contents |
BROWSEINFO.TXT |
Information about the Browse data |
DATA |
Contains .LBL and :TAB files with the plot data as ASCII files |
YEAR |
Directories containing browse plots from each year. See directory structure below. |
BROWSE | |--BROWINFO.TXT |--BROWSE.HTM |--BROWSE.LBL | DATA | +------+------+----... | | | 2004 2005 2006 ... | +----+---... | | MAY OCT ... | +---... | D01 ... | .PNG picture files
Table 6: Geometry Directory Contents
|
|
File Name |
File Contents |
ICALOCATION.JPG |
A figure describing where ICA is located on Rosetta, as well as relating the ICA instrument and spacecraft coordinate systems to each other. |
SECTORS.PDF |
Figure describing the sector (azimuth) and elevation index numbering in the instrument coordinate system. |
ICA_FOV.JPG |
Rosetta seen from the ICA field-of-view in a local latitude-longitude coordinate system explained in figure ICALOCATION.JPG |
Table 7: SOFTWARE Directory Contents
|
|
File Name |
File Contents |
Readica.m |
File to read ICA data into the Matlab workspace. |
Plot_nrm.m |
Sample program to plot ICA data obtained in the NRM modes (on-board mass look-up tables). For high resolution data the program need to be modified, but the general way of working with the data will be the same. |
ICAinitilize.m |
Script to initialize the workspace by for example reading the ICA energy table into the workspace. Needed to run plot_NRM.m |
Table 8: Document Directory Contents |
|
File Name |
File Contents |
DOCINFO.TXT |
A description of the contents of this directory and all subdirectories. |
ICA_EAICD/ |
Directory containing the ICA EAICD document |
ICA_EAICD/ICA_EAICD.HTM |
The ICA Experiment-Archive Interface Control Document as hypertext |
ICA_EAICD/ICA_EAICD.ASC |
The ICA Experiment-Archive Interface Control Document in ASCII text |
ICA_EAICD/ICA_EAICD.PDF |
The ICA Experiment-Archive Interface Control Document in PDF format |
ICA_EAICD/ICA_EAICD.DOC |
The ICA Experiment-Archive Interface Control Document in MSWordXP format |
ICA_EAICD/ICA_EAICD.LBL |
A PDS detached label that describes ICA_EAICD.HTM, ICA_EAICD.ASC, ICA_EAICD.PDF and ICA_EAICD.DOC |
ICA_INST/ |
Directory containing the ICA instrument paper |
ICA_INST/* |
Similar to ICA_EAICD/* |
ICA_CALIB/ |
Directory containing information regarding calibration |
ICA_CALIB/* |
Similar to ICA_EAICD/* |
ICMA_TCTM.PDF |
“The ICA-IMA TC/TM data formats and related software aspects” by Hans Borg |
ICA_CMD_BRIEF.PDF |
“ICA Command Description” by Hans Borg
|
ICA_CAL.PDF |
“ICA calibration report” by Andrei Fedorov |
ICATABLES.PDF |
ICA Tables by A. Fedorov |
Other Documents |
Additional documents describing data processing, etc. |
Other Document labels |
Detached PDS labels for any additional documents |
Table 9: Data Directory Contents
|
|
File Name |
File Contents |
CATINFO.TXT |
A description of the contents of this directory |
RPCICAYYMMDDTHH_000_96L2.LBL, .TAB |
Label and table files containing ICA data |
RPCICAYYMMDDTHH_000_GEOM.LBL, .TAB |
Label and table files with geometry data |
RPCICAYYMMDDTHH_000_HK.LBL, .TAB |
Label and table files with housekeeping data |
See section 3.1.3 for general overview. Note that different processing levels are stored as different DATA_SETs.
Now as defined in section 3.1.3 we have the following structure for each year in the DATA directory.
+---------------+-------------------... DATA_SET_ID1 DATA_SET_ID2 | | DATA DATA | | EDITED CALIBRATED | +---------------------+-------------------+----------... | | | 2004 2005 2006 ... | +------------+-------------+------------... | | | MAR APR MAY ... | +--------------+--------------+---- ... | | | D01 D02 D03 ...
Two levels of data are used: Raw (edited) data and calibrated data. The raw edited data contains the raw counts from the detector, sorted into the PDS format and with UT calculated and a quality parameter set to indicate data quality.
For the calibrated data a background subtraction has been performed and the calibration tables to turn counts ino physical flux units.
The raw data archive also contains the housekeeping data from the TM stream in uncalibrated format.
ICA produce two kinds of data, ion mass energy spectrograms and housekeeping data (HK). The instrument can be run in different modes, resulting in different resolution, but the data format is the same.
A complete set of edited data consists of housekeeping and science data. Calibration files in the CALIB directory will also be needed to interpret the data.
Start time of the observations in Universal Time
Bad HV mask, a status bit which describes if the bad HV mask was enabled (see section 2.4.4)
Shadow mask, a status bit which describes if the shadow mask was enabled (see section 2.4.4). If enabled all azimuth and elevation angle combinations shadowed by the spacecraft are set to zero. If not, the counts need not be exactly zero and may be used to estimate background levels, at the expense of using more telemetry.
PROM section, ICA can run from different PROM software banks. Usually not important for the end user.
Post-acceleration reference level (0-7). Necessary to know in order to interpret the mass channels correctly in terms of physical mass of measured particles. Which mass table to use for a certain post-acceleration reference value is given in the file “ICA_POST_ACC_TABLE_V01.LBL” in the CALIB directory.
Mass table used. ICA may use mass look-up tables on board to bin data into certain physical mass ranges (see section 2.4.3). This column describes which of three on-board tables was used. 0 indicates that no mass look-up table was used, 1 indicates the table for no post-acceleration, 2 low post acceleration and 3 high post-acceleration.
Azimuth index describing the start sector of the sectors binned together to produce the data (see sections 2, 2.4.3 and figures 2 and 3)
Azimuth index describing the last sector of the sectors binned together to produce the data.
Mass channel number describing the start mass channel of those binned together to produce the data. See sections 2 and 2.4.3.
Mass channel number describing the end mass channel of those binned together to produce the data.
Elevation index describing the start elevation angle index of the elevation angles binned together to produce the data (see sections 2, 2.4.3 and figures 2 and 3)
Elevation index describing the end elevation angle index of the elevation angles binned together to produce the data
– 107 contains 96 count-values for the 96 different energy steps in the energy spectrogram. The counts are the sum over all sectors, elevation angles and mass channels used to produce the data according to the previous columns. The energy levels are found in the file ICA_ENERGY_TABLE_V01.LBL in the CALIB directory.
Example of edited raw data detached label file (e.g. RPCICAYYMMDDTHH_000_96L2.LBL)