Menu: Skip within this page

 

How to Prepare PDS4 Data

Preparing data for submission to the PDS includes not only labelling of data files and collection of documentation, but also content verification and usually at least some measure of reformatting to get the data into one of the standard deep archive formats required by the PDS. The formatting and documentation requirements of the PDS were developed in conjunction with astronomers to ensure that the data archived will be accessible long past the lifetime of the mission, its collaborators, or its hardware/software installation. The PDS format is specifically designed to be an archive format, not a working data format. Listed on this page are pointers to information and software tools to assist you in preparing and verifying data for submission to the PDS.

Introductions

The Planetary Data System (PDS) is an archive of data products from NASA planetary missions and supporting ground observations sponsored by the NASA Office of Space Science. A useful mission archive, as required by most NRAs and AOs, includes formatted raw data from each instrument, data calibrated to physical units, and derived products. Derived data products are those based on further processing of the calibrated data, or on combinations of data from different sources, and include things like maps, overlays, and comparative tables. The archive should contain sufficient documentation of the mission, the instruments, and the calibrations so that future generations of scientists can intelligently use and, if required, even recalibrate the data on contemporary computing platforms, without recourse to the original mission personnel and facilities.

The PDS has an Mission Preparer's Archive Guide available to guide new data preparers through the archiving process. Preparing data for submission to the PDS includes not only the labeling of data files and collection of documentation, but also content verification and usually at least some measure of reformatting to get the data into one of the standard deep archive formats required by the PDS. The formatting and documentation requirements of the PDS were developed in conjunction with astronomers to ensure that the data archived will be accessible long past the lifetime of the mission, its collaborators, or its hardware/software installation. The PDS format is specifically designed to be an archive format, not a working data format.

The On-Line Archiving Facility (OLAF) is a tool that data preparers can use to prepare and submit data and supporting documentation to the PDS for review and archiving. OLAF can accommodate tabular ASCII data and simple, single-image FITS files, whether from ground-based or spacecraft observations. OLAF simplifies the generation of the label, index and catalog files required for archiving these simple data formats.

If you are submitting observations of a solar system small body, please follow these Small Body Target Name Conventions to ensure that users can find your targets in the PDS catalog database.

If you are using a coordinate system of a small body and this system has not been approved by IAU or you want to update the IAU approved coordinate system for some object, then SBN should review your coordinate system before you deliver data using the coordinate system. The procedure and requirements are described in the document Coordinate Systems at PDS-SBN.

A Word About Data Formats

It has been our experience that simple data formats have several major advantages over more complex or proprietary formats:

  1. They are immediately accessible to a much wider section of the science community.
  2. They are stable over decades, maintaining accessibility for the projected life of the archive.
  3. They are easier to document and verify, facilitating efficient review and archiving.

Therefore, the Small Bodies Node is a strong advocate of simple data structures, especially in high-level mission data products. The simple data structures are homogeneous arrays and flat (ASCII or binary) tables. Not surprisingly, these are also the formats well supported by the Flexible Image Transport System (FITS) standard, which provides a very general and widely accessible format for exchanging files between various platforms.
Note that, from an archiving point of view, headers (and trailers) that are appended to the data structures present no major problems, as long as they do not interrupt the byte stream which constitutes the observational data. Thus any compliant FITS file can be easily prepared for PDS archiving without modifying the original FITS file because the headers occur before the individual data structures.

PDS Standards Documents

In addition to the printed documents available from the PDS, there are on-line versions accessible by Web browser:

The Archiving Process
The Archiving Process discusses the structure of a PDS4 archive and the archiving process, including the data management plan, archive delivery, and peer review process.
Information for Proposers
The PDS Information for Proposers page includes links to various guides, tools and examples for use in preparing proposal containing archiving elements.
Mission Proposer's Archiving Guide (MPAG)
Individual Proposer's Archive Guide (IPAG)
PDS Standards Reference
The PDS Standards Reference contains the detailed PDS standards for things like archive layout, data label requirements, units of measure, time values and other aspects of the PDS archives.
Data Provider's Handbook
A step-by-step guide guide to developing an archive.
Data Dictionary
The Data Dictionary contains the definitions for all keywords and data structures used in and available for archive datasets. A PDF version is available as well as a web interface to search for specific terms.
Data Dictionary Search.

PDS Tools

A number of tools have been developed for generating and verifying PDS labels. The complete inventory of general PDS utilities is available from the Engineering Node. The tools our data preparers have found most useful are:

SBN Tools

The SBN has also developed suites of small utilities for working with the types of data we most frequently encounter. The majority of these routines are written in Perl; some are written in ANSI-standard C. All are maintained in a linux environment. The source code for these routines is available from the SBN, as is, from our software archives. Utilities available include: