***** File IMAGUIDE.TXT Comet Halley Archive Image Data User's Guide to the Compact Disc Read Only Memory Contents 1. Background and Scope of Compact Disc 2. Accessing the Disc 3. Data Products 4. Supplementary Directories 5. Filenaming Convention 6. Available Software 7. References 1. BACKGROUND AND SCOPE OF COMPACT DISC This compact disc contains wide-field images of Comet Halley obtained by the Large-Scale Phenomena Network (LSPN) of the International Halley Watch (IHW) during the 1985-1986 apparition. The total number of L-SP digitized images (>1500), combined with the typically large size of each datafile, has resulted in a set of CD-ROMs being required for storage of the IHW/LSPN data alone, even in compressed form. Because of this disparity in the volume of data between L-SP and the eight other IHW disciplines, it was decided to deposit the wide-field images on dedicated CD-ROMs, of which this disc is one. The L-SP compressed image CD-ROMs are, of course, part of a larger set which also includes data from the eight other IHW disciplines: Amateur, Astrometry, Infrared Studies, Meteor Studies, Near-Nucleus Studies, Photometry and Polarimetry, Radio Studies, and Spectroscopy and Spectrophotometry. The data from these disciplines, obtained from 1982 through 1989, have been placed on the same CD-ROMs, interleaved together in chronological sequence. These multi-discipline CD-ROMs are sometimes referred to as the "mixed discs", on which also reside subsampled, uncompressed L-SP "browse images"; the rationale was that some representation of ALL the disciplines' data should be in one place. In addition to groundbased data returned from the nine IHW disciplines, spacecraft data from the so-called "Halley Armada" have been collected and placed in the Comet Halley Archive. Much more on the history of the IHW and on the specific operational details of each discipline is given in the appendix and other text files on the mixed discs, and this includes the Large-Scale Phenomena Discipline. However, additional details on L-SP beyond those brief ones mentioned here are also to be found in the various text files in the DOCUMENT directory of these compressed image discs, especially NETLARGE.TXT and ACKNWLDG.TXT. 2. ACCESSING THE DISC These discs have been manufactured following the normal specifications for CD-ROMs as well as the logical structure outlined by the ISO 9660 document. A basic system requirement demands that the CD-ROM reader and software driver support this standard. At this writing (July 1990), most access to the disc will be using IBM-PC or compatible computers. However, software support from vendors for the Apple MacIntosh II and some workstations (SUN, MicroVAX) is now available. Assuming that you are using a PC style machine, the reader that you bought came with the appropriate hardware (usually PC-bus controller board and cable) and software to address your device. Software packages should include a device driver for your machine and extensions to DOS that allow the reader to "look" like another random access disk drive; the de facto standard here is the Microsoft Extensions (MSCDEX 2.0 or higher). Install the specific device driver software as directed by the documentation accompanying the hardware. Usually, this will involve specifying the device driver in a DEVICE= line in the CONFIG.SYS file and invoking the extensions package from your AUTOEXEC.BAT file. This will automatically configure the CD-ROM reader when the computer is booted. Remember that the CD-ROM reader is assigned a drive name which you have either chosen with the appropriate command line switch (/L:H assigns the CD- ROM reader to drive H:) or allowed to default to the next available device letter in your system. Typical PC configurations will have two floppies (A: and B:), and a hard disk (C:). Put your drive at some letter higher than that. Remember that many DOS commands work on the CD-ROM but that it is a read-only medium. Some of the useful DOS functions are: DIR - directory listing CHDIR - change directory (also CD) TYPE - list the contents of a file on the screen; useful for normal text with , delimiters COPY - copy file(s) to another device PRINT - print file on hard copy device Notice that commands such as DEL and MKDIR are not available since the CD-ROM is a read-only medium. Furthermore, some compact disc software packages will invoke screen plots that may depend on the DOS program GRAPHICS.COM. In this case, execute it in your AUTOEXEC.BAT file or before you run the software package. The IHW CD-ROMs have been constructed to allow maximum access to the data using some existing software developed within NASA, as well as user-supplied programs. In particular, each data file was originally supplied in the FITS format, and also has a detached PDS label to describe the data structure. Furthermore, the normal FITS files have had the headers placed in separate files (with extension .HDR) from the data (with extension .IBG, or .IMQ). The headers have been separated to allow unrestricted access to the data by non-FITS programs. The sizes of all header and data files have been preserved as integer multiples of 2880 bytes (required by FITS) in order to facilitate the reconstruction of the original FITS bytestream by concatenating the data file onto the appropriate header file. 3. DATA PRODUCTS Hundreds of scientists worldwide contributed to the success of the IHW Archive generally, and to the Large-Scale Phenomena Network (LSPN) specifically. The participating LSPN observatories, with their IHW system codes, instrument descriptions, and observers, are listed in the file LSPNOBS.TXT in the DOCUMENT directory of this CD-ROM, and are indexed in the file LSPNOBS.IDX of the INDEX directory. The same and additional information about the observations are contained in the individual FITS header files. Details of the L-SP Discipline and Network activities, the types and extent of data processing, etc., are given in various of the text files (especially NETLARGE.TXT and CALIB.TXT) in the DOCUMENT directory of this CD-ROM, as well as in the L-SP Discipline appendix on the "mixed discs". In an effort to reduce the space requirements for the Halley Archive, data compression was applied to the Large-Scale Phenomena images using an algorithm which converts 2-byte pixels to 1-byte pixel-to-pixel differences whenever possible. The result, essentially, is a coded byte stream of 8-bit data. This "previous pixel" technique was used for images of the comet Giacobini-Zinner and placed on an IHW test disc. In practice, previous pixel compression on the L-SP images of Halley has resulted in files only slightly larger than 50% the size of the original images. Details of the algorithm are given in the COMPRESS.TXT file of this directory, and some computer language versions of the code are contained in the SOFTWARE directory. A proposal for handling compressed image data in FITS is given in the file FITSCOMP.TXT in this directory. Due to the modest data transfer rate inherent in the CD-ROM technology, the process of reading an image is quite time-consuming, even though data compression has been applied. For this reason, a set of subsampled or "browse images" is also supplied as a data product on the compressed image discs. These images are restricted to a maximum of 256 pixels on a side and preserve the original sampling geometry. All browse images have been rescaled from their original pixel values to 8-bits; for the vast majority of the images the original values were 10-bits or 2-bytes. For a few images that were already digitized by the observers and were not 2-bytes per pixel, the data were rescaled from 4-bytes to 2-bytes before being compressed or subsampled. Details on the construction of browse images can be found in the BROWSE.TXT file of this directory, and the code used to make the images is given in the MIDGET.FOR file of the SOFTWARE directory. For both the compressed (full-resolution) and browse images, placement of the data on disc and the filenaming scheme have been based on the time-of-observation chronology of the images (refer to Section 5 below). For the compressed images, the directory name is Cyymmmdd, where "yymmmdd" indicates the start date for data on a particular compressed image CD-ROM. For browse images the directory name on all the L-SP CD-ROMs is BROWSE. The average number of data files in a directory is somewhat less than 80 for all the L-SP discs, but note that the FITS headers and PDS labels also appear in each of the directories Cyymmmdd and BROWSE. There is no subdirectory below either of these data directories. Some LSPN observers submitted calibration data which reside on these CD-ROMs as digital images. Those data have been placed both as compressed and browse images in the CALIB and BROWSE directories. Calibration data include step wedges, spot sensitometry, and standard star fields, and the details are contained in the individual FITS headers associated with the calibration datafiles. The linkage between calibration data and specific images of Comet Halley can be found in either the CALIB.TXT file in this directory, or in the CALIB.IDX delimited index table in the INDEX directory. 4. SUPPLEMENTARY DIRECTORIES The volume and directory structures of this disc conform to the Level-1 standard specified by the ISO. This format is widely accepted and used on a variety of machines. The AAREADME.TXT file in the root directory introduces the user to the full extent of the archive and the contents of the individual subdirectories. There are three directories (DOCUMENT, INDEX, SOFTWARE) on the CD-ROM that contain supplementary files. In addition, to correlate chronological observations with physical location of the comet, detailed ephemeris information has been included in a separate subdirectory EPHEM. The table (EPHEM.TAB) contains information at daily intervals, so an interpolation routine is also provided; more information is in the text (EPHEM.TXT) file. The DOCUMENT subdirectory contains text files that give background to this CD-ROM project and a general guide to its use. Descriptions of the FITS and PDS formats are in FITSINFO.TXT and PDSINFO.TXT. IMAGUIDE.TXT is meant as a general overview of the data products. Tables of useful INDEX information have been collected in various forms in order to allow automated searching of the data. For LSPN, the nearly full set of FITS keywords contained in the headers is in a file named NETLARGE.IDX which includes the filename but not the path. A file, PATHTABL.IDX, gives the location of the file separately. We have attempted to make all index tables transportable to relational DBMS by delimiting the tables and providing structure and dBASE-compatible .DBF files. The CALIB.IDX file has a separate entry that relates each calibration to an image or range of images that can be associated. The files EPHEM, CDSTRUCT, and LSPNOBS are also provided as delimited tables. Further information is contained in the file INDXINFO.TXT. The SOFTWARE subdirectory contains program code for a few utilities, in this case, manipulation of images and the interpolation of ephemeris tables for the comet. The Previous Pixel algorithm used to compress the images is described in the file FITSCOMP.TXT. The code was developed by the L-SP Discipline at Goddard Space Flight Center. Pseudocode for this algorithm is listed (PREVPIX.TXT) as well as the IDL procedure (PREVPIX.PRO) originally used to test the compression on comet images. Fortran code to compress and subsample the data was composed for an IBM 3081 environment; the source code and explanation files (BROWSE.TXT and COMPRESS.TXT) is in the appropriate subdirectory under SOFTWARE. For image decompression, both Fortran and C code routines are provided; an MS-DOS program that accesses the split FITS header and data is included as an example of executable programs. An interpolation program was developed by the Astrometry network group for the ephemeris table. The Fortran code, an equivalent C version, and executable file for VAX/VMS and MS-DOS are provided. 5. FILENAMING CONVENTIONS The convention for naming files on the CD-ROM was adopted by the IHW Lead Center to include a running number as a unique data qualifier for chronological sorting. The name for the net (LSPN in this case) has been abbreviated and concatenated with the number to yield the appropriate filename (e.g., LSPNxxxx). For all calibration data, the filename is LSPN4xxx to distinguish from the 3383 images archived for the Comet. The file extensions follow suggestions by the PDS (SPIDS v1.1) for compressed (.IMQ) and subsampled (.IBG) data with detached label (.LBL). Since the data was originally submitted as FITS files and now registered as separate header and data files, a convention for header (.HDR) was adopted. These extensions are listed below: .HDR - FITS header .IBG - subsampled (browse) image .IMQ - compressed data .LBL - detached (PDS) label Note that in the ISO formatting process a version number (1 by default) is attached to the filename. In MS-DOS this information is ignored but for complex systems such as VMS, it is a necessary piece of information. In splitting up the original FITS files, we have attempted to make the data available to a larger community. For transporting data to analysis packages that demand FITS, the header and data files may be concatenated to create a fully valid FITS byte stream, as the original structure of the FITS headers and data have been preserved. 6. AVAILABLE SOFTWARE At the present, some executable software is included with the CD-ROM for programs supporting the ephemeris and image decompression. The data formats are compatible with existing MS-DOS packages and subsequently we will provide such software on floppy disk. We have supplemented the CD-ROM release with an accompanying printed manual for its use. 7. REFERENCES Martin, T., Martin, M., Davis, R.L., Mehlman, R., Braun, M., and Johnson, M.: October 3, 1988, Standards for the Preparation and Interchange of Data Sets, Version 1.1, JPL D-4683. "Information Processing -- Volume and File Structure of CD-ROM for Information Interchange", Reference Number ISO 9660: 1988(E), Produced in coordination with National Information Standards Organization, National Bureau of Standards, Administration 101, Library E-106, Gaithersburg, MD. E. Grayzeck, Jr M. Niedner, Jr. A. Warnock III National Space Science IHW Discipline Specialist IHW Team Member Data Center for Large-Scale Phenomena for Large-Scale Phenomena Interferometrics, Inc Laboratory for Astronomy ST Systems Corp and Solar Physics Code 633 Code 684.1 Code 681 NASA/GSFC NASA/GSFC NASA/GSFC Greenbelt, MD Greenbelt, MD Greenbelt, MD 20771 20771 20771 D. Klinglesmith III M. Aronsson IHW Team Member IHW Software Specialist for Large-Scale Phenomena for the Lead Center Laboratory for Astronomy Jet Propulsion and Solar Physics Laboratory Code 684 MS 169-237 NASA/GSFC 4800 Oak Grove Dr. Greenbelt, MD Pasadena, CA 20771 91109