Specification of the IWF Flat-File Data Base Management System Version 1.01 Date: 18. December 1997 1 Header File 1) The IWF-FLAT Filesystem's Header File (*.HED) is an ASCII-file with 80 bytes printable ASCII characters. Each record starts with a blank character (0x20) at position 1 (i.e. 1st character of record) and ends with a blank character at position 80. Due to historical reason on different operation systems and platforms the Header File have different line end marks. This come from the FORTRAN implementation of the Flat File. With the same source code the header have different line end marks. [ OPEN( .. , ACCESS = 'direct', FORM = 'formatted', .. ) ] On the PC-Platform under MSDOS / Windows a Carriage Return (CR) and Line Feed (LF) will be added each line. So on this Platform the Header File have 82 bytes. ( Position 81: CR, Position 82 LF ) On the VAX and ALPHA under OpenVMS the header have following file format. Record format: Fixed length 80 byte record Record attributes: Fortran carriage control Under Unix on different Platforms the Header have no line end marks. Only 80 bytes per line ( DEC Unix, SUN Solaris and Solarisx86 ) 2) An IWF-Flatfile may contain up to 499 items including the timestamp (i.e. 1 real*8 timestamp plus 498 real*4 variables). 3) Unused space within a header record is filled with blanks (0x20). 4) The detailed header record structure is given in Table 1. 5) The date for entry " date files created: " may be in format "22-AUG-96" or "1996-AUG-22". 6) The timestring for entry " Start time = " and " End time = " may be in format "1977-JAN-01 00:00:00.000" or "31-DEC-77 23:00:00.000". 7) The encoding strings are "PC ", "VAX", "DEC", "DEC". The encoding strings stands for the system on which the file is created. "PC " Personal Computer (MS-DOS, SOLARIS x86) "VAX" VAX or ALPHA Machines under OpenVMS "DEC" DECStation under Ultrix "SOL" SPARC under Solaris The encoding information must be in the first abstract line. 8) Other predefined Abstract keywords: Owner: Owner of Flat File Source: Flat Master File Orbit: Data Orbit Mode: Data Mode Resol: Data Resolution CoordSystem: Coordinate System DataType: Data Type Offset: Data Offset SpaceCraft: Space Craft These predefined Abstracts can stand in the abstract section but the must not. 9) Sample IWF-FLAT Header Files are given in Table 2 and Table 3. Table 1: Field Definition of IWF-FLAT Header File Record Contents From - To Fmt Align 1 " name of header and data files: " 1 - 32 a32 --- FlatFile Base Name (no Extension) 33 - 79 a47 left 2 " date files created: " 1 - 21 a11 --- Date String (-> Rule 6) 39 - 49 a11 right 3 " record length of data file, in bytes: " 1 - 39 a39 --- *.DAT File Record Length 40 - 49 i10 right 4 " number of columns: " 1 - 20 a20 --- Number of Items incl. Time 40 - 49 i10 right 5 " number of rows: " 1 - 17 a17 --- Number of Records in *.DAT File 40 - 49 i10 right 6 " flag for missing data : " 1 - 25 a25 --- Missing Data Flag Value 42 - 50 e9.2 right 7 Blank Line (80 0x20 chars) 1 - 80 a80 --- 8 "#" 4 - 4 a1 --- "name" 8 - 11 a4 --- "units" 22 - 26 a5 --- "source" 36 - 41 a6 --- "type" 67 - 70 a4 --- "loc" 73 - 75 a3 --- 9 " ---------< 75 minus chars >----------" 1 - 76 a76 --- 10 item/variable number (1 <=> Time) 2 - 4 i3 right to item/variable name 8 - 19 a12 left 9 item/variable unit 22 - 33 a12 left + item/variable source 36 - 65 a30 left num item/variable type ( "T" or "R") 68 - 68 a1 --- items item/variable offset in *.DAT record 73 - 76 i4 left 10 + Blank Line (80 0x20 chars) 1 - 80 a80 --- 11 + " NOTES: " 1 - 8 a8 --- 12 + Blank Line (80 0x20 chars) 1 - 80 a80 --- 13 + " Start time = " 1 - 14 a14 --- Time String (-> Rule 7) 15 - 38 a24 left 14 + " End time = " 1 - 14 a14 --- Time String (-> Rule 7) 15 - 38 a24 left 15 + Blank Line (80 0x20 chars) 1 - 80 a80 --- 16 + " ABSTRACT " 1 - 10 a10 --- 17 + " ENCODING: " 3 - 12 a10 --- Encoding String(-> Rule 8) 13 - 15 a3 18 + " This is a sample Abstract text " 3 - 79 a77 left num " which would start at column 3 !!! " items ".. next line is a blank line " to " " 17 + " Note: all abstract text lines " num " start at column 3 " items + num abstract lines 18 + + Blank Line (80 0x20 chars) 1 - 80 a80 --- 19 + + " END" 1 - 5 a5 --- 2 Data File 1) The IWF-FLAT Filesystem's Data File (*.DAT) is an Binary-file with a variable recordlength. 2) Each record starts with double precision floating point value which represent the time. The data values are single precision floating point values. The description of the different data columns in the data file are in the header file. 3) The used floating point format for the data file is the natural floating point format oft the platform. The used floating point format for the PC Flat File data format is the IEEE-P754 standard. For the VAX and ALPHA under OpenVMS only the D_floating format for the double precision floating point value is used. 4) The value for the time in the first column of the data file record are seconds since Jan. 1, 1965 PC-Platform: Floating Point Standard IEEE-P754 IEEE word sign S exponent E mantissa M Format wide wide range wide precision single- 32-bits 1 bit 8 bits 2^(+/-127) 23 bits 7 digits precision double- 64-bits 1 bit 11 bits 2^(+/-1023) 52 bits 16 digits precision M = 1 + m(1)*2^(-1) + m(2)*2^(-2) + m(3)*2^(-3) +... 1 < M < 2 Wide 1 32 64 Single precision: 4 byte s e7 e0 .m1 m23 Double precision: 8 byte s e10 e0 .m1 m52 Table 2: IWF-FLAT Header File with new Timestrings +----------------------------------------------------------------------------+ |0 1 2 3 4 5 6 7 | |1234567890123456789012345678901234567890123456789012345678901234567890123456| +----------------------------------------------------------------------------+ | name of header and data files: TESTFILE | | date files created: 1996-AUG-22 | | record length of data file, in bytes: 60 | | number of columns: 14 | | number of rows: 8760 | | flag for missing data: 1.00E+32 | | | | # name units source type loc | | ---------------------------------------------------------------------------| | 001 UT s Timeline - Epoch T 0 | | 002 Time_PB5-01 year I_4 [1] - Year R 8 | | 003 Time_PB5-02 day I_4 [2] - Day of Year (Jan 1 R 12 | | 004 Time_PB5-03 msec I_4 [3] - Elapsed millisecond R 16 | | 005 Traj_HI-01 AU R_4 [1] - R R 20 | | 006 Traj_HI-02 DEG R_4 [2] - lat R 24 | | 007 Traj_HI-03 DEG R_4 [3] - long R 28 | | 008 B_RTN_c-01 nT R_4 [1] - Br (RTN) R 32 | | 009 B_RTN_c-02 nT R_4 [2] - Bt (RTN) R 36 | | 010 B_RTN_c-03 nT R_4 [3] - Bn (RTN) R 40 | | 011 B_scalar nT R_4 - Scalar B R 44 | | 012 V km/sec R_4 - V R 48 | | 013 N no/cc R_4 - Np R 52 | | 014 temp K R_4 - Temp R 56 | | | | NOTES: | | | | Start time = 1977-JAN-01 00:00:00.000 | | End time = 1977-DEC-31 23:00:00.000 | | | | ABSTRACT | | ENCODING: PC | | Autoprocessed by CDF2FIWF from CDF-File: p10_77 | | -------> Skeleton-file: p10_77.SKT | | | | END | +----------------------------------------------------------------------------+ |1234567890123456789012345678901234567890123456789012345678901234567890123456| |0 1 2 3 4 5 6 7 | +----------------------------------------------------------------------------+ Table 3: IWF-FLAT Header File with old Timestrings +----------------------------------------------------------------------------+ |0 1 2 3 4 5 6 7 | |1234567890123456789012345678901234567890123456789012345678901234567890123456| +----------------------------------------------------------------------------+ | name of header and data files: TESTFILE | | date files created: 22-AUG-96 | | record length of data file, in bytes: 60 | | number of columns: 14 | | number of rows: 8760 | | flag for missing data: 1.00E+32 | | | | # name units source type loc | | ---------------------------------------------------------------------------| | 001 UT s Timeline - Epoch T 0 | | 002 Time_PB5-01 year I_4 [1] - Year R 8 | | 003 Time_PB5-02 day I_4 [2] - Day of Year (Jan 1 R 12 | | 004 Time_PB5-03 msec I_4 [3] - Elapsed millisecond R 16 | | 005 Traj_HI-01 AU R_4 [1] - R R 20 | | 006 Traj_HI-02 DEG R_4 [2] - lat R 24 | | 007 Traj_HI-03 DEG R_4 [3] - long R 28 | | 008 B_RTN_c-01 nT R_4 [1] - Br (RTN) R 32 | | 009 B_RTN_c-02 nT R_4 [2] - Bt (RTN) R 36 | | 010 B_RTN_c-03 nT R_4 [3] - Bn (RTN) R 40 | | 011 B_scalar nT R_4 - Scalar B R 44 | | 012 V km/sec R_4 - V R 48 | | 013 N no/cc R_4 - Np R 52 | | 014 temp K R_4 - Temp R 56 | | | | NOTES: | | | | Start time = 01-JAN-77 00:00:00.000 | | End time = 31-DEC-77 23:00:00.000 | | | | ABSTRACT | | ENCODING: PC | | Autoprocessed by CDF2FIWF from CDF-File: p10_77 | | -------> Skeleton-file: p10_77.SKT | | | | END | +----------------------------------------------------------------------------+ |1234567890123456789012345678901234567890123456789012345678901234567890123456| |0 1 2 3 4 5 6 7 | +----------------------------------------------------------------------------+ Projekt: Flat Flat-File Specification File: flatspec.doc Autor: Harald Ottacher Revision: 1 Datum: 18 Dec 1997 Seite: 1 von 6 Institut fur Weltraumforschung Abt. Experimentelle Weltraumforschung Osterreichische Akademie d. Wissenschaften A-8010 GRAZ / Steyrergasse 17-19 A-8010 GRAZ / Inffeldgasse 12