New Horizons sub-frame WINdow SUMMary Background: The New Horizons (NH) Spacecraft saves high-speed instrument data in the on-board Solid State Recorder (SSR). High-speed data are never telemetered to Earth, but instead either are packetized without compression, or are compressed using either a lossless or lossy algorithm into low-speed data. Low-speed data can be scheduled to be telemetered back to Earth. When that telemetry reaches a Science Operations Center (SOC), they are decommutated and converted back into a data format suitable for the original instrument and detector. For some imaging instruments (LORRI and RALPH/LEISA and RALPH/MVIC), LOSSLESS data may be sub-framed i.e. a rectangular window that is a subset of the complete observation is compressed and telemetered to Earth. For each single observation, up to eight independent sub-frames (a.k.a. windows) may be defined. The pipeline processes each window independently, because each one comes in a single packet set, producing one FITS file per window. A packet set is a sequence of packets: a start packet; zero or more intermediate packets; a stop packet. An alternate packet set is a single packet that is both a start and stop packet. Before March, 2015, each window ended up in a separate FITS file; so only one set of windowing parameters was needed in the FITS header to keep track of the original compression settings. As of March, 2015, the pipeline started to merge each window (packet set) with previously processed windows from the same observation. The implementation keeps a running history of the merge process in a BINTABLE in the FITS file. The history contains three items: 1) the merged data 1.1) non-missing values in new data overwrite old data 2) a history of data keys in the PDU header 3) history mismatches detected when merging data, as a BINTABLE with EXTNAME = 'WINDOWS_MISMATCHES' 3.1) the location of each mismatch 3.2) the numerical difference of each mismatch, new-old 3.3) the data key of each mismatch (from the new data) Each data key comprises seven items, normally stored in each FITS PDU header using the following keywords: WINDOWX - Integer, left-edge offset to sub-frame, compr. params X_1 through X_8 WINDOWY - Integer, bottom-edge offset to sub-frame, compr. params Y_1 through Y_8 WINDOWW - Integer, sub-frame width, compr. params X1_len through X8_len WINDOWH - Integer, sub-frame height, compr. params Y1_len through Y8_len APID - String, Application Process IDentifier (e.g. '0x630') GRT - Float, Ground-Received Time of packet(s) ARCHDATE - String, packet telemetry filename, e.g. '2008/290/2008290.tlm' The data key history, for each packet, set is stored as FITS keywords, in the PDU header, similar to the following: WINX_000 - Original WINDOWX value WINY_000 - Original WINDOWY value WINW_000 - Original WINDOWW value WINH_000 - Original WINDOWH value APID_000 - Original APID value GRT__000 - Original GRT value ARCH_000 - Original ARCHDATE value - the common _000 suffix indicates these values all came from the same packet set. Later suffixes (_001, _002, _003, etc.) indicate separate packet sets. - N.B duplicate data keys are removed; that is, each data key will appear only once in the header - N.B the _nnn suffix does NOT indicate the order in which the packets sets were processed; rather the sub-frame parameters are sorted, and re-sorted each time a new packet set is merged with the data from previous packet sets. In addition to the summary history in the PDU FITS header, the merging implementation creates a BINTABLE FITS eXTENSION of the history of the window processing, plus the locations of any mismatches in the pixel values loaded. Each window added to the data will generate at least one entry (row) in the table, with a location using -999 as the offsets and zero as the delta. There will be ten or eleven columns for 2D or 3D data respectively: MMNAXIS1OFFSET offset along NAXIS1 of data MMNAXIS2OFFSET offset along NAXIS2 of data MMNAXIS3OFFSET offset along NAXIS3 of data (for 3D data only) DELTA delta from current pixel value to pixel being merged WINDOWX Window parameter from data being merged WINDOWY Window parameter from data being merged WINDOWW Window parameter from data being merged WINDOWH Window parameter from data being merged APID Window parameter from data being merged GRT Window parameter from data being merged ARCHDATE Window parameter from data being merged N.B. Each merged window will have at least one entry with the OFFSETs set to -999 and DELTA set to zero. Note that such entries are a superset of the data key list in the PDU header, and that these entries with -999 as the offsets do not indicate any anomalies in the merge process, and most if not all files will have multiple entries with the -999 values.