============================================================================
============================================================================
SOHO 1996
============================================================================
============================================================================
Reviewers: K. Battams, L. Feaga
General
-------
- Need specific provenance for comets in this archive.
- Strongly consider which data products are archived at PDS.
Is a fully-calibrated level 1.0 image required here? All current
publications use level 0.5 data. These data are well understood by
the scientific community and are already available in an archive.
M. Knight's work is the intermediate product for generating light curves.
Ultimately, archive the light curves. This is the most crucial data.
- Recommend taking level 1.0 data product and write a simple routine
to rescale for our work, then we don't need all the calibraiton products,
etc.
- For absolute completeness of the SOHO comet archive at PDS, consider
archiving:
o Comets discovered by SOHO SWAM instrument
o Data from observations of comets by the SOHO UBCS instrument
- Smaller images, 512x256, give a comet position is not on the image,
e.g. data/1996a/1996a2_c3_****.fit. These images should be identified.
This may occur on any image that does not have one dimension that is 1024.
May need to manually look at these images to see if position is outside
the frame. Affects comet post files, not astrometry files. After
1997-1998, this happens infrequently since nearly all data are full-frame.
See note for data/1996a2/ in L. Feaga's list appended below.
- Verify documentation correctly describes what is on the volume.
- Implement changes and corrections noted by K. Battams' (appended below).
Most of these items are duplicated in this section and the following
sections.
- Make corrections noted by L. Feaga (appended below). List includes
typos and inconsistencies in voldesc.cat, aareadme.txt, /document,
/calib, /index, and /data.
- Make corrections noted by A. Raugh (appended below, from "problems.acr").
Images
------
- Problems remain with the LASCO images in the PDS SOHO archive.
As they stand, they are not suitable for release/archiving. M. Knight and
K. Battams are still working on the processing issues (scaling for
dynamic range is not correct). C2 pretty good. C3 needs work because
comet was "erased" in nearly all of the of these images.
Labels
------
- Problem with image start and stop times:
o LASCO is accurate to only to one tenth of a second.
o Absolute times (SOHO clock) are accurate to better than 15 seconds.
Consider removing stop time? See Chapters 7 & 8 of the LASCO C3
calibration paper which discusses this topic in detail.
/document/comet_pos/sub-directory
---------------------------------
pos_1996.tab
- Missing constants are incorrect. Document claims they are of the form
-9999.99 but in table, just ----'s.
- Three different image times are given; reconsider the inclusion of
these time formats in favor of a single corrected start time
- Add the corrected exposure time to this file
- Missing "T" in ISO-formatted dates
pos_1996.txt
- Add note that post Aug 2004, the astrometry files contain extra information
in the first row (SOHO#, Camera, Discoverer)
- Clarify that the column "know" differentiates comet positions (know=0)
from star positions (know=1)
- Remove the comment "For images which do not have an identified comet,
these values are replaced to -9999.99 of the same width as the corresponding
field."? The astrometry file does not have entries with this value.
/document/comet_elements.txt
----------------------------
- Remove inconsistencies in spacing for the name of comet discoverer.
References
----------
- Review and add to list ""Calibration of the SOHO/LASCO C3 White Light
Coronagraph", Morrill et al, Solar Physics (2006) 233: 331-372.
calib/filter/*.tab
------------------
- Some values are negative, do not change to 0; note this in description/
index.lbl
---------
- Add column descriptions.
- Describe how SOLAR_NORTH_POLE_CLOCK_ANGLE is measured in an image.
______________________________________________________________________________
______________________________________________________________________________
Karl Battams' List of Corrections
---------------------------------
Images (Processing)
-------------------
- Problems remain with the LASCO images in the PDS SOHO archive. As they
stand, they are not suitable for release/archiving. There is ongoing
correspondence between myself and Matthew Knight to resolve the processing
issue(s), so unless requested (...??), they will not be discussed further
at this point (with the exception of a brief mention in the summary).
Labels
------
- Image "Start" and "Stop" time: Within the .lbl files for each image
(also the "keyword_glossary" file), the beginning and end time of each
individual exposure is given. (The "STOP_TIME" value is calculated by
"adding the exposure duration value to the start time as extracted from
the FITS file". Additionally, the exposure duration is corrected using the
exposure correction factor.) The "start","stop" and "exposure duration"
values are formatted to 1000ths of a second. It must be noted in the
documentation that after all appropriate corrections:
1. LASCO exposure durations are accurate to better than 1/10th second
2. Absolute times (SOHO clock) are accurate to better than 15 seconds
- The current scheme implies (to me, at least) a precision in time recording
that does not exist within any SOHO data product!
- Consider removing "STOP_TIME" value in data label files (and associated
documentation).
- Chapters 7 & 8 of the LASCO C3 Calibration paper (see below) discuss this
in detail.
Documentation
-------------
/document/comet_pos/ sub-directory
pos_1996.tab
- Missing constants are incorrect. The documentation claims they are of the
form -9999.99 (or variations thereof), but in the actual pos_1996.tab file,
they are just ----'s.
- Three different image times are given:
1. Corrected (calibrated) start time
2. Mid-time (corrected UTC time of s/c at midpoint of image)
3. "Old" time (uncorrected UTC time of s/c at midpoint of image)
- Bearing in mind the aforementioned uncertainties, reconsider the inclusion
of these three time formats in favor of a single (corrected) start time
_ Add the corrected exposure time to this file
pos_expl.txt
- Note that post Aug 2004, the astrometry files contain extra information
in the first row (SOHO#, Camera, Discoverer)
_ Maybe clarify that the column "know" differentiates comet positions
(know = 0) from star positions (know = 1)
- Regarding the statement "For images which do not have an identified
comet, these values are replaced by -999.999 of the same width as the
corresponding field." -- there are no such lines in the astrometry files
as they exist now. Only images with an identifiable comet were used for
SOHO astrometry measurements. If this "missing" data is to be subsequently
added to the astrometry files, it will need to be made very clear that the
new measurements were performed "after the fact", on a different
machine/OS, with a different operator (SOHO measurements can be
subjective...), and potentially with a different version of software.
/document/comet_elements.txt
- <picky> Inconsistency in spacing for the name of comet "discoverer".
Example: "X.Leprette" and "X. Leprette". This could cause issues for
someone who has written a whitespace-dependent program of some kind...
</picky>
References
----------
_ Review (and add to list) "Calibration of the SOHO/LASCO C3 White Light
Coronagraph", Morrill et al, Solar Physics (2006) 233: 331-372
Bottom line
-----------
- The processed images continue to be a problem and are still not suitable
for the PDS archive. Again, I will restate my previous recommendation that
the use of a 'Level-1 variant' data product be reconsidered in favor of
the well- established and widely available level-0.5 data (upon which all
current SOHO comet data and papers are based). Additionally/alternatively,
consider the use of the easily available NRL LASCO Level-1data product,
which would only required a conversion to the desired units.
- Aside from the images, and the few notes described earlier, the SOHO
data archive appears to be in very good shape. Developers should keep
in mind the possible future addition of (the currently
incomplete/unpublished) SOHO comet photometry data. For absolute
completeness of the SOHO comet dataset, additional consideration
should be given to including:
1. comets discovered by the SOHO SWAN instrument, and
2. data from observations of comets by the SOHO UVCS instrument
______________________________________________________________________________
______________________________________________________________________________
Lori Feaga's List of Corrections
--------------------------------
voldesc.cat
- MISSION_CATALOG entry not correct, the file name was shortened.
aareadme.txt
- Missing comet_elements.txt and eventual label should data directories be
listed separately?
- Change Puneet as contact person
document/docinfo.txt
- Be consistent in "PNG", "png", "pdf" as all caps or all lower case
document/capture_dates.txt
- Missing a lot of files that I believe are downloaded from LASCO:
o document/spacecraft_info/time_and_rotation_axis.txt
o document/calib_docs/filter/ 13 plots
o document/calib_docs/quantum_eff/ 2 plots
o document/sun_position/sc_body_axis_avg_roll.tab
o document/sun_position/sunpos.txt
o document/comet_pos/soho*.txt (26 files)
o document/telemetry_stats/tm_plot_1996_* (12 files)
o calib/filter/*.tab (11 files)
o calib/quantum_eff/*.tab (2 files)
o calib/vignetting/*.fit (3 files)
document/comet_elements.txt
- M3 and M2 are "out of order", make sure the rows have the correct data.
YES, Karl said in perihelion order
document/keyword_glossary.txt
- Compared with an actual data label, the following keywords are not defined
here, should they be included?
o PDS_VERSION_ID
o RECORD_TYPE
o RECORD_BYTES
o FILE_RECORDS
document/inst_docs/handbook.txt
- Spelling error from old liens: near the end of the file, "accessable"
should be "accessible"
document/inst_docs/temperature/*.png files
- Not of greatest quality
document/comet_pos/astrometry.txt
- "pos_1996.*" should just be "pos_1996.tab"
document/comet_pos/pos_1996.tab and .lbl
- Missing data values are "-----" in the .tab and are specified as "-999.99"
in .lbl
- Space between date and time, should be one single column w/ "T"
- Impose format for values in table especially sun_comet_dist and
spacecraft_comet_dist
- Label values for RECORD_ BYTES, FILE_RECORDS, ROW_BYTES, ROWS,
BYTES, FORMAT, MINIMUM, MAXIMUM, START_BYTE need to be checked/corrected
document/comet_pos/soho*.txt
- All 26 files need labels
document/sun_position
- Some bad quality images:
o pes_a.png
o bs.png
o 960101sc_roll.png
calib/filter/*.tab
- Small negative numbers (-1<#<0) in the tables for transmittance values,
but in the label, there is a note that these run from 0-100, so note
there is noise
index/index.tab
- Many columns do not have correct description and fix all BYTES,
START_BYTE, FORMAT, etc.
data/1996*/*.lbl
- Check FILE_RECORDS value
data/1996a2/
- All data are 512x256, comet is listed as being in a y pixel out of
the appropriate range in the pos_1996.tab
______________________________________________________________________________
______________________________________________________________________________
Anne Raugh's List of Corrections (from the problems.acr file)
-------------------------------------------------------------
30 March 2006
=============
Known problems with SOHO 1996 update (PDS Standards)
GENERAL
o wrong DSID throughout
aareadme.txt
o Refers to this as separate data set.
o Sentence describing the DATA/ directory makes no sense
o Does not mention document/keyword_glossary
voldesc.cat
o Does not list target catalog files (these are missing)
calib/vignetting
o All files missing labels
catalog/
o Missing all TARGET files
catalog/dataset.cat
o Syntax errors
o Missing DATA_SET_MISSION
catalog/lasco.cat
o Syntax errors
catalog/reference.cat
o Syntax errors
o Incomplete REFERENCE_TEXT for BIESECKERETAL2002
o Incorrect REFERENCE_TEXT for FLECKETAL1995
document/
o Unlabelled files: comet_elements.col, comet_elements.txt
document/docinfo.txt
o Information in here does not match file list in the actual directory.
document/capture_dates.lbl
o RECORD_TYPE, RECORD_BYTES and FILE_RECORDS are incorrect
document/keyword_glossary.lbl
o Invalid RECORD_TYPE
o Incorrect RECORD_BYTES
document/calib_docs/exp_fac/expfac.lbl
o RECORD_BYTES, FILE_RECORDS, RECORD_TYPE incorrect
document/*
o SAME ERRORS AS ABOVE FOUND IN ESSENTIALLY ALL LABELS IN THE DOCUMENT/ TREE
index/index.lbl
o ROWS, ROW_BYTES, RECORD_BYTES, FILE_RECORDS all incorrect.
o index.tab is improperly formatted.
o Column definitions don't work
============================================================================
============================================================================
SOHO 2000
============================================================================
============================================================================
o Some of the LASCO calibration files used were out of date, or will be
shortly. These are now stable, but the calibration will need to be
rerun. There is an IDL library (SolarSoft) available.
o Add a reference to the SolarSoft IDL library with a descriptive note to
the data set description.
o Add a table with orbital elements and family membership (if that makes
sense) for the comets.
o Consider whether this should be one complete data set rather than one
for each year. (This data set will require a special interface to select
data by family.)
o Consider adding Doug Beisecker's published photometry to this data set.
o Include SOHO atrometry files.
o Consider who to credit in the label and how for the people involved in
discovering the comet in the image.
o Consider what to do about "X" comets - uncertain detections (fewer than
five reasonably accurate positions) for which only minimal coordinates
are available.
o Make corrections indicated in K. Battams presentation "Documentation"
slides (appended below).
o Separate vignetting functions for the different filters are not necessary.
One full-resolution function for each band is sufficient.
o Make corrections included in Lori's presentation (appended below).
o Make sure positions in the astrometry table do correspond to comet
positions for faint comets.
o Consider adding a family subdirectory under the "data/" directory.
o Correct typos noted by L. Feaga (on paper in Anne's office).
o Make corrections noted by T. Farnham (appended below).
o Make corrections noted by S. Adams (appended below).
______________________________________________________________________________
______________________________________________________________________________
Karl Battam's Documentation Corrections
---------------------------------------
In the catalog/dataset.cat file:
"Data Set Overview"
- "Nearly all of the SOHO discovered..." should read "Approximately 95% of
the SOHO-discovered..."
- Mention that synoptic programs are full-resolution (1024x1024).
- Note increased LASCO telemetry during 'keyholes'.
- Note reduced FOV during the early part of the mission.
"Processing"
- Level-0.5 and/or Level-1 processing should perhaps be explained with
reference to SolarSoft (software library).
- Clarify the difference processing levels. [Details are on the presentation
slides. See PDS_SOHOComet.pdf -AR]
"Calib"
- Note the ZP's might change (calibration papers from NRL/Marseille due
for publication)
"Processing" (cont...)
- "vignetting corrections have remained constant throughout the mission"
o rephrase
o not true for c3 (post-interrupt)
______________________________________________________________________________
______________________________________________________________________________
Lori Feaga's List of Corrections
--------------------------------
SOHO/calib/filter/*.lbl
- In the DESCRIPTION of transmittance (column 2), range should be 0-100.
- c2_ha.*, c3_ha.* and c3_i75.*
units for wavelength are inconsistent from label to table file
SOHO/catalog/dataset.cat
- In "Processing" is this:
"vignetting corrections have remained constant through the mission"
yet there are versions 1 and 2 for some of the filters and some filters
have multiple files (different filenames).
- In "Data":
"There images are listed in the reduction.log file found in the
documents directory."
but there is no such file.
SOHO/document/capture_dates.txt
- There is an asterisk after the listing for handbook.txt. Why?
- For the list of files in document/calib_docs/exp_fac, only 24 of
192 files were actually available.
SOHO/document/calib_docs/expfac.txt
- In the description of the exposure factor plots, several colars are
mentioned to distinguish the method of calculating the exposure factor,
but I only saw red and green points.
[Note: Using color to encode important information is strongly
discouraged. There are quite a few color-blind astronomers
out there. -AR.]
SOHO/document/calib_docs/filter
- Missing *.tab files as described in filter.txt
- Files reside in /calib/filter/, needs a pointer to that directory.
SOHO/document/calib_docs/filter/c2_all.png
- Plot of filter transmission has 2 orange filter curves with no
explanation.
SOHO/document/calib_docs/filter/c3_all.png
- Doesn't include the transmission curve of the polatization I-75 filter.
SOHO/document/calib_docs/quantum_eff/qe.txt
- ASCII tables are in /calib/quantum_eff/ directory, not here.
- c3_qe.ps plot is also available, but not listed.
SOHO/document/calib_docs/vignetting/vign.txt
- Vignetting images are found in the /calib/vignetting directory, a pointer
to this directory should be added.
- Missing file: c3_cl_vig_951201_v03.fit
SOHO/document/comet_pos/pos_expl.txt
- "values are replaced by dashes of the same width as the corresponding
field" isn't true - they're replaced by -999.999 of same width.
SOHO/index/index.lbl
- Units need to be added to columns 5, 6, 7 and 11.
Liens from previous reviews still unresolved:
/calib/filter
o Should there be a unit mentioned for transmittance, i.e., percent?
o Pointer still needed to the PNG files in the label files.
/calib/quantum_eff
o Should there be a unit for quantum efficiency?
o Pointer still needed to the PNG and PS files in the label files
/document/handbook
o Spelling error remains, "accessable"
Typos and grammatical errors supplied on paper.
______________________________________________________________________________
______________________________________________________________________________
Tony Farnham's Corrections
--------------------------
In AAREADME.TXT
Contact information for Puneet should be changed.
In EXPFAC.TXT file:
Launches right into an explanation about what is done. Should have a
sentence (or reference to another file) that describes why it is being done.
In POS_EXPL.TXT
States that if the comet is not identified in the frame, the phase angle etc
are replaced by dashes. In fact they are replaced by the missing data value
______________________________________________________________________________
______________________________________________________________________________
Steve Adams' Corrections
------------------------
1. root directory
A. dataset.html
1. delete this file on final archive
B. aareadme.txt
1. Directory tree data directory missnamed.
C. voldesc.cat
1. Files listed in CATALOG object do not match what is
on the volume.
2. calib directory
A. Nothing (documentation) on the volume that explains how to
conduct calibration with this data. Fifty or a hundred
years from now, long after we have retired, how will anyone
understand what is here and how to utilize it?
3. catalog directory
A. catinfo.txt
1. Files referenced in this document and what is on the
volume don't match each other.
B. dataset.cat
1. Delete comment after DATA_SET_RELEASE_DATE, not
needed and makes ingestion fail
2. CITATION_DESC =
Move opening quote to the same line as the keyword.
3. missing required object, (i.e., DATA_SET_MISSION)
C. lasco.cat
1. no comments
D. reference.cat
1. List all authors in citation
a. BIESECKERETAL2002
b. BRUECKNERETAL1995
E. soho.cat
1. no comments
F. solar_and_heliospheric_obser.cat
1. no comments
4. document sub-directory
A. docinfo.txt
1. /document/calib/ directory and its sub-directories
listed are not on the volume. See 2.1.
2. /document/comet_pos/ directory listed is missing on
this volume.
5. index sub-directory
A. index.tab
1. index.tab refers to a data directory. There is no
data directory on this volume. Perhaps your "sample"
directory should be named "data"?
6. index sub-directory
A. no comments
7. sample sub-directory (data sub-directory?)
A. no comments