pds4_tools.reader.core module

Functions

pds4_read(filename[, quiet, lazy_load, ...])

Reads PDS4 compliant data into a StructureList.

read_structures(label, label_filename[, ...])

Reads PDS4 data structures described in label into a list of Structure's.

read_byte_data(data_filename, start_byte, ...)

Reads byte data from specified start byte to specified end byte.

Details

pds4_tools.read()

An alias of pds4_read().

pds4_read(filename, quiet=False, lazy_load=False, no_scale=False, decode_strings=True)[source]

Reads PDS4 compliant data into a StructureList.

Given a PDS4 label, reads the PDS4 data described in the label and associated label meta data into a StructureList, with each PDS4 data structure (e.g. Array_2D, Table_Binary, etc) as its own Structure. By default all data structures described in the label are immediately read into memory.

Parameters
filenamestr or unicode

The filename, including full or relative path, or a remote URL to the PDS4 label describing the data.

quietbool, int or str, optional

If True, suppresses all info/warnings from being output to stdout. Supports log-level style options for more fine grained control. Defaults to False.

lazy_loadbool, optional

If True, then the data of each PDS4 data structure will not be read-in to memory until the first attempt to access it. Additionally, for remote URLs, the data is not downloaded until first access. Defaults to False.

no_scalebool, optional

If True, returned data will be exactly as written in the data file, ignoring offset or scaling values. Defaults to False.

decode_stringsbool, optional

If True, strings data types contained in the returned data will be decoded to the a unicode in Python 2, and to the str type in Python 3. If False, leaves string types as byte strings. Defaults to True.

Returns
StructureList

Contains PDS4 data Structure’s, each of which contains the data, the meta data and the label portion describing that data structure. StructureList can be treated/accessed/used like a dict or list.

Notes

Python 2 v. Python 3: Non-data strings (label, meta data, etc) in Python 2 will be decoded to unicode and in Python 3 they will be decoded to str. The return type of all data strings is controlled by decode_strings.

Remote URLs are downloaded into an on-disk cache which is cleared on Python interpreter exit.

Examples

Below we document how to read data described by an example label which has two data structures, an Array_2D_Image and a Table_Binary. An outline of the label, including the array and a table with 3 fields, is given.

>>> # Local file
>>> struct_list = pds4_tools.read('/path/to/Example_Label.xml')

>>> # Remote URL
>>> struct_list = pds4_tools.read('https://url.com/Example_Label.xml')

Example Label Outline:

Array_2D_Image: unnamed
Table_Binary: Observations
    Field: order
    Field: wavelength
    Group: unnamed
        Field: pos_vector

All below documentation assumes that the above outlined label, containing an array that does not have a name indicated in the label, and a table that has the name ‘Observations’ with 3 fields as shown, has been read-in.

Accessing Example Structures:

To access the data structures in StructureList, which is returned by pds4_read(), you may use any combination of dict-like or list-like access.

>>> unnamed_array = struct_list[0]
>>>              or struct_list['ARRAY_0']

>>> obs_table = struct_list[1]
>>>          or struct_list['Observations']

Label or Structure Overview:

To see a summary of the data structures, which for Arrays shows the type and dimensions of the array, and for Tables shows the type and number of fields, you may use the StructureList.info() method. Calling Structure.info() on a specific Structure instead will provide a more detailed summary, including all Fields for a table.

>>> struct_list.info()
>>> unnamed_array.info()
>>> obs_table.info()

Accessing Example Label data:

To access the read-in data, as an array-like (subclass of ndarray), you can use the data attribute for a PDS4 Array data structure, or list-like and the field() method to access a field for a table.

>>> # PDS4 Arrays
>>> unnamed_array.data

>>> # PDS4 Table fields
>>> obs_table['wavelength']
>>> obs_table.field('wavelength')

>>> # PDS4 Table records
>>> obs_table[0:1000]

Accessing Example Label meta data:

You can access all meta data in the label for a given PDS4 data structure or field via the OrderedDict meta_data attribute. The below examples use the ‘description’ element.

>>> unnamed_array.meta_data['description']

>>> obs_table.field('wavelength').meta_data['description']
>>> obs_table.field('pos_vector').meta_data['description']

Accessing Example Label:

The XML for a label is also accessible via the label attribute, either the entire label or for each PDS4 data structure.

Entire label:
>>> struct_list.label
Part of label describing Observations table:
>>> struct_list['Observations'].label
>>> struct_list[1].label

The returned object is similar to an ElementTree instance. It is searchable via Label.find() and Label.findall() methods and XPATH. Consult ElementTree manual for more details. For example,

>>> struct_list.label.findall('.//disp:Display_Settings')

Will find all elements in the entire label named ‘Display_Settings’ which are in the ‘disp’ prefix’s namespace. You can additionally use the Label.to_dict() and Label.to_string() methods.

read_structures(label, label_filename, lazy_load=False, no_scale=False, decode_strings=False)[source]

Reads PDS4 data structures described in label into a list of Structure’s.

Parameters
labelLabel

Entire label of a PDS4 product.

label_filenamestr or unicode

The filename, including full or relative path, of the label.

lazy_loadbool, optional

If True, does not read-in data of each data structure until the first attempt to access it. Defaults to False.

no_scalebool, optional

If True, returned data will not be adjusted according to the offset and scaling factor. Defaults to False.

decode_stringsbool, optional

If True, strings data types contained in the returned data will be decoded to the unicode type in Python 2, and to the str type in Python 3. If false, leaves string types as byte strings. Defaults to False.

Returns
list[Structure]

The PDS4 data Structure’s described in the label.

read_byte_data(data_filename, start_byte, stop_byte)[source]

Reads byte data from specified start byte to specified end byte.

Parameters
data_filenamestr or unicode

Filename, including the full path, of the data file that contains the data for this structure.

start_byteint

The start byte from which to begin reading.

stop_byteint

The start byte at which to stop reading. May be -1, to specify read until EOF.

Returns
str or bytes

The read-in byte data.

Notes

This function will attempt to read until stop_byte, but if that is past the end-of-file it will silently read until end-of-file only.