Common DFOS tools:

dfos = Data Flow Operations System, the common tool set for DFO
*make printable

Calibration memory and AB creation

This page deals with the operational issue of how to fill the calibration memory properly, and with the interplay of createAB and that memory.

The calibration memory is the list of all calibration product files currently available for association. It is used and maintained by the association tool createAB. There are two flavours of calibration products: virtual calibrations and real calibrations.

The two flavours of calibration products are collected in two different pools:

In either case the calibration memory is only a subset of the available dates (for performance reasons). The date range is configured in config.createAB.

The calibration memory is managed by DFOS in the following way:

Virtual calibrations. In routine operations the virtual calibrations, once they are certified and become real, are automatically removed from VCAL and enter MCAL, until they finally become outdated, are replaced by similar, new ones and disappear completely from the calibration memory. The QC scientist usually does not need to take care about these directories, except for fine tuning the memory depth parameters (typically 5-10 days).

There are two exceptional cases when entries in VCAL are removed, or kept, in a user-controlled way:

Criteria for removing or not are:

Cases 1 and 2 are a case-by-case decision which is done as part of certifyProducts. Case 3 is rule-based and hence is configured, as part of moveProducts.

Explicit deletion of virtual calibrations. Only when there is the need to create ABs outside of the daily workflow (e.g. for a historical set of June data while DFO date is already in December), some explicit actions are necessary for calibration memory maintenance:

Then MCAL is correctly filled with June data, and no virtual calibs from December can be associated. If you would call createAB without explicitly specifying -D <reference_date>, it would scan and find the December data instead!

If you want to switch back to December, remove again the files in $DFO_CAL_DIR/VCAL/, and call createAB again in the standard way.

The current content of VCAL and MCAL is displayed in the dfoMonitor output.

Last update: April 26, 2021 by rhanusch