PIPELINES AT ESO

This page provides information about pipeline processing and data types.

Raw data are selected, associated and inserted into a reduction mechanism which produces calibration products, science products and quality control information. This mechanism is the data processing pipeline. There is one such pipeline for each VLT and VLTI instrument.

Find general information about ESO reduction pipelines here.

The main functionalities of the pipelines are:

• create master calibration data,
• reduce science frames,
• extract QC information from the data.

QC Garching creates master calibration data from all raw calibration data. The raw data are stored in the ESO Archive and are public. They are quality-checked and used for data reduction and for trending.

Before October 2011 QC Garching processed science data, using the best available, quality-checked master calibration data. As of October 2011 this service is not offered anymore.

There are two instances of the data reduction pipelines:

• at the instrument workstation on Paranal, running in automatic mode,
• at HQ Garching, run by the Quality Control Group in the optimized mode.

The automatic mode is used for quick look purposes and for on-site quality control. It processes all raw data sequentially, as they arrive from the instrument. If calibration products ("master calibrations") are required for processing science data, these are taken from a database with standard, pre-manufactured calibration products. The automatic mode is not tuned to obtain the best possible results.

The optimized mode is the mode, which uses all data of a night, including the daytime calibrations. The calibration data are sorted and grouped according to their dependencies. Master calibration data are created. Their quality is checked.

The OmegaCAM data processing pipeline is operational since June 2011 (during OmegaCAM commissioning).

The OmegaCAM pipeline is not yet publicly available, but may become so in the future. If so, it will be found at this link: ESO pipelines.
The OmegaCAM pipeline consists of a number of recipes based on the ESO Common Pipeline Library (CPL).

All of OmegaCAM calibrations are acquired and reduced according the OmegaCAM calibration plan. The calibration cascade is given below:

OmegaCAM's only observing mode is imaging.

The pipeline supports all types of calibrations of the calibration plan. There is also a science recipe which reduces science images to the coadded (stacked) level.

Raw data. The OmegaCAM detector mosaic is equipped with 32 e2v blue-sensitve 2048 x 4200 pixel detectors. OmegaCAM raw frames come as fits frames with one primary header and 32 extensions (i.e. MEF), each of which consists of a header containing detector specific information and pixel data.

Products. Pipeline products have can have two formats:

• Fits frames with one primary header and 16 extensions each one consisting of an extension header and pixel data.
• Fits files with one primary header and 16 extensions each one consisting of an extension header and a binary table (e.g. zeropoints and photometric fits tables)