PIPELINES AT ESO

This page provides information about pipeline processing and data types.

Raw data are selected, associated and inserted into a reduction mechanism which produces calibration products, science products and quality control information. This mechanism is the data processing pipeline. There is one such pipeline for each VLT and VLTI instrument.

Find general information about ESO reduction pipelines here.

The main functionalities of the pipelines are:

• create master calibration data,
• reduce science frames,
• extract QC information from the data.

QC Garching creates master calibration data from all raw calibration data. The raw data are stored in the ESO Archive and are public. They are quality-checked and used for data reduction and for trending.

Before October 2011 QC Garching processed science data, using the best available, quality-checked master calibration data. As of October 2011 this service is not offered anymore.

There are two instances of the data reduction pipelines:

• at the instrument workstation on Paranal, running in automatic mode,
• at HQ Garching, run by the Quality Control Group in the optimized mode.

The automatic mode is used for quick look purposes and for on-site quality control. It processes all raw data sequentially, as they arrive from the instrument. If calibration products ("master calibrations") are required for processing science data, these are taken from a database with standard, pre-manufactured calibration products. The automatic mode is not tuned to obtain the best possible results.

The optimized mode is the mode, which uses all data of a night, including the daytime calibrations. The calibration data are sorted and grouped according to their dependencies. Master calibration data are created. Their quality is checked.

The reduction of FLAMES/UVES data at QC Garching is done in a semi-automatic manner: raw data are selected and inserted into a reduction mechanism which produces

• calibration products and
• reduced science data.

This mechanism is the UVES-Fibre reduction pipeline.

The pipeline Data Reduction Software (DRS) has been implemented around the UVES context of MIDAS (which is an evolved version of the ECHELLE context). It is designed to automatically reduce point source object spectra from raw frames to order extraction and final order merging.

The FLAMES/UVES pipeline is public now. Information about downloading it can be found here.

Descriptions of calibration processing, science reduction, and recipes will be available in the future.

There are two instances of the FLAMES/UVES reduction pipeline:

• at the KUEYEN telescope on Paranal, running in automatic mode,
• at HQ Garching run by the Quality Control Team in the optimized mode.

The automatic mode is used for quick look purposes and for on-site quality control. It processes all raw data sequentially, i.e. as they arrive from the instrument. It produces calibration products and reduced science data, but will usually fail to obtain the best possible results. It provides results for all standard and many non-standard UVES settings.

The optimized mode is the mode, which becomes possible when all data of a night have been collected, including the daytime calibrations. Then the calibration data are sorted and assessed independently of their timestamp. The best possible master calibration data are created. Their quality is checked. They are finally applied to the science data of a night. This is the way all calibration data in supported modes are reduced by QC Garching, as are all science data from Service Mode programmes.

The following standard settings are supported by the pipeline:

The main functionalities of the FLAMES/UVES pipeline are:

• create master calibration data,
• reduce science frames,
• provide QC1 parameters.

QC Garching creates

• master calibration data from raw calibration data taken in standard settings, both for Visitor Mode and Service Mode nights. The raw data are stored in the ESO Archive and are public. They are assessed and selected in terms of their quality and used for QC1 checks and trending;
• reduced science data for Service Mode nights. For SM programmes, QC Garching runs the pipeline on all science raw data taken in FLAMES/UVES standard settings, using the best available, quality-checked master calibration data.

Programmes performed in Service Mode receive a set of CD ROMs (or DVDs) which contains

• all raw science data,
• reduced science data,
• suitable calibration data (both raw and master),
• listings and logs,
• QC1 reports.

Raw FLAMES/UVES frames come in the following CCD format:

Only 1x1 binning is supported.

CCD formats are described here.

Master calibration files (master BIAS, master SFLAT) have the prescan and overscan regions chopped off. Master files for the red arm come in twins for either of the red CCD. Usually the CCDs are called REDU = red upper and REDL = red lower. All master files are rotated by 90 degrees. The red masters are in addition flipped. Red master files have the format 4096x2048 px.

Raw calibration files, raw science files, and master calibration files, have the following approximate sizes in MB: