This page provides information about pipeline processing and data types.

Raw data are selected, associated and inserted into a reduction mechanism which produces calibration products, science products and quality control information. This mechanism is the data processing pipeline. There is one such pipeline for each VLT and VLTI instrument.

Find general information about ESO reduction pipelines here.

The main functionalities of the pipelines are:

• create master calibration data,
• reduce science frames,
• extract QC information from the data.

QC Garching creates

• master calibration data from raw calibration data, both for Visitor Mode and Service Mode runs . The raw data are stored in the ESO Archive and are public. They are assessed and selected in terms of their quality and used for QC checks and for trending;
• reduced science data (currently only for Service Mode runs) . For SM programmes, QC Garching runs the pipeline on all science raw data, using the best available, quality-checked master calibration data.

Science data are processed by the pipeline with the best available calibration data. Please note that ESO is not assuming any responsibility in respect to the usefulness of the reduced data. The adopted reduction strategy may not be suitable for the scientific purpose of the observations.

OPERATIONS

There are two instances of the data reduction pipelines:

• at the instrument workstation on Paranal, running in automatic mode,
• at HQ Garching, run by the Quality Control Group in the optimized mode.

The automatic mode is used for quick look purposes and for on-site quality control. It processes all raw data sequentially, as they arrive from the instrument. If calibration products ("master calibrations") are required for processing science data, these are taken from a database with standard, pre-manufactured calibration products. The automatic mode is not tuned to obtain the best possible results.

The optimized mode is the mode, which uses all data of a night, including the daytime calibrations. The calibration data are sorted and grouped according to their dependencies. Master calibration data are created. Their quality is checked. The certified, closest-in-time products are finally associated to the science data of a night. This is the way all calibration data in supported modes are processed by QC Garching, as are all science data from Service Mode programmes.

top SINFONI INFORMATION

The latest version of the SINFONI pipeline is publicly available. The pipeline and its "Users Manual" can be downloaded from here.

The complete reduction of SINFONI data is done at QC in Garching in a semi-automatic manner. Raw data is selected and combined into its associated types and then processed by the pipeline to produce calibration produced and reduced science data. The products and their associated pipeline recipes can are explained in the following links:

calibration products	science products
calibration recipes	science recipes

The SINFONI pipeline consists of a number of recipes based on the ESO Common Pipeline Library (CPL).

A useful reference for getting started with SINFONI data reduction is the SINFONI data reduction cookbook (NOTE: this link opens as a PDF file).

top SUPPORTED MODES AND SETTINGS

Virtually all sinfoni observational modes and techniques are supported by instrument operations and by the pipeline processing. The only exception to this is the use of the sinfoni sky spider and spectral dithering.

The sinfoni pipeline is constantly being improved on to provide the best possible data reduction. The most recent public release, as well as complete documentation can be found at: http://www.eso.org/pipelines

Current pipeline projects include extinction values and zeropoints from the nightly telluric standards, as well as a correction for the atmospheric dispersion within the sinfoni data cubes (i.e. atmospheric dispersion can result in a small shift in the position of a source as a function of wavelength with the data cube. This is most apparent in the largest wavelength coverage and the smallest pixel scale: H+K and 25 mas.

To process the data of Period 84 the sinfoni pipeline version 2.0.8 is being used.

The following is a summary of previous releases of the sinfoni pipeline. Pipeline versions given with a date are those versions used to process data within Garching Quality Control.

2009-09-15 (P83) version 2.0.6/2.08: Bug fixes including repair of missing sinfo_irplib_cpl_wrp.h

version 2.0.5: Bug fixes and repair of some compiler warnings. Fix of some unit test failures.

version 2.0.4: Increased mosaic_max_size default to 14196. This is relevant for science OB's made up of large numbers of individual exposures. This parameter is set in the pipeline configuration file sinfo_rec_jitter.rc and is called sinfoni.objnod.mosaic_max_size. The maximum allowed size for cube mosaics is set as 100 * sinfoni.objnod.mosaic_max_size. The maximum allowable size will strongly depend on the amount of RAM available in your computer.

version 2.0.3: Preliminary implementation of the routine sinfo_utl_efficiency (still to be fully verified) to compute instrument efficiency using telluric standard stars.

version 2.0.2: Improvement in the speed of the kappa-sigma clipping (in the sinfo_rec_jitter recipe).

2009-04-19 (P83) version 2.0.1: Fixed problems in the data reduction of fibre-fed PSF frames. Now returns cpl_error_code sinfo_baryvel.h and sinfo_baryvel.c. Will now properly exit if baryvel fails: sinfo_new_cubes_coadd.c

2009-04-06 (P83) version 2.0.0: Fixed segmentation fault (apparently only appears on ubuntu installations) due to improper array sizing. Now set to -100 QC.STREHL if by chance the value is NAN.

2008-10-17 (P82) version 1.9.8: From this version of the sinfoni pipeline onward, it is now possible to correct the horizontal lines in sinfoni raw frames. These occur during read-out and arise because the hardware only removes the bias offset using an "average" in the overscan area. The pipeline implements the IDL routine described on the final page of the sinfoni cookbook (WARNING: This link opens as a PDF file).

2008-08-01 (P81) version 1.9.5

2008-02-14 (P80) version 1.9.3

2008-01-10 (P80) version 1.8.8

2008-10-03 (P80) version 1.8.4

2007-09-26 (P78) version 1.8.3: Primarily an improvement to the distortion and linearity recipes (also pipeline products did not have the MJD-OBS of the first fits file in the AB)

2007-04-29 (P78) version 1.7.1/1.7.2: Minor bug fixes (repair of OBSJIT pipeline products that had no PRO.CATG headers. . . these now have PRO.CATG=SKY_DUMMY).

2007-02-03 (P77) version 1.7.0: Primarily installed for a more robust OH sky subtraction and an improved strehl ratio calculator.

2007-01-01 (P77) version 1.6.0: Implements a more robust distortion recipe.

2006-11-24 (P75) version 1.5.1: Improvements in the distortion recipe.

2006-08-27 version 1.4.0

2006-07-01 version 1.3.8: An improvement of the "staring mode" data reduction has been implemented

2006-06-06 version 1.3.7: The distortion recipe (sinfo_rec_distortion) and the determination of the slit position have been improved

2006-04-01 version 1.3.1

2006-02-26 version 1.3.0

2006-02-20 version 1.2.8

2005-07-20 version 1.0.8: (for science from 2005-06-15 onward, for calibrations of 2005-06-24 onward). The size of the science COADD products has been minimized. The SKY_MED product is corrected for optical distortion and flat-field. The median collapsed cube products have WCS included.

2005-07-11 version 1.0.7: (for science of 2005-06-14 onward, for calibrations of 2005-06-19 onward). The first frame of the stack is no longer necessarily in the center of the COADD product. Several new QC parameters introduced (linearity). Standard star and PSF products have been modified.

2005-06-17 version 1.0.6: (for science of 2004-04-28 onward, for calibrations of 2005-05-23 onward)

2005-05-24 version 1.0.5: (for science data of ~2005-04-05 onward)

2005-05-13 version 1.0.4: (for data of ~ 2005-03-30 onward)

2005-05-01 version 1.0.3 (for data of 2005-03-13 onward)

2005-02-16 version 0.5.9 (for data of 2004-11-24 onward)

2005-01-01 version 0.5.2

2004-10-01 version 0.4.8

After February 2005, SINFONI raw frames come in the following format:

Header Keyword	Value
HIERARCH ESO DET CHIP ID	ESO-Hawaii2RG
HIERARCH ESO DET CHIP NAME	Hawaii2RG
HIERARCH ESO DET CHIP NX	2048
HIERARCH ESO DET CHIP NY	2048
HIERARCH ESO DET NCORRS NAME	NonDest

Before February 2005, SINFONI raw frames come in the following format: