VLT/VLTI Data Pipelines and Calibration Plans

Last Update: 2011 October 01

	Data pipelines
	VLT instruments: ------------------------------ CRIRES FORS1/2 FLAMES/GIRAFFE FLAMES/UVES HAWKI ISAAC NACO SINFONI UVES VIMOS VISIR X-SHOOTER VLTI instruments: ------------------------------ VLTI/AMBER VLTI/MIDI Survey instruments: ------------------------------ VIRCAM

Introduction

For each VLT or VLTI instrument, including survey cameras, ESO provides a data processing pipeline. This page contains a brief summary of their status. It also lists the current calibration plans. Users are urged to refer to the appropriate user's manuals for complete details.

Information about the public availability of ESO pipelines, including user manuals, can be found here.

For a general statement about the use of pipelines within ESO operations, see Section 8 in the ESO Call for Proposals.

Note: Since October 2011/P88, QC Garching does not process anymore science data, due to manpower shortage. Users may want to install and use the public pipelines to reduce their science products.

Science Product Disclaimer

The accuracy of the science products produced by the ESO pipelines can be limited both by the quality of the available master calibration products and by the algorithmic implementation of the pipelines themselves. In particular, adopted reduction strategies may not be suitable for all scientific goals. Therefore, ESO assumes no responsibility for the usefulness of pipeline reduced data for any specific scientific project.

What's New

2011 October 01: pipeline processing of science data terminated at ESO/QC

2011 July 07: revised for CfP 89

2011 January 23: Revised for CfP 88

2010 July 22: Added section about VIRCAM pipeline

VLT instruments:

top CRIRES    VLT instruments | VLTI | Survey

The CRIRES calibration plan includes day-time measurements of dark exposures, flat-fields, and wavelength calibrations. They are executed with instrument settings matching those used during the night (DITs for darks and reference wavelength of the grating for flat and wavelength calibrations).

The repeatability of the spectrograph settings between night-time and day-time is about 3 pixels. For higher precision wavelength calibration, calibration templates can be executed immediately before or after the science exposures on request by the PI. Calibration exposures either use a ThAr arc lamp or a N2O absorption cell, depending on the setting. For reference wavelengths above 2400nm, the usage of sky lines is usually best. Please refer to the CRIRES user manual for a detailed description of recommended calibration sources.

Telluric and spectro-photometric standard stars are only observed when requested by the PI.

The CRIRES pipeline provides the following calibration steps: dark subtraction, correction for detector non-linearity, flat-fielding, combining of nodding images, spectrum extraction, and wavelength calibration. In general, it gives useful results for single, point-like, continuum-emitting sources with S/N (in the combined image) of higher than about 5. The pipeline supports wavelength calibration using either sky lines or dedicated calibration exposures.

For further information about CRIRES data handling and processing, as well as CRIRES specific quality control parameters, see the CRIRES section of the ESO quality control web pages.

The CRIRES pipeline is available here.

top FLAMES     VLT instruments | VLTI | Survey

FLAMES consists of two major modes: UVES-Fiber and GIRAFFE. These major modes are discussed separately.

The FLAMES calibration plan ensures that ESO maintains and provides master bias, dark, spectroscopic slit and fiber flat field frames. Order definition frames as well as arc lamp spectra are also provided. Flux standard stars are not observed. CCD characteristics like read-out noise and gain are measured on a monthly basis. Daytime arc lamp (ThAr) spectra are used for the wavelength calibration of science frames. When highly accurate radial velocity measurements are required with the Red 580 setting, the first of the eight fibers can be used to record an arc lamp spectrum simultaneously with the science observations. All FLAMES-UVES calibration data taken in standard modes are processed and quality checked. The results are posted on the ESO Quality Control Web pages. The following settings are supported (all in 1x1 binning):

Red 520 nm (CD#3)
Red 580 nm (CD#3)
Red 860 nm (CD#4)

The following corrections of the science spectra are available for each fiber: bias subtraction, inter-order background subtraction, flat field correction, order extraction, rebinning to wavelength scale, and order merging. The two detectors are processed independently. An optimum extraction algorithm with cosmic rejection is applied. Average extraction is also available. FLAMES/UVES pipeline recipes are provided as part of the UVES pipeline.

For further information about FLAMES/UVES data handling and processing, as well as FLAMES/UVES specific quality control parameters, see the FLAMES/UVES section of the ESO quality control Web pages.

The UVES pipeline is available here.

FLAMES/GIRAFFE

The GIRAFFE calibration plan foresees fiber flats and arc lamp spectra to be taken on a daily basis for each science setup used in the previous night. For ARGUS, the fiber flats are taken during the night using the Nasmyth screen. For all other modes, they are taken as daytime calibrations using the robotic arm. Bias frames are taken daily, as are fiber flats in one selected setup per fiber system. Dark frames and other technical calibrations to monitor the CCD performance are taken about monthly. Flux standard stars are measured routinely with ARGUS, and on request during the night (or in twilight) with one of the IFUs. To provide flux calibration for the other IFUs, Nasmyth flats are measured. There are also technical calibrations to monitor overall efficiency of the instrument and the accuracy of radial velocity astrometry. Pipeline support is available for all three modes (MEDUSA, IFU, ARGUS). Extracted, wavelength calibrated and flat-field corrected spectra are provided. Data are currently extracted with a fixed numerical filter. Optimal extraction is possible but not used in data flow operations.

For further information about FLAMES/GIRAFFE data handling and processing, as well as quality control procedures and parameters, see the FLAMES/GIRAFFE section of the ESO quality control Web pages.

The GIRAFFE pipeline is available here.

top FORS1/2     VLT instruments | VLTI | Survey

Note: FORS1 has been de-commissioned in April 2009. All public FORS1 data can be accessed here.

The calibration plan of FORS2 ensures that ESO maintains and provides detector biases on a regular basis. In imaging (IMG) mode, twilight flat field frames and photometric standard stars are regularly obtained for the BVRI filters. From the standard star observations, zero points and first-order extinction terms are determined. Spectro-photometric stars are observed when narrow-band imaging science observations are obtained.

In long-slit spectroscopic (LSS) mode, screen flats and arc-lamp frames are obtained with the same slit as the science data. Spectra of spectro-photometric standard are also obtained for the determination of response curves.

In multi-object spectroscopic modes (MOS and MXU), flat-field and arc-lamp frames are obtained for all slit configurations (MOS) or masks (MXU).

The FORS pipeline is used for processing FORS2 data. The imaging pipeline generates master bias and imaging flat-field frames as well as photometric zeropoints. When processing science frames, it applies bias and flat field corrections as well as error estimates. The FORS spectroscopic pipeline produces a wavelength calibration table and and normalized flat fields for all the slits. It also derives response curves from spectrophotometric standard star observations. When processing science frames, it applies bias and flat-field corrections to all slits (MOS, MXU, and LSS). The wavelength calibration table is then used to produce a wavelength distortion corrected frame with linear dispersion. In addition the spatial distortion is corrected for MOS and MXU mode. If a sufficient number of spatially distributed slitlets is present (more than 12) the global distortion is also determined. The pipeline supports all grisms. In the SCIENCE spectroscopic observations, the object spectra are detected and optimally extracted and flux-calibrated if appropriate response curves are provided.

The FORS pipeline supports the processing of spectro-polarimetric data with 2 or 4 files at different angles for circular polarization and with 4 or 8 files at different angles for linear polarization. After applying the same corrections as for MOS data it then creates additional results frames for the various Stokes parameters.

Support for jittered observations is planned.

For further information about FORS2 data handling and processing, as well as specific quality control parameters (e.g. zeropoints and color terms), see the FORS2 section of the ESO quality control Web pages.

The FORS pipeline is available here.

top HAWK-I     VLT instruments | VLTI | Survey

The HAWK-I calibration plan ensures that ESO maintains and provides dark frames for all DITs and the maximum NDIT for a given DIT used during the previous night's science and/or standard star observations. Twilight flat-field frames are obtained for the four broad band filters (Y, H, J, Ks) and the six narrow band filters (Brackett gamma, CH4, H2, 1.061 μm, 1.187 μm, 2.090 μm), usually within a few days of the science observations, if not already available from within a few days beforehand.

Photometric zero-points are measured every night HAWK-I is used for the broad band filters, based on supplementary observations of standard stars from the faint UKIRT standards.

The calibration plan forsees the observation of one standard star for the night for zero point determination.

The calibration plan does NOT forsee the zero points for the narrow band filters. Full photometric calibration, including extinction determination and colour terms is NOT forseen within the calibration plan.

For further information about HAWK-I data handling and processing, as well as specific quality control parameters (e.g. zeropoints and color terms), see the HAWK-I section of the ESO quality control Web pages.

The HAWK-I pipeline is available here.

top ISAAC     VLT instruments | VLTI | Survey

The ISAAC calibration plan ensures that ESO maintains and provides dark frames for the DITs used during the observations. In SW imaging (SWI) mode, twilight flat-field frames and photometric standards are regularly obtained for the JsJHK filters. In other SWI filters, flat-fields and standard star observations are obtained as needed. The latter is also true for LW imaging (LWI) mode. For long-slit spectroscopy, screen flats and arc-lamp frames are obtained during the day as needed for the setups (slits and grating position) used at night. Telluric absorption standards are observed as needed, see the user's manual for more details. Star trace frames are obtained periodically to determine the optical distortion in the spatial (y) direction. The response function is determined approximately once per semester via the observation of spectro-photometric standards. Detector linearity is monitored regularly for the Aladdin array.

JHK broad-band imaging with the LW-arm Aladdin array is calibrated with twilight flats and zero points.

In SW imaging (SWI) mode, the data reduction pipeline handles creation of master dark frames with the appropriate DIT, twilight flats, and photometric zero points. It supports co-addition of multiple frames taken in the AutoJitter, AutoJitterOffset and FixedSkyOffset modes.

In SW spectroscopy (SWS) mode, the pipeline handles the creation of master dark frames, spectroscopic flats, and arc frames to correct optical distortion and to apply a wavelength dispersion solution. It supports co-addition, optical distortion rectification, and wavelength dispersion correction of multiple frames taken with the AutoNodOnSlit and StandardStar modes. Startrace calibration tables are created and applied to correct the optical distortion in y-direction. In these templates, the pipeline also extracts a wavelength calibrated spectrum of the brightest source.

In LW imaging mode (LWI), the pipeline handles the creation of dark frames for the DITs used during the observation, twilight flats (in non-chopping modes) and sky flats (for chopping modes) in the filters used during the observation. It also processes detector non-linearity correction and standard star observations. It supports co-adding of multiple frames taken in modes AutoJitter, AutoJitterOffset (non-chopping modes for filters NB_3.21 and NB_3.28), AutoChopNod (chopping modes with filters L and M), and StandardStar.

In LW spectroscopy (LWS) mode, the pipeline handles the creation of master dark and spectroscopic flat-field frames, as well as the processing of arc frames (in first-order for L-band and third-order of M-band) to determine optical distortion and dispersion solution in y-direction. The pipeline supports the co-addition of multiple frames taken with the AutoChopNod and StandardStar templates in chopping mode, and the AutoNodOnSlit and StandardStarNod templates in non-chopping modes. Startrace calibration tables are created and applied to correct the optical distortion in y-direction.

The current version of the ISAAC pipeline is based on procedures available within CPL, which is available for use by external users.

For further information about ISAAC data handling and processing, as well as ISAAC specific quality control parameters, see the ISAAC section of the ESO Quality Control Web pages.

The ISAAC pipeline is available here.

The NAOS-CONICA (NACO) Calibration Plan includes all the offered observing modes: imaging (also in the cube mode), spectroscopy, coronography, also Apodizing Phase Plate (APP) coronography, polarimetry, Simultaneous Differential Imaging (SDI), Space Aperture Interferometric Masks (SAM) imaging, and SAM combined with polarimetry, SAMPol. The plan ensures that ESO maintains and provides dark frames for the DITs and cameras used during the night, internal lamp and/or sky flat field frames, and standard star observations. Flat fields are obtained for the default readout-modes defined for every template. The twilight flats are taken without the focal plane masks, the polarimetric masks and without the polarizer.

No cube mode calibrations are offered.

For spectroscopy, ESO obtains spectroscopic flats (from internal lamps) in all SW spectroscopic modes, slits and readout modes. The arc-lamp exposures are taken in all spectroscopic modes and slits, however, the LW arcs are not supported. Observations of telluric standard stars, with the setups used for the science target, are performed whenever the grisms are used.

In the current version, the NACO data reduction pipeline  supports only the imaging, imaging-cube and spectroscopy. The pipeline support is not available for any other NACO observing mode. For further information about NAOS-CONICA data handling and processing as well as NAOS-CONICA quality control parameters, see the NACO section of the ESO quality control Web pages.

The NAOS pipeline is available here.

top SINFONI       VLT instruments | VLTI | Survey

The SINFONI calibration plan ensures that ESO maintains and provides dark frames for the DITs used during the observations. Spectroscopic screen flats and arc-lamp frames with the same grating and pre-optics are taken during the day. Telluric absorption standards are observed as needed, see the user's manual for more details. Fiber frames are obtained periodically for each grating to determine the optical distortion and the positions of the slitlets in order to reconstruct the data cube. Detector linearity is monitored regularly for the array.The data reduction pipeline handles creation of dark frames, spectroscopic flats, and arc-lamp frames to identify the positions of the slitlets and to apply a wavelength solution. It extracts and applies optical distortion correction and aligning of slitlets. It supports co-addition of calibration stars (telluric standard stars and PSF calibration stars) and science observations and sky subtraction. For science stacks without embedded SKY frames the sky is extracted from the OBJECT frames. Currently the science recipe generates 3D spectrum, a series of intermediate non-coadded science cube products and sky frames.

Information about SINFONI data handling and processing, as well as SINFONI specific quality control parameters, see the SINFONI section of the ESO quality control Web pages.

The SINFONI pipeline is available here.

The UVES Calibration Plan ensures that ESO maintains and provides bias, dark, spectroscopic flat field, and order definition frames and arc lamp spectra. Flux standard stars are observed in three dedicated settings only (346+580, 390+564, and 437+860). They are used to create master response curves (valid for relative flux calibration over certain periods) and to monitor the overall instrument efficiency. CCD characteristics like read-out noise and gain are measured on a monthly basis.

Daytime calibration lamp (ThAr) spectra are used for the wavelength calibration of science frames. When highly accurate radial velocity measurements are required, additional calibration spectra have to be taken during the night. All UVES calibration data taken in standard modes are processed and quality checked. The results are posted in the UVES section of the ESO Quality Control Web pages.

Health check data are taken on a daily basis in the following standard modes using the 1x1, low gain readout mode:

Blue 346 nm (CD#1),
Blue 437 nm (CD#2),
Red  580 nm (CD#3),
Red  860 nm (CD#4).

The primary purpose of these data is to assure that the instrument is operating within specification. QC parameters derived from pipeline processing of these data are trended on the UVES Health Check website.

A set of eight interference filters is offered in Visitor Mode. The filters can be used in the red arm to isolate certain échelle orders so that the full slit length of 30 arcsec can be chosen. There is no pipeline support for this UVES long-slit spectroscopy mode.

Blue 346 nm (CD#1),
Blue 437 nm (CD#2),
Red  520 nm (CD#3),
Red  580 nm (CD#3),
Red  600 nm (CD#3, for use with iodine cell only),
Red  860 nm (CD#4),
Dic1 346+580, Dic1 390+564, Dic1 346+564, Dic1 390+580,
Dic2 346+760, Dic2 390+760, Dic2 437+760,
Dic2 346+860, Dic2 390+860, Dic2 437+860.