VLT Instrument Pipelines
Introduction
In collaboration with the various instrument consortia, the Pipeline Systems Department has undertaken to implement data reduction pipelines for the most commonly used VLT/VLTI instrument modes. These data reduction pipelines have three main purposes:
- Data quality control - pipelines are used to produce the quantitative information necessary to monitor instrument performance (see the ESO Quality Control web pages for more information).
- Master calibration product creation - pipelines are used to produce master calibration products (e.g. combined bias frames, super-flats, wavelength dispersion solutions).
- Science product creation - using pipeline-generated master calibration products, science products are produced for supported instrument modes (e.g. combined ISAAC jitter stacks; bias-corrected, flat-fielded FORS images, wavelength-calibrated UVES spectra). The accuracy of the science products can be limited both by the quality of the available master calibration products and by the algorithmic implementation of the pipelines themselves. In particular, adopted reduction strategies may not be suitable for all scientific goals. Therefore, ESO assumes no responsibility for the usefulness of reduced data for any specific scientific project.
Pipelines Running Environments: Reflex, Gasgano and Esorex
Home users can use Gasgano, a Java-based data file organizer developed and maintained by ESO. It can be used to manage and organize in a systematic way the astronomical data observed and produced by all VLT compliant telescopes. Gasgano is a graphics-based software tool for organising and viewing data files produced by the VLT Control System. Gasgano offers functionalities for data viewing, grouping, sorting, classification, searching, and filtering of data. And, of course, Gasgano will run any requested CPL recipe on the selected data.
Reflex is a new workflow-based data reduction tool that allows easy organization and grouping of the data. It contains interactive elements that provides very quick assesment of the reduction quality. Check this table to know which instruments are supported.
Esorex, a command-line utility for running pipeline recipes is also available. It may be embedded by users at their home institute into data reduction scripts for the automation of processing tasks.
Both Gasgano and Esorex are included in the pipeline public release packages.
Latest News:
May 16, 2012: New release of Reflex 2.1 based workflows of UVES (5.0.7) and X-Shooter (1.4.10) pipelines
April 20, 2012: Updated release of the X-shooter pipeline (1.4.6) and of the corresponding Reflex-based workflows.
April 5, 2012: New release of the UVES (5.0.3) pipeline and X-shooter pipeline (1.4.5) and of the corresponding Reflex-based workflows.
February 7, 2012: Release of the VIMOS pipeline version 2.6.9 for CPL 5.3.1.
November 2, 2011: Release of the SINFONI pipeline version 2.2.9 for CPL 5.3.1.
October 31, 2011: Release of the UVES pipeline version 4.9.8 for CPL 5.3.1.
July 7, 2011: Release of the X-shooter pipeline version 1.3.7 for CPL 5.3.1.
June 22, 2011: Release of the GIRAFFE pipeline version 2.8.9 for CPL 5.3.1.
May 12, 2011: New release of the VISIR (3.4.4) pipeline using the latest CPL (5.3.1).
May 11, 2011: New release of the CRIRES (2.1.3) and SOFI (1.5.4) pipelines using the latest CPL (5.3.1).
May 03, 2011: New release of the X-shooter pipeline (1.3.0) using the latest CPL (5.3.1).
April 15th, 2011: New release of the UVES (4.9.5) pipeline using the latest CPL (5.3.1) and of the corresponding Reflex-based UVES Red and Blue Arm Echelle workflow.
April 14th, 2011: New release of the Amber (3.3.0) and Midi (2.7.0) pipelines using the latest CPL (5.3.1).
April 7th, 2011: New release of the SINFONI (2.2.5), VIMOS (2.6.3), FORS (4.8.7), and EFOSC (2.2.2) pipelines using the latest CPL (5.3.1).
April 4th, 2011: New release of the NACO (4.2.3) and ISAAC (6.0.6) pipelines using the latest CPL (5.3.1).
Pipeline Reflex-based workflows
This table shows the pipelines released with a Reflex workflow. See the Release Notes of the new Reflex 2.1 workflows.
| Pipeline With links to the instrument pages |
Release Information | Release Kit | Documentation | Demo Data Sets | Installation script (It installs Reflex, the pipeline and demo data sets) |
|---|---|---|---|---|---|
| UVES |
Reflex 2.1 workflows release notes
5.0.7 pipeline release notes |
5.0.7 (2012-05-15) |
UVES Reflex Tutorial
UVES Pipeline User Manual 20.0 |
Echelle data
(976M, downloaded by installation script) |
install_reflex |
| X-SHOOTER |
Reflex 2.1 workflows release notes
1.4.10 pipeline release notes |
1.4.10 (2012-05-15) |
XSHOOTER Reflex Tutorial
X-SHOOTER Pipeline User Manual 9.0 |
X-shooter demo data
(2.5G, downloaded by installation script) |
install_reflex |
Pipeline Releases Based on the most Recent CPL Release
The Pipeline Systems Department makes also these data reduction pipelines available to the user community. The table below provides access to the pipelines currently available.
Installation of Multiple Pipelines
The procedure for a simultaneous installation of multiple pipelines on a single account is:- Download the desired pipeline kits from the "Release File" column in the table below into an empty, temporary directory.
- Download the Pipeline-kit installer into the same directory.
- If necessary, make it executable:
chmod u+x install_pipelinekit - Execute the command:
./install_pipelinekit *-kit-*.* - After the installation the temporary directory with its contents can be deleted.
The below pipelines are delivered with CPL 5.3.1, gasgano 2.4.0 and esorex 3.9.0, they can be installed together using the above procedure.
Previous Pipeline Releases
The Pipeline Systems Department also makes these data reduction pipelines available to the user community. They are based on older versions of CPL and can in general not be installed together.
| Pipeline With links to the instrument pages |
Release Information | Release File Includes CPL,Gasgano,EsoRex, and static calibration data |
Users Manual and Documentation | Data Sets |
|---|---|---|---|---|
| UVES (DISCLAIMER) | 2.9.7 (2007-10-12) | 2.9.7 (MIDAS based) |
UVES Pipeline User Manual 2.9.7
FLAMES-UVES Pipeline User Manual 2.9.7 |
A set of reference and master calibration data is included in the pipeline package. |
UVES RED arm data reduction after UVES MIT CCD upgrade, done on 6-14 July 2009, is not supported by current MIDAS based UVES pipeline release due to change of orientation of the CCD.
Publications and Links
VLT Science Products Produced by Pipelines, 2004, D.R. Silva & M. Peron, ESO Messenger 118.The ESO Quality Control Group processes VLT data and monitors the instrument and data quality and also provides a summary of current pipeline status.
The detector monitoring user manual is available here.