PIPELINES AT ESO

This page provides information about pipeline processing and data types.

Raw data are selected, associated and inserted into a reduction mechanism which produces calibration products, science products and quality control information. This mechanism is the data processing pipeline. There is one such pipeline for each VLT and VLTI instrument.

Find general information about ESO reduction pipelines here.

The main functionalities of the pipelines are:

• create master calibration data,
• reduce science frames,
• extract QC information from the data.

QC Garching creates master calibration data from all raw calibration data. The raw data are stored in the ESO Archive and are public. They are quality-checked and used for data reduction and for trending.

Before October 2011 QC Garching processed science data, using the best available, quality-checked master calibration data. As of October 2011 this service is not offered anymore.

There are two instances of the data reduction pipelines:

• at the instrument workstation on Paranal, running in automatic mode,
• at HQ Garching, run by the Quality Control Group in the optimized mode.

The automatic mode is used for quick look purposes and for on-site quality control. It processes all raw data sequentially, as they arrive from the instrument. If calibration products ("master calibrations") are required for processing science data, these are taken from a database with standard, pre-manufactured calibration products. The automatic mode is not tuned to obtain the best possible results.

The optimized mode is the mode, which uses all data of a night, including the daytime calibrations. The calibration data are sorted and grouped according to their dependencies. Master calibration data are created. Their quality is checked.

The GIRAFFE pipeline is able to support both the old and the new CCD, with some modified static calibration files describing the changes in alignment. The geometry of the new CCD is the same as for the old (2kx4k). It features increased efficiency in the red, and reduced fringing. It is in operations since 2008-05-26. Last science night with the old CCD is 2008-03-13. The main changes for the pipeline is a modified strategy for removing the BIAS.

The GIRAFFE data processing pipeline is operational since June 2004. A list of existing and supported setups is maintained here. Until October 2011 all science data were processed by QC Garching. This has been stopped now, and only calibration data are pipeline-processed by QC Garching.

The GIRAFFE pipeline is publicly available (check out here). Under this link you also find the pipeline Users Manual.

It is based on the GIRAFFE Baseline Data Reduction System (BLDRS) of the Observatoire de Geneve.

Find the description of GIRAFFE data processing and pipeline recipes here:

GIRAFFE pipeline
calibration
science reduction

Both in Service Mode and Visitor Mode, GIRAFFE observations are supported in certain standard settings:

GIRAFFE standard settings

Since October 2011 QC Garching processes only calibration data from GIRAFFE. Before that date, all calibration and science were processed.

Raw data. GIRAFFE has a single CCD with 2048 photon-sensitive columns and 4096 rows. To these add 50 pre-scan and 50 post-scan columns. Until 2008-03-13, the CDD was an EEV chip optimized for the blue ("Bruce"). Since 2008-05-26, GIRAFFE has an e2v CCD ("Carreras") with much higher red efficiency and lower fringing.

Find more details about the GIRAFFE CCD format and properties here.

In total, GIRAFFE raw frames have 2148x4096 pixels. Pixel size is 15x15 microns. There is only one read-mode available, which is unbinned, low gain. Raw frames have a size of 17.4 MB each. Find example (reference) frames here.

Bruce had three columns with hot pixels, at x=416-418, starting at y=1466. Carreras has none.

Extensions. Raw data come as FITS files with three HDUs (header units). The main plane (HDU1) has the header with all primary keywords and telescope, positioner, detector, observation, instrument etc. keywords, as well as the pixel data. Two extensions (HDU2 and HDU3) host binary tables. The first binary table is the OzPoz table. It includes the association of the objects and the fibre buttons, and also contains fibre positioner keywords. This is the PAF file generated by FPOSS (the fibre positioner observation support software tool), with some OzPoz information added (such as R, Theta and errors).

The second binary table contains the fibre description and the association between fibre buttons and fibre position in the slits, and also includes lab-measured fibre transmission values. This table is static.

Find here a description of the GIRAFFE binary tables. The Users Manual has also a detailed description of the GIRAFFE raw data structure.

Products. Pipeline products have varying formats. Typical for almost all products beyond the extraction step is that the X axis records the fibre index, while the Y coordinate records pixels or wavelength bins. Hence typically there is one column per fibre. More details are described here.

Except for the BIAS and DARK calibration frames, all GIRAFFE data come in a specific data format which is due to the fibre systems being projected onto the detector. The format is described here.

The GIRAFFE spectrograph receives light from the fibre positioner called OzPoz. It consists of two plates which are functionally identical. Each plate hosts three fibre systems (Medusa, IFU, UVES), one plate accomodates another IFU system called 'Argus'.

The Medusa system has nominally 137 fibres which are projected into one slit called Medusa<n> (where n is 1 or 2). The slit is subdivided into

• 8 subslits with nine object fibres each
• 5 subslits with 12 object fibres and 1 calibration fibre each.

The calibration fibres can be used for simultaneous calibrations (SimCal mode) or as object fibre (OzPoz mode).

Find more about the MEDUSA spectral format here.

Schematic layout of the MEDUSA spectral format. Blue solid lines: object fibres. Red dotted lines: calibration fibres. The directions of the increasing fibre number in the slit (FPS) and of increasing wavelength are indicated.

The IFU fibre system has 320 fibres being projected into one slit called IFU<n> (where n is 1 or 2). The slit is subdivided into

• 10 subslits with 20 object fibres and 1 sky fibre each,
• 5 subslits with 19 object fibres and 1 sky fibre and 1 calibration fibre each.

The calibration fibre can be used for simultaneous calibrations (SimCal mode) or as object fibre (OzPoz mode).

Find more about the IFU spectral format here.

Schematic layout of the IFU spectral format. Blue solid lines: object fibres. Red dotted lines: calibration fibres. Green dots: sky fibres. The directions of the increasing fibre number in the slit (FPS) and of increasing wavelength are indicated.

The Argus fibre system is offered since P72 (October 2003). It is similar to the IFU system. It also has 15 sky fibres.

Find more about the ARGUS spectral format here.

Schematic layout of the ARGUS spectral format. Blue solid lines: object fibres. Red dotted lines: calibration fibres. Green dots: sky fibres. The directions of the increasing fibre number in the slit (FPS) and of increasing wavelength are indicated. Note that the direction of FPS is inverted with respect to IFU and Medusa.