mirror sites: PL (internal link) HQ     [?]

Quality Control and
Data Processing

CAL | HC | refs | QC
Trending & QC1
general info
QC links:
General information | Data format

General information


Pipelines are used at ESO to process both calibration data and science data and to retrieve quality information. There is a dedicated pipeline for each VLT and VLTI instrument. Find general information about ESO reduction pipelines here.


The main functionalities of the pipelines are:

  • create master calibration data,
  • reduce science frames,
  • extract QC information from the data.
QC Garching creates master calibration data from all raw calibration data. The raw data are stored in the ESO Archive and are public. They are quality-checked and used for data reduction and for trending.

For selected instrument modes, we offer science-grade data products processed with the pipelines.


There are two instances of the data reduction pipelines:

  • at the instrument workstation on Paranal, running in automatic mode,
  • at HQ Garching, run by the Quality Control Group in the optimized mode.
The automatic mode is used for quick look and for on-site quality control. It processes all raw data sequentially, as they arrive from the instrument. If calibration products ("master calibrations") are required for processing science data, these are taken from a database with standard, pre-manufactured calibration products. The automatic mode is not tuned to obtain the best possible results.

The optimized mode is the mode, which uses all data of a night, including the daytime calibrations. The calibration data are sorted and grouped according to their dependencies. Master calibration data are created. Their quality is checked.

HAWKI part

The HAWK-I data processing pipeline is operational since April 2008. All operational HAWK-I observing modes are supported although with some caveats:

  • JITTER offsets greater than 1500 can not be handled
  • Limitations of the OS may impose limits on the number of RAW frames that can be handled in one recipe exection.
The HAWK-I pipeline is publicly available (check out here). Under this link you also find the pipeline Users Manual.

Changes to the pipeline

The following is s summary of some changes in the pipeline. Please note that the list may not be complete:

  • 1.4.2-Public Release:
    • Monolithic Science recipe hawki_img_jitter split into several component recipes to allow use with Reflex.
    • MASTER DARK product is now divided by DET.DIT value, i.e. pixel values are now [ADU/sec] (previously were [ADU])
    • A number of bugs and improvements in hawki_img_zpoint
  • 1.3.5: Better memmory allowing the hawki_img_jitter recipe to handle up to ~135 frame stacks
  • 1.3.5: Median level and RMS level filtering for auto selection of Flatfield sample
  • 2015-10-01 New, CASU prepared pipeline (v.2.0.2) adopted. It is dramatically different from previous versions. New master calibrations are not compatible with the old ones. In particular, measurements of the zeropoints differ from previous values. For more information go here


Find the description of HAWK-I data processing and pipeline recipes here:

ESO internally processed HAWKI science data products are available on the phase3 archive interface.

Data format
General information | Data format


HAWK-I has a 2x2 mosaic of 4 2048x2048 photon-sensitive pixels. There are NO pre- or post- overscan columns or rows.

Further details regarding the chips can be found in Appendix B of the User Manual.

In total, HAWK-I raw frames have 4096x4096 pixels. There is only one read-mode available, which is unbinned, un-correlated. Raw frames have a size of 65MB each.

Extensions: Raw data come as FITS files with FIVE HDUs (header units), so-called Multi-Extension FITS files, or MEFs. The primary HDU has the header with all primary keywords and telescope, positioner, detector, observation, instrument etc. keywords. The pixel data, plus chip-specific header info for each of the FOUR chips are then stored in four further HDUs. HAWKI data obtained in the AO (Adaptive Optics) mode have additional 1 or 2 extensions containing information from the GRAAL AO system module.

Products:Pipeline products are either FITS images or FITS tables, and either have a single HDU (i.e. traditional FITS files) or 5 HDUs as in case of the RAW data, i.e. a primary HDU containing the header with all primary keywords and telescope, positioner, detector, observation, instrument etc. keywords and then, pixel or table data, plus chip-specific header info for each of the FOUR chips in four further HDUs. Further information can be found here.

Send comments to <qc_hawki@eso.org> powered by QC [webCMS v1.0]