[ ESO ]
UVES pipeline:
science reduction

shortcuts:
general
raw data
reduction
assumptions

calib. products

QC checks
precision
problems
 
UVES PIPELINE: SCIENCE REDUCTION

The UVES pipeline is designed to reduce raw SCIENCE spectra taken in echelle mode to a level almost free from instrumental signature.

On Paranal, the quick-look pipeline makes an attempt to automatically reduce all science data taken in echelle mode. The reduction is performed using standard calibration solutions from a local calibration database which is refreshed every one or two months. All standard wavelength settings are supported, in all four binnings. Furthermore, for non-standard wavelength settings, a complete set of calibration files is processed in order to obtain solutions. This often works fine, but since the number of possible non-standard UVES configurations is very high, there is no experience with the pipeline behaviour in these cases. Generally any pipeline processing on the site is done on a best-effort basis.

At QC Garching, all SCIENCE data taken in Service Mode are pipeline-reduced, using the best available calibration solutions (quality-checked and closest in time, following the calibration cascade). The products of science reduction are also quality-checked. If any irregularity in the reduced data is found, an attempt is made to improve on this. If not possible, information about the poor quality is made available in the SM package. Present policy is to not suppress these data.

 

   [top]
SCIENCE RAW DATA






FITS headers
frame
.txt
SCIENCE
[science header]

 



Raw science data are usually accompanied by slit viewing images. These types of raw science files exist:


frame DPR TYPE Purpose
SCIENCE OBJECT science exposure (observed before 2002-06-29)
SCIENCE OBJECT,POINT science exposure (point-like source)*
SCIENCE OBJECT,EXTENDED science exposure (extended source)*
TEST SLIT slit acquisition image (observed before 2002-11-23)
ACQUISITION SLIT slit acquisition image
*The destinction between point-like and extended sources is made by the user during the Phase II proposal. For extended sources, an additional reduction step (2D extraction, see below) is performed for Service Mode data.
 [top] REDUCTION STEPS

 

science reduction
complete product list


Recipe. A science raw file is pipeline-processed by the pipeline recipe uves_obs_scired. In short, data are corrected for bias, interorder background, sky background, sky emission lines and cosmic ray hits. They are flattened, optimally extracted and finally merged. A response correction is also performed if an appropriate response curve exists for the setting. Find here a detailed record of the reduction steps.

Extraction. The UVES pipeline knows three extraction modes:

  • optimum (OPT)
  • average (AVG)
  • two-dimensional (2D)

Optimum extraction is the default extraction mode for "ECHELLE" and "ECHELLE,ABSORPTION-CELL" observations; average extraction for "ECHELLE,SLICER" and "ECHELLE,ABSORPTION-CELL,SLICER" observations. A two-dimensional extraction is performed by QC Garching in addition if the DPR TYPE keyword is set to "OBJECT,EXTENDED".

Optimum extraction provides an optimized signal-to-noise ratio and is suitable for point sources. Average extraction is an alternative mode which simply averages any signal along the slit and above the sky background level. It may be a good choice in case of failure of the optimum extraction, e.g. at high exposure level. Two-dimensional extraction actually does not extract the signal at all, but provides a flattened and wavelength-calibrated solution with the flux redistributed into wavelength-slit coordinate space. This means that any spatial information along the slit is preserved and can be manually extracted later. Find more information here.

Optimum extraction. UVES science data reduced for Service Mode programmes are processed in optimal extraction mode (OPT) if the image slicers have not been used. The UVES pipeline assumes a Gaussian profile for the cross-dispersion flux distribution. This procedure

  • yields optimum S/N,
  • automatically discriminates against any non-Gaussian components in the cross-dispersion profile.

This means specifically

  • automatic sky subtraction (since the sky background is treated as a pedestal),
  • automatic cosmic ray rejection (whenever the hit has a non-Gaussian distribution),
  • automatic sky emission line removal (since these fill up the whole slit and have a rectangular profile).

All Service Mode spectra reduced by QC Garching are flattened in order-bin space, i.e. with extracted flats. Flat extraction is done with the weight factors derived from the science spectrum.

The UVES pipeline also offers flattening in pixel space (the science spectrum is divided by the flat field before extraction). Checks done on high-S/N, featureless spectra show that the pixel-space flattening may yield slightly (less than 10%) higher S/N than extracted flattening. However, telluric lines in the flats are found to propagate much stronger into the extracted science spectrum in the case of pixel-space flattening. Since telluric lines in the flats fill the whole slit, they are effectively removed by the optimum extraction.

Flattening corrects for the blaze function within the orders. In the red settings (860 nm), flattening also efficiently removes fringing.

Since version 1.6 of the pipeline, an automatic response correction (flux calibration) can be applied to the extracted spectra. This additional step is performed for all Service Mode files observed after 2002-11-06, with the exception of settings with 520nm and 600nm central wavelength. Users interested in flux-calibrating older spectra can use the master response curves derived from standard star spectra (check out for more information and downloads here). Raw standard star measurements, if available, are also delivered in the 'calib' directories of the Service Mode package.

Products. Service Mode programmes receive the following reduction products per CCD (i.e. 1 BLUE for the blue arm, 1 REDLower and 1 REDUpper for the red arm). For optimal extraction, if flux-calibrated spectra are provided:

product category (PRO CATG) product number* format comments
RED_SCI_POINT or RED_SCI_EXTENDED 0000 (+0013) 1D (wavelength space) final science spectrum
(RV corrections to heliocentric and barycentric scale are provided in FITS header)
ERRORBAR_SCI_POINT or ERRORBAR_SCI_EXTENDED 0005 (+0018) 1D (wavelength space) errorbar file** corresponding to RED_SCI
(contains the standard deviation of the extraction as error estimate for the extraction procedure)
ORDER_TRACE 0008 (+0021) tfits table position and FWHM information per order
CRMASK 0009 (+0022) tfits table cosmic ray mask table
FLUXCAL_SCIENCE 0011 (+0024) 1D (wavelength space) flux-calibrated science spectrum (in 10E-16 erg/s/cm**2/A)
FLUXCAL_ERRORBAR 0012 (+0025) 1D (wavelength space) errorbar file corresponding to FLUXCAL_SCIENCE
* The numbers in brackets refer to REDU CCD, others to BLUE or REDL.
** defined by the variance of the chi-square fit to the signal obtained during optimum extraction. The error is determined by the boundary of the chi ... chi+1 domain. The errorbar file then gives the size of the bar which is symmetrical around the most likely value.

If flux calibration is not provided:

product category (PRO CATG) product number* format comments
RED_SCI_POINT or RED_SCI_EXTENDED 0000 (+0011) 1D (wavelength space) final science spectrum
(RV corrections to heliocentric and barycentric scale are provided in FITS header)
ERRORBAR_SCI_POINT or ERRORBAR_SCI_EXTENDED 0005 (+0016) 1D (wavelength space) errorbar file** corresponding to RED_SCI
(contains the standard deviation of the extraction as error estimate for the extraction procedure)
ORDER_TRACE 0008 (+0019) tfits table position and FWHM information per order
CRMASK 0009 (+0020) tfits table cosmic ray mask table
* The numbers in brackets refer to REDU CCD, others to BLUE or REDL.
** defined by the variance of the chi-square fit to the signal obtained during optimum extraction. The error is determined by the boundary of the chi ... chi+1 domain. The errorbar file then gives the size of the bar which is symmetrical around the most likely value.

For slicer observations reduced with average extraction:

product category (PRO CATG) product number* format comments
RED_SCIENCE_SLICER 0000 (+0010) 1D (wavelength space) final science spectrum
(RV corrections to heliocentric and barycentric scale are provided in FITS header)
ERRORBAR_SCIENCE_SLICER 0005 (+0015) 1D (wavelength space) errorbar file corresponding to RED_SCIENCE
(contains the standard deviation of the extraction as error estimate for the extraction procedure)
FLUXCAL_SCIENCE 0008 (+0018) 1D (wavelength space) flux-calibrated science spectrum (in 10E-16 erg/s/cm**2/A)
FLUXCAL_ERRORBAR 0009 (+0019) 1D (wavelength space) errorbar file corresponding to FLUXCAL_SCIENCE
* The numbers in brackets refer to REDU CCD, others to BLUE or REDL.

For slicer observations without flux-calibration:

product category (PRO CATG) product number* format comments
RED_SCIENCE_SLICER 0000 (+0008) 1D (wavelength space) final science spectrum
(RV corrections to heliocentric and barycentric scale are provided in FITS header)
ERRORBAR_SCIENCE_SLICER 0005 (+0013) 1D (wavelength space) errorbar file corresponding to RED_SCIENCE
(contains the standard deviation of the extraction as error estimate for the extraction procedure)
* The numbers in brackets refer to REDU CCD, others to BLUE or REDL.

Complete product list : The complete suite of products is available if the pipeline is installed and run at the user's home workstation.

 

[top] ASSUMPTIONS MADE


optimum extraction

 


The UVES pipeline, running on Service Mode data in optimum extraction mode, implicitly makes the following assumptions on the raw spectra:

  • the object is a point source,
  • the object is centred on the slit,
  • the object has some continuum, i.e. is not a pure emission line object,
  • the signal-to-noise ratio is not too high ( < 50 approximately).

Only if these assumptions are readily met by the nature of the observations, can useful results be expected. While some of these assumptions are checked as part of the QC checks done for assessment of the pipeline products, there is no alternative approach chosen (e.g. average extraction). This is the sole responsibility of the user.

 

 [top] CALIBRATION PRODUCTS USED

[UVES calibration scheme]

UVES calibration
cascade


[UVES reduction scheme]

UVES reduction
scheme


The successful execution of science reduction requires a complete set of calibration products available.

All science data processed by QC Garching are reduced with a quality-checked set of calibration files which are usually generated from daytime calibrations taken immediately after the science night. In cases of 'attached calibrations' (WAVE and FLAT calibrations taken during nighttime), these are used for the reduction of the science data from the same OB.

 

 [top] QUALITY CHECKS

 

QC1 report for Service Mode programmes


QC Garching checks on the quality of the reduced science data. The following items are checked:

  • proper selection of calibration data

  • any peculiarities in the raw data
    e.g. unusual bias level.

  • the cross-dispersion profile
    Here the exposure level, the centering and the sky background are controlled. These parameters are essential for assessing the quality of the optimum extraction.

  • proper dispersion solution
    A comparison to the expected positions of frequent emission lines, e.g. the Balmer lines, reveals at least larger errors in wavelength calibration.

  • proper extraction
    The full spectrum and its variance are plotted and checked visually for anomalies in the extraction (e.g. ripple patterns with order periodicity etc.)

  • alarm flags
    Three parameters (temperature difference between science and wavelength calibration file; proper object centering; average S/N ratio) are controlled and flagged if threshold values are violated.

The QC process is continuously improved and adapted to experience.

QC1 reports: Since August 2001, extensive QC1 reports about SCIENCE reduction and signal tracing are delivered as part of the Service Mode package.

 

[top] PRECISION

 

quality checks for wavelength calibration

 


Wavelength calibration.
The precision of the wavelength calibration in the course of science reduction is determined by

  • the precision of the dispersion solution used for resampling,
  • the difference of temperature between the science observation and the calibration solution used.

Typical values for the standard deviation (in milli-Angstroem) of the dispersion solution are given here. They are of the order of 2-8 milli-Angstroem.

The overall UVES spectrum shifts on the detector mainly depend on the air temperature in the UVES enclosure. The shift in the red arm is, in first approximation, 0.35 pixels/Degree C in the dispersion direction. One pixel is ~1.2 km/sec. The corresponding value in the blue arm is about 0.05 pixel/Degree C. Since 2002-01, an active compensation for such drifts has been implemented.

The temperature information is recorded in the file headers and reported in the 'TEMP' columns of the list files. By comparing the temperatures at the times of the science exposure and of the wavelength calibration applied to the data, the magnitude of the possible shift can be estimated from the gradients given above.

CCD defects. The UVES pipeline does not apply any correction for cosmetic defects of the CCDs. These are minimal because of the good cosmetic quality of the devices. However, there are a few defects which show up, notably in long integrations on faint objects on the two CCDs of the red arm.

There are four trails of hot pixels which appear in long exposures in the blue-side quadrant of the EEV chip (bluer side of the red arm mosaic, X coordinates 3896,3963, 4052 and 4140 in an unbinned fits file, middle of the chip in y). They occupy a single column and are almost parallel to the orders. They appear as broadish emission in the bluer part of the extracted spectrum of a faint object.

In the MIT-LL chip (red side of the CCD mosaic of the red arm) there is a trap in the column X1609 which might show up as a slight depression over ~130 pixels in the extracted spectrum of one order. In long, binned exposures this chip shows also an emission band starting on the red side and extending over the rows 2790-2850 with decreasing intensity toward the blue side of the echelle format. Since this band is perpendicular to the spectrum, it is usually well subtracted in the sky subtraction step of the optimal extraction.

 

[top] ACTUAL PIPELINE PROBLEMS AND ISSUES

 

tutorial on pipeline problems

 


High signal.
Optimum extraction may face problems in cases of high signal, i.e. when the differences between the true shape of the instrumental profile and the assumed Gaussian start to become systematic rather than random. These problems are found more frequently in the blue than in the red regions.

Two kinds of artefacts may show up then:

  • ripple-like patterns repeating with the orders. These are due to the non-Gaussian shape effect being stronger in the centre of an order than at the edges. Hence optimum extraction loses more flux in the centre of an order. Since usually flattening removes these order-period ripples completely, their occurrence is a good indicator of the "high-signal" problem. In such cases, average extraction will produce better results.
  • Small-scale scatter with a period of about 20-30 pixels. This problem is due to problems in the order-tracing algorithm which become stronger in case of high signal and good seeing, hence small FWHM. Since the echelle orders are inclined by a few degree, their centre "jumps" vertically by one pixel every 20-30 pixels. This affect degrades the S/N ratio of the spectrum considerably. Again, average extraction will produce better results.
    This problem is much stronger for UVES pipeline products obtained until October 2000. The recipe has been improved considerably since then.

Some of these cases are detected by quality check procedures, and a short note is included in the Service Mode package. However, this is not generally true, and we strongly recommend to carefully check the results if they could be affected by this problem.

More information about problems with high signal can be found in the tutorial on optimum extraction.

High airmass. In cases of blue spectra taken at high airmass (> 1.5) without the ADC, atmospheric dispersion may shift the bluest orders out of the extraction window, and their flux is lost. Even at airmass 1.1, differential extinction spreads the spectrum by about 3 pixels for the bluest setup (346 nm).

More information about problmes with high airmass can be found in the tutorial on optimum extraction.

Order identification. In a few cases problems have occured with order identification in settings with 346nm central wavelength: the orders found by the pipeline in the order defintion flat and during extraction/wavelength calibration of the science frame may be different. This is not recognised by the pipeline and the lines in the final wavelength-calibrated spectrum appear shifted by one order. An example is given here where the overlap between an observation with 346nm and 437nm central wavelength is shown. The problem may be resolved by choosing a different order definition flat.

Sky emission lines. Optimum extraction may over/under-corrected bright sky emission lines by 5% or more.

Radial velocity corrections. Until 2001-09-17 (date of acquisition), a bug in the radial velocity correction as calculated by the pipeline may have caused wrong values. Please check carefully (FITS keywords HIERARCH ESO QC VRAD BARYCOR and HELICOR; also stated in the listing file 'list_of_all_red.txt'). Note that no RV correction is applied to the data anyway.
This bug is present in version 1.2 and earlier of the uves pipeline, and solved with version 1.3.

 


[UVES pipeline][Observing Facilities and Operations][ESO][Index][Search][Help][News]