The UVES pipeline is designed to reduce raw SCIENCE spectra taken in echelle mode to a level almost free from instrumental signature.

On Paranal, the quick-look pipeline makes an attempt to automatically reduce all science data taken in echelle mode. The reduction is performed using standard calibration solutions from a local calibration database which is refreshed every one or two months. All standard wavelength settings are supported, in all four binnings. Furthermore, for non-standard wavelength settings, a complete set of calibration files is processed in order to obtain solutions. This often works fine, but since the number of possible non-standard UVES configurations is very high, there is no experience with the pipeline behaviour in these cases. Generally any pipeline processing on the site is done on a best-effort basis.

At QC Garching, all SCIENCE data taken in Service Mode are pipeline-reduced, using the best available calibration solutions (quality-checked and closest in time, following the calibration cascade). The products of science reduction are also quality-checked. If any irregularity in the reduced data is found, an attempt is made to improve on this. If not possible, information about the poor quality is made available in the SM package. Present policy is to not suppress these data.

Raw science data are usually accompanied by slit viewing images. These types of raw science files exist:

Recipe. A science raw file is pipeline-processed by the pipeline recipe uves_obs_scired. In short, data are corrected for bias, interorder background, sky background, sky emission lines and cosmic ray hits. They are flattened, optimally extracted and finally merged. A response correction is also performed if an appropriate response curve exists for the setting. Find here a detailed record of the reduction steps.

Extraction. The UVES pipeline knows three extraction modes:

• optimum (OPT)
• average (AVG)
• two-dimensional (2D)

Optimum extraction is the default extraction mode for "ECHELLE" and "ECHELLE,ABSORPTION-CELL" observations; average extraction for "ECHELLE,SLICER" and "ECHELLE,ABSORPTION-CELL,SLICER" observations. A two-dimensional extraction is performed by QC Garching in addition if the DPR TYPE keyword is set to "OBJECT,EXTENDED".

Optimum extraction provides an optimized signal-to-noise ratio and is suitable for point sources. Average extraction is an alternative mode which simply averages any signal along the slit and above the sky background level. It may be a good choice in case of failure of the optimum extraction, e.g. at high exposure level. Two-dimensional extraction actually does not extract the signal at all, but provides a flattened and wavelength-calibrated solution with the flux redistributed into wavelength-slit coordinate space. This means that any spatial information along the slit is preserved and can be manually extracted later. Find more information here.

Optimum extraction. UVES science data reduced for Service Mode programmes are processed in optimal extraction mode (OPT) if the image slicers have not been used. The UVES pipeline assumes a Gaussian profile for the cross-dispersion flux distribution. This procedure

• yields optimum S/N,
• automatically discriminates against any non-Gaussian components in the cross-dispersion profile.
  

This means specifically

• automatic sky subtraction (since the sky background is treated as a pedestal),
• automatic cosmic ray rejection (whenever the hit has a non-Gaussian distribution),
• automatic sky emission line removal (since these fill up the whole slit and have a rectangular profile).

All Service Mode spectra reduced by QC Garching are flattened in order-bin space, i.e. with extracted flats. Flat extraction is done with the weight factors derived from the science spectrum.

The UVES pipeline also offers flattening in pixel space (the science spectrum is divided by the flat field before extraction). Checks done on high-S/N, featureless spectra show that the pixel-space flattening may yield slightly (less than 10%) higher S/N than extracted flattening. However, telluric lines in the flats are found to propagate much stronger into the extracted science spectrum in the case of pixel-space flattening. Since telluric lines in the flats fill the whole slit, they are effectively removed by the optimum extraction.

Flattening corrects for the blaze function within the orders. In the red settings (860 nm), flattening also efficiently removes fringing.

Since version 1.6 of the pipeline, an automatic response correction (flux calibration) can be applied to the extracted spectra. This additional step is performed for all Service Mode files observed after 2002-11-06, with the exception of settings with 520nm and 600nm central wavelength. Users interested in flux-calibrating older spectra can use the master response curves derived from standard star spectra (check out for more information and downloads here). Raw standard star measurements, if available, are also delivered in the 'calib' directories of the Service Mode package.

Products. Service Mode programmes receive the following reduction products per CCD (i.e. 1 BLUE for the blue arm, 1 REDLower and 1 REDUpper for the red arm). For optimal extraction, if flux-calibrated spectra are provided:

If flux calibration is not provided:

For slicer observations reduced with average extraction:

For slicer observations without flux-calibration:

Complete product list : The complete suite of products is available if the pipeline is installed and run at the user's home workstation.

The UVES pipeline, running on Service Mode data in optimum extraction mode, implicitly makes the following assumptions on the raw spectra:

• the object is a point source,
• the object is centred on the slit,
• the object has some continuum, i.e. is not a pure emission line object,
• the signal-to-noise ratio is not too high ( < 50 approximately).

Only if these assumptions are readily met by the nature of the observations, can useful results be expected. While some of these assumptions are checked as part of the QC checks done for assessment of the pipeline products, there is no alternative approach chosen (e.g. average extraction). This is the sole responsibility of the user.

UVES calibration
cascade

UVES reduction
scheme

The successful execution of science reduction requires a complete set of calibration products available.

All science data processed by QC Garching are reduced with a quality-checked set of calibration files which are usually generated from daytime calibrations taken immediately after the science night. In cases of 'attached calibrations' (WAVE and FLAT calibrations taken during nighttime), these are used for the reduction of the science data from the same OB.

QC Garching checks on the quality of the reduced science data. The following items are checked:

• proper selection of calibration data
  
  
• any peculiarities in the raw data
  e.g. unusual bias level.
  
  
• the cross-dispersion profile
  Here the exposure level, the centering and the sky background are controlled. These parameters are essential for assessing the quality of the optimum extraction.
  
  
• proper dispersion solution
  A comparison to the expected positions of frequent emission lines, e.g. the Balmer lines, reveals at least larger errors in wavelength calibration.
  
  
• proper extraction
  The full spectrum and its variance are plotted and checked visually for anomalies in the extraction (e.g. ripple patterns with order periodicity etc.)
  
  
• alarm flags
  Three parameters (temperature difference between science and wavelength calibration file; proper object centering; average S/N ratio) are controlled and flagged if threshold values are violated.

The QC process is continuously improved and adapted to experience.

QC1 reports: Since August 2001, extensive QC1 reports about SCIENCE reduction and signal tracing are delivered as part of the Service Mode package.

Wavelength calibration. The precision of the wavelength calibration in the course of science reduction is determined by

• the precision of the dispersion solution used for resampling,
• the difference of temperature between the science observation and the calibration solution used.

Typical values for the standard deviation (in milli-Angstroem) of the dispersion solution are given here. They are of the order of 2-8 milli-Angstroem.

The overall UVES spectrum shifts on the detector mainly depend on the air temperature in the UVES enclosure. The shift in the red arm is, in first approximation, 0.35 pixels/Degree C in the dispersion direction. One pixel is ~1.2 km/sec. The corresponding value in the blue arm is about 0.05 pixel/Degree C. Since 2002-01, an active compensation for such drifts has been implemented.

The temperature information is recorded in the file headers and reported in the 'TEMP' columns of the list files. By comparing the temperatures at the times of the science exposure and of the wavelength calibration applied to the data, the magnitude of the possible shift can be estimated from the gradients given above.

CCD defects. The UVES pipeline does not apply any correction for cosmetic defects of the CCDs. These are minimal because of the good cosmetic quality of the devices. However, there are a few defects which show up, notably in long integrations on faint objects on the two CCDs of the red arm.

There are four trails of hot pixels which appear in long exposures in the blue-side quadrant of the EEV chip (bluer side of the red arm mosaic, X coordinates 3896,3963, 4052 and 4140 in an unbinned fits file, middle of the chip in y). They occupy a single column and are almost parallel to the orders. They appear as broadish emission in the bluer part of the extracted spectrum of a faint object.

In the MIT-LL chip (red side of the CCD mosaic of the red arm) there is a trap in the column X1609 which might show up as a slight depression over ~130 pixels in the extracted spectrum of one order. In long, binned exposures this chip shows also an emission band starting on the red side and extending over the rows 2790-2850 with decreasing intensity toward the blue side of the echelle format. Since this band is perpendicular to the spectrum, it is usually well subtracted in the sky subtraction step of the optimal extraction.

One Angstrom Offset: A bug causing the wavelength calibration of Echelle mode SCIENCE products to be offset by 1.0 AA was introduced into the UVES pipeline in version 4.2.3, and subsequently corrected as of version 4.3.0 (the version currently available (as at 2009-03) from www.eso.org/pipelines). None of the buggy versions of the pipeline were ever made public, so this bug only affects Service Mode packages sent to PIs. All products for RAW data acquired during the nights from 2008-04-01 till 2008-08-27 are affected by this bug. Products created with versions of the pipeline sufferring from this bug can be corrected simply by adding 1.0 to the CRVAL1 keyword in the FITS header of the following products produced by the uves_obs_scired recipe:

• ERRORBAR_SCI_POINT_<BLUE|REDL|REDU>
• FLUXCAL_SCI_POINT_<BLUE|REDL|REDU>
• FLUXCAL_ERRORBAR_SCI_POINT_<BLUE|REDL|REDU>
• RED_SCI_POINT_<BLUE|REDL|REDU>

One Angstrom offset. In black the original spectrum produced by processing with version 4.2.3 of the pipeline. In blue, the same RAW data reprocessed with exactly the same set of calibrations, but with version 4.3.0 of the pipeline. In red, but almost completely covered over by the blue spectrum, the product generated by version 4.2.3 with 1.0 added to the CRVAL1 keyword of the header. The offset 4.2.3 product and the 4.3.0 product are not exactly identical, but the difference is clearly much smaller than the noise in the spectrum.

Residual fringes in the red: Many red spectra (in particular those taken with the standard setup at central wavelength 860 nm) display arteficial ondulations in the flat-fielded and extracted spectra. Their strength increases towards redder wavelengths. They have a typical spatial scale of 0.2-0.3 nm and a typical amplitude of roughly 10%. They result from residual fringes, i.e. an imperfect fringe cancellation.

The pipeline default flat-fielding method is called "extract". It was also used here and consists of the following sequence: 1. optimum extraction of science spectrum; 2. optimum extraction of the flat-field, using the same spatial profile; 3. divide both by each other. This method does not completely suppress fringes, as was realized after finishing the reprocessing project. An alternative method, called "pixel", essentially avoids the residual-fringes problem. It first divides the flat-field into the science spectrum pixel by pixel (hence the name), and then provides the extraction. The figure below demonstrates the difference:

Residual fringes problem. The upper spectrum shows the result spectrum of the default flat-fielding method ("extract"), as compared to the lower (scaled and shifted) spectrum obtained with the "pixel" method. The upper spectrum displays the residual fringes as 2-3 A wide ondulations at the 10% level. The lower spectrum is free from these artefacts.

As a drawback, there is only a rather simple order-merging mechanism available with the pixel method, which truncates the signal from the orders at specified positions, thereby causing larger interorder gaps in the redmost spectral parts. But generally this method yields greatly improved S/N for red spectra. It is currently available with the public release 3.4.0+ , but not with the archived product spectra.

High signal. Optimum extraction may face problems in cases of high signal, i.e. when the differences between the true shape of the instrumental profile and the assumed Gaussian start to become systematic rather than random. These problems are found more frequently in the blue than in the red regions.

Two kinds of artefacts may show up then:

• Ripples: Under certain conditions small-scale ripples appear in the extracted spectra. Especially affected are high-S/N data with a small FWHM (good seeing). The effect scales roughly as (S/N)/FWHM². It may be due to inappropriate virtual sampling of the cross-dispersion profile, or due to the non-linear response of the CCD. By default the cross-dispersion profile is sampled with 1/5 pixel size. If the profile is only a few pixels wide this results in a quasi-periodic sampling error due to the finite size of the numerical grid. If due to the extraction algorithm, this effect can be avoided by a smaller sampling size. Pipeline versions 3.2.x and later have a self-adaptive mechanism which finds and sets the appropriate sampling size after a coarse initial estimate of S/N. Earlier pipeline versions do not have this mechanism and have been executed always with sampling size 1/5. Those data are therefore more subject to the ripple effect.
  An alternative in those cases is AVERAGE extraction which does not have sampling problems. However the products then may include cosmic rays. Within the QC processing environment there is no option to dynamically decide between AVERAGE and OPTIMUM extraction, but with an exported pipeline the user may decide to use this alternative extraction scheme.

• Small-scale scatter with a period of about 20-30 pixels. This problem is due to problems in the order-tracing algorithm which become stronger in case of high signal and good seeing, hence small FWHM. Since the echelle orders are inclined by a few degree, their centre "jumps" vertically by one pixel every 20-30 pixels. This affect degrades the S/N ratio of the spectrum considerably. Again, average extraction will produce better results.
  This problem is much stronger for UVES pipeline products obtained until October 2000. The recipe has been improved considerably since then.
  

Some of these cases are detected by quality check procedures, and a short note is included in the Service Mode package. However, this is not generally true, and we strongly recommend to carefully check the results if they could be affected by this problem.

Flux-calibrated spectra not always available: master-response curves are required to proceed from the extracted spectra to the flux-calibrated spectra. These are not available for all settings but only for the standard settings (the ones used in Service Mode) centred on: 346, 390, 437, 520, 564, 580, 600, 860 nm.

Wavelength range of flux-calibrated spectra: some of the master response curves used for flux calibration have the initial and the final 25 Angstrom cut off, hence the flux-calibrated spectra have a slightly smaller wavelength range than the extracted ones.

Flat-fielding and wavelength dispersion before 2001-10-01: in the early history of UVES data processing, the proper association between calibrations and science was done with all but one relevant instrument parameter: slit width. This means that master flats (UV_MFLT), and the dispersion tables (UV_PLI1,2,3), were correctly created and archived but only for one value of slit width per setting, although maybe raw science and calibration data with several such values were existing. In other words: raw calibration data were complete from the beginning of UVES operations, but master calibrations in the archive are incomplete, for certain dates before 2001-10-01. With the operational setup of the UVES reprocessing project, master calibrations are not recreated but just downloaded from the archive. Therefore, the correct match of slit width and calibrations cannot be guaranteed for science data before that date. Usually, this has little impact on the data quality, but in extreme cases (e.g. science data taken with the 0.3 arcsec slit, master_flat with 1.5 arcsecs) this may affect the extraction quality and may explain artefacts, especially with a period of one echelle order. Check here for a particularly impressive example.

Order gaps: The 860 REDU spectra have incomplete spectral coverage (echelle orders are truncated by the chip). This is a feature of the instrument.

Object centering: if the object is not well-centered in the slit, extraction may fail or produce all kinds of artefacts, including extraction ripples (here).

Object not point-like: if the object has a complex structure (binary, background, extended object), the extraction may fail or produce artefacts. The algorithm assumes a point source, with a near-gaussian profile shape (although the profile is measured and fitted, for stability, with a more complex function). It locks to the brightest signal in the slit. Complex sources should better be extracted with the EXTENDED method and treated individually.
The certification process has not been very sensitive to complex object structure. It may have happened that corrupted product spectra of an extended object, or a binary object, have passed certification without comment!

Emission-line objects: without a traceable continuum, the pipeline may produce artefacts. In many cases such products can be recognized automatically by an exceptionally small (< 0.5") or large (> 3") FWHM, and a comment is provided. Still the spectra may be useful since the emission lines may be extracted correctly (e.g. see here). But such data have to be inspected with care.

Order identification. In a few cases problems have occured with order identification in settings with 346nm central wavelength: the orders found by the pipeline in the order defintion flat and during extraction/wavelength calibration of the science frame may be different. This is not recognised by the pipeline and the lines in the final wavelength-calibrated spectrum appear shifted by one order. An example is given here where the overlap between an observation with 346nm and 437nm central wavelength is shown. The problem may be resolved by choosing a different order definition flat.

Sky emission lines. Optimum extraction may over/under-corrected bright sky emission lines by 5% or more.

Radial velocity correction: The wavelength-rebinned spectra are not corrected for barycentric and heliocentric motion. If you want to apply these corrections, their values may be found in the header, as keys HIERARCH.ESO.QC.VRAD.BARYCOR and HELICOR, resp. Moreover, Until 2001-09-17 (date of acquisition), a bug in the radial velocity correction as calculated by the pipeline may have caused wrong values. Please check carefully (FITS keywords HIERARCH ESO QC VRAD BARYCOR and HELICOR; also stated in the listing file 'list_of_all_red.txt'). This bug is present in version 1.2 and earlier of the uves pipeline, and solved with version 1.3.