[ ESO ]
UVES:
recipe uves_obs_scired
 

CAL | HC | refs | QC
QUALITY CONTROL
    HOME
UVES QC
Trending & QC1
Pipeline
ECH: science recipe
Data Packages
Data Management
Solar&sky spectra
QC links:
UVES-Echelle UVES-MOS

DPR CATG = SCIENCE, DPR TYPE = OBJECT

[science.gif 10K]
Raw science frame

Recipe. The pipeline recipe uves_obs_scired performs the full reduction of UVES science frames.

Scheme. The following steps are performed:

  • preparation of data;
  • de-bias with master bias;
  • subtract interorder background (background table);
  • extract orders (order table), modes OPT or AVG; if OPT, store weight map;
  • extract master flat in same way as science spectrum (OPT or AVG); if OPT, use weight map; normalize extracted flat by its median;
  • divide extracted science spectrum by extracted and normalized master flat;
  • resample science spectrum using line table(s);
  • merge the orders;
  • flux-calibrate the final spectrum if an appropriate master response curve is available .

The pipeline processing of science data starts with data preparation: the raw input frames are rotated and in the case of the red mosaic data, split for each detector. The prescan and overscan regions are trimmed. The instrumentís background light is corrected in two steps. A master bias frame is subtracted. The interorder background light is estimated by a minimum filter being applied in small boxes at the background positions defined in the background table. The optimal extraction provides the object signal and its variance. The flat-field correction is performed in pixel-order space on the extracted data. The signal is then resampled to constant wavelength bins. Finally the orders are merged into a single spectrum.

A flux-calibrated spectrum can be produced in addition to the extracted spectrum if a response curve is provided. Details about flux calibration and master response curves are given here.

The UVES pipeline run on science data by QC Garching (before October 2011) used OPT extraction if the image slicer had not been used, and AVG extraction for image slicer data. Flat fields were applied in pixel-order space. Flux-calibrated spectra were provided for all settings except for those with 520nm and 600nm central wavelength.


Extraction. Extraction methods are

  • OPT = optimum,
  • AVG = average,
  • 2D = two-dimensional.

Optimum extraction. Optimum extraction fits a profile to the cross-dispersion flux distribution and extracts the flux per bin, using weights which are derived from the fitted profile. The UVES pipeline assumes a Gaussian cross-dispersion profile. The result data are automatically sky subtracted and free from sky emission lines and cosmic ray hits.

Optimal extraction is a minimal variance estimator of the flux at each wavelength. The algorithm introduced by Horne (1986) for long-slit data assumes that the illumination fractions of the spatial profile vary smoothly with the wavelength and can be adjusted by low-order polynomials. This assumption does not generally hold in echelle spectroscopy due to the inclination of the orders. Resampling the data along the spatial profiles would introduce periodic profile distortions and noise. Different methods have been developed for cross-dispersed spectroscopy (Marsh, 1989; Mukai, 1990) involving data resampling for the sole purpose of estimating the weights.

The optimal extraction algorithm of the UVES pipeline is based on profile fitting by chi-square minimization. It estimates the object signal and sky background and performs cosmic ray rejection under the assumption of a Gaussian profile of the light distribution along the slit. The extraction is performed independently on each order. The Gaussian shape is a reasonable assumption for low to medium signal-to-noise data.

First the full-width at half-maximum and the position of the object is estimated for different subsections along the order. Then through a chi-square minimization, the best values for the amplitude and the background level are obtained and optimized through a kappa-sigma clipping routine.

Cosmic rays are interpreted as strong deviations from the Gaussian shape in their vicinity (where 'strong' means 5 sigma deviation). They are recognized and filtered out, if they satisfy the condition:

(profile - fit)^2/variance > THRESH**2 where THRESH = 5 + alpha*S/N

where profile is the cross-dispersion profile of the raw data, fit is the Gaussian fit, and S/N is the local signal-noise ratio (in cross-dispersion direction). alpha is an empirical parameter of order 1.

Sky emission lines, since filling the whole slit, are removed during the extraction as they contribute to the linear (constant) term of the spatial profile, hence do not contribute to the extracted total intensity. Tests have shown that the sky line subtraction achieves an accuracy better than 5%. Strong sky emission lines may remain visible in the extracted spectrum as residual light and of course by the larger variance of the extracted spectrum.

One of the common difficulties in optimal extraction schemes is to prevent the rejection of valid data samples by the cosmic ray rejection methods in particular on high signal-to-noise data. In the UVES pipeline the rejection threshold is therefore adjusted to the signal-to-noise so that the cosmic ray rejection is relaxed for increasing signal-to-noise data, converging to an average extraction scheme for high signal-to-noise data. Presently this method appears to be appropriate for data with a signal-to-noise per bin up to fifty.

Optimal extraction supports the propagation of variance. The initial variance is estimated on the raw images by a noise model including read-out and photon noise. The variance images are then transformed together with the data frames on each processing step. Finally this information is used to optimally merge the orders and to deliver error bar estimates on the result spectrum.

Horne, K., 1986, An optimal extraction for CCD spectroscopy, PASP, 98:609-617
Marsh, T.R., 1989, The extraction of highly distorted spectra, PASP, 101:1032-1037
Mukai, K., 1990, Optimal extraction of cross-dispersed spectra, PASP, 102:183-189

Average extraction. The UVES pipeline can also be run in AVG and 2D extraction modes. In AVG mode,

  • the slit is divided into three equally sized subwindows (sky1, object, sky2);
  • the flux in the object window is extracted without weights;
  • the averaged flux from the two sky windows is used to provide a sky correction;
  • no correction for cosmic ray hits is done.

For higher signal-to-noise, average extraction is recommended rather than optimum extraction because then deviations of the cross-dispersion profile from the fitted Gaussian become systematic rather than random. Also the grid effect due to the inclined echelle orders may cause high-frequency ripples. In average extraction, the source is centered on the slit and sky windows are determined. The object contribution is summed in the object sub-window, and the sky contribution is averaged and subtracted from the object. A drawback of average extraction is the missing cosmic ray rejection. However, in practice average extraction will mostly be used for bright sources with correspondingly short exposure times.

The propagation of variance is also supported for average extraction.

In case of image slicer data, the subwindows showing only the sky contribution are in general too small for an automatic procedure. Therefore, sky subtraction is not performed. Average extraction gives the sum over the slicer length (after subtraction of the inter-order background).

2D extraction. This mode is available for extended or multiple sources. The raw data are pre-processed as in the case of average or optimum extraction (i.e. flattened and wavelength-calibrated), but instead of the flux extraction a rebinning of the flux into a two-dimensional wavelength-slit coordinate grid is performed, hence preserving all spatial information along the slit. Naturally, there is no background subtraction. The products are intended for further processing by hand since knowledge about the physical nature of the source is required to select the best extraction strategy.


Order trace table. In the course of the optimum extraction procedure, positions and widths of the Gaussian fit to the signal are determined and stored in a product table (PRO.CATG = ORDER_TRACE), as well as the linear offsets. These data can be very useful for assessing the quality of the extraction, especially in critical cases (faint sources, emission line sources, high exposure level, extended or multiple sources).

Check here for a description of ORDER_TRACE tables.

Cosmic ray table. The optimum extraction routine identifies features with non-Gaussian flux distribution as cosmic ray hits. Their corresponding X and Y coordinates (in order-bin space, i.e. after de-rotation and per order in Y slit coordinates) and their associated flux values are stored in the cosmic ray table CRMASK. Check here for a description.


Flattening. The UVES pipeline offers two modes for the flattening procedure (plus the option to omit flattening):

  • extract the master flat before dividing it into the science data,
  • apply the master flat in pixel space and then extract the flattened science data.

In case of extracted flat, extraction is done using the weights from the extracted science spectrum.

In case of flattening in pixel space, the science spectrum is first divided by the flat and then extraction of the ratio is done. Checks done on high-S/N, featureless spectra show that the pixel-space flattening may yield slightly (less than 10%) higher S/N than extracted flattening. However, telluric lines in the flats are found to propagate much stronger into the extracted science spectrum in the case of pixel-space flattening. Since telluric lines in the flats fill the whole slit, they are effectively removed by the optimum extraction.

Sampling. Sampling to wavelength space uses the dispersion solution from the line table. The step size is set to 2/3 of the minimum bin size in Angstrom. This bin size, in most cases, is smaller than the "optimum" number derived from the effective resolution. In order to optimize the S/N ratio of the product, one may wish to finally rebin the products to

Dl = lmin/(2*R)

where lmin is the minimum wavelength of the spectrum, and R is the effective resolving power for the slit width used, as measured as QC parameter in the arc lamp spectra.

Blaze function. Proper merging of the adjacent echelle orders requires the correction of the blaze function. In standard pipeline reduction, the blaze function is estimated from a flat-field exposure, which after extraction, wavelength calibration and smoothing enables merging of adjacent orders with an accuracy of a few percent.

The blaze function may also be determined from a spectrophotometric standard star, but this is not part of the standard UVES science reduction.

Response. A response correction (flux-calibration) can be performed if a response curve is provided to the pipeline. Information on response curves is available here.

Merging. The orders are merged into a single spectrum with OPT or AVG option. Overlapping regions are merged using weighted means. No parts of the spectrum are chopped.

Products. The following products are created by the recipe (if OPT extraction is used and flux-calibrated spectra are produced):

product category (PRO CATG) product number* format comments
RED_SCI 0000 (+0013) 1D (wavelength space) final science spectrum (RV corrections to heliocentric and barycentric scale are provided in FITS header)
MERGED_SCI 0001 (+0014) same (for OPT, this is identical to RED_SCIENCE)
WCALIB_SCI 0002 (+0015) 2D (order, wavelength) 2D wavelength calibrated spectrum, unflattened
WCALIB_FF_SCI 0003 (+0016) 2D (order, wavelength) as WCALIB_SCIENCE, flattened
WCALIB_FLAT_OBJ 0004 (+0017) 2D (order, wavelength) FLAT file, identical in format to WCALIB_SCIENCE
ERRORBAR_SCI 0005 (+0018) 1D (wavelength space) errorbar file corresponding to RED_SCI (contains standard deviation of extraction)
VARIANCE_SCI 0006 (+0019) 2D (order, bin) variance of extraction (not merged)
BKG_SCI 0007 (+0020) 2D (X,Y pixels) fitted background (used for extraction)
ORDER_TRACE 0008 (+0021) tfits table order positions, FWHM values from OPT extraction
CRMASK 0009 (+0022) tfits table

cosmic ray table (since 2001-09-17)

MERGED_SKY 0010 (+0023) 1D (wavelength space) merged relative sky emission
FLUXCAL_SCIENCE 0011 (+0024) 1D (wavelength space) flux-calibrated science spectrum (in 10E-16 erg/s/cm**2/A)
FLUXCAL_ERRORBAR 0012 (+0025) 1D (wavelength space) errorbar file corresponding to FLUXCAL_SCIENCE
* numbers in brackets refer to REDU CCD, others to BLUE or REDL CCD. The numbering scheme refers to UVES pipeline version 1.6 or higher. Previous versions do not have the MERGED_SKY, BKG_SCI files, ORDER_TRACE, CRMASK tables, and/or FLUXCAL files.

If flux-calibrated spectra have not been produced you will find the following files (OPT extraction):

product category (PRO CATG) product number* format comments
RED_SCI 0000 (+0011) 1D (wavelength space) final science spectrum (RV corrections to heliocentric and barycentric scale are provided in FITS header)
MERGED_SCI 0001 (+0012) same (for OPT, this is identical to RED_SCIENCE)
WCALIB_SCI 0002 (+0013) 2D (order, wavelength) 2D wavelength calibrated spectrum, unflattened
WCALIB_FF_SCI 0003 (+0014) 2D (order, wavelength) as WCALIB_SCIENCE, flattened
WCALIB_FLAT_OBJ 0004 (+0015) 2D (order, wavelength) FLAT file, identical in format to WCALIB_SCIENCE
ERRORBAR_SCI 0005 (+0016) 1D (wavelength space) errorbar file corresponding to RED_SCIENCE (contains standard deviation of extraction)
VARIANCE_SCI 0006 (+0017) 2D (order, bin) variance of extraction (not merged)
BKG_SCI 0007 (+0018) 2D (X,Y pixels) fitted background (used for extraction)
ORDER_TRACE 0008 (+0019) tfits table order positions, FWHM values from OPT extraction
CRMASK 0009 (+0020) tfits table

cosmic ray table (since 2001-09-17)

MERGED_SKY 0010 (+0021) 1D (wavelength space) merged relative sky emission
* numbers in brackets refer to REDU CCD, others to BLUE or REDL CCD. The numbering scheme refers to UVES pipeline version 1.3 or higher. Previous versions do not have the MERGED_SKY, BKG_SCI files, ORDER_TRACE and/or CRMASK tables.

For slicer observations reduced in AVG mode, the following files are produced (pipeline version 1.6 or higher, with flux-calibrated spectra):

product category (PRO CATG) product number* format comments
RED_SCIENCE_SLICER 0000 (+0008) 1D (wavelength space) final science spectrum (RV corrections to heliocentric and barycentric scale are provided in FITS header)
MERGED_SCIENCE_SLICER 0001 (+0009) same (this is identical to RED_SCIENCE_SLICER)
WCALIB_SCIENCE_SLICER 0002 (+0010) 2D (order, wavelength) 2D wavelength calibrated spectrum, unflattened
WCALIB_FF_SCIENCE_SLICER 0003 (+0011) 2D (order, wavelength) as WCALIB_SCIENCE, flattened
WCALIB_FLAT_OBJ 0004 (+0012) 2D (order, wavelength) FLAT file, identical in format to WCALIB_SCIENCE
ERRORBAR_SCIENCE_SLICER 0005 (+0013) 1D (wavelength space) errorbar file corresponding to RED_SCIENCE (contains standard deviation of extraction)
VARIANCE_SCIENCE_SLICER 0006 (+0014) 2D (order, bin) variance of extraction (not merged)
BKG_SCI 0007 (+0015) 2D (X,Y pixels) fitted background (used for extraction)
FLUXCAL_SCIENCE 0008 (+0018) 1D (wavelength space) flux-calibrated science spectrum (in 10E-16 erg/s/cm**2/A)
ERRORBAR_SCIENCE 0009 (+0019) 1D (wavelength space) errorbar file corresponding to FLUXCAL_SCIENCE
* numbers in brackets refer to REDU CCD, others to BLUE or REDL CCD.

If flux-calibrated spectra have not been produced then you will find the following files for slicer observations:

product category (PRO CATG) product number* format comments
RED_SCIENCE_SLICER 0000 (+0008) 1D (wavelength space) final science spectrum (RV corrections to heliocentric and barycentric scale are provided in FITS header)
MERGED_SCIENCE_SLICER 0001 (+0009) same (this is identical to RED_SCIENCE_SLICER)
WCALIB_SCIENCE_SLICER 0002 (+0010) 2D (order, wavelength) 2D wavelength calibrated spectrum, unflattened
WCALIB_FF_SCIENCE_SLICER 0003 (+0011) 2D (order, wavelength) as WCALIB_SCIENCE, flattened
WCALIB_FLAT_OBJ 0004 (+0012) 2D (order, wavelength) FLAT file, identical in format to WCALIB_SCIENCE
ERRORBAR_SCIENCE_SLICER 0005 (+0013) 1D (wavelength space) errorbar file corresponding to RED_SCIENCE (contains standard deviation of extraction)
VARIANCE_SCIENCE_SLICER 0006 (+0014) 2D (order, bin) variance of extraction (not merged)
BKG_SCI 0007 (+0015) 2D (X,Y pixels) fitted background (used for extraction)
* numbers in brackets refer to REDU CCD, others to BLUE or REDL CCD.