mirror sites: PL (internal link) HQ     [?]

Data Processing and Quality Control

CAL | HC | refs | QC
Trending & QC1
science data
Data Management
QC links:

GIRAFFE pipeline: science data

general GIRAFFE


On Paranal, the quick-look pipelines make an effort to automatically reduce all science data. The reduction is performed using standard calibration solutions from a local calibration database which is refreshed every few months. Generally any pipeline processing on the site is done on a best-effort basis. Its purpose is to offer a quick look to assess data quality etc.

Until the end of P87 (September 2011), science data were processed by QC Garching with the best possible (certified) calibrations solutions. The products were ingested into the Science Archive and delivered to the PIs. This service has been terminated with the begin of October 2011.

For some instruments, science-grade data products are created by QC Garching, using certified pipelines. Currently these are 1D spectral data products from UVES (Echelle mode), XSHOOTER (Echelle mode), and GIRAFFE (Medusa MOS modes). See the QC pages for those instruments for more information.

For GIRAFFE Medusa1/2 modes, more than 1 mio. science-grade 1D spectral data products are available covering the entire data history. More ...


1.0 since December 2005 same all setups (except for 3) pipeline supported
2.2 since June 2007 delivers data cube for Argus data as above
2.3 since October 2007 removes dark (incl. CCD glow); the dark and glow removal is applied to the reduced data in the SM data packages as of P80  
2.5 since January 2008 SimCal correction applied OBJECT,SimCal data
2.6 since April 2008 for SIMCAL data, barycentric, heliocentric, geocentric corrections are given in a binary table (not applied!) OBJECT,SimCal data
2.6.2 since May 2008 new set of static calibrations; modified bias removal strategy (method "PROFILE"); dark subtraction suspended (not required for new CCD) all
2.6.7 since August 2010 bug fix for S/N calculation; improved bias subtraction method ("PROFILE+CURVE") all
2.8.9 since April 2011 for SIMCAL and OZPOZ data: barycentric, heliocentric, geocentric corrections are given in a binary table (not applied!) all


GIRAFFE science data come in two main types: OzPoz (just the OBJECT and SKY fibres illuminated on the sky), and SimCal (with the additional SIMCAL fibres illuminated by the calibration unit). The data types are identified by the DPR TYPE keyword of the FITS headers. Find information about the general GIRAFFE data format and about its CCD here. The information contained in the simultaneous calibration fibres is applied by the pipeline since January 2008.

The GIRAFFE science frames come per slit which can be MEDUSA1, MEDUSA2, IFU1, IFU2, or Argus.

The SCIENCE varieties further split into two different gratings (HR and LR) with a set of central wavelengths (which are listed here).

Note: It is not visible from the DPR TYPE if a FLAMES OB uses UVES and GIRAFFE at the same time, or just GIRAFFE stand-alone. Data from these two spectrographs taken simultaneously are reduced, packed, and shipped independently.
DPR CATG DPR TYPE DPR TECH description FITS headers
SCIENCE OBJECT,OzPoz MOS (for Medusa) or IFU (for IFU and Argus) only science and sky observations; no simultaneous calibrations taken science_ozpoz.hdr
SCIENCE OBJECT,SimCal MOS (for Medusa) or IFU (for IFU and Argus)
simultaneous calibrations taken


Overview of science recipe and QC plots

Raw frame recipe description
SCIENCE, OzPoz; all slits giscience debias, extract, wavelength-calibration
since April 2005: flat-fielding
since October 2007: dark and CCD glow removal
SCIENCE_RBNSPECTRA, SCIENCE_RBNERRORS (rebinned spectra and their errors); SCIENCE_EXTSPECTRA, SCIENCE_EXTERRORS (extracted spectra and their errors)
SCIENCE, SimCal as above same as OzPoz, plus correction from simultaneous calibration applied (since 2008-01-18)
as above
SCIENCE, IFU and Argus only as above there is in addition the reconstructed image

as above, plus SCIENCE_RCSPECTRA, SCIENCE_RCERRORS (2D reconstructed image and error);
since June 2007: plus SCIENCE_CUBE_SPECTRA and SCIENCE_CUBE_ERRORS (3D reconstructed image and error), for Argus only

The GIRAFFE calibration scheme, including SCIENCE data reduction, is shown here.

Recipe. A science raw file is pipeline-processed by the pipeline recipe giscience. Data are corrected for bias. Optionally a bad pixel map is used to correct for bad pixels (this is not done in QC operations; this step is not required for the new CCD). For the old CCD, the background (composed of dark current and CCD glow) was subtracted using a recent master_dark frame; the new CCD (since 2008-05-26) does not require this. The fibres are then extracted using the localization and width solutions from the flat field data. They are wavelength-calibrated using the dispersion-solution from the arc-lamp (wave) data. The spectra are then flat-fielded, to correct for overall efficiency, relative (fibre-to-fibre) efficiency, and fringing. For IFU and Argus data, a reconstructed image and a full 3D datacube are calculated.

Bias removal. For the new CCD (since 2008-05-26), the bias is averaged from the overscan region (columns 1-50) into a 1D profile, and subtracted as a 2D expanded function. This method is called PROFILE (--bsremove-method=PROFILE). It has the advantage that the overall BIAS level is always correct, despite the small fluctuations observed to sometimes occur between BIAS calibration files and SCIENCE files. On the other hand, the overscan region has small residual fluctuations which are probably of random nature. To avoid propagating them as dark and bright stripes and bands into the products of the giscience recipe (and actually also the gistandard recipe), there is the method PROFILE+CURVE which applies a 5th order polynomial fit to the 1d profile before applying the result to the science data. This improved method is used since version 2.6.7 (August 2010). It is applied both to science and to standard star data.

Result of bias removal method PROFILE. The displayed spectrum is an extracted but not yet binned ARGUS standard star product STD_EXTSPECTRA. Each column is an extracted fibre. The artefacts due to the low-amplitude bands in the overscan region are visible as horizontal structures (columns are still in pixel space). The spectrum displayed is extracted from the vertical cut marked by the white solid line.
Result of bias removal method PROFILE+CURVE. This is the same spectrum after applying the bias method PROFILE+CURVE with a 5th order polynomial. There are much less bands visible, most are removed by the smooth polynomial.
Overscan BIAS profiles. Black is the raw BIAS profile (vertical overscan signal, collapsed over the first 50 columns), red is a smoothed version with a filter, green is the 5th order polynomial fit used by the PROFILE+CURVE method.

Extraction. By default, the GIRAFFE pipeline sums up all pixels within the mask defined in the flat-field localization file (extraction method 'SUM'). An optimum extraction algorithm is available (methods 'HORNE' or 'OPTIMAL'). This extraction mode has been developed to better handle the scattered light component in the background. The scattered light is more an issue for the IFU modes than for the Medusa data. Also, it performs very slow and does not always deliver stable solutions. For Medusa, the sky fibres are not marked in a standard scheme so the pipeline cannot find the and subtract the sky signal. Finally, this extraction mode requires a different flat field product than the one used for the SUM method. Therefore optimum extraction is not applied in the pipeline scheme used at QC.

Sky. There is no sky subtraction provided by the pipeline. The sky fibres (either the dedicated ones for IFU and Argus, or the user-defined ones for Medusa) are processed in exactly the same way as the target fibres. If sky subtraction is desired, the user has to identify the sky fibres, fit a model to their signal, and subtract this from the pipeline-provided solution. Find more information about the stability of the fibre transmission here.

Fibre-to-fibre efficiency. Differences of the fibre-to-fibre efficiency are corrected for by the flat-fielding step. The coefficients used for the correction are stored in the product binary table, column TRANSMISSION. (If you are unhappy with the applied correction, divide these factors into the fibre signals for undoing the correction.)

Fringing. Fringing is removed by the flat-fielding.

Cross-talk and background. With version 2.3, the pipeline can correct for the notorious glow of the old CCD using a close-in-time master_dark.

There is presently no pipeline correction for cross-talk.

Image reconstruction. In the IFU modes (IFU and Argus), a reconstructed image is calculated. It is obtained by collapsing the spectra in dispersion direction, and re-arrange the signal according to the X and Y sky coordinates as provided in the fibre table. With version 2.3, there is also the full 3D datacube delivered for Argus (X,Y,wavelength).


All setups (except for three very blue ones) are pipeline-supported.

There is no scattered light correction.

The sky is not subtracted, sky emission lines and sky continuum are imprinted on the science data. The sky signal is available for subtraction by the user from the user-defined SKY fibres.

The following issues with the science reduction are known:

  • wavelength calibration covers not always the maximum possible wavelength range (especially at the long end)
  • the old CCD ("Bruce", used until 2008-03-13) had an area with a bright glow of variable intensity on its upper right edge which does not cancel out upon flat-fielding; it caused an artificial background for fibres 105 - 125 (in the Medusa case) and 250+ (ARGUS, IFU), resp. This glow can be reasonably well removed with a master dark frame. There is no such issue with the new CCD, no master-dark subtracted.
  • three hot columns (at X coordinates 368 to 370, corresponds to fibre #24 for Medusa) are presently not suppressed. There is no meaningful signal in these columns. The extracted signal in this area may be corrupt.
  • The pipeline extracts only complete fibres, i.e. the last one or two at the right edge of the chip are not extracted.
  • The new CCD "Carreras" has a slightly more variable BIAS than the old one; therefore, it has been decided that the BIAS is removed from all raw frames, incl. SCIENCE data, by averaging the OVERSCAN region into a 1D column, expanding it, and removing it from the raw frames. No master bias is subtracted. However, created master bias frames are archived.
Science-grade data products

Since 2014, more than 1 mio. science-grade 1D spectral data products are available for the entire data history of GIRAFFE Medusa1/2 mode. New data are added on a monthly basis; data products older than one year are freely accessable, younger data products are available to PIs. More ...

Find the phase3 spectral archive interface here.