On Paranal, the quick-look pipelines make an effort to automatically
reduce all science data. The reduction is performed using standard calibration
solutions from a local calibration database which is refreshed every few
months. Generally any pipeline processing on the site is done on a best-effort
basis. Its purpose is to offer a quick look to assess data quality etc.
Until the end of P87 (September 2011), science data were processed by QC Garching with the best possible (certified) calibrations solutions. The products were ingested into the Science Archive and delivered to the PIs. This service has been terminated with the begin of October 2011.
For some instruments, science-grade data products are created by QC Garching, using certified pipelines. Currently these are 1D spectral data products from UVES (Echelle mode), XSHOOTER (Echelle mode), and GIRAFFE (Medusa MOS modes). See the QC pages for those instruments for more information.
For GIRAFFE Medusa1/2 modes, more than 1 mio. science-grade 1D spectral data products are available covering the entire data history. More ...
GIRAFFE SCIENCE DATA REDUCTION
since December 2005
all setups (except for 3) pipeline supported
since June 2007
delivers data cube for Argus data
since October 2007
removes dark (incl. CCD glow); the dark and glow removal
is applied to the reduced data in the SM data packages as of P80
since January 2008
SimCal correction applied
since April 2008
for SIMCAL data, barycentric, heliocentric, geocentric corrections are
given in a binary table (not applied!)
since May 2008
new set of static calibrations; modified bias removal strategy
(method "PROFILE"); dark subtraction suspended (not required for new CCD)
since August 2010
bug fix for S/N calculation; improved bias subtraction method ("PROFILE+CURVE")
since April 2011
for SIMCAL and OZPOZ data: barycentric, heliocentric, geocentric
corrections are given in a binary table (not applied!)
OF RAW SCIENCE DATA
GIRAFFE science data come in two main types: OzPoz (just the OBJECT
and SKY fibres illuminated on the sky), and SimCal (with the additional SIMCAL
fibres illuminated by the calibration unit). The
data types are identified by the DPR TYPE keyword of the FITS headers.
Find information about the general GIRAFFE data format and about its CCD
here. The information
contained in the simultaneous calibration fibres is applied
by the pipeline since January 2008.
The GIRAFFE science frames come per slit which can be MEDUSA1, MEDUSA2,
IFU1, IFU2, or Argus.
The SCIENCE varieties further split into two different gratings (HR and
LR) with a set of central wavelengths (which are listed here).
Note: It is not visible from the DPR TYPE if a FLAMES OB uses
UVES and GIRAFFE at the same time, or just GIRAFFE stand-alone. Data from these two
spectrographs taken simultaneously are reduced, packed, and shipped independently.
MOS (for Medusa) or IFU (for IFU and Argus)
only science and sky observations; no simultaneous
debias, extract, wavelength-calibration
since April 2005: flat-fielding
since October 2007: dark and CCD glow removal
(rebinned spectra and their errors); SCIENCE_EXTSPECTRA, SCIENCE_EXTERRORS
(extracted spectra and their errors)
same as OzPoz, plus correction from simultaneous
calibration applied (since 2008-01-18)
SCIENCE, IFU and Argus only
there is in addition the reconstructed
as above, plus SCIENCE_RCSPECTRA,
SCIENCE_RCERRORS (2D reconstructed image and error);
since June 2007: plus SCIENCE_CUBE_SPECTRA and SCIENCE_CUBE_ERRORS
(3D reconstructed image and error), for Argus only
The GIRAFFE calibration scheme, including SCIENCE data reduction, is
Recipe. A science raw file is pipeline-processed by the pipeline
Data are corrected for bias. Optionally a bad pixel map is used to correct for bad
pixels (this is not done in QC operations; this step is not required for the new CCD).
For the old CCD, the background (composed of dark current and CCD glow) was subtracted
using a recent master_dark frame; the new CCD (since 2008-05-26) does not require
this. The fibres are then extracted using the localization and width solutions from
the flat field data. They are wavelength-calibrated using the dispersion-solution
from the arc-lamp (wave) data. The spectra are then flat-fielded, to correct for
overall efficiency, relative (fibre-to-fibre) efficiency, and fringing. For IFU
and Argus data, a reconstructed image and a full 3D datacube are calculated.
Bias removal. For the new CCD (since
bias is averaged from the overscan region (columns 1-50) into a 1D profile, and subtracted
as a 2D expanded function. This method is called PROFILE (--bsremove-method=PROFILE).
It has the advantage that the overall BIAS level is always correct, despite the small
fluctuations observed to sometimes occur between BIAS calibration files and SCIENCE
files. On the other hand, the overscan region has small residual fluctuations which
are probably of random nature. To avoid propagating them as dark and bright stripes
and bands into the products of the giscience recipe (and actually also the gistandard
recipe), there is the method PROFILE+CURVE which applies a 5th order polynomial fit
to the 1d profile before applying the result to the science data. This improved method
is used since version 2.6.7 (August 2010). It is applied both to science and to standard
Result of bias removal method PROFILE. The
displayed spectrum is an extracted but not yet binned ARGUS standard star
product STD_EXTSPECTRA. Each column is an extracted fibre. The artefacts
due to the low-amplitude bands in the overscan region are visible as horizontal
structures (columns are still in pixel space). The spectrum displayed is
extracted from the vertical cut marked by the white solid line.
Result of bias removal method PROFILE+CURVE.
This is the same spectrum after applying the bias method PROFILE+CURVE
with a 5th order polynomial. There are much less bands visible, most are
removed by the smooth polynomial.
Overscan BIAS profiles. Black is the raw BIAS profile
(vertical overscan signal, collapsed over the first 50 columns), red is a
smoothed version with a filter, green is the 5th order polynomial fit used
by the PROFILE+CURVE method.
Extraction. By default, the GIRAFFE pipeline sums up all pixels
within the mask defined in the flat-field localization file (extraction method 'SUM').
An optimum extraction algorithm is available (methods 'HORNE' or 'OPTIMAL'). This
extraction mode has been developed to better handle the scattered light component
in the background. The scattered light is more an issue for the IFU modes than for
the Medusa data. Also, it performs very slow and does not always deliver stable solutions.
For Medusa, the sky fibres are not marked in a standard scheme so the pipeline
cannot find the and subtract the sky signal. Finally, this extraction mode requires
a different flat field product than the one used for the SUM method. Therefore optimum
extraction is not applied in the pipeline scheme used at QC.
Sky. There is no sky subtraction provided by the pipeline.
The sky fibres (either the dedicated ones for IFU and Argus, or the user-defined
ones for Medusa) are processed in exactly the same way as the target fibres.
If sky subtraction is desired, the user has to identify the sky fibres, fit a model
to their signal, and subtract this from the pipeline-provided solution. Find more
information about the stability of the fibre transmission here.
Fibre-to-fibre efficiency. Differences of the fibre-to-fibre efficiency
are corrected for by the flat-fielding step.
The coefficients used for the correction are stored in the product binary table, column TRANSMISSION. (If you are unhappy with the applied correction, divide these factors into the fibre signals for undoing the correction.)
Fringing. Fringing is removed by the flat-fielding.
Cross-talk and background. With version 2.3, the pipeline can
correct for the notorious glow of the old CCD using a close-in-time master_dark.
There is presently no pipeline correction
Image reconstruction. In the IFU modes (IFU and Argus), a reconstructed
image is calculated. It is obtained by collapsing the spectra in dispersion
direction, and re-arrange the signal according to the X and Y sky coordinates
as provided in the fibre table. With version 2.3, there is also the full 3D datacube
delivered for Argus (X,Y,wavelength).
PROBLEMS AND ISSUES
All setups (except for three very blue ones)
There is no scattered light correction.
The sky is not subtracted, sky emission lines and sky continuum are imprinted on the science data. The sky signal is available for subtraction by the user from the user-defined SKY fibres.
The following issues with the science reduction
wavelength calibration covers not always the maximum possible wavelength
range (especially at the long end)
the old CCD ("Bruce", used until 2008-03-13) had an
area with a bright glow of variable intensity on its upper right edge which does not cancel out upon flat-fielding; it caused
an artificial background for fibres 105 - 125 (in the Medusa case) and 250+ (ARGUS,
IFU), resp. This glow can be reasonably well removed with a master dark frame. There is no such issue with the new CCD, no master-dark subtracted.
three hot columns (at X coordinates 368 to 370, corresponds to fibre
#24 for Medusa) are presently not suppressed. There is no meaningful
signal in these columns. The extracted signal in this area may be corrupt.
The pipeline extracts only complete fibres, i.e. the last one or two
at the right edge of the chip are not extracted.
The new CCD "Carreras" has a slightly more variable BIAS
than the old one; therefore, it has been decided that the BIAS is removed from
all raw frames, incl. SCIENCE data, by averaging the OVERSCAN region into a 1D
column, expanding it, and removing it from the raw frames. No master bias is subtracted.
However, created master bias frames are archived.
Science-grade data products
Since 2014, more than 1 mio. science-grade 1D spectral data products are available for the entire data history of GIRAFFE Medusa1/2 mode. New data are added on a monthly basis; data products older than one year are freely accessable, younger data products are available to PIs. More ...