Data packages have been delivered for period P87 (April-September 2011) and before. For new data, acquired after the begin of October 2011, data packages are no longer created. Users can access their raw data in electronic form through the ESO User Portal.

Data for VLT pre-imaging runs are processed and delivered as before.

For completeness, the structure of the historical packagesis described below.

Science data have been processed by the pipelines with the best available calibration data. Please note that ESO is not assuming any responsibility in respect to the usefulness of the reduced data. The adopted reduction strategy may not be suitable for the scientific purpose of the observations.

The top-level structure of the data package is as follows:

For each observation block (OB) that has been executed on Paranal, you find all measured raw data (FITS files) in a directory named by the OB number (FITS key HIERARCH.ESO.OBS.ID). If pipeline products exist, these are also added in the OB directory.

The GEN_CALIB directory collects all those calibration files (raw and products) that have been measured as part of the regular calibration plan, and calibration frames of a general nature (like static line tables). The GEN_INFO directory has general information, like data reports and night logs.

The tree shown above is the logical structure, which means that this is the way the data have been organized before they have been put onto media. Depending on the size of your package, the directories may be distributed across several media. It is a good idea to create the original tree on your local disk and then copy all files from the media into this tree.

<OBS_ID> (e.g. 179211)

For each executed observation block of your run, the package contains a directory with all measured data from that OB. All data under <OBS_ID> carry your run ID.

Note that some of your OBs may have been executed more than once. In particular, if time permitted, observatory staff try to re-execute OBs which produced data clearly out of the specified constraints. Check out the NIGHTLOG.html file for details (go to "OB information"). All data from OBs that have been executed multiple times are found in the same directory.

Each OB directory is further subdivided into subdirectories for science frames, calibration frames, and log files. In many cases, there will be science data only, but there may also be OBs with attached calibration data:

<OBS_ID>

acq sci_raw sci_proc cal_raw cal_proc logs

<OBS_ID>/acq

All acquisition frames (DPR.CATG=ACQUISITION) from the OB are contained in this directory. This directory only exists if such data exist.

<OBS_ID>/sci_raw

All raw science frames (DPR.CATG=SCIENCE) from the OB are contained in this directory.

<OBS_ID>/sci_proc

Here you find the pipeline-processed science data. The naming scheme can be found here.

<OBS_ID>/cal_raw

If measured, raw calibration frames (DPR.CATG=CALIB) produced by the OB are contained in this directory ("attached calibrations"). These are the ones which have been taken upon user's request in addition to the ones from the calibration plan.

Calibrations measured as part of the regular calibration plan are stored under the GEN_CALIB directory.

<OBS_ID>/cal_proc

The pipeline products of the raw attached calibrations are delivered here.

The CALIB products are renamed. The naming scheme can be found here.

<OBS_ID>/logs

This directory holds logging information about processing and packing of your data:

• Association Blocks (.ab)
• association logs (.alog)
• logs of the pipeline processing (extension .rblog)
• scoring results (.html) (optional)
• extraction from the nightlog, OB grade, QC0 report (.qcm)
• a reduction comment (.cmt) (optional)

Description:

• Association Blocks (ABs) are text files which contain all the information required to pipeline-process and pack data. This information includes the reduction recipe, the input raw file(s), the calibration products needed for processing, and the names of the final products. More ...
• Association logs are delivered since P80. They are a simplified version of ABs, designed to provide the association information essential for the user. More ...
• The pipeline processing log is a record of the science reduction process, with a detailed log of reduction steps, results etc.
• The scoring report is intended to give some feedback about the data quality. It is still experimental. More ...
• The nightlog file is an extracted version (per OB) of the summary qc0 report and the NIGHTLOG.html file (see below). More ...

That directory in addition holds QC plots, if available.

---

GEN_CALIB

This directory collects all calibration frames from the regular calibration plan that are associated to your science data. It also contains their pipeline products, and calibration frames of a general nature (like static line tables). Calibrations that have been measured by user-defined OBs and that have been used for pipeline processing of science data may be included here in addition.

The directory has four subdirectories (gen, logs, proc, raw), two of which have further fine-structure:

GEN_CALIB
raw	proc	gen	logs
cal1 cal2 cal3 cal4	cal1 cal2 cal3 cal4

GEN_CALIB/raw

Raw calibration files. These divide into raw file types (e.g. BIAS, FLAT etc.; see instrument specific section below).

GEN_CALIB/proc

Calibration products derived from the raw calibrations. These divide into file types like the raw calibration files, see instrument specific section below.

The CALIB products are renamed. The naming scheme can be found here.

GEN_CALIB/logs

Association Blocks, association logs and processing logs for the calibration files under GEN_CALIB. There might also be scoring logs (.html files).

GEN_CALIB/gen

General calibration data of static nature.

Additional or missing raw calibration files may be retrieved anytime from the generic ESO Archive form, or from the instrument specific forms.

Calibration data are public immediately while SCIENCE data normally have a proprietary period of one year.

---

GEN_INFO

This directory hosts some general information. It has the following subdirectories:

DIRECTORY	CONTENT
ObservingReports	nightlogs, OB report (HTML files), association report
scripts	executable scripts (presently one: print_all_reports)

 

The data package contains the following report files:

FILE	CONTENT	WHERE
README.html	the package portal page: point your browser here to find all information	top
ServiceMode.html	this file	top
product_codes.html	a table describing the naming scheme for product files	top
archive_<RUN_ID.txt	list of all proprietary files (SCIENCE, attached CALIBs) as read from the archive	GEN_INFO
qc0_<RUN_ID>.txt	list of all SCIENCE files, containing the comparison between the user constraint set and the actual values	GEN_INFO
NIGHTLOG.html	set of html files with nightlog, OB and association information	GEN_INFO/ObservingReports
list_sciRaw_<OBS_ID>.txt etc.	summary report of the fits files in each directory (these files are provided in text [*.txt] and PostScript format)	all data directories

The executable script print_all_reports under GEN_INFO/scripts can be used to print all postscript files in your package.

Archive report: archive_<RUN_ID>

While the above listings are about files in the package, the archive report is the result of a query to the ESO Archive. It is useful as a check on the completeness of the data package. All files created by OBs which have been generated by the PI are listed here. The list includes all SCIENCE files, and the attached calibrations, and acquisitions, if applicable.

archive report
keyword table
sample file

---

QC0 report: qc0_<RUN_ID>

This report is sent only for Service Mode runs.

This file contains a report of quality control parameters ('QC level 0' where level 0 stands for Quality Control without pipeline processing) for your raw SCIENCE files. These parameters are airmass, seeing, moon distance, and fractional lunar illumination. They have been measured on site (column 'msrd'). They are compared to the required values as defined in your OBs ('targt') and flagged (OK/NOK).

The list is intended to give a rough indication of whether or not the required constraints have been fulfilled. They should not be interpreted in a too formal way, however. E.g., there may be cases where the seeing was worse than required, but this was compensated by a longer exposure time. Check the night reports for details.

Note that the seeing values reported here are DIMM seeing values, they are not measured on the frame. If the alarm flag ("NOK") is set in the SEEING column, the DIMM seeing value was larger than your seeing constraint during the indicated obseration. However, in many cases, the delivered seeing in the instrument focal plane is better than the DIMM seeing. Whenever possible, the on-site astronomer has measured the focal plane immediately after or during execution to determine the success or failure of your observation. Thus, your observation may have been completed within your specifications, even if the SEEING alarm flag is set. Please review the affected observation carefully and check the night reports for details.

QC0 report
keyword table
sample file (.txt)

---

Night logs, OB logs and Association report

This is a set of HTML files with night log information, OB grading information and data association information. All relevant information about the nights contained in your package is included here, as well as information about each OB in your delivery.

Point your browser to GEN_INFO/ObservingReports/NIGHTLOG.html (or start from the package portal page, README.html) and navigate per night (labeled as 1), per OB (2) or per set of files (3).

The HTML files also come as stripped-down, printer-friendly versions. The files are organized to have a summary on top, and details below.

You can use either the navigation bar to jump to a specific night/OB/set of files, or use the up/down arrows (night logs only) to browse sequentially. The OB navigation bar (2) uses colour coding to give you a quick impression about OB grading. There are additional links to ambient condition information.

The association report (3) organizes your data and their association. It has two main levels: the OB (observing block), and the AB (association block) which collects raw file(s) and associated information like product files, calibration files, log files etc. This report gives you an impression how the data in your package are logically linked, while the listings in each directory give you a table of contents. File names in the association report may show up several times, e.g when a calibration file has been used for processing more than one science file.

NOTE:

• The external links (like the ASM links: seeing, sky transparency etc.) will only work with network connection.
• The ASM links require java-enabled browsers.
• The navigation bars read best with style-sheets and java-enabled browsers.

Sample nightlog files
NIGHTLOG.html

Known IRAF problems

• Filename Length. To display or manipulate the FITS files with older versions of IRAF (before 2.11), you can:
  - copy these FITS files to your hard-disk and rename them with filenames <= 32 characters in length;
  - create symbolic links with filenames <= 32 characters in length to your DVD files.
  
  
• Header Interpretation. ESO FITS files use the ESO HIERARCH FITS keyword extensions standard to all ESO telescopes. Note that IRAF treats all ESO HIERARCH header lines as COMMENT lines, i.e. IRAF and IDL cannot automatically interpret the information provided in ESO HIERARCH header lines. The problem may be solved using the tool hierarch28. Find information about this tool here.
  
  
• RA, DEC. Please note that the RA and DEC keywords are recorded in degrees. To translate these keywords so that they can be used by IRAF you have to use the asthedit task in the noao.astutil package. The help file for this task gives an example of how to translate the ESO format to the IRAF format.

Stand-alone FITS handling tools

Runs performed in Service Mode receive a set of data on media which presently are DVDs or CD-ROMs. A description on how to read media produced by the ESO/ST-ECF archive is available here.

This page contains an overview of the structure and content of the data packages for VIMOS.

For further information about VIMOS please look at:

• VIMOS Data Processing and Quality Control Homepage
• General description of VIMOS data management with a list of FAQs
• VIMOS pipeline processing
• VIMOS data types: science raw | science products | calibration raw | calibration products.

VIMOS GEN_CALIB DIRECTORY

GEN_CALIB

The GEN_CALIB directory for VIMOS has the following structure:

GEN_CALIB
gen	logs	proc	raw
		DET IMG_FLAT IMG_STD MOS_ARC MOS_FLAT MOS_STD IFU_ARC IFU_STD	DET IMG_FLAT IMG_STD MOS_ARC MOS_FLAT MOS_STD IFU_ARC IFU_STD

GEN_CALIB/proc

All calibration frames are divided into a few general types. These types can be found as subdirectories of the proc and raw directories:

TYPE/SUBDIRECTORY	VIMOS FLAVOUR	CONTENT	DPR.TYPE
DET	all	Detector bias frames	BIAS
IMG_FLAT	Imaging	Twilight and lamp flat field exposures	FLAT,SKY FLAT,LAMP
IMG_STD	Imaging	Photometric standard star exposures	STD
MOS_ARC	MOS	Arc lamp exposures through MOS mask	WAVE,LAMP
MOS_FLAT	MOS	Flat field exposures through MOS mask	FLAT,LAMP
MOS_STD	MOS	Spectroscopic standard star exposures	STD
IFU_ARC	IFU	Arc lamp and flat field exposures through IFU fibre system	WAVE,LAMP FLAT,LAMP
IFU_STD	IFU	Spectroscopic standard star exposures IFU fibre system	STD

Some notes on bias frames

Bias frames usually come in sets of five raw files which are stacked to one master bias. Please note that imaging and spectroscopic modes use different CCD read-out modes and that, therefore, two different kinds of bias frames are measured.

Exchange of video board of FIERA A (change of gain)

On 1 March 2011, the video board of FIERA A was exchanged. This implied a change of the gain for quadrants 2 and 3. Affected are all observations after 1 March 2011. Please note that not all delivered raw fits files (and their derived pipeline products) have corrected CONAD values (which are written to the header keyword DET.OUT1.CONAD). New CONAD values (in e/ADU) have been measured for all quadrants and are listed below:

The gain values have been adjusted again around 15 April 2011. All files recorded after 25 April 2011 have correct CONAD values in their headers. These values are:

Instrument upgrade 2010

Several parts of VIMOS have been upgraded between May and July 2010. In particular, new detectors have been installed which give increased sensitivity in the red and have dramatically reduced fringing. An active flexure compensation system has also been installed which will be used to minimize flexures within the instrument due to rotation. Please see the instrument news page for further details.

AFC (Active Flexure Compensation) files

The Active Flexure Compensation system creates short calibration exposures before a science or an on-sky calibration template is executed. This process results in additional fits files with the header keyword DPR.TYPE='LAMP,AFC'. The files may have the programme ID of the following science files/exposures. They are, however, not distributed within PI data packages.

Reduced spectral resolution in quadrant 4

MOS and IFU observations using quadrant 4 show a reduced spectral resolution on the upper part of the detector after the upgrade. For observations with the LR_red grism, a resolving power R of about 220 (instead of 240) at 8000A can be expected. With the HR_blue grism, the resolving power may be only 1300 instead of 2200 at 5875A (for a 1 arcsec slit). The problem was solved by fixing the tilt of the focal plane during the intervention in May 2011.

Known IMAGING reduction problems

• Twilight flats in quadrant 3 show instabilities which can reach a few per cent. This may affect flat-fielding of IMG science data in that quadrant.
• Twiligth flat fields: When extended saturation patterns due to bright stars appear in one of the CCDs, sometimes residuals are not removed by the median stacking of the raw flats and a good master is not produced. In these cases the science data of the relative quadrant are reduced with the closest in time good master flat.
• Combination of jitter sequences and removal of fringing patterns are not supported by automated data reduction of data by QC. Every IMG science frame is individually processed.

Known MOS observation and reduction problems

New pipeline recipes. Since pipeline version V2.3.9, two new recipes for the MOS part are available (called vmmoscalib and vmmosscience). These recipes have been applied to all data observed MOS data since August 2010. They use a pattern-matching approach for wavelength calibration and reduce flat-field and wavelength calibration within one single step. The quality of the wavelength solution is in general improved compared to the results from the previous recipes. The results for the LR_blue grism suffer, however, from the low density of arc lamp lines and the possible slit multiplex. An improvement of the vmmoscalib recipe for this setting is under development.

Bug in sky subtraction. Reduced spectra of MOS observations may suffer from a bug with sky subtraction in the new recipe vmmosscience. The bug appears if fringing correction is requested (if more than one input science frame is provided). The pipeline output file MOS_SCIENCE_SKY shows remnants of the object spectra in addition to the sky spectrum. These remnants are subtracted from the object spectra which can reduce the extracted flux by about 5%. Affected can be MOS science data that have been reduced with pipeline versions V2.3.9 to V2.5.6, i.e. data measured between August 2010 and February 2011, but only if fringing correction has been applied. This can be checked in the pipeline log by searching for a MOS_SCI_FRINGES frame. The MOS_SCIENCE_SKY frame can be used to verify whether object remnants are indeed present.

Line identification. It is crucial for pattern matching to identify the arc lines and to distiguish them from random noise. The initial line identification can be controlled with the pipeline parameter peakdetection (of the recipe vmmoscalib) which gives the level above the background that a line must have in order to be considered. The parameter values that are used by the automated reduction are reasonable but in case of highly multiplexed observations fine-tuning may be useful so that the first order spectrum is better distinguished from contaminating -1 order spectra.

Large slit widths. The pipeline recipes vmmoscalib and vmmosscience have been verified so far with masks having slits of less than 1.5 arcsec width. Pipeline reduction for larger slits may fail. A known example for a failure is the combination of the LR_red grism with 2 arcsec slits.

Arc lamp exposures

• With the HR-blue grism, the spectra of slits with y-mask coordinate > 20 mm (0,0 is the mask center) can not be calibrated in wavelength because the red part of the spectrum containing the arc-lamps lines is not imaged, and the blue part does not contain enough emission lines.
• The arc lamp frames taken with low resolution grisms present minus one (-1) and zero order contamination. Sometimes, they also show contamination from the first order of the near multiplexed slits. For this reason the identification of the arc lines is difficult and the RMS of the residuals may be high (1.5 pixels or larger). The problem may be solved by a very accurate "first guess" of the distorsion models in the frame header.

  Contamination from order zero and -1 in multiplexed spectra. On the left, the first order slit spectra A and B are shown, together with the zero and the -1 order of spectrum A. If spectra A and B are multiplexed, as shown on the right, spectrum B is contaminated by the zero and -1 orders of spectrum A.

Spectrophotometric standards

• Spectrophotometric standards are always saturated in LR_red without OS_red filter because the exposure time can not be lower than 1 sec.
• The reduction of spectrophotometric standard stars is pipeline-supported; the resulting response curves are not applied during processing of Service Mode data. Averaged master response curves are used instead.
• Standard star observations can be subject to slitt losses, especially in quadrant 4. See also the note on specphot variability.

Science reduction (notes applicable for data obtained and reduced until May 2010)

• LR_red and HR_red data are strongly affected by fringing. Fringing correction is pipeline-supported if two or more jittered science frames are available. For reduction of Service Mode data, fringing correction is only applied if the offest between any two jittered frames is at least 4 pixels.
• Frames with spectra that are superimposed in some regions are in many cases not well pipeline-reduced. Pipeline results are, however, still included in the Service Mode packages since they can be useful for quick-look purposes. Possible reasons of spectra superposition are grism misalignment or imperfect mask insertion. In multiplexed (LR) data, an imperfect mask insertion results in not aligned first order spectra.

  Raw MOS HR-orange frame showing spectra superposition due to an imperfect mask insertion.

  Raw MOS arc frame showing superposition of spectra due to imperfect mask insertion.

  
  
• Sky subtraction: the sky level at each wavelength is assumed to be the median value of the slit signal in the spatial direction after excluding the object. This assumption is wrong for slits near to the CCD border and extending for most part in the vignetted region, where the median sky level is underestimated. The sky level is instead overestimated in slits with bright objects.
• 1-dimensional object extraction does not work properly in LR_red data because it is disturbed by the fringing.

Known IFU observation and reduction problems

Lost Fibres. The theoretical number of 400 fiber spectra per pseudo slit is not reached. Two fibers in the middle of each block of 80 fibers are typically missing because they are vignetted by the IFU head shutter. There is also vignetting present at one border of each CCD. The actual number of lost fibers cannot be predicted since the positions of the fibers on the raw images vary within 2 to 5 pixels. Depending on the quadrant, 20 to 40 fibers are typically lost on each pseudo slit.

IFU fringing. Internal reflections within the IFU unit produces interference; the origin is not completely understood. Fringing is almost zero at 400 nm and increases to about 7% of the flux level towards longer wavelengths. The pattern cannot be completely repoduced. Details can be found in the paper of Jullo et al., 2008, in: The 2007 ESO Calibration Workshop, eds. Kaufer/Kerber, p. 343.

Elongation of point sources. The VIMOS IFU head is positioned at the edge of the total field of view (see e.g. VIMOS IMG FOV). Point-like sources are, therefore, imaged slightly elongated (basically in y direction).

The efficiency of quadrant 2 was lower than expected of about 30% between 2004-10-04 and 2004-11-10 due to a bad positioning of the IFU mask on the focal plane.

The IFU masks are not well fixed because their hinges are worn out. The effect on the data is that the position on the CCD of the fibers changes depending on the instrument rotation. The changes of the fiber positions between calibrations taken at different instrument rotation angles can reach 11 pixels. To minimize the fiber shifts between science and calibration data a night-time calibration template is executed after the science template acquisition, without changing the instrument position. However, even during a single science template, if the template is long and requires important instrument rotation, the fiber positions on the CCD may change significantly between the first and the last science exposure. On 2004-10-01 the IFU masks have been fixed and afterward the fibers position on the CCD changes of only 2-5 pixel, according to mechanical flexures. With 2-5 pixel fiber movements the requirement of using night calibration for data reduction remains unchanged.
We reduce science data with the pipeline using the night-time data. When the shift of the fiber position in the cross-dispersion direction between science and flat field data is more than 2.5 pixels the pipeline mis-identifies fibers. The results are: 1) the transmission correction applied is not accurate and 2) the re-constructed image presents "zig-zag" patterns. Instead, if we have a shift in the dispersion direction of about 7-8 pixels between the science and the arc-lamp, it is the wavelength calibration applied that may not be accurate (when this happens it is reported in the file NOTES.txt in the directory GEN_INFO of the data package).

Contaminations in Low-Resolution grisms. In LR observations all 4 pseudo-slits are used. This implies second order contaminations between different pseudo-slits. The pipeline may not distinguish between real object spectra and spectra produced by second order contamination. When a pseudo-slit is strongly contaminated by second order spectra, the reconstructed image by the pipeline may contain dummy objects. An example of contamination in quadrant 2 in the raw frame and in the reconstructed image is shown here. The raw frame (on the left) contains the spectra of a bright object in pseudo-slit 2 and the second order contaminations are in pseudo-slits 3 and 4. In the reconstructed image (on the right), in addition to the real bright object (bottom right), there are also the dummy objects resulting from the second order contamination in pseudo-slit 3 (top right) and in pseudo-slit 4 (bottom left).

Raw (left) and reconstructed (right) images of QUAD 2 in LR-red. Pseudo-slit 2 contains the spectra of a bright object that produce second order contaminations in pseudo-slits 3 and 4. The contaminations are preserved in the re-constructed image and appear as dummy objects.

Pick-up noise

Since April 2008, several observations have been affected by pick-up noise. It is mainly visible in quadrant 1 of IFU observations using the HR_blue grism. It is present with a periodic pattern in dispersion direction of the raw frames with an amplitude of about 10 ADUs. Its intensity is varying so that it usually cannot be corrected with bias frames. Its presence can decrease the attainable S/N for faint targets.

Example raw IFU observation affected by pick-up noise (horizontal stripes).

Example reduced IFU observation affected by pick-up noise (dispersion direction is from left to right).

Focus problems in quadrant 3

Quadrant 3 suffers from focus problems in IFU mode which depend on the rotator angle. As a result of this, the reachable spectral resolution can be worse by 20% for LR grisms and 30% for HR grisms around RA = 120 degress. Other quadrants are not affected.

Mesured resolution at blue end (4471A) of IFU wavelength calibrations using HR_blue grism versus rotator angle. Data from January 2008 to April 2009.

Spectro-photometric standard stars. Spectro-photometric standard stars are pipeline reduced; the resulting response curves are, however, not applied during reduction of science data. Averaged master response curves are used instead. See also the note on specphot variability.