FORS2:  Service Mode data package
	HOME INDEX SEARCH HELP NEWS

Navigate:   General Directory structure Content of directories List files  Data files Photometric data Actual problems

Programmes performed in Service Mode receive a set of CD ROMs. Find here an overview of the structure and content of the data package.

Directory Structure

Content of Night directories

raw

all raw data of the night from your PROG_ID (of types 'SCIENCE' and 'TEST').

reduced

pipeline processed data from your PROG_ID (type 'SCIENCE').
list of all reduced data (list_of_all_red.txt)

calib

processed master calibration files
raw calibration files
list of all corresponding calibration data (list_of_all_calibs.txt) 

logs

Reduction Blocks (extension .rX);
reduction logs per Reduction Block (extension .longlog).

Reduction Blocks are ASCII files which contain all the information required to launch a reduction job with the pipeline. This information contains the reduction recipe, the input raw file(s), and master calibration files retrieved from the Calibration Database.

While Reduction Blocks are a technical piece of information, the reduction log is a user-oriented account of the reduction process, with a detailed log of reduction steps, results etc. The most important ones will be those associated to SCIENCE data. The CALIBRATION reduction logs (those which created master calibration files) may also contain some valuable information.

In directory GEN_INFO you will find the following text files:
 

file	content	tree level
ServiceMode.html	this file	top (1)
README.txt	short additional information	top (1)
list_SM_PACKAGE	listing of all Science/Calibration files included in the Service Mode Package (text and PostScript format)	top(1)
nightlog_<date>.txt	extractions from the night log when the data were taken	<date> (2)
list_of_all_obs.txt	summary of the fits files in each directory (text and PostScript format)	raw, reduced, calib (3)
raw_<PROG_ID>.txt	listing of all raw files with the programme_ID in the ESO Archive (text and PostScript format)	top (1)
list_OB.txt	listing of all raw and reduced files per OB on your CD-ROMs (text and PostScript format)	top (1)
OB_status_<PROG_ID>.txt	summary of the executed OBs and status	top (1)
qc0_report	log containing the comparison between the constraint set (airmass, moon distance, etc) and the actual values (text and PostScript format)	top(1)
SKYTRANS_yyyy-mm-dd.gif	Plot of the sky transparency for the night	top(1)

Raw data summary file: list_of_all_obs

The .txt file prints well with UNIX a2ps (landscape format, font size 5.7).
 

keyword table    sample file (.txt)    sample file (.ps)

Reduced data summary file: list_of_all_red

keyword table    sample file (.txt)    sample file (.ps)

Calibration data summary file: list_of_all_calibs

a listing of raw calibration files similar to the raw data summary file (list_of_all_obs).
a listing of master calibration files.
 

keyword table     sample file (.txt)    sample file (.ps)

Find here a overview about the naming convention for master calibration data.

This file lists all ObservingBlocks, raw, reduced, and programme-specific calibration data files on the SM package (CD ROMs).

This log contains all the data in an OB-related way, rather than the above listings which sort the data in a night-related way.
 

keyword table    sample file (.txt)

While the above listings contain files on the CD ROMs, this listing is the result of a query to the ESO Archive. It is useful as a check on the completeness of the ServiceMode package.
 

keyword table     sample file (.txt)

Science Raw Data

All the FITS files produced by user-created Observation Blocks and that could be executed are delivered (categories SCIENCE and TEST). Some of the OBs may have been executed more than once. In particular, if time permitted, those OBs which produced data clearly out of the specified constraints were tried to be re-executed. Hence there may be more raw files per OB than originally specified.

Appropriate calibration data (these may be raw and/or processed [master] calibration data) are always delivered.

Master calibration data, if available, are delivered according to the instrument mode of the science raw data. Only exception are BIAS and DARK files and a bad pixel table, which are not mode-specific. Their names contain information about their function and instrumental modes.

The following master calibration data are in general included:
 

• bias frames,
• twilight sky flats (IMG),
• nightflats (IMG; derived from SCIENCE exposures during the night),
• normalized screen flats (LSS),
• wavelength calibration files (LSS),
• reduced photometric standard star exposures and zeropoint tables (IMG).

Raw calibration data, again mode-specific, are delivered if they fulfil at least one of the following conditions:
 

• they are specific to your programme (i.e. have been created by your OBs),
• they have not been used to construct master calibration files,
• they are standard star files and your science data are IMG mode,
• the instrument mode selected by you is presently not pipeline-supported.

An example: your science data are measured in the IMG and the MOS mode. You have created a specific calibration OB for z-Gunn skyflats. You can expect the following calibration data (per date):
 

• master BIAS files,
• master calibration data for IMG mode (master skyflats, master nightflats),
• specific master and raw calibration files from your OB,
• any MOS raw calibration files (since MOS is not yet pipeline-supported).

Check the FORS2 pipeline pages, specifically for calibration data.

If you have measured in the IMG or the LSS mode, you can expect to receive pipeline-reduced files. Generally, all SCIENCE taken in IMG and LSS mode are reduced if they fulfil the following conditions:
 

• they are measured in standard modes (UBVRI filters, 1x1 binning, no windowing),
• master calibration data were available from the same night or at least within a few nights,
• LSS data: the 1port readout mode has been used.

Quality checks for the data reduction are applied. The quality of flattening and removal of 4port structure is checked. The degree of overexposure is determined. No file is rejected on the basis of these checks, however.
 

Science data are processed by the pipeline with the best available calibration data. Please note that ESO is not assuming any responsibility in respect to the usefulness of the reduced data. The adopted reduction strategy may not be suitable for the scientific purpose of the observations.

Check the FORS2 pipeline pages, specifically for more details on science reduction.

Photometric data included for IMG mode

For imaging projects, photometric standards are observed on every clear night through the filters being used that night. Clear nights are defined as nights when no clouds are visible. However, we can not certify these nights as photometric.

When available,
 

• the raw standard star files (DPR CATG = STD),
• the reduced standard star files (PRO CATG = REDUCED_STANDARD_IMG),

Pipeline-generated zeropoints per night are obtained by averaging all available standard star measurements. They are available here.

NOTE: some of the STANDARD star files (raw or reduced) may contain saturated pixels. So check them carefully if you want to use them. Pipeline-produced zeropoints are checked for overexposure.

Filename Length Problem

To display or manipulate the FITS files with older versions of IRAF (before 2.11), you can:

1. copy these FITS files to your hard-disk and rename them with filenames <= 32 characters in length,
2. create symbolic links with filenames <= 32 characters in length to your CD-ROM files.

The FITS files use the ESO HIERARCH FITS keyword extensions standard to all ESO telescopes. Note that IRAF treats all ESO HIERARCH header lines as COMMENT lines, i.e. IRAF and IDL cannot automatically interpret the information provided in ESO HIERARCH header lines.

Please note that the RA and DEC keywords are recorded in degrees. To translate these keywords so that they can be used by IRAF you have to use the asthedit task in the noao.astutil package. The help file for this task gives an example of how to translate the ESO format to the IRAF format.

Several keywords have been added to the files to make them understandable for IRAF and IDL. The main keywords are OBJECT, FILTER, GRISM, SLIT, AIRMASS, UT, ST, and IMAGETYP. These are translations from some of the hierarchical keywords. The tasks in ccdred and xccdred understand these keywords if the instrument file is set up correctly. The image sections for 4-port readouts are defined as well in the headers and are understood by the tasks in xccdred.

Change of master calibration distribution

Master screen flats (MSFI) are not longer distributed on CD ROMs. Their information about small-scale fixed-pattern noise is fully contained in the master sky flats (MKFI) which are routinely contained in the data packages. This change is effective from 2000-01-18 on.

Data related issues

QC home 
QC and trending 
pipeline 

Send comments to <smoehler@eso.org>  Last update: Aug 23, 2000