[ ESO ] FORS1 Data Management:
general information
HOME INDEX SEARCH HELP NEWS

CAL | HC | refs | QC
QUALITY CONTROL
    HOME
FORS1 QC
Trending & QC1
History
Pipeline
Data Management
general info
QC links:
general FORS1

This section is not maintained anymore, but is the frozen version used last with FORS1.

DATA MANAGEMENT AT ESO

Once a VLT/VLTI observing run is finished, the QC Group creates a data package which is then delivered by the Archive Operations Group to the PI. Data packages are created for all VLT and VLTI instruments for observations done in Service Mode.

Data packages for VLT pre-imaging runs are created and delivered just like ordinary data packages.

The data packages contain raw and processed data (provided there is pipeline support for the instrument mode used), calibration data as well as science data. To distribute the data correctly, the FITS keyword 'HIERARCH ESO OBS PROG ID' is evaluated (where PROG.ID actually is a RUN.ID).

The following rules apply when data are distributed in a data package:

  • SCIENCE and ACQUISITION files: a strict check on PROG_ID is made. PIs receive all science files which carry that PROG_ID, and only those files.
  • CALIB files: PIs receive by default shared calibration files, i.e. those ones which were produced by the Observatory as part of the daily calibration plan. If in addition the user has defined specific calibrations all these files will carry their PROG_ID and will also be included in the data package.
 File naming schemes

Throughout the data flow system, files are named following certain conventions. The following naming schemes are relevant for files in the data packages:
  • archive file name (FITS keyword: ARCFILE): this is the unique name of a raw file which consists of the parent instrument name and the timestamp of its generation. Raw FITS files produced at ESO always have extension '.fits'.
  • original file name (FITS keyword: ORIGFILE): this is the name of a raw file generated on the mountain on the instrument workstation. It consist of:
    - the name of the parent instrument,
    - the category (e.g. BIAS, FLAT, OBS, ... ),
    - the day sequence number within the year,
    - and a sequence number counted within the night.

    Original file names are not unique (they will usually repeat after a year, sometimes even after a day).
  • data product file name (FITS keyword: PIPEFILE): this is the name of any pipeline product. It is composed of:
    • the prefix 'r.',
    • the ARCFILE of the parent raw file stripped off by the extension,
    • and a sequence number '_0000' etc. (since there may be multiple pipeline products per raw file).

    Product files have the extension '.fits'. In some cases, ESO product files also may have the extension 'tfits' where 't' stands for 'table'. These are ordinary FITS files just like the ones with extension 'fits'.

    In case that more than one raw file has been used to generate the product, the ARCFILE of the first one is used.

  • calibration product name: this is a name of calibration data products which has been chosen to have explicit information about the type of file, the date of creation, its version and the instrument parameters in the file name. It is generated after the pipeline product has been created. The FITS keyword PIPEFILE is preserved and can be used to map the two naming conventions.
  • science product names: with period 75 (April 2005), science products have also been renamed to a scheme which allows the user to recognize from the file name the parent OB, the product type and setting parameters.
 FAQs

What is the general policy of data distribution?

Check out here.

Do I have to submit a request to receive my data package?

No. Service Mode (SM) programmes are declared completed by the Users Support Department (USD) (info | mail). This signal is sent to the Quality Control Group which then prepares the data package. The data are copied onto media and sent to the PI as soon as possible.

When can I expect to receive my data package?

Once a run has been finished (VM) or closed by USD (SM), the following series of steps can be expected:

  • Raw data are transfered over the internet and are available in Garching essentially in real time.
  • They are processed, quality-checked and packed by QC Garching as quickly as possible.
  • Once all data for a specific run have been processed, a signal is sent to the archive group to copy this data package on media (presently DVDs).
  • The complete set of data is mailed to the PI.

In total this chain of events is finished on average a few days after the last frames have been measured. Usually this time span is dominated by the shipping time to the PI.

General information about Service Mode Observing can be found here.

top FORS1 INFORMATION

The processing sequence for FORS1 data and the association between calibration and science data can be found here. Usually Service Mode packages should contain raw and processed science and calibration data (if the mode is supported by the pipeline). Sometimes, however, the automatic pipeline fails. In this case information should be available in the GEN_INFO directory.

 


 
[FORS1 QC][Observing Facilities and Operations][ESO][Index][Search][Help][News]