Data Distribution

OVERVIEW

Observing in Service Mode:
Philosophy and Scheduling

Archiving of the data

The data produced at ESO telescopes during each observing night, as well as their daytime calibrations, are stored in the ESO Archive. With the ESO Archive Query Form, users can obtain information on the observations done with ESO telescopes and their instruments.

All ESO scientific data are subjected to a proprietary period of one year since the date when they were distributed to the Principal Investigator, or since the date when they were obtained in the case of Visitor Mode observations. After the proprietary period, the data become public and anyone can retrieve them by means of an archive request. Calibration data are not subjected to any proprietary period and they can be requested as soon as they are available in the Archive.

The data obtained with the VLT are also sent to the Data Flow Operations Group, where the observations done in instrument modes supported by data reduction pipelines receive master calibration frames and are used to produce reduced data. A detailed quality assessment of the data is made by the Data Flow Operations Group beyond the quick-look quality classification made by the astronomer executing Service Mode observations as they are obtained.

Data release

As a rule, Service Mode science data are issued to PIs when:

An observing run is completed, or
The observing period is completed.

The interval between the completion of the run or of the period and the release of the data package to the PI is typically a few weeks, depending on the instrument mode and the data volume.

As of 01 April 2008 Principle Invesigators can download their proprietary raw data from the Science Archive Facility as soon as the data have been ingested into the archive. Please note that an archive query may return a result for data which are actually not yet in the archive (and hence are currently unavailable for download). This is caused by the fact that

the information returned for queries is derived from FITS header information which is available very soon after the corresponding data file has been created, and
the ingestion of the data themselves into the archive can only be done after they are physically transferred to Garching.

This latter process can take some time, but is typically 10-14 days from the time the data were obtained. If a data request is made for data that appear in a query result but are not yet in the archive those files requested will not be available and the requester will be informed that they have N status ("Not available"). The request will be closed and the user should simply try again after waiting for an appropriate time.

It is important to note that the one year proprietary period begins when the corresponding archive request is successfully completed (data files are made available to the requester).

Note that this new data delivery policy does not change in any way the rapid data release procedure used in certain cases (e.g. Targets of Opportunity, pre-imaging for spectroscopy, or data presenting possible technical problems in which a quality assessment by the PI is needed).

Data packages

Complete data distribution packages include the raw science and calibration data, as well as several README and log files. For many instrument modes, processed calibration and science data are also included. These data distribution packages are organized by the ESO Data Flow Operations Group.