ESO Data Processing System (EDPS)

What is EDPS?

EDPS is a system to automatically organise data from ESO instruments for pipeline processing and executing this processing. EDPS is currently being developed, and this is pre-alpha release. It will eventually replace the older ESOREFLEX environment, as well as several ESO internal systems to run the pipeline.

 

ESOREFLEX is currently the recommended enviroment for most ESO pipeline based reduction because of its rich GUI, interactive features and detailed documentation including tutorials. However, for some users and in limited circumstances, EDPS is already now the best way to run ESO pipelines. We recommend to use EDPS for users 

  • who want to learn and experiment with the future environment to run ESO pipelines.
  • who want to process large amounts of data in batch mode mostly using standard pipeline setup, in particular if these data are taken over an extended period of time.The advantage of EDPS is this situation is:
    • EDPS is written in Python and is much more robust then ESOREFLEX
    • EDPS takes better advantages of available computational resources. In particular, recipes are run in parallel when possible, taking into account the internal parallelization of the recipes.  Depending on circumstances, this might lead to an order of magnitude faster executions.
    • Unlike ESOREFLEX, EDPS takes into account changes in the instrument hardware when organizing the data ("break points"). 
  • who want to (re-)process calibration data independent of science data. ESOREFLEX is not set up to do this.
 

Latest Release

A Pre-alpha release meant for experimental usages is planned for October 15. Instructions how to install and use it will be available from this webpage.  

 

Important Note

EDPS is already currently used for Quality control at ESO. For this reason, interfaces and general features are considered as stable and well tested. However, we can not garantee at this point that there will not be some changes. In particular, workflows written for the current version of EDPS might need to be modified to run in future version of EDPS.

 

Documentation

Documentation for EDPS is still work in progress. At this point, there is only the Quick Start Guide that should be sufficient to install EDPS and run the provided workflows.

 

Installation

  Prerequisites

 

  Instructions