A release of the ERIS pipeline recipes is now available !

A release of the ERIS pipeline recipes is now available.

Together with the data processing modules, we also provide standalone applications for running them:

  • EsoReflex, a GUI which provides an easy and flexible way of runing pipeline execution workflows.
  • Gasgano, a GUI for data browsing and organisation, providing an intuitive and comprehensive interface to pipeline recipes that are based on the ESO Common Pipeline Library (CPL).
  • EsoRex, a command-line driven utility to launch pipeline recipes, which can be used either as an interactive tool, or can be embedded into scripts for automating some of the data reduction tasks.

Changes:

Version 1.4.1 Public Release (19 Jun 2023)

  • Updated the public release of the ERIS pipeline.

Improvements

  • This release focus on improvements for users, but also improves operations.
      SPIFFIER data reduction improvements:
    • Implemented recipe eris_ifu_combine_hdrl to allow combination of 3D data cubes created by the jitter recipe.
    • Changed cube resampling to do a plane by plane cube resampling. It largely reduces the RAM usage and allows a speed-up of 10%. This generates slightly less noisy data in certain resampling modes.
    • The output grid size of resampling is calculated from the axis increments using the full CD matrix, which takes the rotation into account.
    • Found that too many pixels are flagged as bad, in particular on the dark frames. As this may affect the quality of the extracted spectrum, creating spikes that are not present in the data, we decided filter out the flagged pixels before doing the extraction. This solved the problem of the spikes present in the extracted spectrum in previous releases.
    • Fixed a bug that was preventing the generation of flux calibrated data cubes.
    • Updated static frames controlling the response computation to extend the wavelength range of the response spectrum. This allows to extend the wavelength range of the flux resampled spectrum.
    • Added the parameter derot_corr to eris_ifu_stdstar and eris_ifu_jitter recipes to correct a (small) shift along X direction depending on the rotation angle.
    • Added the parameter aj-method to eris_ifu_stdstar and eris_ifu_jitter recipes, which allows users to estimate the sky from the next in MJD-OBS object frame in case there is no sky frame taken.
    • Fixed a bug for the OH alignment when building the data cubes. Previously the shift was calculated but not applied due to the bug.
    • Added quality control parameters on the recipes eris_ifu_distortion, eris_ifu_flat, eris_ifu_wavecal, eris_ifu_persistence_monitor and fixed some minor bug in the determination of a few QC parameters.
    • Improved robustness when computing Strehl, and of the eris_ifu_stdstar recipe in general.
    • Updated Reflex OCA rules to enable processing of PSF STD stars. We now offer more parameters to the user in the interactive workflows that control reduction of standard stars and science objects.
    • Removed ’outgrid’ and some other parameter from the eris_ifu_stdstar, and eris_ifu_jitter recipes as they were useful only for the recipe maintainer for debugging purposes and not for desktop users.
    • Added checks on required input frames. However now the pipeline stops in case a required input is not provided. Some input maybe optional, and is recommended to read the user manual, Section 12.
    • Updated user manual
      NIX data reduction improvements:
    • Improved WCS accuracy.
    • Fixed a bug saving products of data obtained with DET.FRAM.FORMAT = ’cube’
    • Fixed a problem reducing darks of windowed data.
    • The recipe eris_nix_cal_det is now faster by a factor 2.8. Further speed-up can be obtained disabling the computation of the pipeline products checksums, that means running the recipe as: esorex --nodatamd5=true --no-checksum=true my_recipe myfiles.sof. For operations quick-look, we also added a parameter that, changes reduction by collapsing the data cube before applying the detector signature corrections, in this way drastically reducing the computation time.
    • Added more quality control parameters for calibrations recipes.
    • Added check on required input frame tags.
    • Updated user manual

ERIS pipeline recipes public version:

The public version of the ERIS pipeline recipes is 1.4.1, and it is included in the pipeline distribution kit eris-kit-1.4.1.tar.gz together with the following packages:

  • the CFITSIO library, version 4.2.0
  • the Common Pipeline Library (CPL), version 7.3.2
  • Gasgano version 2.4.8
  • EsoRex, version 3.13.7
  • Third-party libraries used by CPL (wcslib, fftw)
  • Third-party libraries required by the ERIS pipeline

ERIS pipeline documentation

User Manual: 1.4.1 (ERIS-NIX)
User Manual: 1.4.1 (ERIS-SPIFFIER)
EsoReflex Tutorial: 1.4.1 (ERIS-NIX)
EsoReflex Tutorial: 1.4.1 (ERIS-SPIFFIER)

ERIS pipeline recipes System Requirements

The ERIS pipeline kit version 1.4.1 is verified and supported on the VLT target platform:
  • CentOS 7 (x86_64), using gcc v4.8.5.
It is also verified that the pipeline kit can be built on the following platforms (x86_64, unless specified otherwise):
  • CentOS 7
  • Rocky Linux 8
  • Fedora 31
  • Fedora 34
  • Fedora 35
  • Fedora 36
  • Fedora 37
  • Ubuntu 20.04
  • Ubuntu 22.04
  • Ubuntu 22.10
  • Debian 10
  • Debian 11
  • macOS 10.15
  • macOS 11
  • macOS 12 (Intel & Apple Silicon)
  • macOS 13 (Intel & Apple Silicon)

Installing and running the ERIS pipeline recipes 1.4.1

There are several ways to install the pipeline on your machine. The recommended installation procedure depends on whether you are working on a Mac OSX or on Linux.

Installation using RPM repositories

For Fedora 35/36/37, CentOS 7 and Scientific Linux 7, it is recommended to install the pipeline from our RPM repository. The RPM repositories include not only the pipeline recipes but also the library dependencies (CPL, etc..), the EsoReflex workflow, demo data and the EsoReflex tool itself.

Please refer to the instructions provided here.

Installation using MacPorts repositories

For Apple macOS 11/12/13, it is recommended to install the pipeline with MacPorts. The MacPorts repositories include not only the pipeline recipes but also the library dependencies (CPL, etc..), the EsoReflex workflow, demo data and the EsoReflex tool itself.

Please refer to the instructions provided here.

Installation using the EsoReflex installation procedure

Please refer to EsoReflex Software Prerequisites and Installation Instructions for detailed instructions

Installation using the public pipeline kit

To install the ERIS pipeline recipes 1.4.1 you will need to retrieve the pipeline distribution kit, unpack and install it.

Prerequisites:
  • Please check the software prerequisites here.

Installation procedure:

  1. Change directory to where you want to retrieve the ERIS pipeline recipes 1.4.1 package. It can be any directory of your choice but not:
    • $HOME/gasgano
    • $HOME/.esorex
  2. Download the software and the static calibration data (line catalogs etc.) necessary for running the pipeline recipes:
  3. Unpack using for example the following command:     tar xzf eris-kit-1.4.1.tar.gz
  4. Install: after moving into the installation directory,     cd eris-kit-1.4.1 it is recommended to perform the installation using the supplied installer script:     ./install_pipeline (BEWARE: The installation may take several minutes). By default the script will install the ERIS recipes, Gasgano, EsoRex, all the necessary libraries, and the static calibration tables, into a directory tree rooted at $HOME. A different path may be specified as soon as the script is run. For instance (user input is boldfaced):
    $ ./install_pipeline
    I am about to install the following software packages:
    • cfitsio4.2.0.tar.gz
    • cpl-7.3.2.tar.gz
    • esorex-3.13.7.tar.gz
    • gasgano-2.4.8.tar.gz
    • eris-kit-1.4.1.tar.gz
    with the following pipeline calibration file(s):
    • eris-calib-1.4.1.tar.gz
    The software installation will be organised in bin/, lib/ and include/ directories, while the calibration files will be installed in a calib/ directory. Where should I install the software packages ? [/home/userid] /home/dummy/pipelines
    Where should I install the pipeline calibration files ? [/home/dummy/pipelines] /home/dummy/calibrations
    In this case the software is installed under the directory /home/dummy/pipelines and the static calibration tables under /home/dummy/calibrations/eris The only exception to all this is the Gasgano tool, that will always be installed under the directory $HOME/gasgano Note that the installer will move an existing $HOME/gasgano directory to $HOME/gasgano.old before the new Gasgano version is installed.
    Important: the installation script would ensure that any existing Gasgano and EsoRex setup would be inherited into the newly installed configuration files (avoiding in this way any conflict with other installed instrument pipelines).
    Please follow the instructions shown at the end of a successful installation script: they will indicate how to set your PATH and CPLDIR variables in your machine in order to run the pipeline.
    Alternatively, it is possible to perform a manual installation (experienced users only): the README file located in the top installation directory contains more detailed information about a step-by-step installation.

Executing the pipeline 1.4.1

Using EsoReflex

  • To run the EsoReflex workflow, please refer to the ERIS workflow tutorial available in this table under column Documentation.

Using Gasgano

  1. To run the Gasgano GUI, just add $HOME/gasgano/bin to your PATH environment variable:     export PATH="$HOME/gasgano/bin:"$PATH You should also define an environment variable CPLDIR to point to the same path specified for the installation. Possible files to update are:     $HOME/.bashrc     $HOME/.profile Finally, enter the command     gasgano
  2. If the Gasgano main panel appears, then you have successfully installed Gasgano. Make sure, however, that the expected Gasgano 2.4.8 release is running: the version number is listed at the top of the main Gasgano panel.
  3. Refer to the Gasgano User's Manual to learn how to browse data and run pipeline recipes. Refer to the Pipeline User Manual(s) for detailed information about the recipes usage.

Using EsoRex

  1. The EsoRex executable should be available in the bin directory under your installation path (see above). The PATH environment variable should be upgraded accordingly (similarly as done for Gasgano).
  2. Refer to the EsoRex web page for details about the related features and options.

Documentation

The EsoReflex Users' Manual is availale for download

The GASGANO Users' Manual (~1.2 Mb, 66 pages) is available for download in the Gasgano web page.

On the EsoRex web page some online documentation about EsoRex can be found.

The CPL manuals are available on the CPL web pages.

In case of problems when opening the documents directly from your web browser, the files may be first saved on disk, and then opened with Acrobat Reader.

Bug Reports

If you experience an unexpected behavior of any component of the ERIS pipeline recipes package, please, first verify that you are using one of the above mentioned supported platforms and refer to the list of known problems and limitations in the pipeline manual of the current ERIS pipeline release.

For any other issues or requests, please, send a report to the ESO User Support Department mentioning ERIS pipeline in the subject, describing:        

  • the ERIS pipeline version (currently 1.4.1), and the version of other components (e.g., Gasgano, EsoRex, ...) you are using
  • the version of your OS and C compiler.
  • the exact sequence of actions that were performed before the problem occurred
  • what were precisely the symptoms and the possible error message(s)
  • whether the problem is repeatable