The GIRAFFE pipeline recipes v2.16.7 have been released!

This is a maintenance release of the GIRAFFE pipeline recipes.

Changes:

2.16.7 (2020-06-03):
  • Update of the Reflex tutorial.
  • Updated the Reflex workflow to be compatible with Python 3 and Python 2, and with Reflex 2.10.
2.16.5 (2019-05-20):
  • Do not treat simultaneous calibration fibers as ancilliary data when creating products in SDP format.
  • Improved version of the Reflex workflow:
    • Improved science and standard star interactive GUIs including implementation of (optionally-interactive) sky-subtraction.
    • Support for multiple grouping (or no grouping) strategies in a single workflow.
2.16.3 (2018-04-20):
  • Fixed value and unit of the TUCD1 FITS keyword in the phase 3 products.
2.16.2 (2017-04-26):
  • Improved version of the EsoReflex workflows with interactivity.


The distributed recipes can be used for all three observing modes of the GIRAFFE spectrograph:

  • Medusa
  • IFU
  • Argus

Together with the data processing modules, we also provide two standalone applications for running them:

  • Gasgano, a GUI for data browsing and organisation, providing an intuitive and comprehensive interface to pipeline recipes that are based on the ESO Common Pipeline Library (CPL).
  • EsoRex, a command-line driven utility to launch pipeline recipes, which can be used either as an interactive tool, or can be embedded into scripts for automating some of the data reduction tasks.

GIRAFFE pipeline recipes public version:

The public version of the GIRAFFE pipeline recipes is 2.16.7, and it is included in the pipeline distribution kit giraf-kit-2.16.7.tar.gz together with the following packages:

System Requirements

This release of the GIRAFFE pipeline kit is verified and supported on the VLT target platforms:
  • CentOS 7 (x86_64), using gcc v4.8.5
It is also verified that the pipeline kit can be built on the following platforms (x86_64):
  • Scientific Linux 7
  • Fedora 28
  • Fedora 29
  • Fedora 30
  • Fedora 31
  • Ubuntu 16.04
  • Ubuntu 18.04
  • Ubuntu 18.10
  • Ubuntu 19.04
  • Ubuntu 20.04
  • Debian 9
  • MacOS 10.11
  • MacOS 10.12
  • MacOS 10.13
  • MacOS 10.14
In addition the GIRAFFE pipeline has been built successfully on the following systems using the default system compiler:
  • openSUSE Leap 15.1 (x86_64)
  • openSUSE Leap 42.3 (x86_64)
  • openSUSE Leap 42.2 (x86_64)

Installing and running the GIRAFFE pipeline recipes 2.16.7

There are several ways to install the pipeline on your machine. The recommended installation procedure depends on whether you are working on macOS or on Linux.

Installation using RPM repositories

For Fedora 28/29/30/31, CentOS 7 and Scientific Linux 7, it is recommended to install the pipeline from our RPM repository. The RPM repositories include not only the pipeline recipes but also the library dependencies (CPL, etc..), the EsoReflex workflow, demo data and the EsoReflex tool itself.

Please refer to the instructions provided here.

Installation using MacPorts repositories

For Apple macOS 10.11/10.12/10.13/10.14, it is recommended to install the pipeline with MacPorts. The MacPorts repositories include not only the pipeline recipes but also the library dependencies (CPL, etc..), the EsoReflex workflow, demo data and the EsoReflex tool itself.

Please refer to the instructions provided here.

Installation using the EsoReflex installation procedure

Please refer to EsoReflex Software Prerequisites and Installation Instructions for detailed instructions

Installation using the public pipeline kit

To use the GIRAFFE pipeline recipes you will need to retrieve the pipeline distribution kit, unpack and install it.

Prerequisites:

  • The default system C/C++ compiler for one of the OS versions listed above, and standard GNU tools including make
  • To use the latest graphical front-end Gasgano version 2.4.8 a Java Development Kit (JDK) version 1.8.0 or later must be available on your system. You may use either OpenJDK, or Oracle JDK. Most likely, OpenJDK is already part of your Linux distribution and can be installed from its software repositories. The environment variable JAVA_HOME must be set correctly, and the related java executable must be in your PATH.

    Gasgano is known to be resource-intensive: see details of memory and CPU usage in Appendix A of the Gasgano User's Manual.
  • Please check for further software prerequisites here

Installation procedure:

  1. Change directory to where you want to retrieve the GIRAFFE pipeline recipes 2.16.7 package. It can be any directory of your choice but not: $HOME/gasgano
    $HOME/.esorex
  2. Download the software and the static calibration data necessary for running the pipeline recipes:
  3. Unpack the distribution file, using the following command:
        tar -zxvf giraf-kit-2.16.7.tar.gz
  4. Install: after moving into the top level directory of the unpacked distribution,
        cd giraf-kit-2.16.7 it is recommended to perform the installation using the supplied installation script:
        ./install_pipeline This may take a few minutes. Please be patient!

    Note for macOS users:
    On macOS the environment variable JAVA_HOME should be set to JAVA_HOME=/System/Library/Frameworks/JavaVM.framework/Versions/CurrentJDK/Home prior to running the installation script in order to avoid compilation errors during the installation.

    By default the script will install the GIRAFFE recipes, Gasgano, EsoRex, all the necessary libraries, and the static calibration tables, into a directory tree rooted at $HOME A different path may be specified as soon as the script is run. For instance (user input is boldfaced):
    $ ./install_pipeline
    I am about to install the following software packages:
      gasgano-2.4.8.tar.gz
      cfitsio-3.47.tar.gz
      wcslib-4.24.tar.bz2
      fftw-3.3.4.tar.gz
      cpl-7.1.3.tar.gz
      esorex-3.13.3.tar.gz
      giraf-2.16.7.tar.gz

    - with the following pipeline calibration file(s):
      giraf-calib-2.16.7.tar.gz

    The software installation will be organised in bin/, lib/ and include/
    directories, while the calibration files will be installed in a calib/ directory.

    Where should I install the software packages ? [/home/dummy] /home/dummy/pipelines
    Where should I install the pipeline calibration files ? [/home/dummy/pipelines] /home/dummy/calibrations
     In this case the software is installed under the directory /home/dummy/pipelines and the static calibration tables under /home/dummy/calibrations/calib/giraf-2.16.7
    The only exception to all this is the Gasgano tool, that will always be installed under the directory $HOME/gasgano Note that the installer will move an existing $HOME/gasgano directory to $HOME/gasgano.old before the new Gasgano version is installed.

    Important: the installation script would ensure that any existing Gasgano and EsoRex setup would be inherited into the newly installed configuration files (avoiding in this way any conflict with other installed instrument pipelines).

    Alternatively, it is possible to perform a manual installation of the individual components (experienced users only): the README file located in the top installation directory contains more detailed information about a step-by-step installation.

Setup of the GIRAFFE calibration files

In order to use the GIRAFFE pipeline recipes, you may want to download additional calibration files. The GIRAFFE Standard Calibration Data Sets contain all calibrations which are necessary run the GIRAFFE pipeline immediately on scientific observations.

While these data sets are useful to get a first glance on the processed data, it is recommended to use the calibrations which have been specifically prepared for a given observation, in order to obtain the best possible results. Such calibrations may be obtained from the ESO archive using the CalSelector service.

If you decide to re-create the calibration using the GIRAFFE pipeline recipes, certain processing steps will still require some of these calibrations as an initial guess. Please check the GIRAFFE Pipeline User Manual for details.


Installing the GIRAFFE Standard Calibration Data Sets:

The latest release of the standard calibration data sets is still version 2.8.7. However, these data sets can be used together with the GIRAFFE pipeline version 2.16.7.

It is recommended to install these calibrations in the same place where the static calibration tables are located and simply unpack the downloaded archive files. Continuing with the above example, you have to execute the following commands to install the calibration files for the Medusa1 fibre system:

    $ cd /home/dummy/calibrations/calib/giraf-2.16.7
    $ tar -zxvf giraf-data-medusa1-2.8.7.tar.gz
The files will be copied to the
    cal
subdirectory of the directory
    /home/dummy/calibrations/calib/giraf-2.16.7


Using the GIRAFFE pipeline recipes

Using Gasgano

  1. To run the Gasgano GUI, just add $HOME/gasgano/bin to your PATH environment variable:
        export PATH="$HOME/gasgano/bin:"$PATH You should also define an environment variable CPLDIR to point to the same path specified for the installation. Possible files to update are:
        $HOME/.bashrc
        $HOME/.profile     Finally, enter the command
        gasgano
  2. If the Gasgano main panel appears, then you have successfully installed Gasgano. Make sure, however, that the expected Gasgano 2.4.8 release is running: the version number is listed at the top of the main Gasgano panel.
  3. Refer to the Gasgano User's Manual to learn how to browse data and run pipeline recipes. Refer to the GIRAFFE Pipeline User Manual for detailed information about the recipes usage.

Using EsoRex

  1. The EsoRex executable should be available in the bin directory under your installation path (see above). The PATH environment variable should be upgraded accordingly (similarly as done for Gasgano).
  2. Refer to the EsoRex web page for details about the related features and options.

Documentation

The GIRAFFE Pipeline User Manual (ca. 869 kB, 95 pages), is available here for download. Note that the manual is also included in the GIRAFFE pipeline kit.

The GASGANO Users' Manual (1.2 MB, 66 pages) is available for download in the Gasgano web page.

On the EsoRex web page some online documentation about EsoRex can be found.

The CPL manuals are available on the CPL web pages.

In case of problems when opening the documents directly from your web browser, the files may be first saved on disk, and then opened with Acrobat Reader.

Bug Reports

If you experience an unexpected behavior of any component of the GIRAFFE pipeline recipes package, please, first verify that you are using one of the above mentioned supported platforms and refer to the list of known problems and limitations in the current GIRAFFE pipeline release, in chapter 5 of the pipeline manual.

For any other issues or requests, please, send a report to the ESO User Support Department, describing:        

  • the GIRAFFE pipeline version, and the version of other components (e.g., Gasgano, EsoRex, ...) you are using
  • the version of your OS and C compiler.
  • the exact sequence of actions that were performed before the problem occurred
  • what were precisely the symptoms and the possible error message(s)
  • whether the problem is repeatable