Subject:
About DFS, CPL, pipelines and other beasts
From:
Carlos Guirao <cguirao@eso.org>
Date:
Thu, 06 Mar 2008 11:40:04 +0100
To:
Quality Control <qc@eso.org>


Hi,
Below some info about DFS and pipelines installed on
new DFO and QC blades:


1-The DFS release used by pipeline accounts is identified
  with the environment variable:

     $DFS_RELEASE

  This variable is defined in your file ~/.pecs/misc-all.env
  and, by default, is set to "dfs".

  This variable identifies the directory used by DFS
  under /home/flowmgr. In particular the value of "dfs"
  is a soft link pointing to the official
  DFS release (on new DFO blades, it is pointing to
  dfs-5_1_1 with CPL 4.0, so same version as at Paranal)

2-We can have several DFS releases under /home/flowmgr, like
  recently the newer dfs-5_3beta5 with CPL 4.1

3-Pipelines are, by default, compiled in the quality account
  with DFS_RELEASE=dfs, i.e. the DFS release indicated by
  soft link /home/flowmgr/dfs.

  I can however, as quality, change the DFS_RELEASE value and
  force a pipeline to be compiled with any other DFS release,
  i.e. the newer dfs-5_3beta5 if it requires the new CPL 4.1.

4-After an installation, pipelines are located in the directory:

    /home/quality/pipelines/<instrument>-M.m.p.

  The soft link:
    /home/quality/pipelines/<instrument>

  is always pointing to the last (not the newest!) installation
  of the pipeline with IPIP.

5-Pipelines executables are bound to the CPL libraries that
  were used at compilation. You can find which libraries are
  to be used with the command "ldd". Example:

   cd /home/quality/pipelines/naco/lib/naco/plugins/naco-3.8.1
   ldd naco_img_detlin.so
   ...
   libcplcore.so.5 => \
   /home/flowmgr/dfs-5_3beta5/cpl/lib/libcplcore.so.5 (0x0012c000)
   ...

  It is IMPORTANT to know that this is independent of which
  DFS_RELEASE your account is set to use. And partially independent
  of the value of your LD_LIBRARY_PATH environment. Let me
  explain this: LD_LIBRARY_PATH is ignored if the original
  libraries are still available, otherwise it uses LD_LIBRARY_PATH
  to locate the same libraries somewhere else.

6-For pipeline accounts, e.g. isaac, naco, etc.., you
  can specify which particular pipeline release you want to use
  by changing in your ~/.esorex/esorex.rc file, the entry:

  A) esorex.caller.recipe-dir=/home/quality/pipelines/<instrument>-M.m.p

  you can also specify the last compiled release by referring
  to its soft link:

  B) esorex.caller.recipe-dir=/home/quality/pipelines/<instrument>

  I particularly recommend the option A) for the following reason:
  your pipeline session would not be affected with new installation
  of pipelines that changes the pipeline soft link after compilation
  is completed.

7-The executable "esorex" is located under the
  /home/flowmgr/$DFS_RELEASE/bin
  You can find which "esorex" will be executed with the command:

     % which esorex
     /home/flowmgr/dfs/cpl/bin/esorex

8-HOWEVER, and with "esorex", you may get warnings like:
     [WARNING] esorex: Run-time version 4.0 of CPL is lower
               an the compile-time version (40100)

  This happens because you are using an "esorex" compiled with
  CPL 4.0 (remember dfs-> dfs-5_1_1) to execute pipeline
  recipes compiled with CPL 4.1 (located in dfs-5_3beta5)

  I do not know how serious are those warnings, but you can avoid
  them by forcing your account to use the same release of "esorex",
  used to compile your pipeline (see 5). You have
  two alternatives:

  A) Temporarily, execute:
       export DFS_RELEASE=dfs-5_3beta5
       . /home/flowmgr/$DFS_RELEASE/admin/dataflowUser.sh

  B) Permanently, modify the value of DFS_RELEASE to "dfs-5_3beta5"
     in your ~/.pecs/misc-all.env file. But you need to logout-login
     for this change to take place though!!

Hope it helps,
Carlos