Common DFOS tools:
|dfos = Data Flow Operations System, the common tool set for DFO|
This is a description of the maintenance of the dfos system. It applies to both subcomponenents, dfos and tqs. The python library pyQC is maintained by Burkhard Wolff.
The dfos system is currently maintained by me under the operational account giraffe on the operational server muc02. The root for dfos maintenance is $HOME/distrib.
|all dfos sources|
|all versions of dfos sources|
|short history description which is displayed under 'history' on the dfos main page|
|template configuration files which are contained in the distribution; same substructure as ($DFO_CONFIG_DIR)|
|$HOME/distrib/doc||all help files (the ones under $DFO_DOC_DIR)|
|$HOME/distrib/makefiles||all makefiles (as contained in the distribution); substructure: .../<tool name>/makefile|
Get authorization from me. In an emergency, login as qcshift@muc02 and from there to giraffe@muc02 (just type 'giraffe').
How is it maintained?
Save current and earlier versions. A cronjob $HOME/crontabs/cronDfosVersion is called once a day and puts a copy of the latest version of each dfos tool (under $HOME/distrib/bin) into $HOME/distrib/versions. That version gets the name <tool_name>_v<version>. Effectively this is a version control system.
Distribute a new version. For each tool, there is a little shell script under $HOME/distrib called make_tar_<tool_name>. That script finds all components of the distribution package (e.g. tool source, template config file, help file, history file, makefile; maybe other tools) and adds a README file. All components are put in a tarball named <tool_name>.tar and ftp'ed to the QC dfos server (http://www.eso.org/~qc/dfos/tools).
These tar files are linked to the dfos main pages http://www.eso.org/~qc/dfos/details.html and http://www.eso.org/~qc/tqs/details_tqs.html.
The installation of a new tool is described here.
Development style and rules
The development and maintenance of the common dfos tool suite is of key importance for day-to-day QC operations. Therefore it is restricted. Currently the responsibility for system maintenance is with me. Contributions by group members are usually marked.
All dfos tools are written as bash shell scripts. The scripting approach provides extreme flexibility which is required due to the ever-changing operational conditions. On the short side, tools are generally not optimized for performance, but for stability.
Attempts are made to develop tools with the same structure, the same syntax and the same style. For instance, there are many different ways to code a DO loop or an IF statement in a shell script. Some developers prefer a concise style. dfos has a "transparent" style, simple-minded, certainly inelegant, but (hopefully) easy to understand by others who may need to maintain the system.
The tools try to follow the workflow. They have an initial section, containing the definition of variables (either hard-coded or read from a configuration file); evaluation of command options; procedures and helper scripts. Sections are made visible in the code. The main part of the code with the workflow steps always starts with section 1.0. An attempt has been made to use parameter and option names which are common among the tools.