make printable	new:	see also:
	v1.6: - new tool supported: autoDaily (for forced mode only)	Check here how to set up cronjobs in general. Check autoDaily documentation for more about the forced autoDaily, and calChecker for more about the forced calChecker.
	v1.7: - supporting 'calChecker -I'	databases none dfos tools wrapper around dfos tools output used by none upload/download none

dfosCron

Description

This tool is an infrastructure tool to support the cronjob management of dfos tools. It supports the following dfos tools:

• ngasMonitor; calChecker; dfosExplorer; dfosBackup
• HdrDownloader; createReport [in mode -d]; filterRaw [-m SCAN]
• $DFO_JOB_DIR/JOBS_TREND (generic job file) with daily updates of trending reports (or any other daily job)
• call of forced modes of autoDaily and calChecker (checking for trigger files).

It manages automatically, in the background, the front-end part of the dfos system (new-data detection, header download, report creation).

The tool logs into $DFO_MON_DIR/CRON_LOGS. There, for each day a log file is created in which all dfosCron activities are logged.

JOBS_TREND. This is a jobs file that should be called at least once per day. It should be filled with all HC (trendPlotter) jobs, for type HISTORY and HEALTH. This to ensure that these plots are updated at least daily. This is necessary since autoDaily triggers updates only of the HEALTH plots displaying new data. This additional dfosCron call then ensures that all plots are up-to-date within a day.

Check out here how to set up cronjobs.

How to use

dfosCron -v|-h     gives version number and short help

dfosCron -t[ool] <name> calls the specified tool (this interactive call makes sense for testing only)

[00 08 * * * dfosCron -t dfosExplorer] If this line is defined by using crontab -e, the dfosExplorer will be refreshed daily at 08:00 UT.

Configuration file

None.

Operational considerations

• The tool sources .bashrc, and thereby also .qcrc and .dfosrc. In the cronjob call, do not source .qcrc and .dfosrc again.
• Make sure that .qcrc defines the following environment variables properly:
  export DISPLAY=<name of your display>
  export PATH=<name of your $DFO_BIN_DIR>:${PATH}
  e.g.: export PATH=${HOME}/bin:${PATH} if your $DFO_BIN_DIR is defined as $HOME/bin

Workflow description

1. setup: do some DATE calculations, define log (CRON_$DATE.log where DATE is the date of operation = TODAY)

2. Launch the supported tools

2.1 Simple launch: call the tool (e.g. ngasMonitor, dfosExplorer)
2.2 Complex launch: do a bit more (JOBS_TREND: call createReport; HdrDownloader: manage the entries in DFO_STATUS)
2.3 Special launch of autoDaily and calChecker (both by calling '-t autoDaily'!):

2.3.1 Look for trigger files HC_ENFORCE and CAL_ENFORCE on $DFO_WEB_SERVER (in wrapped mode, checking for TIMEOUT and exit after 20 sec if connection fails)
2.3.2 No trigger file found --> exit
2.3.3 Trigger file found: check for other instance of autoDaily or calChecker
2.3.4 None found --> call 'autoDaily -F' or 'calChecker -I'
2.3.5 Another instance found --> exit

For the HdrDownloader call, note that the tool calls 'HdrDownloader -t 24' which is optimized for speed. It downloads incrementally the headers of the last 24 hours. It does not download the headers of the last template if they are younger than 1 hour (to avoid the situation that processing starts before the template is completely available).

Please note that the cronjob with 'dfosCron -t autoDaily' checks for both trigger files (HC_ENFORCE and CAL_ENFORCE) and therefore can launch both tools (autoDaily and calChecker). The only reason for having the same syntax (mode) for both tools is simplicity (no need to change anything in the existing cronjob).

Sketch of 'dfosCron -t autoDaily' launching the forced autoDaily run ('autoDaily -F'):

Sketch of the forced mode of autoDaily: The user creates a request for running autoDaily, using the php interface and creating a file HC_ENFORCE. The dfosCron cronjob on the muc machine discovers this "trigger file" and then launches 'autoDaily -F' (this part is shaded blue).

The same sketch for the launch of the forced calChecker run (with the same command 'dfosCron -t autoDaily'):

Sketch of the forced mode of calChecker: The user creates a request for running calChecker, using the php interface and creating a file CAL_ENFORCE. The dfosCron cronjob on the muc machine discovers this "trigger file" and then launches 'calChecker -I' (this part is shaded blue).