The calChecker monitor is developed and maintained by QC Garching. It provides information about the calibration completeness and calibration quality.

Update information and documentation

Last update:	2008-02-22T07:12:07 (UT)	(0d 00h:02m ago)	Paranal date*: 2008-02-21
Last header: GIRAF.	2008-02-22T05:21:51.420.hdr		*Date on this monitor changes at 21:00 UT

Last update: timestamp of creation of this page (UT)

Last header: name of last header file currently available; compare this timestamp to 'Last update' to see if monitor is up-to-date

(0d 00h:02m ago)		current age of the monitor page (time elapsed since last update); it is green if the last refresh is less than 2 hours ago (requires javascript enabled)
(0d 13h:36m ago)		a problem has occurred (could e.g. be: problem with hosting machine, or network, or database)

	transfer	monitors the health of the bbcp data transfer process from Paranal to Garching Archive (ngas); green if currently no CALIB delayed by more than 1 hour, otherwise red; red alert may explain processing problems
	ngas	monitors the QC access to the archive (ngas); green if download possible, otherwise red; red alert may explain processing problems

calChecker is supposed to run for every instrument once per hour, 24/7. If it is not refreshed within 2 hours, this flag will tell you (it is calculated by your browser; to verify, download and wait, or change the system time). The first thing to check is: does this problem show up only for the current instrument, or for others as well? If so, then it's most likely a network or a database problem. If you think the problem is on the Paranal side, check first with DHA. If you think it is specific to one instrument / QC account, send a mail ("contact") if you expect the QC scientist to be able to read it; to the general list qc@eso.org, if not. On a weekend generic problems will most likely be picked by the Garching maintenance teams, they provide service 7 days a week during daytime. Again ask DHA for support, they know whom to contact.

What to do if the "transfer" or the "ngas" watch flag are red? (intended for our Paranal partners)

Both flags watch the health of the data transfer process and the data access process for QC to NGAS. They are not of prime importance for calChecker but for data processing and for the "products" links displayed.

The transfer watch flag checks the health of the bbcp data transfer between Paranal and Garching Primary Archive. It turns red if CALIB files are found with delivery delays of more than 1 hour. If it is red, please consult first the DataTransferMonitor linked to the calChecker, or consult DHA. They will analyze if the problem is on the Paranal or the Garching side, and contact the right people if necessary.

The ngas watch flag monitors the QC data access to NGAS, the primary archive system in Garching. If access is not possible, data processing is not possible and may explain certain effects on the AB product monitor. For both watch flags it is true that if they show red, they should be red on all instrument versions of the calChecker, modulo execution timestamp of course. So you may want to check this first. Contact DHA and/or the QC scientist(s) for further investigation.

Paranal date: logical current date for this tool (change of date occurs at 21:00 UT in order to have daytime calibrations sorted under the same date as night-time science). Note that this convention differs from the one for some other processes (e.g., Paranal nightlog and opslogs change date at 12:00 LT).

Further information:

Links:

DataTransferMonitor | BandWidth | history ... | contact

This is a link to the DataTransferMonitor (powered by SDD), in case you want to check which data have already been transferred to the Garching archive and are accessible to QC Garching.

Under history you can find older calChecker result pages.

The contact link can be used to send a mail to one of the QC accounts (which is read by the primary and secondary QC scientist for that instrument).

Result table

Dates with science data are color marked and contain the SM and/or VM flag. Dates without science data have a gray column in the result table but no SM/VM flag. The report and NR labels link to the raw data report and the nightlog. The nightlog section requires a password for login. Note that these nightlog's are read from the PANL2 database and may differ in structure and content from the ones distributed via email: their date starts with the nighttime observations and includes all following daytime calibrations. The nightlog's are updated several times during the QC workflow and are therefore more up-to-date than the distributed, static ones.

The last DATE column has a field indicating the status of daytime calibrations (pending; ongoing; finished). This is relevant for the judgment of missing calibrations for that date: any MISS or NOK in case of "finished" is significant, while otherwise these calibrations are likely to still be in the calOBBuilder queue. Also, this field might be helpful to detect cases when a calibration queue is broken or has been interrupted.

The row labeledas CAL Product quality links to the product data report ("AB monitor" where AB stands for association block, which is the fundamental QC processing job). Once processed, also the science data results are displayed.

The AB monitor shows the current status of data processing at QC Garching, including quality scores (which are also used on the Health Check monitor) and QC reports. The AB monitor reflects the calibration quality, while the calChecker reflects calibration completeness. Processing of the calibrations is done automatically. However, it may be delayed anytime by data delivery delays.

The column "action required?" contains types of missing calibrations, and a link to the association rules used. The last Setup column lists the setup of missing calibrations.

Data type (Mode):	Setup:
SCIENCE_MED	Medusa1_H525.8B			ok					ok	ok		all ok
	Medusa1_L682.2		ok							ok		all ok
	Medusa1_L881.7	ok					ok	ok			nok [not yet analyzed]	FLAT | WAVE	Medusa1_L881.7
	Medusa2_H525.8B				ok				ok	ok	ok	all ok
	Medusa2_L682.2									ok	ok	all ok
	Medusa2_L881.7					ok	ok	ok	ok			all ok

Data types are defined by mode and setup. Each science data type has one row, each date has one column. These two coordinates define a box which is evaluated by the tool as ok, nok or miss:

ok	available CALIB data are complete and all within the validity time range.
nok	available CALIB data are complete but at least one CALIB file type is outdated. New measurement should be done asap. The corresponding CALIB file type is listed in the rightmost column.
miss	CALIB data are incomplete, at least one CALIB file type is missing. New measurement should be done asap. Missing files are listed in the rightmost column.

Whenever a "missing" calibration is indicated, this is a reminder for obtaining them as soon as possible. Otherwise the corresponding SCIENCE observation may be lost (graded 'C' or 'D').

Analysis notes. Whenever possible, the QC scientist makes an attempt to analyze a situation with NOK or MISS. The analysis result is then displayed below the result table, as a separate table labeled"Analysis notes".

Instrument information and configuration

Below the result table there is additional information about the data types and setups used for the calChecker:

This table contains information on how a setup is defined etc.

Number of days scanned: this parameter defines how many days are scanned for science data, starting with the current date.It will usually be in the range 4-10 days.

Range of days for the calibration memory: this number is usually higher than the one before. It defines the depth of the calibration memory, i.e. the number of days scanned for calibration data.

Days in the calibration memory: List of days in the calibration memory.

Long-term and maintenance calibrations

Long-term calibrations are maintenance, technical and Health Check calibrations. Their distinguishing feature is that their acquisition is not driven by science data but by a maintenance calibration plan. Typically they are taken rarely (say every month or so), but there exist also cases (often Health Check calibrations) which are scheduled every week, or even more often.

They are monitored on the calChecker in the following way:

• they have a configured validity (e.g. 30 days)
• the last occurrence of each data type is scanned in the raw data reports; the tool iterates backwards in time, to a maximum of one year
• the current age of the last instance is evaluated and flagged as
  • OK/green: if it is younger than 70% of the validity
  • soft REMINDER/yellow: younger than validity but within 30%; recommended action: refresh on next occasion
  • REMINDER/yellow: outdated but by not more than 30%; recommended action: refresh on next occasion
  • REMINDER/red: outdated by more than 30%, refresh as soon as possible

All cases (if any) which flag yellow or red are displayed in the summary table on the main calChecker page. So these are the ones where some action needs to be taken (soon or now).

The complete overview is available behind the link "complete overview here". As an example, you may want to inspect the GIRAFFE case. It lists:

• the tag defining the type of calibration (e.g. DETECTOR_MON for the detector monitoring sequence), and a description of what that means; that description is meant to be specific enough to identify clearly the particular data set. It may contain the template name, a set of DPR keys etc.
• the validity and age in days
• a slightly more extensive version of the recommended action

The long-term calibration monitor is implemented by scanning the raw data reports (the ones which are linked to the calibration completeness monitor as report ). Scanning starts with the current date and goes backwards month by month, up to a full year. The first raw file matching the selection criteria will be evaluated in terms of age and validity.

There is no check for quality, nor for correctness of the header entries.