The calChecker monitor is developed and maintained by QC Garching. It provides information about the calibration completeness and calibration quality.
Find general information about the process here.

Science and CAL4CAL checks
science		cal4cal	If you see these two tabs enabled, calChecker for this instrument is configured for CAL4CAL checks (calibrations for calibrations) and comes with two main result pages.
science		no cal4cal	If only the 'science' tab is enabled, this instrument is not configured for CAL4CAL checks (because they make no sense here).

cal_scores: The completeness flags (OK, NOK or MISS, see below) are cumulated into overall calibration scores (cal_scores), along the same ideas as the quality scores on the HC monitor.

	all calibrations existing and within validity time range
	issue with outdated or missing calibrations but daytime calibs not yet finished
	issue with outdated or missing calibrations

tellTracker: A few instruments have a third module of calChecker enabled, called tellTracker:

science		cal4cal		tellTracker

The tellTracker is designed to follow up time-critical nighttime calibrations, like the telluric standard stars. Find more information about tellTracker here.

Update information and documentation

Last update:	2008-02-22T07:12:07 (UT)	(0d 00h:02m ago)	Paranal date*: 2008-02-21
Last header: GIRAF.	2008-02-22T05:21:51.420.hdr		*Date on this monitor changes at 21:00 UT

Last update: timestamp of creation of this page (UT)

Last header: name of last header file currently available; compare this timestamp to 'Last update' to see if monitor is up-to-date

(0d 00h:02m ago)		current age of the monitor page (time elapsed since last update); it is green if the last refresh is less than 2 hours ago (requires javascript enabled)
(0d 13h:36m ago)		a problem has occurred (could e.g. be: problem with hosting machine, or network, or database)

	transfer	monitors the health of the bbcp data transfer process from Paranal to Garching Archive (ngas); green if currently no CALIB delayed by more than 1 hour, otherwise red; red alert may explain processing problems
	ngas	monitors the QC access to the archive (ngas); green if download possible, otherwise red; red alert may explain processing problems

calChecker is supposed to run for every instrument once per hour during daytime and twice every hour during nighttime, 24/7. If it is not refreshed within 2 hours, the "ago" box will tell you (it is calculated by your browser and will turn red after 2 hours). The first thing to then check is: does this problem show up only for the current instrument, or for others as well? If so, then it is most likely a network or a database problem. If you think the problem is on the Paranal side, check first with DHA. If you think it is specific to one instrument / QC account, send a mail (use the "contact" link if you expect the QC scientist to be able to read it; to the general list qc@eso.org, if not. On a weekend general problems will most likely be picked by the Garching maintenance teams, they provide service 7 days a week during daytime. Again ask DHA for support, they know whom to contact.

What to do if the "transfer" or the "ngas" watch flag are red? (intended for our Paranal partners)

Both flags watch the health of the data transfer process and the data access process for QC to NGAS. They are not of prime importance for calChecker but for data processing and for the "products" links displayed.

The transfer watch flag checks the health of the bbcp data transfer between Paranal and Garching Primary Archive. It turns red if CALIB files are found with delivery delays of more than 1 hour. If it is red, please consult first the DataTransferMonitor linked to the calChecker, or consult DHA. They will analyze if the problem is on the Paranal or the Garching side, and contact the right people if necessary.

The ngas watch flag monitors the QC data access to NGAS, the primary archive system in Garching. If access is not possible, data processing is not possible and may explain certain effects on the AB product monitor. For both watch flags it is true that if they show red, they should be red on all instrument versions of the calChecker, modulo execution timestamp of course. So you may want to check this first. Contact DHA and/or the QC scientist(s) for further investigation.

Paranal date: logical current date for this tool (change of date occurs at 21:00 UT in order to have daytime calibrations sorted under the same date as night-time science). Note that this convention differs from the one for some other processes (e.g., Paranal nightlog and opslogs change date at 12:00 LT).

Further information:

News: there is a "general news" section and an instrument-specific news section:

Links:

analyze ISSUES   HELP | ASSOC-RULES   DataTransferMonitor | BandWidth | history ... | contact

"analyze ISSUES" links to the analysis editor.

HELP links to this help page. The link ASSOC-RULES links to the association rules used on the calChecker.

There is a link to the DataTransferMonitor (powered by SDD), in case you want to check in detail which data have already been transferred to the Garching archive and are accessible to QC Garching. The Bandwidth monitor link (powered by DHA) monitors the current and past bandwidth usage.

Under history you can find older calChecker result pages.

The contact link can be used to send a mail to one of the QC accounts (which is read by the primary and secondary QC scientist for that instrument).

Result table

DATE*: [color if science data acquired]		2008-12-09 SM 11 report | NR	2008-12-10 SM 87 report | NR	2008-12-11 report | NR	2008-12-12 SM 23 report | NR	2008-12-13 report | NR	2008-12-14 VM 217 report | NR	2008-12-14 daytime calibs ...	action required? [if not green: take these data types ...	Setup: ... for these setups]
P... Product quality:     [?]		products	products	products	products	products	products	[daytime calibrations ongoing, started 13:33 UT]

Dates with science data are color marked and contain the SM and/or VM flag. The total number of science files is indicated. Dates without science data have a grey column in the result table but no SM/VM flag. The report and NR labels link to the raw data report and the nightlog. The nightlog section requires a password for login. Note that these nightlogs are read from the PANL2 database and may differ in structure and content from the ones distributed via email: their date starts with the nighttime observations and includes all following daytime calibrations. The nightlogs are updated several times during the QC workflow and are therefore more up-to-date than the distributed, static ones.

The last DATE column has a field indicating the status of daytime calibrations (pending; ongoing; finished). This is relevant for the judgment of missing calibrations for that date: any MISS or NOK in case of "finished" is significant, while otherwise these calibrations are likely to still be in the calOBBuilder queue. Also, this field might be helpful to detect cases when a calibration queue is broken or has been interrupted.

The row labeled as Product quality links to the product quality monitor (also called the "AB monitor" where AB stands for association block, which is the fundamental QC processing job).

The product quality monitor shows the current status of data processing at QC Garching, including quality scores (which are also used on the Health Check monitor) and QC reports. The AB monitor reflects the calibration quality, while the calChecker reflects calibration completeness. Processing of the calibrations by QCG is done automatically and within the hour unless there are data delivery delays.

The links labelled "Products quality" are marked products if the CALIBs for that date have already been certified by the QC group (to indicate to PSO that the product quality has been checked and potential issues have already been raised). Those nights which are not yet certified are marked products .

An exclamation mark ! products marks nights with data coming in unsupported modes. These data have no products and need a visual check by Paranal daytime astronomers.

The column "action required?" contains types of missing calibrations, and a link to the association rules used. The last Setup column lists the setup of missing calibrations. These two columns contain the necessary information about missing calibrations.

Data types are defined by mode and setup. Each science data type has one row, each date has one column. These two coordinates define a box which is evaluated by the tool as ok, nok or miss:

ok	available CALIB data are complete and all within the validity time range.
nok	available CALIB data are complete but at least one CALIB file type is outdated. New measurement should be done asap. The corresponding CALIB file type is listed in the rightmost column.
miss	CALIB data are incomplete, at least one CALIB file type is missing. New measurement should be done asap. Missing files are listed in the rightmost column.

Whenever a "missing" calibration is indicated, this is a reminder for obtaining them as soon as possible. Otherwise the corresponding SCIENCE observation may be lost (graded 'C' or 'D').

Analysis notes. Whenever possible, the QC scientist or the Paranal astronomer makes an attempt to analyze a situation with NOK or MISS. The analysis result is then displayed underneath the result table, as a separate table labeled "Analysis notes". More ...

Detail pages. Click on any of the links in the result boxes, and see detailed results per DATE: every single SCIENCE file is listed with its OB ID, grade and comment, and the details of its associated calibrations, including the time delay and the calChecker flag. These detailed pages are intended to help with the analysis of problems. For instance, they tell you immediately that an OB has been graded C and therefore in principle does not need complete and valid calibrations.

Analysis notes

Analysis notes are comments about a NOK or MISS case. Often the automatic assessment by calChecker is inappropriate and can be overridden. A typical example is a missing calibration referring to an OB graded as 'C'. The analyzed result then is 'OK', with a comment like "no issue, OB graded C". The analysis notes are displayed in the corresponding section of the main result table.

As the QC scientist or the Paranal daytime astronomer, you can provide analysis notes. Use the "analyze ISSUES" link.

analyze ISSUES   HELP | ASSOC-RULES | DataTransferMonitor | BandWidth | history ... | contact

You need first to authenticate (use the contact link if you do not know the credentials). The interface offers you all cases currently open for analysis (multiple selection possible). Enter the result, the new grade, your analysis note, and your email. The analyzed case will be be taken up by calChecker upon its next scheduled execution.

Here is an example:

Instrument information and configuration

Below the result table there is additional information about the data types and setups used for the calChecker:

This table contains information on how a setup is defined etc. In the CAL4CAL version, this information applies to the setups of the (triggering) calibrations.

Number of days scanned: this parameter defines how many days are scanned for science data, starting with the current date. By agreement this is 7 days.

Range of days for the calibration memory: this number is usually higher than the one before. It defines the depth of the calibration memory, i.e. the number of days scanned for calibration data.

Days in the calibration memory: List of days in the calibration memory.

Long-term and maintenance calibrations

Long-term calibrations are maintenance, technical and Health Check calibrations. Their distinguishing feature is that their acquisition is not driven by science data but by a maintenance calibration plan. Typically they are taken rarely (say every month or so), but there exist also cases (often Health Check calibrations) which are scheduled every week, or even more often. They are monitored on the calChecker in the following way:

• they have a configured validity (e.g. 30 days)the last occurrence of each data type is scanned in the raw data reports; the tool iterates backwards in time, to a maximum of one year
• the current age of the last instance is evaluated and flagged as
  • OK/green: if it is younger than 70% of the validitysoft REMINDER/yellow: younger than validity but within 30%; recommended action: refresh on next occasionREMINDER/yellow: outdated but by not more than 30%; recommended action: refresh on next occasion
  • REMINDER/red: outdated by more than 30%, refresh as soon as possible

All cases (if any) which flag yellow or red are displayed in the summary table on the main calChecker page. So these are the ones where some action needs to be taken (soon or now).

The complete overview is available behind the link "complete overview here". As an example, you may want to inspect the GIRAFFE case. It lists:

• the tag defining the type of calibration (e.g. DETECTOR_MON for the detector monitoring sequence), and a description of what that means; that description is meant to be specific enough to identify clearly the particular data set. It may contain the template name, a set of DPR keys etc.the validity and age in days
• a slightly more extensive version of the recommended action

The long-term calibration monitor is implemented by scanning the raw data reports (the ones which are linked to the calibration completeness monitor as report ). Scanning starts with the current date and goes backwards month by month, up to a full year. The first raw file matching the selection criteria will be evaluated in terms of age and validity.

There is no check for quality, nor for correctness of the header entries.

CAL4CAL checks and tellTracker link

Science and CAL4CAL checks
science		cal4cal	If you see these two tabs enabled, calChecker for this instrument is configured for CAL4CAL checks (calibrations for calibrations) and comes with two main result pages.
science		no cal4cal	If only the 'science' tab is enabled, this instrument is not configured for CAL4CAL checks (because they make no sense here).

For instruments configured for CAL4CAL checks, there are two main result pages instead of one, both are identical in format. The difference is the content. For the normal calChecker page the science data are used to trigger checks for calibration completeness. For the CAL4CAL result page, certain calibrations trigger those checks (hence 'calibrations for calibrations' or CAL4CAL for short). Classical example are standard stars which require their own set of calibrations, this set not being covered by the normal checks. Another example are IR twilight flats which may require darks in setups (DIT_NDIT combinations) which are different from the ones needed for science. We call these calibrations 'triggering calibrations' while the DARKs or FLATs are 'dependent calibrations'.

The CAL4CAL output format and colour-coding is identical to the one for science data. The CAL4CAL part of calChecker has its own configuration, ASSOC-RULES, NEWS file and ANALYSIS section.

Important: There is no logical link between the CAL4CAL part and the science part. This means, if a calibration for a triggering calibration is flagged MISS on the CAL4CAL page, and if that triggering calibration is required for science, the corresponding box for the science file is not marked as MISS. Hence, you need to always check both pages independently. If the science page displays only OKs but the CAL4CAL page has found an issue, it might be true that this issue indirectly affects also the science data. (It is simply not possible to transfer that information from the CAL4CAL page to the science page for all possible cases.)

science

cal4cal

If an issue has been found (a NOK or MISS result), the CAL4CAL navigation tab is marked by an exclamation mark ' !'. If there is no such flag, you can be sure that there are no issues on the CAL4CAL page.

tellTracker link
science		cal4cal		tellTracker	A few instruments have a third module of calChecker enabled, called tellTracker.

The tellTracker is designed to follow up time-critical nighttime calibrations, like the telluric standard stars. Find more information about tellTracker here.