Help for Health Check monitor

last update: October 12, 2009 | contact

topics: Update information and process monitor | navigation | result table | plot symbols | statistical functions | scores & quick-look

The Health Check monitor is developed and maintained by QC Garching. It provides information about the health of the instrument and about calibration quality.

Update information and process monitor

Last update: 2011-11-11T09:35:18 (UT) (0d 02h:05m ago)
now: 2007-11-11T11:37:22 (UT)

Last update: timestamp of creation of this page and plot (UT)

Now: current timestamp (UT) (press Ctrl+R to enforce reloading of the page, it may be cached)

(0d 20h:05m ago): difference of both timestamps (requires javascript enabled)

ftp monitor. For HC plots with OPSLOG data, there is a monitor for the ftp process (OPSLOG files from Paranal to Garching) and the parsing process (executed on the QC machines). A javascript calculates the difference between NOW and the respective timestamps of these two processes. If these differences are less than 2 hours, the corresponding boxes are coded green, otherwise red. If after 12 UT no OPSLOG fle has arrived, the code is grey, since then it is unknown if there is a problem or if no OPSLOG data are available (because e.g. all daytime calibrations have already been processed before 12 UT).

Possible values:
ftp
parser
ftp and parser results not older than 2 hours
ftp
parser
ftp result older than 2 hours, may indicate a problem; parser is up and running
ftp
parser
ftp and parser results older than 2 hours; the parser process is stopped; the ftp process cannot be monitored (so it could be ok)
... ftp
parser
no ftp result since 12 UT, parser up and running, cannot decide if ftp is broken or no OPSLOG file availabe yet

What to do if HealthCheck monitor page is outdated? (intended for our Paranal partners)

The Health Check monitor pages are refreshed upon discovery of new data, but at least once in 24 hours. If a page is found to be older than 24 hours (as indicated by the "... ago" field) this might indicate a problem (this field is calculated by your browser).

The first thing to check is: does this problem show up only for the current report, or for all reports of a given instruments, or for all instruments? In the latter case, there is most likely a network or a database problem. It may be known already, check first with DHA. If the problem is specific to one instrument, send a mail ("contact") if you expect the QC scientist to be able to read it; to the general list qc@eso.org, if not. On a weekend generic problems will most likely be picked by the Garching maintenance teams, they provide service 7 days a week during daytime. Again ask DHA for support, they know whom to contact.

If there is a problem with one report only, this is most likely due to the process on the QC machine. Pls contact the QC scientist in charge.

What to do if the "ftp" or the "parser" watch flags are red? (intended for our Paranal partners)

Both flags watch the health of the opslog data transfer and the parsing process on the QC side (see above). If the ftp flag is red, pls. ask DHA to check. It may help to click on "MORE" first, there you see all watch flags. This will tell you about a general problem. A gray ftp flag does not indicate a problem.

If the parser flag is red, send a mail ("contact") if you expect the QC scientist to be able to read it; to the general list qc@eso.org, if not.

Any problem related to these two flags is not considered very urgent since the opslog data are a fallback solution in case of trouble with the data transfer process.


[ top ] Navigation

HELP USERS-GUIDE MORE

This navigation links to:

  • this file;
  • the process description (users guide);
  • a file with additional information (time zone information, link to ASM, technical monitor of ftp and parser process).

Group navigation. Plots are often grouped together. For instance, plots with detector properties could be grouped like:

same group: bias   dark linearity&gain fp-noise contamination

The most important plots are marked:   daily/often; important to check These are the ones which are fed daily by dedicated health check calibrations. These are also the ones which should be checked daily. The other ones are also of interest but are not filled so frequently.

The second line displays the report scores (see here).

OPSLOG information. If OPSLOG data are included, their numbers are collected here. There is the option to inspect or download these data.

last OPSLOG data:
2007-11-06
Plot: 1 2 3 4 5   latest files
ftp
parser
N_data: number of recent OPSLOG entries plotted here
N_data: 7 7 7 - - availability

Quick-look navigation, history, tutorial, contact. If enabled for QUICK-LOOK versions, the HC plots have a link labelled as 'less', which has its counterpart on the QUICK-LOOK version ('more'). Here you can toggle between these two views.

less ... | history ... | plot tutorial ... | contact                 daily/often; important to check     [?]

History. Many HC plots have a HISTORY link. This link opens the set of HISTORY plots in a separate window.

The HISTORY plots come in the same format as the HC plot and present the historical evolution. Click on the red arrows to move forth and back in time, or on the symbols for direct navigation:
close window history: 2003 2004 2005 2006   2007   2008   HEALTH FULL
  ... ... ...

"HEALTH" is the HC plot, the only one to contain OPSLOG data (if configured and if available).
"FULL" is a link which sometimes exist and combines all historical data points in one plot.

Tutorial. Often this link exists and provides additional information about the QC items, the data, the algorithms etc. These pages are in the instrument QC section.

Contact. Send an email to the QC scientist responsible for this particular HC page.


[ top ]Result table

Plot
?
Symbol
?
Source* OPS?** Average ? Thresholds ? N_data QC1 parameter Data downloads Remarks
method value unit method value
1 QC1DB yes (6) MEAN 151.2 ADU 3SIG 1.051 51 flux this | last_yr | all median BIAS level
plot number
plot symbol
data source
OPSLOG entries (same as OPSLOG table)
applied methods, trending results, units, number of data
links to download the data (this time range; last 365 days; all data)
remarks

Find in this table more details about the applied statistics and the current trend.


[ top ]Plot symbols

The symbols used in the plots are displayed in the column "Symbols". An attempt has been made to display them in the HTML pages as they are shown on the MIDAS plots, but some symbols do not exist as HTML code and show up here as bracketted number, like [20]. The table below lists all possible plot codes and reveals that [20] stands for a filled triangle.

SYMBOL meaning   SYMBOL meaning
1 dot   11 lozenge ◊
2 open circle   12 horizontal bar –
3 square   13 vertical bar |
4 triangle   14 right arrow →
5 plus +   15 arrow up ↑
6 cross x   16 left arrow ←
7 asterisk   17 arrow down ↓
8 star *   18 filled hexagon
9 plus square   19 filled square
10 cross square   20 filled triangle
      21 filled lozange

[ top ] Statistical functions

Average:
method implementation
MEAN average (with clipping if combined with certain threshold option, see below) MIDAS: stat/tab; outputr(3)
MEDIAN median (with clipping if combined with certain threshold option, see below)   MIDAS: stat/ima; outputr(8)
NONE nothing applied  

The average values are listed in the result table (together with their units and the number of data points evaluated). They are indicated in the plots by a solid line. They are clipped (outliers beyond the thresholds are removed after first calculation, then the calculation is repeated with the cleaned data set) if combined with certain threshold options (see below).

Thresholds:
method implementation clipping
1SIG standard deviation MIDAS:
if Average=MEAN: stat/tab; outputr(4)
if Average=MEDIAN: stat/ima; outputr(4)
yes
3SIG standard deviation 3times 1SIG yes
PER=val1 relative thresholds percentage of average; e.g., PER=10: thresholds are 10% of average value yes
OFF=val1 relative thresholds as PER, but fixed offset; e.g., OFF=0.05: thresholds are average ± OFF yes
VAL=val1,val2 fixed thresholds thresholds are independent of average; val1, val2 listed in the result table no
none nothing applied   no

The threshold values are listed in the result table. These values are indicated in the plots by broken lines, and are used to detect and mark statistical outliers. 1SIG, 3SIG, PER and OFF thresholding is relative to the average value. 1SIG and 3SIG is dynamic and based on the statistical results for rms, PER and OFF are fixed relative to the average value. VAL thresholding is completely static and might represent e.g. a specification or an alarm level based on experience or specifications.

Outliers. Statistical outliers are marked if thresholds are defined and plotted. Marking is done by a red asterisk (to become blue if the symbol color is red).

Data points outside the Y range. These data points are marked by a red arrow.

OPSLOG data points. The data points from OPSLOG files (produced by the on-line Paranal pipelines) are always marked by a filled blue circle .

LAST data point. If configured, all data points from the last date are highlighted by a large magenta circle .

Data source:
method
QC1DB QC1 database (final, certified QC1 parameters from pipeline processing by QC)
OPSLOG QC1 ops logs (preliminary QC1 parameters from automatic pipeline processing on Paranal)
LOCAL Local data source (QC1 parameters not yet ingested into QC1 database, e.g. since they are under development; not considered a permanent option!)


[ top ] Scores and quick-look

Quick-look version. Clicking on scores ... brings you to the score overview (QUICK-LOOK page). It displays the score result table:

DARK parameters (last 7 days)
1: DARK_current
1|
2: GLOW
2|

It displays scores for each HC report. Scores are flags assessing the compliance of data points with configured thresholds. A green score symbol indicates that the corresponding instrument component performs OK in the monitored time range, a red one may indicate a problem.

Each plot in the report has one score table cell. Each cell displays a set of scores. Scores are displayed for the last 7 days only. This time range is aligned with the calChecker page which also has a standard depth of 7 days.

Scores are based on QC1 parameters. They are checked against an upper and lower threshold. If compliant, the score is 0, otherwise 1. An outlier is scored as 1.

All scores are aggregated to plot scores and report scores. If no outlier is found, the corresponding plot is scored OK and marked green. Outliers get a NOK score and are marked red:

2: GLOW
2|
This plot has two QC data points in the monitored time range, both scored OK.   2: GLOW
2|
... a NOK score.

Plot scores. If the last QC data point for a plot is found NOK, the total score for the plot is NOK and marked red (see below). If any other QC data point is found red but the last one is green, the total score for the plot is OK and marked green (since it is assumed that the problem is solved in the meanwhile).

2: GLOW
2|
Both data points are scored OK. The plot score is OK.   2: GLOW
2|
Same plot, with one data point scored NOK (but not the last one) . The plot score is OK. 2: GLOW
2|
Same plot, with the last data point scored NOK. The plot score is NOK. 3: ALL_GLOW
multiple dataset
Note: Plots with multiple data sets cannot be scored.

Report scores. Total scores per plot are accumulated to report scores and displayed at larger size (see example above). Report scores can be OK, NOK or grey ("not implemented"). If no report score is red (some may be grey), the plot score is OK and displays green. A single red plot score will turn the report score also red. Reports without scoring are marked grey and are not evaluated further.

Other possible tags:

multiple dataset more than one dataset in a single report; since scores are always for a report, this situation is usually avoided but may make sense exceptionally
computed values as displayed are computed after download from the database; for technical reasons, these values cannot be scored

Report scores are repeated in the group navigation bar, for quick reference:

same group: bias   dark linearity&gain fp-noise contamination

Group scores. Likewise, report scores are accumulated to group scores. They are displayed in the vertical navigation bar at left:

FLAMES/GIRAFFE:

If you want to monitor the current health of an instrument, you start with its navigation bar. Either all items are green, or you find at least a red one. In that case, follow any red score to the group level, find the corresponding report and plot scores. There you may want to check the detailed HC plot. All in all 2 levels to see the QC data point which causes the alert, and 3 to see all details.