Quality Control and
Data Processing

HC PLOTS
RMS of master flat
RMS of ratio flat
counts (min,max)
counts range
QC1 database (advanced users): browse | plot

The vircam calibration plan includes twilight flats to calibrate science exposures. The derived QC parameters measure operational properties, e.g. if the covered flux range is optimal, or heavily underexposed. No instrument specific property is monitored.

QC1_parameters

FITS key	QC1 database: table, name	definition	class*	HC_plot**	more docu
QC.FLATRMS	vircam_twilight..qc_twil_rms	rms in master flat	ENG		[docuSys coming]
*Class: KPI - instrument performance; HC - instrument health; CAL - calibration quality; ENG - engineering parameter **There might be more than one.

Trending

The rms of the master flat is monitored for all filters: NB118, NB980, Y, Z, J, H ,K separately. The K-band is monitored here.

Scoring&thresholds RMS of master twilight flat

There is no scoring.

History

No remarks.

Algorithm RMS of master twilight flat

The QC.FLATRMS is a measure of RMS in ADU obtained from master twilight flat products. It is the median of the absolute deviations from the median ( = MAD ) times 1.48 derived from the master twilight flat.

QC1_parameters

FITS key	QC1 database: table, name	definition	class*	HC_plot**	more docu
QC.FLATRATIO_RMS	vircam_twilight..qc_ratiotwil_rms	rms of ratio flat	ENG		[docuSys coming]
*Class: KPI - instrument performance; HC - instrument health; CAL - calibration quality; ENG - engineering parameter **There might be more than one.

The vircam twilight flat pipeline recipe allows to specify a reference flat, which can be any master flat. The pipeline divides the master flat by the submitted reference flat and extracts some QC parameters of which the RMS of the ratio flat is the most important. The pixel-to-pixel relative response differences (= the fixed pattern noise), which dominate the RMS of the master flat, are canceled out in the ratio flat. The RMS of the ratio flat is therefore sensitive to other changes in the master flat, like:

• changes in the relative gain of the 16 read-out channels (16 vertical stripes each one 128 pixel wide). Note the comment for the dome flats. Since early 2020 channel gain variations increased. In this period the values of qc_ratiotwil_rms are dominated by channel gain variations and are no longer sensitive to the detector characteristics originally intended to be monitored, in particular for detectors #6 and #10, #12 and #14.
• new dust grains
• new bad pixel (e.g. 2013-03--01 on chip #3)
• controller issues

Trending

The RMS of the ratio twilight flat is monitored here.

Scoring&thresholds RMS of ratio flat

The values are not scored

History

The maintenance of the reference flat means it has to be updated from time to time, typically twice a year and after interventions. The twilight flat QC1 DB contains the name of the reference twilight flat (qc_twflat_refname) and the time stamp of the reference twilight flat (qc_twflat_refdate).

Algorithm RMS of ratio flat

The master flat is divided by the reference flat, the RMS is taken.

QC1_parameters

FITS key	QC1 database: table, name	definition	class*	HC_plot**	more docu
counts in faintest flat	vircam_twilight..qc_twflat_medmin	minimum counts	ENG		[docuSys coming]
( max - min ) / 2	vircam_twilight..qc_twflat_medavg	average counts	ENG		[docuSys coming]
counts in brightest flat	vircam_twilight..qc_twflat_medmax	maximum counts	ENG		[docuSys coming]
max - min	vircam_twilight..qc_twflat_medrng	counts range	ENG		[docuSys coming]
*Class: KPI - instrument performance; HC - instrument health; CAL - calibration quality; ENG - engineering parameter **There might be more than one.

Trending

The counts in the faintest flat is monitored as red diamond symbols, the counts in the brightest flats is monitored as black square symbols. The qc_twflat_medavg is monitored in the same plot as green circles. Due to the different responses per detector, the detector with lowest response of the brightest raw flat can show less counts that the highest response detector of the faintest raw flat.

The counts range of the raw flat sequence is monitored here. The counts in the brightest flat can be divided by the DET.SATLEVEL to check how close the counts of brightest flat are at the saturation level. The values are scored for the individual, but there is not trending plot provided, as being a pure product quality and not an instrumental property.

In case raw flats are rejected by the pipeline recipe because being too bright or too faint, the master flat of that chip is composed of a lower number of raw flats, which is expressed by the PRO.DATANCOM key. This value is scored on the product level, but there is no trending plot provided.

Scoring&thresholds Minmax of twilight flat series

The values are not scored.

History

No remarks.

Algorithm Minmax of twilight flat series

min,max,avg,range: The counts of the twilight flat raw frames vary as the sky brightness varies during dusk or dawn. For each raw frame the median counts is taken. Out of this sample of raw frame medians the minimum, the maximum, the average, the standard deviation and the range is extracted.

fracsat: Among all median flux values obtained from the raw fats, the highest flux value is multiplied by (MINDIT+DIT) / DIT to get the flux value on the chip before the detector reset was subtracted by the acquisition system. The value is compared with the saturation level. Details are given on the problems page.

Each detector has its own response. Therefore twilight flats and dome flats show the following relative count rates:

chip13	chip 14	chip 15	chip 16
0.79	1.04	1.17	0.91
chip 9	chip 10	chip 11	chip 12
1.04	1.13	1.03	1.13
chip 5	chip 6	chip 7	chip 8
1.08	1.19	1.26	1.01
chip 1	chip 2	chip 3	chip 4
1.24	0.79	0.86	1.04

chip #1 and #13 showing the smallest response and chip #1 and #7 showing the largest response. Although the response of chip #5 is close to the array average, it is the first to saturate because of its low saturation level.

QC1_parameters

FITS key	QC1 database: table, name	definition	class*	HC_plot**	more docu
NONE	vircam_twilight..qc_twflat_fluxdep	flux depression (for chip #16)	ENG		[docuSys coming]
*Class: KPI - instrument performance; HC - instrument health; CAL - calibration quality; ENG - engineering parameter **There might be more than one.

The upper part of detector #16 shows the following effect when being illuminated:

• In J band and NB118 shows about 75% of the flux when compared to the remaining area of the chip. The level of the flux depression was rather constant within the investigated time interval (2009-11 ... 2010-02).
• The H and K band flats are not affected.
• The Z and Y band images show a remaining flux of 60% to 80% . The flux depression is variable in time and the variability of the flux level is not correlated between Y and Z band. The flux depressed pixel area on chip #16 is not homogeneous, but the area remains constant in time.
• In addition to the large scale fractal-shape area with flux depression there are a considerable number single pixels (isolated pixels) with unreliable response.
• As a consequence source detection as part of the pipeline science recipe is affected for the upper part of chip #16.

The QC1 parameter qc_twflat_fluxdep, as obtained from detector #16 master twilight flats. The parameter represents the median flux in the upper part of the detector in the pixel area: 1500,2048: 2048,2048. The abscissa is the MJD-OBS (modified Julian Date).

Trending

The depression of the upper part of detector #16 is monitored for all filters.

Scoring&thresholds Chip #16 depression

The parameter is not scored.

History

No remarks.

Algorithm Chip #16 depression

Using the (normalized) master twilight flat, the median counts in the following window is given:

xmin=1 ymin=1500 xmax=2048 ymax=2048 , which covers almost the upper part of the detector. Setup: INS.FILT1.NAME = Ks (2.15), H (1.65), J (1.25), NB118 (1.185), Y (1.02) NB980 (0.98), Z (0.88)

As the upper region of detector #16 shows a rather fractal pattern, the minimum counts in the analyzed region are much less than the median value monitored.The purpose of the plot is to monitor the unpredictable variations in the blue bands.

QC1_parameters

FITS key	QC1 database: table, name	definition	class*	HC_plot**	more docu
None	vircam_twilight..qc_twflat_expfitrms	rms of skybrightness fit	ENG		[docuSys coming]
*Class: KPI - instrument performance; HC - instrument health; CAL - calibration quality; ENG - engineering parameter **There might be more than one.

Quality control report for chip #10 of a twilight flat. Black is reported counts and blue points are counts corrected for the MINDIT effect. The saturation level, 90% of the saturation level and the pipeline lower rejection threshold is marked. Right: least square fit of flux(time)= A + B * exp ( - C * (D - time)) to the data points.

Trending

The cirrus parameter is monitored

Scoring&thresholds Twilight cirrus

This parameter is not scored

History

In 2019-01 two cases were found where one of the raw frames of the sequence showed in several but not all detectors a controller issue (an unillumnated image with a peculiar fixed patter noise). The inclusion of the fake raw frame cannot be detected in the QC parameters extracted from the master twilight flat. Only the cirrus parameter is sensitive to these events. Controller issues can show up in cases of high ambient temperature.

Algorithm Twilight cirrus

Under photometric and clear conditions the sky brightness is (de)inreases during (dusk)dawn in a continuous manner. Thin or thick cirrus moving through the FOV disturb the continuous flux trend. The function flux(time)= A + B * exp ( - C * (D - time)) is fit to the raw flux sequence and the RMS of the fit is considered as a measure of the sky brightness continuity within the FOV. The CirrusTest parameter is the RMS of the exponential fit to fluxes retrieved from the twilight flat raw frames. It is a quality parameter of the master twilight flat.

2009-10-3 ... 2009-11-03: twilight flat saturation

The acquisition of photons by near infrared detectors consists of an initial reset, a first non-destructive read out followed by one or more further reads. The signal recorded between the reset and the first read is subtracted by the acquisition system before being written to file. The time between the reset and the first read amounts to 1.0011 sec and is called the minimum possible DIT (= MINDIT). The time between the first and the final read is called DIT. The fits frame contains the counts accumulated within DIT seconds, but the pixels on the detector hosted more counts accumulated within DIT + MINDIT seconds and might saturate the chip. This internal saturation can occur for high fluxes and when a DIT of the order of MINDIT is used. This effect occurred for a number of raw twilight flats in some periods during the commissioning (2008-10-01 - 2009-10-25), science verification (2009-10-15 - 2009-11-03) and dry runs (2009-11-04 - 2010-02-15). Assuming linear response the signal on the detector before reset subtraction can be estimated :

counts[detector] = counts[rawframe] * ( MINDIT + DIT ) / DIT

The counts[detector] values are monitored for twilight flats. The maximum values within a raw frame stack is given in units of the individual detector saturation level by the dimensionless QC1 parameter qc_twflat_fracsat. The values are scored.

	Counts of a stack of twilight flat raw frames with DIT=1.0011 sec for detector #5. The black dots show the counts[rawframe] values and are well below the saturation level. The blue dots show the counts[detector] values of which the first few frames show saturation effects. The saturation level and the scoring threshold (= 90% of saturation level) are shown.
	The same stack for detector #11 with a higher saturation level.