NACO: Detector monitoring

HC PLOTS 
saturation level 

linearity 

gain 

fixed pattern noise 

QC1 database (advanced users):
browse 
plot

The NACO detmon template acquires a series of flat field pairs and
corresponding dark frames in order to retrieve the detector gain and detector
nonlinearity. Up to 2016 the following four read modes are monitored:
 Double_RdRstRd_HighSenistivity
 Double_RdRstRd_HighDynamic
 FowlerNsamp_HighSensitivity
 Uncorr_HighWellDepth
Notes on the history
Data acquired in 2015 are based on a new detector, for which the template is
not optimized. 2015 detmon data are less useful.
In 201601 Double_RdRstRd_HighSensitivity is replaced by Uncorr_HighDynamic,
and the monitoring is restarted again. Since the detector suffers since
201501 from a dead column effect in the lower left quadrant, only the upper
right detector quadrant is used for the detmon analysis. Furthermore it turned
out (for the period 2015 up to 201601) that flats with DIT=3sec and Uncorr
mode show additional fixed pattern noise which biases the photon transfer
curve when retrieving the gain. These DIT=3sec flats are ignored when
processing detmon frames in Uncorr read mode.
Status since 20160501
The following read modes are monitored, values are for the upper right 512x512
pixel detector quadrant:
read mode  Double_RdRstRd_HighDynamic  FowlerNsamp_HighSensitivity  Uncorr_HighWellDepth  Uncorr_HighDynamic 
DIT range  0.5  15 sec  1.8  14 sec  0.25  26 sec  0.25  15 sec 
saturation level  11600 ADU  3230 ADU  16400 ADU  9500 ADU 
offlamp  100 ADU  30 ADU  3000 ADU  3000 ADU 
full well  11500 ADU  3200 ADU  19400 ADU  12500 ADU 
comment  LR quadrant contaminated  LR quadrant contaminated  read window = 512x512  oddevenrow effect 
DetMon recipe     
DM linearity range  11500 ADU  2900 ADU  8500 ADU  8500 ADU 
DM linearity reference  8000 ADU  2000 ADU  8000 ADU  8000 ADU 
DM gain range  10000 ADU  2000 ADU  8000 ADU  8000 ADU 
Status 201707:
The detector controller was reconfigured, that the saturation
level of the Fowler sampling read mode is again at 4000 ADU.
20181107: Two sequences with DET.NCORRS.NAME=Uncorr and DET.MODE.NAME=HighBackground have been
acquired once. This is the read mode used for Mband imaging only. The following QC1 parameters were retrieved: gain = 11.7 ADU/e, nonlinearity = 0.099 at 20000 ADU, fixed pattern noise = 0.03,
(short DIT) dark level at 1300 ADU and saturation level of 24300 ADU.
related QC report (restricted access).
Saturation level
In optical CCDs there is a fixed saturation level of 65335 ADU (=
2^(16)1) and the small nonlinear component of the response can be
described by a polynomial in the count range of up the discontinuity at
65335 ADU. In IR detectors the nonlinearity is stronger, starts at lower
counts and the saturation level is reached more smoothly when compared to
CCDs. The detmon pipeline recipe is configured to use for each of the four
monitored read modes an optimized set of recipe parameters. These command
line parameter sets are tuned to retrieve the gain and the nonlinearity
but not the saturation level. Since in 201510 the saturation level
changed in one of the read modes, the following procedure was implemented
to monitor the saturation level (see the magenta points here):
The median values of the raw flat frames is fit by the following function:
f(x) = [1/2  atan((xx0)/s)] * (a+b*x) + [1/2 + atan((xx0)/s)] * c
with a linear part (a+b*x) dominating for x << x0 and a constant part c
dominating for x >> x0, where x0 is the transition zone between both functions
and s is a smoothing width for the transition zone.
The left and the right [ ]  bracket are numerical weighting factors that
change at x=x0 from 0 to 1 or vice verse. This composed function f(x) fits the
linear part at low DIT and the constant (saturation part) at high DIT.
a,b,c, and x0=transition point and s=transition width are best fit parameters.
See the following QC report as an example.

The median values versus DIT have been fit two times, once
(green f(x) ) with a free width and a second time (blue g(x) ) where x0 is
fixed from f(x) and the width s is fixed to a 5 times smaller value of f(x),
which results in a more stable values of c.

QC1_parameters
FITS key 
QC1 database: table, name 
definition 
class* 
HC_plot** 
more docu 
[derived from QC procedure]  naco_detmon..brightest_counts  counts in brighest flat generated by template  HC   [docuSys coming] 
[derived from QC procedure]  naco_detmon..max_count  maximum counts used for linearity estimate  HC   [docuSys coming] 
[derived from QC procedure]  naco_detmon..fit_c0  fit: saturation in ADU  HC   [docuSys coming] 
*Class: KPI  instrument performance; HC  instrument health; CAL  calibration quality; ENG  engineering parameter
**There might be more than one. 
Trending
The trending plot covers the three parameters explained
above, to check that a) the frame sequence covers the
saturation level, b) the recipe is configured to cover the
saturation lavel and that c) the saturation level of the detector
is stable (it was not in 2016).
Scoring&thresholds Saturation level
The values are not scored
History
Date 
event 
201501  start of monitoring 
2015  In late 2015, the full well depth of the Fowler sampling read mode decreased, see Fig. History1 
201603  detmon template finetuned 
201707  detector full well depth reconfigured for Fowler sampling 

History1: This plot shows the median counts in imaging flats
as dependent on the exposure time (DIT) acquired with the detmon
template. The read mode is FowlerNsamp. The DIT values of the sequence
from 201603 have been scaled by 0.55, that the linear part of the
detector response matches in both calibration sequences. The sequence
acquired in 201509 (red) saturates at ~ 4300 ADU, while the sequence
from 201603 (green) saturates at ~3400 ADU.
The same issue happened again in 201611 with a smaller change.

Algorithm Saturation level
The counts are directly read from the raw frames.
Linearity
The nonlinearity of the detector response is calculated by the detmon recipe.
A higher order polynomial is fit to the dark subtracted flat frame median as a
function of DIT. See:

This figure shows an extraction of the larger QC report. The
third order polynomial fit (green) and a corresponding linear component is
shown. The QC parameter returned from the recipe is the deviation from
linearity at a reference level of 8000 ADU (for Double_RdRstRd_HighDynamic).

Dividing the green line by the blue line and expressing DIT by ADU, the
deviation from linearity is shown:

The figure shows that in 201605, the relative deviation from
nonlinearity at R=8000 ADU is about 0.04, which is monitored as QC parameter
in this HC plot. The nonlinearity is a function of count level.

QC1_parameters
FITS key 
QC1 database: table, name 
definition 
class* 
HC_plot** 
more docu 
QC.LIN_EFF  naco_detmon..non_linear  nonlinearity at reference flux level  HC   [docuSys coming] 
*Class: KPI  instrument performance; HC  instrument health; CAL  calibration quality; ENG  engineering parameter
**There might be more than one. 
Trending
See gain
Scoring&thresholds Linearity
This parameter is not scored.
History
See gain
Algorithm Linearity
Details are given in the
detector monitoring pipeline user manual.
Gain
The detector gain value is calculated by the detmon recipe, where a counts
threshold for excluding bright flats is applied in order confine the analysis
(the photon transfer curve method is used) to the linear part of the detector
response. The following figure shows an extraction of the QC report.

The left box shows the variance of the flat frames (minus the
variance of the offlamp frames) as a function of twice the dark subtracted
flat counts.
The inverse of the best fit linear function slope is the gain, which
corresponds in this case to 13.4 e/ADU.
The right figure shows the gain values for each DIT, where two flats and two
dark frames have been used (red circles).
For comparison, the gain retrieved from the left figure is plot (blue line).

Note: The variances show a slight nonlinear behavior. A nonlinear fit would
result in lower gain results (steeper slope) for low counts and higher gain
values for the less steep high count range.
The offset of 169 ADU is a known unreliable linear extrapolation of the photon
transfer curve and is meaningless.
QC1_parameters
FITS key 
QC1 database: table, name 
definition 
class* 
HC_plot** 
more docu 
QC.GAIN  naco_detmon..gain  gain (e/ADU)  HC   [docuSys coming] 
*Class: KPI  instrument performance; HC  instrument health; CAL  calibration quality; ENG  engineering parameter
**There might be more than one. 
Trending
The Gain s derived from the detmon recipe is monitored for four detector setups.
Scoring&thresholds Gain
The gain is scored. The thresholds have been set such, that data contamination by sporadic events of the
8column noise in the upper right quadrant exceed the thresholds and give a red score.
History
Date 
event 
201501  The detector is exchanged. The newly installed detector is the Aladdin detector formerly used by ISAAC LWarm. It is mounted in NACO by 90deg rotated with respect to ISAAC and it is operated with the NACO read modes. 
201507  The upper right quadrant is subject to a pattern where every 8th column is dead. 
201510  The saturation level of the Fowler sampling read mode changed from 4300 ADU to ~3400 ADU. 
201512  The upper right quadrant is ok, but the lower left quadrant is subject of a pattern where every 8th column is dead. 
201606  The statistical noise of the two Uncorr read modes increased for larger DIT, resulting in a reduced gain value ( 25% less ), see Fig. History2. 
201611  The saturation level (full well depth) becomes smaller in Fowler sampling mode and partially in Double ResetReadRead mode. 
201707  The detector controller was reconfigured, that the saturation level of the Fowler sampling read mode is again at 4000 ADU. 
201709  From time to time (in single nights), the saturation level is found to be lower by several hundreds of ADU for the Double and Fowler read modes 
201802  It can happen from time to time that the flats acquired with DIT=2sec are contaminated by statistical noise of unknown origin. In this case the flat pair is discarded from the sequence, the event is documented and the shortened data set is reprocessed to monitor the gain and nonlinearity. 
20180515_and_20180426_and_20180328  The flat difference frames show 8column noise in the UR quadrant, contaminating the statistical noise and hence the gain calculation. Gain thresholds set to + 2 e/ADU, to become sensitive to stripe pattern in upper right detector quadrant. 

History2: On 20160603 statistical noise as read from flat difference frames changed its relation to the exposure time (or to the flux level).
The noise became larger for larger DIT resulting in a steeper photon transfer curve and a lower gain. For Uncorr HighWellDepth read mode the gain reduced from 10 e/ADU to 7.5 e/ADU.
This Figure shows the photon transfer curve for the Uncorr HighWellDepth read mode from and of May 2016 (red) with a derived gain value of ~10 e/ADU and from early June 2016 (blue) with a derived gain value of ~7.5 e/ADU.

Algorithm Gain
The photon transfer curve is used. Details are given in the
detector monitoring pipeline user manual.
