16.3 Basic Data Reduction: calnica

Figure 16.1: Conceptual calnica Pipeline; CDB is the Calibration DataBase

Figure 16.2 shows the portion of a calibrated NICMOS science file header containing the switches and reference file keywords that pertain to the processing performed by calnica. The accompanying flow chart (Figure 16.3) shows the sequence of calnica calibration steps, the input data and reference files and tables, and the output data file. Each calibration step is described in detail in the following sections.

Figure 16.2: Partial NICMOS Header

Figure 16.3: Pipeline Processing by calnica

ZOFFCORR

The pipeline will subtract the zeroth read image from all readouts, including the zeroth read itself. Furthermore, the self-subtracted zeroth-read image will be propagated through the remaining processing steps and included in the output products, so that a complete history of error estimates and data quality (DQ) flags is preserved. After this step is performed, the science data are in the same form as the raw science data from any other observing mode and are processed the same way throughout the remaining steps of calnica. No reference files are used by this step.

MASKCORR

BIASCORR

The BIASCORR step therefore searches for pixel values in the range -26000 to -32768 and adds an offset of 65536 DN to these pixel values to reset them to their original real values.

No reference files are used by this step.

NOISCALC

Eq. 16.1

where _rd is the read noise in units of electrons, adcgain is the analog-to-digital conversion gain factor (in electrons per DN) and counts is the signal in a pixel in units of DN. Noise is computed in units of electrons, but the result is converted to units of DNs for storage in the error image. The detector read noise is read pixel-by-pixel from the NOISFILE reference image and depends on the read rate of the observation, as well as the number of initial and final reads (NREAD). Separate NOISFILEs are required for each combination of read rate and NREAD. The data quality flags set in the DQ image of the NOISFILE are propagated into the DQ images of all image sets (imsets) being processed.

For RAMP mode observations, the error estimate computed by the instrument (and therefore, present in the raw science file) is a variance. For these observations the NOISCALC step simply computes the square-root of the raw error image so that it is on the same basis as other modes.

Throughout the remaining steps in calnica, the error image is processed in lock-step with the science image, getting updated as appropriate. Errors are mostly propagated through combination in quadrature.

DARKCORR

A simple scaling of a single dark reference image to match the exposure time of the science data is, unfortunately, not possible due to the non-linear behavior of the dark current as a function of time. Therefore, a library of dark current images is maintained, covering the range of exposure times of the MULTIACCUM sequences and a subset of ACCUM exposure times and NREAD values (see the NICMOS Instrument Handbook). The reference dark appropriate for the exposure time and sequence used in MULTIACCUM, or the exposure time and NREAD values used in ACCUM, is determined and subtracted from the data. For non-standard MULTIACCUM sequences (see the NICMOS Instrument Handbook) and for some ACCUM exposure times, the appropriate dark image is interpolated from existing dark files. There is one reference file (DARKFILE) per detector, which contains the set of dark images (at various exposure times) for that detector. Error estimates of the dark current, stored in the ERR images of the DARKFILE, are propagated in quadrature into the ERR images of all processed IMSETs. Data quality (DQ) flags set in the DARKFILE are also propagated into the DQ images of all processed imsets.

Sets of synthetic dark reference files have been recently generated to populate the calibration database. These dark files reproduce all the characteristics of on-orbit darks, which do not suffer from the pedestal (see Chapter 17).

Dark subtraction is skipped in BRIGHTOBJ mode, because the short exposure times should result in insignificant dark current.

NLINCORR

• At low signal levels the response is linear and no correction is needed; the low signal level for NICMOS is pixel- and camera-dependent, and is about 14,500 DN and below, with a standard deviation of about 400 DN.
• At intermediate levels the detector response deviates in a linear fashion from the incident flux and is easily correctable via the expression: where c1 and c2 are the correction coefficients, F is the uncorrected flux (in DN) and F_c is the corrected flux.

• At high signal levels-as saturation sets in-the response becomes highly non-linear and is not correctable to a scientifically useful degree; the saturation level is about 30,500 DN, with a standard deviation of about 2,000 DN. The NLINCORR step applies no correction to pixels in the low signal regime, applies a linear correction term to intermediate level pixels, and applies no correction to pixels in the high signal regime but rather flags them in the DQ image as saturated. This step uses the NLINFILE reference file, which consists of a set of images containing correction coefficients and variances at each pixel. Error estimates on the correction applied to intermediate-level pixels are propagated into the ERR images of all imsets processed. Data quality flags set in the NLINFILE are also propagated into the processed DQ images. There is one NLINFILE per detector.

UNITCORR

PHOTCALC

CRIDCALC

This step identifies and flags pixels suspected of containing cosmic ray (CR) hits. For MULTIACCUM mode observations, this step also combines the data from all readouts into a single image. In MULTIACCUM mode, the data from all readouts are analyzed pixel-by-pixel, searching for and identifying the data from individual readouts that appear as outliers using an iterative sigma-clipping technique. In each pixel, the signal from each readout is ascribed a standard deviation given by the combination of readnoise and Poisson noise. Values which deviate more than 5 from the mean slope of the counts-versus-time relation are identified as outliers, and the corresponding pixels in the DQ images of the intermediate MULTIACCUM (*_ima.fits) file are flagged, however, the pixel values themselves in the SCI and ERR images are unchanged. Once all outliers have been identified, a final, combined value is computed for each pixel using only non-flagged samples. The result of this operation is stored as a single imset in the output *_cal.fits file in which the number of unflagged samples used to compute the final value for each pixel and the total exposure time of those samples will be reflected in the SAMP and TIME images. The variance ascribed to the final mean countrate is the uncertainty in the slope of the counts-versus-time relation at each pixel location. Pixels for which there are no unflagged samples, e.g., permanently hot or cold pixels, will have their output SCI, ERR, SAMP, and TIME values set to zero, with a DQ value that contains all flags that were set. The cosmic ray rejection threshold, 5 , is currently hardwired in the calnica code, and seems to perform a reasonable job of identifying and rejecting cosmic rays.

The algorithm is not yet completely defined or implemented for ACCUM, BRIGHTOBJ, and RAMP mode observations, therefore the *_cal.fits output file for these modes will still contain cosmic ray hits.

BACKCALC

WARNCALC (User Warnings)