NACO:
Association Rules

1. Introduction

Data Classification and Association is a general requirement for data processing. In the VLT DataFlow raw NACO frames (FITS files) are classified in Garching to be processed by the data reduction pipeline and to be distributed to the individual PIs. Data can be classified as either science data or calibrations . Other classification schemes use the instrument mode and the NACO raw data are processed at two places in the VLT DataFlow:

On Paranal: raw data arrive iduring the night on the pipline workstation in the same order as they are generated by the VLT. They are processed for quick-look purposes wihout requiring further calibration frames.

In Garching: all frames of the night including the day-time calibrations of all previous nights are avilable when the science data of a night are processed.

CLASSIFICATION: Raw files must be classified in order to associate raw frame stacks with the appropriate data reduction recipe. In any case the FITS header keywords are used to cross check against a classification table. In the case of NACO this task is performed by the Data Organizer (DO) on Paranal. In Garching we use scripts to classify raw data. The most important key is the HIERARCH ESO TPL ID and the HIERARCH ESO DPR keywords. The end of template event acts like closing condition for frame stacks; it occurs when HIERARCH ESO TPL EXPNO exceeds HIERARCH ESO TPL NEXP or when the HIERARCH ESO TPL ID has changed with respect to the previous file. Note that all raw frame of a night can be sorted according to the time stamp (the ARCFILE key). As a consequence any classification system must have knowledge of the classification of the previous frame. After all files of a certain period, e.g. a complete night or many nights have passed a classification scheme the processing stage can be described as follows:
- There is a number of file stacks, each stack is a set of files ready to be submitted to a recipe The last file of the stack triggered the end of template event.
- Each file stack is associated to a reduction recipe
ASSOCIATION: Data association occurs at two placed within the Data Flow Operations in Garchig. First when calling a reduction recipe and second when packing a Service mode data package. The tasks are very similar but not identical Data association requires rules and in this section we describe the association rules used with NACO data.

2. NACO Association Rules

2.1 NACO Association Rules for Imaging and Coronography

triggering template	associated raw/products	match parameter	comments
NACO_img_obs_AutoJitter NACO_img_obs_AutoJitterOffset NACO_img_obs_GenericOffset NACO_img_obs_FixedSkyOffset NACO_coro_obs_Stare NACO_coro_obs_AutoChopNod
	NACO_img_cal_Strehl	OBS.PROG.ID, OBS.ID	two Strehl ratio frames for NAOS
	NACO_img_acq_MoveToPixel	OBS.PROG.ID, OB.ID	CONICA acquisition
	NACO_all_cal_Darks	DET.NCORRS.NAME DET.MODE.NAME DET.DIT	Three keywords are required for unique identification. Note that darks use a weak association in time. If not taken in the same night, take the closest in time.
NACO_img_cal_TwFlats NACO_img_cal_SkyFlats		DET.NCORRS.NAME DET.MODE.NAME INS.OPTI3-6.NAME INS.OPTI7.NAME AOS.INS.DICH.POSNAM	OPTI3-6 is one of the filters, OPTI7 is the objective/camera, AOS.INS.DICH.POSNAM is the dichroic name and NCORRS is again the read mode. Mode is the Gain tag. Twlight flats will be offered in the following three modes: Broad band < 2.5mu in Double_RdRstRd; NB+IB < 2.5mu in FowlerNsamp and Uncorr for > 2.5mu. If other combinations are used in the science frames only the NCORRS will be matched. Note that the twflat recipe produces in addition a badpixel map a error map and a interception map, for which the same association rules apply. Only the objective the filter and the dichroic must match. The detector setting must be known for the lamp flats
NACO_img_cal_LampFlats		DET.NCORRS.NAME DET.MODE.NAME INS.OPTI3-6.NAME INS.OPTI7.NAME	In addition to either twilight flats or skyflats, internal lamp flats are provided in any case. In case a neutral density filter (INS.OPTI3.NAME=ND_Short or ND_Long) is used in addition to BB filters, this key is not matched. The lampflat is matched with the readout mode of the science but not with the DIT of the science. Lampflats do not require dark calibrations in addition.
NACO_img_cal_StandardStar			collect all zeropint measurements in all standard filters (J,H,Ks, and L), but no IB,NB filters. The match is strong in time: only those of the same night are taken.
	NACO_all_cal_Darks	DET.NCORRS.NAME DET.MODE.NAME DET.DIT	Parameters must match those of the Standard star
	NACO_img_cal_TwFlats NACO_img_cal_SkyFlats	INS.OPTI3-6.NAME DET.NCORRS.NAME INS.OPTI7.NAME AOS.INS.DICH.POSNAM DET.MODE.NAME	Parameters must match those of the Standard star
	NACO_all_cal_Darks	DET.NCORRS.NAME DET.MODE.NAME DET.DIT	Parameters must match those of the twilight flat
	NACO_img_cal_LampFlats	INS.OPTI3-6.NAME DET.NCORRS.NAME INS.OPTI7.NAME AOS.INS.DICH.POSNAM DET.MODE.NAME	In addition to either twilight flats or skyflats, internal lamp flats are provided in any case. In case a neutral density filter is used in addition to BB filters for the bright Standard Star, this key is not matched.

Table Notes:

The broad band filters are in INS.OPTI4.NAME and INS.OPTI6.NAME; narrow band filters and intermediate band filters can also be in INS.OPTI5.NAME. For the association only one of the keys has a valid entry (e.g. Ks) while the other keys have values like 'Full' , 'empty' or 'closed'. In any case four keywords have to be evaluated to get the filter.
DET.NCORRS.NAME is one of Double_RdRstRd (= double read-reset-read mode), FowlerNsamp (= multiple Fowler sampling) and Uncorr (= uncorrelated readout mode). For the association these values are abbreviated to FS, UCand DR.
DET.MODE.NAME is one of HighSensitivity, HighDynamic, or HighBackground, and is abbreviated to HS, HD, or HB.
INS.OPTI7.NAME is the iobjective/camera and take one of the follwing values S13, S27, S54, L27, L54.
AOS.INS.DICH.POSNAM is optical path of the NACO dichroics provided for CONICA. It is either VIS, or N20C80.
neutral density filters, if showing up, occur in INS.OPTI3.NAME and can take values ND_Short, or ND_Long. If such a filter shows up it is not used as an additional association parameter. The other three remaining filter wheels 4-6 still contain the normal filter. The filter can occur in data product names (e.g. NC_ICLF_021001A_H+NDS_DRHS_VIS_S13.fits, in case neutral densitiy filters are used in addition they will be tagged as filter+neutraldensity, e.g. H+NDS for H and ND_Short.
Corrupt frames with incomplete fits headers will be ignored, if not rejected in earlier steps of the dataflow

2.3 NACO Association Rules for Polarimetry (Wollaston)

These rules differ only in the handling of the standard star with respect to the imaging mode.

triggering template	associated raw/products	match parameter	comments
NACO_pol_obs_GenericOffset
	NACO_img_cal_Strehl	OBS.PROG.ID, OBS.ID	two Strehl ratio frames for NAOS
	NACO_img_acq_MoveToPixel	OBS.PROG.ID, OB.ID	CONICA acquisition
	NACO_all_cal_Darks	DET.NCORRS.NAME DET.MODE.NAME DET.DIT	Three keywords are required for unique idetification. Noten that darks use a weak assicition in time. If not takern in the same night, take the closest in time.
NACO_img_cal_TwFlats NACO_img_cal_SkyFlats		INS.OPTI3-6.NAME DET.NCORRS.NAME INS.OPTI7.NAME AOS.INS.DICH.POSNAM DET.MODE.NAME	OPTI3-6 is one of the filters, OPTI7 is the objective/camera, AOS is the dichroic name and NCORRS is a again the read mode. Mode is the Gain tag. Twlight flats will be offered in the following three modes: Broad band < 2.5mu in Double_RdRstRd; NB+IB < 2.5mu in FowlerNsamp and Uncorr for > 2.5mu. If other combinations are used in the science frames only the NCORRS will be matched. Note that the twflat recipe produces in addition a badpixel map a error map and a interception map, for which same association rules apply.
NACO_img_cal_LampFlats		INS.OPTI3-6.NAME DET.NCORRS.NAME INS.OPTI7.NAME AOS.INS.DICH.POSNAM DET.MODE.NAME	Neutral density filtersvare not matched.
NACO_pol_cal_LampFlats		INS.OPTI3-6.NAME DET.NCORRS.NAME INS.OPTI7.NAME AOS.INS.DICH.POSNAM DET.MODE.NAME	Internal lamp flats with Wollaston prism are provided in any case. In case a neutral density filter is used in addition to BB filters, this key is not matched.
NACO_pol_cal_StandardStar		OBS.PROG.ID, OBS.ID	Polarimetric Standard star, used to determine instrument polarisation.
	NACO_all_cal_Darks	DET.NCORRS.NAME DET.MODE.NAME DET.DIT	Parameters must match those of the Standard star
	NACO_img_cal_TwFlats NACO_img_cal_SkyFlats	INS.OPTI3-6.NAME DET.NCORRS.NAME INS.OPTI7.NAME AOS.INS.DICH.POSNAM DET.MODE.NAME	Parameters must match those of the Standard star
	NACO_all_cal_Darks	DET.NCORRS.NAME DET.MODE.NAME DET.DIT	Parameters must match those of the twilight flat
	NACO_img_cal_LampFlats	INS.OPTI3-6.NAME DET.NCORRS.NAME INS.OPTI7.NAME AOS.INS.DICH.POSNAM DET.MODE.NAME	In addition to either twilight flats or skyflats, internal lamp flats are provided in any case.

2.3 NACO Association Rules for Spectroscopy

triggering template	associated raw/products	match parameter	comments
NACO_spec_obs_AutoNodOnSlit NACO_spec_obs_GenericOffset
	NACO_img_cal_Strehl	OBS.PROG.ID, OBS.ID	two Strehl ratio frames for NAOS
	NACO_img_acq_MoveToSlit	OBS.PROG.ID, OB.ID	CONICA acquisitioin with a through slit image
	NACO_all_cal_Darks	DET.NCORRS.NAME DET.MODE.NAME DET.DIT	Three keywords are required for unique identification. Note that darks use a weak assocition in time. If not taken in the same night, take the closest in time.
	NACO_spec_cal_Arcs	INS.OPTI1.NAME INS.OPTI4.NAME INS.OPTI6.NAME INS.OPTI7.NAME	OPTI1 is the slit, OPTI4 is the grism, OPTI6 is the filter, and OPTI7 is the objective/camera. Arcs and flats are matched with a weak constraint, meaning they can be taken from other days than the science observation day. Arcs are not taken for the wavelngths lower than 3.0 micron, meaning no Arcs are provided for settings using L54 in SEQ.SPECMODE.
	NACO_spec_cal_LampFlats	INS.OPTI1.NAME INS.OPTI4.NAME INS.OPTI6.NAME INS.OPTI7.NAME	OPTI1 is the slit, OPTI4 is the grism, OPTI6 is the filter, and OPTI7 is the objective/camera
NACO_spec_cal_StandardStar		OBS.PROG.ID, OBS.ID or INS.OPTI1.NAME INS.OPTI4.NAME INS.OPTI6.NAME INS.OPTI7.NAME	User defined standards (standard stars observed under the same RUN.ID) are included, even they do not match with the parameters of the science target
	NACO_all_cal_Darks	DET.NCORRS.NAME DET.MODE.NAME DET.DIT	The parameters must match those of the standard star.
	NACO_spec_cal_Arc	INS.OPTI1.NAME INS.OPTI4.NAME INS.OPTI6.NAME INS.OPTI7.NAME	The parameters must match those of the standard star.
	NACO_spec_cal_LampFlats	INS.OPTI1.NAME INS.OPTI4.NAME INS.OPTI6.NAME INS.OPTI7.NAME	Parameters must match those of the standard star
slit position file

Table Notes:

Only Slit_86mas is offered in P70 which we tag as S86. For slitless spectroscopy mask FLM_13 is used, which is tagged as F13.
Only a limited number of spectroscopic modes is offered, for which the wavelength is a fixed, hence the central wavelength is not a required matching parameter.
The SEQ.SPECMODE keyword is a composition of objective/camera, grism number and filter (e.g. S54_2_SK). This means the two keys SEQ.SPECMODE and INS.OPTI1.NAME (the slit) are sufficient to identify a spectroscopic mode.

Send comments to <Danuta.Dobrzycka@eso.org>
Last update: August 07, 2006