WFPC2 Associations Maintenance

The WFPC2 Associations Maintenance consits of two sets of procedures:

1. the Jitter Extraction Pipeline (JEP)
2. and the Association Builder (AB)

Once in a while, after having received a new bunch of CD-ROMs from STScI, the archive operator submits an archive request to run the WFPC2 Associations Maintenance.

The submission takes place via a script that:

• finds out (from the dads_received table) which are the jitter files to be retrieved from the ECF juke-box (

  select dataset_name+".[CJ][IM]__"+gen_date_str+"_OMS%"
  from dads_received
  where curr="Y" and (jitter = " " or jitter is null)
  and cdrom = "Y"
  and dataset_name like "U%J" 
  and archive_class = "OMS"
  

  ).
• and actually issues an archive request with a proper set of actions.

The ECF Request Handler picks up such request:

1. rh_get retrieves the necessary jitter files (with the -O option to leave unchanged the file names preserving the gen_date_str in the file name)

2. rh_proc runs (on each individual jitter pair) the jitter extraction script, which:
   • Extracts necessary keywords from jitter headers;
   • Computes the averages and standard deviations for the Right Ascension, Declination and Roll angle
   • Stores all the extracted parameters and the gen_date_str of the given jitter file in the jit_member table
   • Changes the jitter field in the dads_received table from " " to "Y"
     (for archive_class = OMS)

3. rh_proc runs the script (overall action) which computes the associations (Association Builder).
   

What is triggering the (re)computation of an association ?

• if a new jitter file is available,
  
• if a new version of a jitter file is available,
  
• if a CAL dataset gets rearchived,
  
• if there is a change in one of the descriptors of the association (e.g., if the exposure time or the Filter of an exposure gets updated in the hstdads..science table)
  

The association builder script will:

• discard all the associations (and their members) affected by any of the above-mentioned cases;
• (re)build associations for all the pep_ids involved

Which datasets have to be picked up by the association builder ?

dads_received before jitter curr OMS " " "Y" CAL Anything "Y"	dads_received before jitter curr OMS "Y" "Y" CAL " " "Y"	dads_received before jitter curr OMS "Y" "Y" CAL Anything "Y"
JEP	SQL	CHDB
dads_received after jitter curr OMS "Y" "Y" CAL "R" "Y"	dads_received after jitter curr OMS "Y" "Y" CAL "R" "Y"	dads_received after jitter curr OMS "Y" "Y" CAL "R" "Y"

Jitter field value "R" for the archive_class=CAL means (Re)build the association for this dataset.

(Re)building Associations

More precisely, due to the fact that an association is computed for all the datasets belonging to the same proposal, the association building procedure must run on all the pep_ids which those "R" datasets belong to.

Generation Dates

The gen_date_str (and the respective gen_date) of the CAL entry in dads_received is stored in the hst_association_members table.

The gen_date_str (and the respective gen_date) field computed as the maximum of all the gen_date_str of the members is stored in the hst_associations table.

What triggers the Preview pipeline ?

The field gen_date_str (in dads_received) gets populated by the hstscience program by looking at the gen_date field of the hst_associations table.

The following table shows for each of the maintenance drivers:

• what are the maintenance scripts to be run,
• when the gen_date field gets changed in the hst_associations table
  (but also in the other tables).

When is gen_date changing ?

Driver:	OMS New		OMS Re-Archived		CAL Rearchived	Change in the DB
Maintenance script:	JEP+AB				SQL	CHDB
Possible cases:	CAL New	CAL Unchanged	CAL New	CAL Unchanged	OMS any	CAL Re-Archived	CAL Unchanged
jit_member gen_date:	New	No sense	New	New	any	Already taken in consideration by SQL	any
hst_association_members gen_date:	New	No sense	New	Unchanged	New		Unchanged
hst_associations gen_date:	New	No sense	New	Unchanged	New		Unchanged
Preview Pipeline:	Run!	-	Run!	!Re Run!	Re Run!	-	No *

* Only the description of the association changes, not the data, nor the shifts

The gen_date in the hst_associations table is not able to trigger the execution of the Preview pipeline when a jitter file gets re-archived!

Solution

hstscience will need to look at this new field!

Is this acceptable for you, CADC ???