Guide to the SM97 HDA database
Guide to the HDA database and related changes
Return to Requirements Home Page
Introduction
The current HDA database is construed in the following as divided into two parts:
a science part and a instrument part. The science part of the HDA database contains
the tables generally viewed by the user. All of the tables in this part will have
the same kind of entries as the science table. The level of generality of the science
HDA database is driven by the level of generality of the science table. The science
table will continue to have an entry for each "single exposure" that is archived.
(A "single exposure" is one that is not archived as a member of an association. All current
instruments have only single exposures archived.) In the era of associations, the
science table will only have entries for the product or products of associations.
The exposures that are used to make the associations will not be in this table. Such exposures
will not be in the science of the HDA database at all.
The instrument part of the HDA database contains the tables which are less generally
used. It contains the "instrument tables". All of the entries in this part of the
database will be exposures. There will be no products of associations here.
The primary reason for the "falling apart" of the DADS database into two parts is
the introduction of products with STIS and NICMOS. (The fact that STIS and NICMOS
package their products in different ways is of no relevance to this database design.)
With the coming of products we have entities for the archives which have no engineering
information connected with them: products can't go in the instrument database. With
the coming of products we have exposures--the constitutive exposures of the products--which should not be in the science database. These constitutive exposures should be
available to the user but not as prominent as the products.
The instrument part of the HDA will eventually be migrated to a data warehouse. In
examining this proposal, attention should be given to the use of the HDA database
when it is split into two physically distinct databases.
There are many tables in the HDA database which deal with non-science data: engineering
data, house keeping, mission schedules, etc. These tables are not addressed here.
They will not be changing in the redesign of the HDA database.
This proposal must be examined very carefully. Its main goal is to maintain the intelligibility
of the HDA database as we transition into the era of associations. Along with trying
to accomplish that main goal, other changes are suggested that will improve the HDA database. (All such secondary changes are labeled: Secondary change--to
be reviewed.)The main thing to be examined is the basic division between the science
and instrument databases. Does this division work? Will we have to have StarView
screens that make complicated joins between the science and instrument databases?
In the following, we will first present the science database, then the instrument
database, then the joining between the two. The design optimizes working within
the science or instrument databases and provides a way to join between the two of
them. There is a separate section later in this proposal that focuses on how the two parts of
the database are joined and what we can do to facilitate this joining.
It is proposed that the same key be used within the science database, instrument database
and between the two. This key will be called the "pppssooo_key" and is composed of
the following:
program_id (from keyword PROGRMID)
obset_id (from keyword OBSET_ID)
obsnum (from keyword OBSERVTN)
This, in some form, is the primary connecting link among the tables in the science
database and among the tables in the instrument database and between the science
and instrument databases. We cannot join in a general way between the two databases
by use of the dataset name. OMS files have a different name, PDQ files can have a different
name, STIS buried exposures don't have a unique dataset name. We must use the pppssooo_key.
This key will exist in the archive_data_set_all table with each of the relevant tuples.
It will be present for the best version of the data belonging to the following classes:
CAL, AST, OMS, PDQ, ASN. (The association table goes in the ASN class.) The DADS ingest logic will continue to null this value out for worse versions of datasets.
Such datasets will essentially be hidden from view.
The archive_data_set_all table is not considered as part of the science or instrument
database.
The present database will have to be migrated to the new form.
Science Database
The science database has the same granularity throughout as the science table. The
science table has single exposures and the products of associations. Therefore all
of the tables in this part of the database have that level of generality, even if
that level of generality has to be forced. For instance, much of the information for the
several products in a NICMOS association is the same. Little of it is product specific;
it is association specific. This information will be repeated for each NICMOS product that is in the science table. (NICMOS can have several products.) No attempt
will be made to have association level tables that contain information common to
associations, the way the proposal table contains information common to exposures
in the same proposal.
All of the tables in this part of the database only keep one version for an pppssooo_key.
When a "better" one comes along, the tuples corresponding to the "worse" one are
deleted in ALL of the tables in this part and replaced by values from the better
dataset. The instrument database will also be updated when a "better" pppssooo_key comes
along.
Proposal
Notes
This table carries the proposal level information.
The primary index
pro_pep_id -- joins to a field in the science table--and in other tables (OBS and
ADS)
The fields
pro_program_id (first part of pppssooo_key)
pro_pi_first_name
pro_pi_initial
pro_pi_last_name (also in science table)
pro_proposal_type
pro_proposal_title
Science
The primary indices
pppssooo_key
assoc_id
Notes
Stefi wants to add a field for the keyword ORIENTAT to the science table. This field
would be for all instruments. It would not be backfilled for the old instruments.
(Secondary change--to be reviewed)
The Alias's from the target_synonym table may be added to this table. (Secondary
change--to be reviewed)
Any field common across an association may be moved from into this table.
The fields
sci_asn_id (new)
sci_data_set_name (of CAL or AST dataset)
sci_join_field
sci_archive_class (CAL or AST)
sci_pep_expnum
sci_targname
sci_target_descrip
sci_broad_category
sci_ra
sci_dec
sci_ra_v1
sci_dec_v1
sci_expflag
sci_start_time
sci_actual_duration
sci_instrume
sci_instrument_config
sci_operating_mode
sci_aper_1234
sci_spec_1234
sci_spectral_res
sci_spectrum_end
sci_spectrum_start
sci_fgslock
sci_mtflag
sci_pep_id
sci_pi_last_name
sci_release_date
sci_pdq_comment_1
sci_pdq_comment_2
sci_pdq_quality
sci_v3_pos_angle
sci_costar_deploy
sci_aec (used in setting up conflicts table. If "C" ignore)
sci_generation_date
sci_config_id
sci_orientat [new??]
fixed_target
The primary indices
pppssooo_key
Notes
These values are constant across associations. They will be made constant across
products.
The fields
fit_dec_proper_motion
fit_parallax
fit_radial_velocity
fit_ra_proper_motion
fit_redshift
fit_type
moving_target_position
The primary index
pppssooo_key
Notes
These values are constant across associations. They will be made constant across
products.
The fields
mtp_level
mtp_line_number
mtp_spec_text
target_keyword
The primary index
pppssooo_key
Notes
The information in this table is also in the science table. It is loaded into these
two tables from different keywords.
The fields
tak_broad_category
tak_keyword_text
target_synonym
Notes
Should this table be dropped? This information does not seem to be used. It has
not been migrated to the science table. (Secondary change--to be reviewed)
The primary index
pppssooo_key
The fields
tsy_name
scan_parameters
The primary index
pppssooo_key
Notes
Should this table be in the science or the instrument database? Daryl is looking at
this. (Secondary change--to be reviewed)
The fields
scp_angle_between_sides
scp_number_of_lines
scp_points_per_line
scp_position_angle
scp_scan_coord
scp_scan_length
scp_scan_rate
scp_scan_type
scp_scan_width
scp_seconds_per_dwell
reference file tables
The primary index
pppssooo_key
Notes
There will be one of these tables for each instrument.
The information in these tables was part of the instrument tables. With the coming
of associations, this no longer works. We do not need to have the reference file
information for the exposures used to make NICMOS or STIS products. We only need
this information for the products of associations. According these tables belong with the science
table--at that level of generality. The keywords that determine the "mode" vary
from instrument to instrument. These keywords will be taken out of the instrument
table which is their current home. Their values are constant across associations. Also
the reference files are constant across associations.
The tables are primarily used by getref/bestref software. Indeed there will be fields
in this table which are populated by that software. The design of these tables assumes
that users will want the best reference file information only for those exposures/products that are entries in the science table. (If a user does want to get the
best reference files for a NICMOS exposure which is not in this part of the database
because it is an exposure used to make an association, he will be able to get this
information--but not easily: find the pppssooo_key in the instrument database, find the
association it is part of, find any product that is in that association, find that
product in one of these reference file tables.)
The fields
ref_data_set_name
ref_(modes)
ref_(table or file names)
fields for use of best ref and get ref
Instrument Database
This part of the database contains information about all of the exposures actually
take by the spacecraft--and about none of the products made from those exposures
(in NIMCOS and STIS associations.)
The pppssooo_key will be used to join all of the tables in this part of the database.
Generally all of the information in this part of the database will be at the pppssooo_key
level.
We can NOT use dataset name to join the tables in this part of the database. STIS
embedded exposures cannot be uniquely indentified by the use of the dataset name.
(They are all packaged in the same dataset.) OMS dataset names always end in "J"
and cannot be joined with the names of the corresponding CAL or AST datasets.
shp_data
Notes
All association level information will be replicated for each exposure. Association
level information would come from the primary header of the spt file in an association.
The primary indices
pppssooo_key
assn_id
The fields
shp_data_set_name
shp_join_field
etc.
observation (going away)
Notes
All fields in the table will be moved to either the science table or the shp_data
table. See PR.31342 for the details.
instrument tables
Notes
There will be one (or more) of these tables for each instrument.
The instrument table will be different from all others because STIS will give us the
same exposures (wavecals with the same ipppssooo_keys) in different associations.
We will archive such separately given exposures as many times as they are given
to us. We will insert a tupple in the stis_data table each time the exposure is given to us.
The primary key for the stis_data table will have to be the ipppssooo_key PLUS something
like the dataset name or join key.
The primary index
pppssooo_key
The fields
xxx_data_set_name
xxx_join_field
etc.
oms_data
Notes
The association level information connected to this table (about 14 keywords) will
be replicated for each exposure in the association.
The primary indices
pppssooo_key
assoc_id
The fields
oms_data_set_name
oms_join_field
etc.
optical_configuration
Notes
Many of the values in this table are computed, not loaded from values in the headers.
For the new instruments, all of these values will be loaded directly from the headers.
The NICMOS and STIS calibration software will compute them.
This table joins to the other parts of the database by means of the config_id. Currently
there is a config_id in the observation table and in the science table. This design
was used to control the width of the observation/science table. Not at the granularity of the science database or the instrument database.
The primary index
config_id
The fields
The following also shows the sources of the fields.
N indicates that the keyword is present for NICMOS, S indicates it is present for
STIS. A * indicates that it is absent for the instrument.
SHH.APER_1 --------> OPC_APERTURE_1 (N, S)
SHH.APER_2 --------> OPC_APERTURE_2 (N, S)
SHH.APER_3 --------> OPC_APERTURE_3 (N, S)
SHH.APER_4 --------> OPC_APERTURE_4 (N, S)
SHH.CONFIG --------> OPC_INSTRUMENT_CONFIG (N, S)
SHH.OPMODE --------> OPC_OPERATING_MODE (N, S)
SHH.SPEC_1 --------> OPC_SPECTRAL_ELT_1 (N, S)
SHH.SPEC_2 --------> OPC_SPECTRAL_ELT_2 (N, S)
SHH.SPEC_3 --------> OPC_SPECTRAL_ELT_3 (N, S)
SHH.SPEC_4 --------> OPC_SPECTRAL_ELT_4 (N, S)
FOS.C1H/D0H.MAXWAVE --------> OPC_SPECTRUM_END (*, S)
HRS.C1H/D0H/D1H.MAXWAVE --------> OPC_SPECTRUM_END
FOS.C1H/D0H.MINWAVE --------> OPC_SPECTRUM_START (*, S)
HRS.C1H/D0H/D1H.MINWAVE --------> OPC_SPECTRUM_START
INTERNAL: S/W --------> opc_bandwidth (*, *)
INTERNAL: S/W --------> opc_central_wavelength (*, *)
INTERNAL: S/W --------> opc_config_id
INTERNAL: S/W --------> opc_dispersion (*, *)
INTERNAL: S/W --------> opc_optical_range (*, *)
INTERNAL: S/W --------> opc_pixel_res (*, *)
INTERNAL: S/W --------> opc_spectral_res (*, *)
Association tables (between the two databases)
association_member
Notes
Contains information about the members of an associaiton.
The primary index
assoc_id
The fields
archive class (OMS,CAL, ASN, PDQ)
pppssooo_key (or assoc_id for OMS and PDQ data)
member type (product type, exposure type, OMS or PDQ)
dataset name
association_status
Notes
Contains association level information
The primary index
association id
The fields
association status
association id
products present (y/n?)
association generation date
oms_count
pdq_count
association_orphan
Notes
Contains information about orphans.
The primary index
association id
The fields
association member type
association id
pppssooo_key
dataset name
orphan type (straggler/failed product/single exp/incomplete)
class of orphan (OMS or CAL or ASN or PDQ)
Linking the instrument and science databases--the join field
There will be a need to join from the instrument database to the science database
and from the science database to the instrument database. We have elected to do this
by means of a "join field" that will be shared between the important tables in the
science and instrument databases. The value of the "join field" will be same for all the
tuples that are meant to be connected. For single exposures, the value of the join
field will be the pppssooo_key of the exposure. For associations, the value of the
join field will be the pppssooo_key of the primary
product of that association.
Tables containing the join field
science (science database)
shp_data (instrument database)
instrument tables for particular instruments (instrument database)
oms_data (instrument database)
Single exposures
For single exposures, when you join on the join field, you will find the single exposure
in the science database that matches the single exposure in the instrument database.
STIS
For STIS, when you join on the join field, you find the single product in the science
database that corresponds to the multiple exposures in the instrument database.
NICMOS
For NICMOS, when you join on the join field you will find all of the products in the
science database that match all of the constituent exposures in the instrument database.
Introducing redundancy
In addition to the primary keys i n the science and instrument databases, we will
add other, redundant, fields to ease the finding of certain information.
Dataset name
You have to be able to retrieve the data corresponding to an pppssooo_key. We could
only keep this information in the archive_data_set_all table. It is adequate to
have dataset name in only one place. It might require several joins to find the
dataset that goes with an pppssooo_key in the instrument database but it could be done. Or
we could keep it there and also distribute it through out the other tables to make
query construction easier. The only reason for having the dataset name scattered
through out the database is for convenience. There is no compelling logical reason to do
this. (But convenience is important.) Our general game plan, then, will be to put
dataset name in as few other places as necessary. We will not proliferate this without
good reason.
In the science database, we will put the dataset name in the science table. The values
in the science table are derived from TWO different datasets: the PDQ file and the
CAL or AST class dataset. The field will contain the name of the CAL or AST dataset.
In the instrument database, we will put this in oms_data ,shp_data and in the tables
for the particular instruments.
This field will also be in the association_member table.
assoc_id
It is useful to know whether a pppssooo_key in the instrument or science database
is part of or a product of an association. This information can always be found
by searching the association_member table for a match against the pppssooo_key.
When it is deemed convenient we will add the field assoc_id. If it is null, then that pppssooo_key
is not part of an association. If it is not null, it will contain the association
id and can be used to access more information about that item in the association
tables.
This field will be in the science table in the science database and the shp_data and
oms_data table in the instrument database.
Things you can't do with the new database
There are some things that will be difficult or impossible to do with this new design.
This section is an attempt to cull those things out.
- Having different reference files for different exposures or products within an association
The database is designed so that this kind of information cannot even be stored
in it.
- Finding the reference files for an exposure used in making a STIS or NICMOS product:
first find the pppssooo_key of the product from the tuple in the instrument database.
then go to the science database and find the corresponding entry in the appropriate reference file table.
- Finding the exposures that go with the STIS products: first join the science table
with the association_member table on pppssooo_key. this will give you a list of
the pppssooo_keys. you could use any of these keys to find an exposure in the instrument database.
Rearchiving Associations
What is the "best" in the case of associations?
The last character of the name will not be one of the usual ones, but will be a numeral
for all products.
The rule for associations will be that associations always overlay assocations solely
based on DATE. If newer, then assummed better, scrape all the association tables
clean--including the orphan table if appropirate--and repopulate.
The best role here would be to first check to see whether the ingest request is for
a single exposure or for an association and then to do the checking about replacement
after this. It is not a good idea to determine whether a dataset is an association
by looking at the last character or numeral of the dataset root name.
Examples
Single Exposure
the data
pppssooo type of entry
AAA,WW,01 exposure
AAA,WW,01 will be in the instrument database
AAA,WW,01 will be in the science database
Joining the science and instrument tables:
- Can use the dataset name
- Can use the pppssooo key
- Can use the join key
the tables
archive_data_set_all:
dataset_name pppssooo
WAAAWW01T AAA,WW,01
science:
dataset_name pppssooo association_id join_key
WAAAWW01T AAA,WW,01 AAAWW01
instrument:
dataset_name pppssooo association_id join_key
WAAAWW01T AAA,WW,01 AAAWW01
association_member:
No entry
NICMOS Association
the data
pppssooo type of entry
BBB,YY,01 first exposure
BBB,YY,02 second exposure
BBB,YY,03 third exposure
BBB,YY,010 primary product
BBB,YY,011 first non-primary product
BBB,YY,012 second non-primary product
BBB,YY,010, BBB,YY,011 and BBB,YY,012 will be in the science database
BBB,YY,01, BBB,YY,02 and BBB,YY,03 will be in the instrument database
Joining the science and instrument tables:
- Can't use dataset name or pppssooo since products and exposures have different dataset
names and pppssooos.
- Can join from the science table to the association_member table using association
id and then join to the instrument table also using the association id.
- Can use association id directly.
- Can use the join key
Note: In the 3 joining methods above, when joining from science to instrument tables,
all the exposure information belonging to an association will be returned. When
joining from instrument to science tables, all product information belonging to an
association will be returned.
the tables
archive_data_set_all:
dataset_name pppssooo
NBBBYY01T BBB,YY,01
NBBBYY02T BBB,YY,02
NBBBYY03T BBB,YY,03
NBBBYY010 BBB,YY,010 (primary product)
NBBBYY011 BBB,YY,011
NBBBYY012 BBB,YY,012
science:
dataset_name pppssooo association_id join_key
NBBBYY010 BBB,YY,010 NBBBYY010 BBBYY010
NBBBYY011 BBB,YY,011 NBBBYY010 BBBYY010
NBBBYY012 BBB,YY,012 NBBBYY010 BBBYY010
instrument:
dataset_name pppssooo association_id join_key
NBBBYY01T BBB,YY,01 NBBBYY010 BBBYY010
NBBBYY02T BBB,YY,02 NBBBYY010 BBBYY010
NBBBYY03T BBB,YY,03 NBBBYY010 BBBYY010
association_member:
dataset_name pppssooo association_id
NBBBYY010 BBB,YY,010 NBBBYY010
NBBBYY011 BBB,YY,011 NBBBYY010
NBBBYY012 BBB,YY,012 NBBBYY010
NBBBYY01T BBB,YY,01 NBBBYY010
NBBBYY02T BBB,YY,02 NBBBYY010
NBBBYY03T BBB,YY,03 NBBBYY010
STIS Association
the data
pppssooo type of entry
CCC,ZZ,01 first exposure
CCC,ZZ,02 second exposure
CCC,ZZ,03 third exposure
CCC,ZZ,010 product
CCC,ZZ,01 , CCC,ZZ,02 and CCC,ZZ,03 will be in the instrument database
CCC,ZZ,010 will be in the science database
Joining the science and instrument tables:
- Can't use pppssooo since products and exposures have different pppssooos
- Can use dataset name
- Can join from the science table to the association_member table using association
id and then join to the instrument table also using the association id.
- Can use association id directly.
- Can use the join key
Note: In the 3 joining methods above, when joining from science to instrument tables,
all the exposure information belonging to an association will be returned. When
joining from instrument to science tables, all product information belonging to an
association will be returned.
the tables
archive_data_set_all:
dataset_name pppssooo
OCCCZZ010 CCC,ZZ,010
science:
dataset_name pppssooo association_id join_key
OCCCZZ010 CCC,ZZ,010 OCCCZZ010 CCCZZ010
instrument:
dataset_name pppssooo association_id join_key
OCCCZZ010 CCC,ZZ,01 OCCCZZ010 CCCZZ010
OCCCZZ010 CCC,ZZ,02 OCCCZZ010 CCCZZ010
OCCCZZ010 CCC,ZZ,03 OCCCZZ010 CCCZZ010
association_member:
dataset_name pppssooo association_id
OCCCZZ010 CCC,ZZ,010 OCCCZZ010
OCCCZZ010 CCC,ZZ,01 OCCCZZ010
OCCCZZ010 CCC,ZZ,02 OCCCZZ010
OCCCZZ010 CCC,ZZ,03 OCCCZZ010
General Case
the data
pppssooo type of entry
AAA,WW,01 single exposure
BBB,YY,01 NICMOS first exposure
BBB,YY,02 NICMOS second exposure
BBB,YY,03 NICMOS third exposure
BBB,YY,010 NICMOS primary product
BBB,YY,011 NICMOS first non-primary product
BBB,YY,012 NICMOS second non-primary product
CCC,ZZ,01 STIS first exposure
CCC,ZZ,02 STIS second exposure
CCC,ZZ,03 STIS third exposure
CCC,ZZ,010 STIS product
AAA,WW,01, BBB,YY,01, BBB,YY,02, BBB,YY,03, CCC,ZZ,01, CCC,ZZ,02,CCC,ZZ,03
will be in the instrument database
AAA,WW,01, BBB,YY,010, BBB,YY,011, BBB,YY,012, CCC,ZZ,010
will be in the science database
Joining the science and instrument tables:
- Can't use dataset name or pppssooo since products and exposures have different dataset
names/ and pppssooo's.
- Can't join from the science tables to the association_member table using association
id and then join to the instrument tables also using the association id where the
entries are exposures since this does not include single exposure cases
- Can't use the association_id since this does not include the single exposure case
- Can only use the join key
the tables
archive_data_set_all
: association_member:
dataset_name pppssooo dataset_name pppssooo association_id
WAAAWW01T AAA,WW,01 NBBBYY010 BBB,YY,010 NBBBYY010
NBBBYY01T BBB,YY,01 NBBBYY011 BBB,YY,011 NBBBYY010
NBBBYY02T BBB,YY,02 NBBBYY012 BBB,YY,012 NBBBYY010
NBBBYY03T BBB,YY,03 NBBBYY01T BBB,YY,01 NBBBYY010
NBBBYY010 BBB,YY,010 NBBBYY02T BBB,YY,02 NBBBYY010
NBBBYY020 BBB,YY,020 NBBBYY03T BBB,YY,03 NBBBYY010
NBBBYY030 BBB,YY,030 OCCCZZ010 CCC,ZZ,010 OCCCZZ010
OCCCZZ010 CCC,ZZ,010 OCCCZZ010 CCC,ZZ,01 OCCCZZ010
OCCCZZ010 CCC,ZZ,02 OCCCZZ010
science:
dataset_name pppssooo association_id join_key
WAAAWW01T AAA,WW,01 AAAWW01
NBBBYY010 BBB,YY,010 NBBBYY010 BBBYY010
NBBBYY020 BBB,YY,020 NBBBYY010 BBBYY010
NBBBYY030 BBB,YY,030 NBBBYY010 BBBYY010
OCCCZZ010 CCC,ZZ,010 OCCCZZ010 CCCZZ010
instrument:
dataset_name pppssooo association_id join_key
WAAAWW01T AAA,WW,01 AAAWW01
NBBBYY01T BBB,YY,01 NBBBYY010 BBBYY010
NBBBYY02T BBB,YY,02 NBBBYY010 BBBYY010
NBBBYY03T BBB,YY,03 NBBBYY010 BBBYY010
OCCCZZ010 CCC,ZZ,01 OCCCZZ010 CCCZZ010
OCCCZZ010 CCC,ZZ,02 OCCCZZ010 CCCZZ010
OCCCZZ010 CCC,ZZ,03 OCCCZZ010 CCCZZ010
Matrix of Join Possibilities
dataset_name pppssooo association_id join_key
Single Exposure yes yes no yes
NICMOS Assoc no no yes yes
STIS Assoc yes no yes yes
General Join no no no yes
Database Migration Plan
The following steps will have to be taken
- fill in the PDQ entries in archive_data_set_all with pppssooo_key values. They are
not currently filled in at all
Consequences for Auto_PI
Auto PI will have to wait for a single OMS file for complete associations but for
a PDQ file for each product.
Consequences for StarView
The Association ID, the Product Number and dataset names
The FINAL Minutes from the Product number/name Meeting
A meeting was held on Tuesday 2 April 1996 to consider certain issues relating
to the interface between the NICMOS calnicb task and the OPUS and HDA systems.
This follows from an action at the previous pipeline bi-weekly.
Attended by: Stefi Baum, John Baum, Ed Hopkins, Jim Rose, Chris Skinner,
Daryl Swade, and John MacKenty
Issues and resolutions:
1. Does calnicb generate an ASC table or modify the ASN table in place?
Ans: a) Yes, a ASC table is created by calnicb, archived, and distributed as
a product. It will not be opened and used by HDA.
This table will be a member of the dataset of the primary NICMOS
product. Hence it will be a member of the CAL class not the ASN
class. (Note to Ray: this is the only identified NICMOS
calibration "product" that we want to be sure to get in the test
data.)
b) The ASN table will be created by gc/collector (i.e. OPUS prior
to calnicb) with the assumption that calnicb will work correctly.
If a calnicb exception occurs, OPUS will modify the ASN table
as appropriate.
Thus there will be no need for CALNICB to change the ASN table.
2 Where (who) generates the names for NICMOS products?
Ans: Jim Rose when the ASN table is created.
3. Naming convention for products, the association id and product numbers
The proposal of Swade et al to continue to use the ipppssoot (9 char) naming scheme
was accepted with some modifications. This means the earlier SM97 baseline of IPPPSS_nn_TARG_EXT.FITS
format is dropped!
a) naming conventions for products
The new format is:
I PPP SS OO T_EXT.FITS for exposures
(where OO is the observation number and "T" can be a variety of values
depending on the "source" of the data)
and
I PPP SS ID N_EXT.FITS for products
(where ID = ans_id from the qeassociation table--a running
sequence from 01 to ZZ within each obset and N = product type
flag (always 0 for STIS; for NICMOS: 0 = Target pointing. Since
NICMOS can only have one target pointing and 9 backgrounds
and STIS always has only one value, N should never get greater
than "9". For Nicmos, 1 = first offset position, 2=second
offset positions, etc.)
b) Association id
The association ID which is used as the value of the keyword ASN_ID and in many other
ways is NOT the asn_id field from qeassociation. There is a field in qeassociation--asn_name--which
contains what is usually considered the association id. It is always the same as the full root name of the primary product for an association.
Thus in the name above
I PPP SS ID 0_EXT.FITS
I PPP SS ID 0 is the association id.
This is for STIS and NICMOS.
This association id will also be used for:
the value of the ASN_ID keyword
the root name of the primary product
the root name of the OMS association product
the PDQ file root name for associations.
c) Product numbers
There will be PRODUCT numbers which play the same role in the HDA as the observation numbers
for exposures. They are part of the unique key that identifies the product or exposure.
The product number will be the same as the last three characters of the product name. The product number will placed in the existing keyword OBSERVTN and this keyword will be filled in by the time the file is archvied.
(Note for Jim and Chris:
OPUS will fill in the value of OBSERVTN for STIS for products.
CALNICB will fill in the value of OBSERVTN for NICMOS for products.
(OPUS does not see the product header, so CALNICB has
to fill in this value for ALL products when it makes the
headers. CALNICB will get this number by parsing
the product names that are passed to it by OPUS.)
4) Deriving product and exposure names from keyword values
Product names AND exposure names are formed in the exact same way: (using the keyword
names for the pieces in most cases)
Combine the following:
Instrument identifier (single character)
+
PROGRAM_ID (three characters)
+
OBSET_ID (two characters)
+
[(OBSERVTN + transmission source) or
OBSERVTN
] (three characters)
5. Exceptions for calnicb?
Ans: calnicb will make all products expected of it or fail and go to trouble. In
the case of the failure, OPUS will still archive all of the exposures and the association
table (ASN) in the same ingest request. DADS will have the task of breaking them
apart.
6. Two collectors is the working assumption at this time.
7. No new open issues where identified.