9 Data in the HST Archive

Overview / 213
Science Data Files / 214
Archive Classes / 220

Overview

According to the above prescription, the files stored in the archive all have unique file names, the exception being that recalibrated and reprocessed observations may generate files with names identical to those produced by the original calibration or processing. Such files are distinguished by their generation dates. Typically, only archive operators at the Space Telescope Science Institute (STScI) will be concerned about these different versions. Archive researchers should use only the most recent versions of the files, which is the default for retrievals.

In the sections below, we describe the conventions used to name the different data files and archive classes in the HST archive.

File Names

Most of the files that you will encounter in the archive are FITS files containing the science data for an observation. Science data files are described in the next section. There are also non-science data consisting of unidentified science data files, dump data files, engineering data, etc. File naming conventions for these non-science file types are not of interest to the typical archive user. These can be found on the WWW at http://sesd.stsci.edu/~hopkins/non-science-data.htm.

Science Data Files

At a minimum, for each observation the OPUS pipeline produces a standard header packet file, a science data file, and its corresponding data quality file; and stores appropriate engineering data and ancillary information in the unique data log file (UDL).¹ The standard header packet (SHP) contains telemetry data, spacecraft operation data, and instrument-specific data. The UDL contains the command values for each instrument, such as exposure parameters and aperture commands. For the STIS and NICMOS instruments, both the SHP and UDL information are contained in the SPT file. The science data files contain the observed image stored as a FITS file. The number of science data files, the size of those files, and the number of dimensions vary across instruments and observing modes. Each science data file has an accompanying data quality file to identify bad data values in the science data. In addition to these files, additional files are typically produced by the calibration software in the OPUS pipeline.

The dataset root name consists of 9 characters of the form

IPPPSSOOT

The IPPPSSOOT file naming convention is applied by OPUS and is defined in Table 9.1. We note that this convention applies only to science data files (ASN, AST, CAL, and PDQ classes: see Table 9.13) and observation log (OMS) files.

Character	Meaning
I	Instrument Used, will be one of: F - Fine Guidance Sensors J - Advanced Camera for Surveys N - Near Infrared Camera Multi Object Spectrograph O- Space Telescope Imaging Spectrograph U - Wide Field Planetary Camera-2 V - High Speed Photometer W - Wide Field Planetary Camera X - Faint Object Camera Y - Faint Object Spectrograph Z -High Resolution Spectrograph
PPP	Program ID; can be any combination of letters or numbers (46,656 combinations possible)
SS	Observation set ID; any combination of letters or numbers (1,296 possible combinations)
OO	Observation ID; any combination of letters or numbers (1,296 possible combinations)
T	Source of transmission (RSDP environment) J- OMS data R- Real time (not recorded) T - Tape recorded M - Merged real time and tape recorded N - Retransmitted merged real time and tape recorded O - Retransmitted real time P - Retransmitted tape recorded 0 - (zero) Primary association product (NICMOS/STIS/ACS) 1-8 NICMOS background association products

Table 9.1: IPPPSSOOT Root File Names

Common Extensions for Instrument Data Files

Extension	File Type
_a1f.fits	FGS1 group data file
_a2f.fits	FGS2 group data file
_a3f.fits	FGS3 group data file
_dmf.fits	Science data and proposal information header file. Also, database indices for astrometry and engineering data

Table 9.2: Extensions for FGS Astrometry Data Files

Extension	File Type
_d0f.fits	Uncalibrated data file
_shf.fits	Standard header packet
_q0f.fits	Data quality file1
_ulf.fits	Unique data log
*.trl	Trailer file

1 _q*f.fits for FOS.

Table 9.3: Extensions for FOC, FOS, GHRS, WFPC,
and WFPC-2 Raw Data Files

Extension	File Type
_c0f.fits	Calibrated science image file
_x0f.fits	Extracted engineering file
_q1f.fits	Data quality for extracted engineering file
_c1f.fits	Data quality for calibrated science image
_c2f.fits	Histogram of science image pixel values
_c3f.fits	Saturated and missing pixel map (WFPC)

Table 9.4: Extensions for WFPC and WFPC-2 Calibrated
Data Files

Extension	File Type
_c0f.fits	Dezoomed, geometrically corrected data, with photometry
_c1f.fits	All of the above, plus flatfielded data

Table 9.5: Extensions for FOC Calibrated Data Files

Extension	File Type
_c0f.fits	Calibrated wavelengths
_c1f.fits	Calibrated fluxes
_cqf.fits	Wavelength/flux data quality
_x0f.fits	Extracted data
_xqf.fits	Extracted data quality
_c2f.fits	Propagated statistical errors
_c3f.fits	Calibrated special diodes
_c4f.fits	Special diodes data quality
_c5f.fits	Background
_d1f.fits	Return-to-Brightest acquisition fluxes
_q1f.fits	Return-to-Brightest data quality

Table 9.6: Extensions for GHRS Calibrated Data Files

Extension	File Type
_raw.fits	Raw science data
_spt.fits	File containing SHP and UDL information
_asn.fits	Association table

Table 9.7: Extensions for NICMOS Raw Data Files

Extension	File Type
_cal.fits	Calibrated science data
_ima.fits	Intermediate multiaccum calibrated science data
_mos.fits	Mosaiced target or background images
_asc.fits	Post-calibration association table
_trl.fits	Trailer file
_pdq.fits	Processing Data Quality File

Table 9.8: Extensions for NICMOS Calibrated Data Files

Extension	File Type
_raw.fits	Raw science data
_tag.fits	Timetag event list
_spt.fits	Support file (planning and telemetry information)
_wav.fits	Associate wavecal exposure
_wsp.fits	Wavecal support file (planning and telemetry information)
_asn.fits	Association table
_trl.fits	Trailer file (input)
_lrc.fits	Local rate check image
_lsp.fits	Local rate check support file
_jit.fits	OMS file
_jif.fits	OMS file
_pdq.fits	Processing Data Quality File

Table 9.9: Extensions for STIS Raw Data Files

Extension	File Type
_flt.fits	Flatfielded science data
_crj.fits	Cosmic ray-rejected, flatfielded science data
_sfl.fits	Association table
_x1d.fits	1-D extracted spectra
_x2d.fits	2-D extracted data
_sx1.fits	Summed 1-D extracted spectra
_sx2.fits	Summed 2-D extracted spectra
_trl.fits	Trailer file (output)

Table 9.10: Extensions for STIS Calibrated Data Files

Extension	File Type
_c0f.fits	Calibrated wavelengths
_c1f.fits	Calibrated fluxes
_x0f.fits	Science header line
_xqf.fits	Science header line data quality
_d1f.fits	Science trailer line
_q1f.fits	Science trailer line data quality
_cqf.fits	Calibrated science data quality
_c2f.fits	Propagated statistical errors
_c3f.fits	Special statistics
_c4f.fits	Count rate
_c5f.fits	Flat fielded object spectra
_c6f.fits	Flat fielded sky spectra
_c7f.fits	Background spectra
_c8f.fits	Flatfielded object minus smoothed sky spectra

Table 9.11: Extensions for FOS Calibrated Data Files

Extension	File Type
_d0f.fits	Science data: digital star
_d1f.fits	Science data: digital sky
_d2f.fits	Science data: analog star
_d3f.fits	Science data: analog sky
_q0f.fits	Data quality: digital star
_q1f.fits	Data quality: digital sky
_q2f.fits	Data quality: analog star
_q3f.fits	Data quality: analog sky
_c0f.fits	Calibrated science data: digital star
_c1f.fits	Calibrated science data: digital sky
_c2f.fits	Calibrated science data: analog star
_c3f.fits	Calibrated science data: analog sky

Table 9.12: Extensions for HSP Calibrated Data Files

Archive Classes

Class	Contents
ACC	STIS MAMA Accumulated Image Data
ANC	Ancillary data; includes POD files (raw science data)
ASN	Association table (NICMOS, STIS, ACS)
AST	Astrometry data from the Fine Guidance Sensor (FGS)
CAL	Calibrated and uncalibrated science files
CDB	Calibration reference files
CTB	Calibration reference tables
DIA	Engineering NICMOS, STIS and ACS diagnostic FITS memory dumps
DLG	DADS log files
DMP	On-board computer memory dumps (HSP, FOC, FOS)
EDT	Raw edited versions of science files, used by the Post Observation Data Processing System (OPUS) to generate CAL files
ENG	Reconstructed engineering telemetry data
MSC	Mission schedules
MTL	Mission Timeline Data
OCX	Observer comment files
OMS	Observation log files (including HST jitter motion)
ORB	Definitive ephemeris files for HST orbit
PDQ	OPUS data quality comment files
PRB	Problem science data
SMS	Science mission specification files (the command files used to control the observations)
SUB	Engineering subset files
VLA	VLA FIRST data (Faint Images of Radio Sky at 20 cm)

Table 9.13: Archive Classes