SofI Pipeline

This page describes the SofI Pipeline. It is a clone of the ISAAC, adopted to fit the SofI features, i.e. crosstalk, orientation of the slit, etc. Contributions and suggestions are encouraged.

What does the pipeline do?

Quick-look on-line data reduction. This means that the pipeline automatically classifies every file produced by SofI, and reduces it according to pre-defined rules, with one of the specific SofI recipes. The recipes are specialized data reduction tools, similar to the IRAF task but more comprehensive. They are written in C. The SofI pipeline recipes are available upon request starting the beginning of P78.
The data product is transferred to the off-line workstation wg5off where the user can review the file. Note: to view the newly arrived files on wg5off with Gasgano, one has to "refresh" the content of the Gasgano file list.
All file transfers are transparent to the user.

Purpose of the pipeline

  • To provide reasonably well reduced data that can be reviewd by the observer in nearly real time to facilitate making decisions about the quality of the acquired data.
  • To provide data quality control parameters measured from the data itself, such as: zero points, central wavelength for the given spectroscopic setup, readout noise, gain, illumination level for the flats, dark/bias level, etc. These parameters are used for both data quality control and for instrument health check.

What type of data does the pipeline reduce?

As a rule of thumb, the pipeline fares well on good, high signal-to-noise data and tends to fail on poor quality data. The parameters of the recipes are tuned to encompass as larger variety of scientific observations as possible. Of course, it is not possible to address all observing strategies, so do not expect the pipeline to reduce your complicated imaging program that alternates between four objects and a sky field. The pipeline is likely to fail if you do spectroscopy of faint objects, or objects with weak continuum, or a galaxy without sharp central peak. Polarimetry is not supported.
Most likely the pipeline will reduce AutoJitter-taken imaging and spectroscopy of reasonably bright stars obtained with nodding along the slit.
Consult your support astronomer to determine whether you need the pipeline products, what are the chances of getting good reduction or any other pipeline related issues and all imaging and spectroscopic daytime calibrations.

What recipes are available?

Here is a short description of the recipes, with some data quality parameters that they calculate:

  • sofi_img_dark - combines dark frames, calculates bias/dark level, RoN and gain
  • sofi_img_domeflat - reduces SofI imaging special dome flats, calculates the illumination level
  • sofi_img_illum - imaging illumination correction
  • sofi_img_jitter - reduces imaging taken with jittering templates, calculates the sky level
  • sofi_img_zpoint - calculates the photometric zero points for observations of standard stars produced with the dedicated standard star template (if other imaging templates are used the recipe will ignore the standards and treat them as any science imaging data)
  • sofi_spc_arc - wavl. calibrations for arcs, calculates the central wavl. for the given setting
  • sofi_spc_flat - combines the spectrosocpic flats, calculates the illuminaiton level
  • sofi_spc_jitter - combines 2-dimensional spectra taken with nodding along the slit after geometic correction for the distortions, extracts wavl. calibrated 1-dimensional spectrum for the brightest point source in the field

Where can I get a detailed description of the recipes? -- Use the ISAAC pipeline page, which can be found here and here. There are some differences such as the treatment of the crosstalk and the current release of the SofI pipeline still is lacking certain recipes. Ask the SofI instrument scientist for details.

How to review the pipeline products?

The easiest way is to use Gasgano on the astro@wg5off. The raw data are in sub-dir /data/raw/2006-04-19 or similar, and the reduced data are in sub-dir /data/reduced/2006-06-24 or similar. The quality control parameters, incl. the zero points are written in *.tfits or *.paf files. The names of the reduced files starts with "r.*". Remmeber to "refresh" the Gasgano content to see the latest data products.

How to get the pipeline products? -- The pipeline products are not part of the visitor's back up. The users can ftp the reduced data from wg5off or use the CD/DVD burner on that machine to back them up by themselves. Talk to your support astronomer about this.