30 August 2004
Survey ..................... XMM
Telescope .................. ESO/MPG 2.2m
Instrument ................. WFI
Program IDs ................ 170.A-0789; 70.A-0529; 71.A-0110
Origin ..................... ESO/EIS
Number of regions .......... 12
Region ..................... Selected XMM Serendipitous Survey fields
Number of Fields ........... 12
Passbands .................. B; V; R; I
EIS Release Number ......... 19
Version .................... 0.5
Total Data Volume .......... 79.9 Gb
Release Date ............... August 2004
Release prepared by ........ EIS/PSG team, J. Dietrich, A. Schwope
Product Type ............... Nightly Products
Number of Reduced Images ... 146
Data Volume ................ 79.9 Gb
PRODUCTS IN PREVIOUS RELEASE
Based on ideas submitted by the ESO community and evaluated by ESO's Survey Working Group (SWG), the XMM survey (administered by the ESO Imaging Survey (EIS) and the Public Survey Group (PSG) teams) consists of follow-up optical observations of the XMM-Newton Serendipitous Sky Survey (XSSS) using the ESO Wide Field Imager (WFI) at the ESO/MPG 2.2m telescope at the La Silla Observatory. This survey seeks to obtain WFI images in B-, V-, R- and I- passbands (for minimum spectral discrimination and photometric redshift determination) of XMM fields publicly available in the XMM archive. The fields were selected and prioritized by a collaboration of interested parties from the XMM Survey Science Centre (SSC), the Bonn group (led by P. Schneider) and an appointed committee of the SWG. The observations were originally proposed to cover a total area of approximately 10 square degrees (40 fields) to a limiting magnitude of 25 (AB, 5 sigma, 2 arcsec aperture, see original proposal presented by the SWG to the OPC).
Data for the XMM survey are being provided by:
The present release consists of data from the above programs accumulated until October 16, 2003, corresponding to a total of roughly 130 and 720 science OBs and exposures, respectively, and 80 hours on-target integration. About 15% of the B-band and 85% of the R-band data are from the contributing programs.
With the present release 83% of the data accumulated as part of the Public Survey over 15 fields (roughly 3.75 square degrees) have been reduced. On the other hand, the observations for the XMM survey correspond, at the time of writing, to less than 35% of the original goal of 40 fields.
The 720 raw exposures were converted into 160 fully calibrated reduced images, of which 146 are being released. Of the remaining 14, 10 were observed with wrong coordinates, three (XMM-05 (R), XMM-06 (I), XMM-12 (V)) were rejected after visual inspection and one (XMM-12) was discarded due to a very short integration time (73 sec), associated to a failed OB. From these numbers one finds that the volume of data can be reduced by a factor of at least 5 by converting raw into nightly products and by 16 when final products (stacks) are produced. This information is of interest when dealing with data gathered by wide-field imagers.
The accuracy of the astrometric calibration is estimated to be better than 0.2 arcsec. The photometric solutions obtained by the automatic photometric pipeline, a sub-system of the EIS Survey System, are in excellent agreement with the one computed by the WFI team at La Silla. The estimated error of the absolute calibration, based on the scatter of the available solutions, ranges from 0.05 to 0.08 mag, with passbands B and R having the largest errors. However, these values are still poorly determined due to the relatively small number of nights available. Other effects, associated with a multi-chip detectors, may also impact the error budget. Improvements will require more extensive tests. There are also three anomolous night solutions in three distinct passbands (V, R, I), with the most significant (> 3 sigma) being in I (0.5 mag), with the R and I likely to be affected by fringing.
For more information about the terminology and conventions used in this document refer to the WEB README pages.
This is the first official release of reduced data for the XMM-Newton follow-up survey. The data set being released was accumulated in service mode during ESO observing periods 70 and 71, and observations are still on-going.
The present release consists of 146 fully calibrated ESO/MPG 2.2m WFI images in B- (36), V- (32), R- (43) and I- (35) passbands for the 12 fields listed below at high (8; XMM-03 through XMM-10) and low (4; XMM-01, XMM-02, XMM-11 and XMM-12) galactic latitude. The table below gives: in column (1) a sequence number; in column (2) the EIS field name; in column (3) the name normally used to identify the field, and in columns (4)-(7) the number of fully calibrated images (and in parenthesis the number of independent nights on which they were observed) being released in each of the passbands.
|#||EIS Field Name||Other||B||V||R||I|
|1||XMM-01||RX J0925.7-4758||3 (1)||3 (2)||3 (1)||3 (3)|
|2||XMM-02||RX J0720.4-3125||3 (1)||3 (1)||3 (1)||3 (1)|
|3||XMM-03||HE 1104-1805||3 (1)||3 (2)||5 (3)||3 (1)|
|4||XMM-04||MS1054.4-0321||3 (1)||3 (2)||4 (2)||3 (2)|
|5||XMM-05||BPM 16274||3 (1)||3 (2)||5 (1)||3 (2)|
|6||XMM-06||RX J0505.3-2849||3 (1)||3 (2)||6 (4)||3 (2)|
|7||XMM-07||LBQS 2212-1759||3 (2)||3 (2)||6 (4)||3 (2)|
|8||XMM-08||NGC 4666||3 (1)||3 (1)||3 (2)|
|9||XMM-09||QSO B1246-057||3 (1)||3 (2)||3 (2)|
|10||XMM-10||PB 5062||3 (1)||5 (3)|
|11||XMM-11||Sgr A||3 (1)||3 (2)||3 (1)||5 (4)|
|12||XMM-12||WR 46||3 (1)||2 (1)||3 (1)||3 (2)|
Official EIS products can be retrieved via two alternate routes, both originating at the EIS home page. These procedures are described elsewhere (see Retrieving EIS Products). It is worth reiterating that to request data users have to be registered with the ESO Science Archive.
In the case of the present release (involving reduced images only) the "data release information section" is followed by a section detailing the "contents" of the release, listing:
For each survey product the EIS system prepares an extensive ``product'' log from which the process log and the configuration file used can be easily accessed and inspected. This infrastructure provides the means of ensuring the uniqueness of the ingested products. It also provides the information required to understand differences between versions of the same product.
Typically, the product logs consist of seven distinct sections represented in the rendering of the HTML as separate tabs. These sections consist of:
For the time being, a preliminary implementation of this infrastructure is available for the single chip instruments. Work is under way to upgrade and generalize this infrastructure to support all instruments.
In the upper right corner of the data release WEB page there are two buttons (completeness and properties) which link to plots showing the completeness of the dataset being released relative to the original strategy. It also presents different visualizations of the attributes associated with the images being released (for details see Data Quality Assessment section below).
Since the infrastructure is still under development, currently plots are being produced without adequate description. It is foreseen that these plots will be embedded into HTML files providing captions and statistics.
Since WFI was first offered as a common user instrument at the ESO/MPG 2.2m telescope in 1999, the available filter set and naming conventions have evolved somewhat. To avoid confusions it is instructive to state explicitly the filter set used when conducting the present survey. The current La Silla Science Operations naming convention assigns all filters which are permanently available at the WFI an ESO identity number. If the filter does not have a special name (U, B, V etc.) then the assigned name is "CWL/FWHM" where CWL is the truncated central wavelength (nm) and FWHM is the truncated full-width at half-maximum (nm). If the filter has a special name, the FWHM of the filter (in nm) is appended to the special name. A full description of the filter names can be found at the WFI Filter Characteristics page of the La Silla Science Operations.
The present survey makes use of the following WFI filters:
|Passband||Filter Name||ESO identity number|
Due to the on-going nature of the survey, the passband range, depth and completeness of the products in the present release is not homogeneous. In fact, the completeness of each field may vary, as some fields have not yet been completely observed. The is shown in the completeness plot accessible from the release WEB page. This figure shows for each field two histograms of time versus passband, the latter represented by the ESO filter name convention. The hatched histogram represents the total integration time required by the original strategy (in seconds). The full colored histograms represents the sum of the integration time (TOT-EXPT keyword in the header) of all reduced images in a particular filter. The color code adopted for the filters (passbands) here and elsewhere is as follows: B (blue); V (green); R (red), I (magenta). Note that in some cases the R-band greatly exceeds the originally proposed integration time. This is because of the contributing programs targeting the high-galactic latitude fields.
As explained elsewhere (see EIS Definitions and Conventions) the conventions adopted by EIS to identify a pointing both internally and externally (e.g. OB name, object name, target name, field name) and their graphical representations on the EIS WEB pages have evolved considerably as new procedures were introduced, making it impossible to strictly maintain consistency.
Field names can normally be found in the OBJECT keyword of the FITS header. Since they are, normally, inherited from ``Observation Blocks'' (OBs) which have been prepared by multiple team members, the naming convention may not be homogeneous, and thus it is recommended that the value of the OBJECT keyword should not be, in general, used to rename files.
In the particular case of this survey one should be warned that the contents of the OBJECT keyword do not necessarily correspond to the name convention listed above (e.g. images with OBJECT=SHARC-2 do not correspond to the second listed cluster of the SHARC collaboration but it is instead an internal convention adopted by the Bonn group).
This release consists of ``reduced'' images, thus for given field and filter there may be more than one image product either from the same night (resulting from differently grouped raw images or several similar OBs) or from different nights.
This release represents a first attempt to photometrically calibrate WFI data using the EIS Survey System. A proper calibration involves correction for CCD-to-CCD gain variations, illumination correction and absolute zeropoint determination. For the current release no illumination correction has been applied and the verification of the zeropoint determination has been compared with values provided by the 2p2 telescope team. On the other hand, relative photometry and astrometry have been compared to results obtained by the Bonn group (Schneider et al. ) using independent software.
The WFI data were reduced using the EIS/MVM image processing library. This software package is publicly available and can be retrieved starting from the EIS survey release page.
Each field is the co-addition of a number of sky-subtracted frames (the TOT_IMAG keyword in FITS image header) grouped in a number of "Reduction Blocks" (RB). The basic reduction methodology is described in Vandame (2004). Significant improvements have been made among which:
The astrometric calibration was derived using the GSC2.2 reference catalog and a distortion model described by a polynomial of second order. Comparisons yield typical rms scatter in astrometry of 0.2 arcseconds (Vandame 2004). The WCS in the image header is reported using the CD-matrix notation. The projection adopted is TAN and the orientation is north up and east to the left.
This release is the first for which photometrically calibrated WFI images are being provided. It is important to stress that the photometric pipeline being used to automatically extract catalogs, measure fluxes at different apertures, identify those corresponding to known standard stars, and derive from linear fits to these measurements zeropoints, extinction and color terms (if applicable) was originally developed to deal with single chip imagers - only recently systematic tests have been conducted for multi-chip cameras. A number of improvements to the code and more experimentation with the many configurable parameters involved are still possible. Therefore, the solutions presented in this 0.5 release should not be considered final.
In accordance with the established EIS standard procedures, photometric calibration of the science fields is performed on the Vega magnitude system. Zeropoints are obtained from linear fits to flux measurements of standard stars in Landolt (1992) fields taken at the same night as the science frames. These fits were obtained using from one (taking some specified values for the extinction and color term) to three free parameters depending on the available airmass and color coverage provided by the Telescope Team calibration plan (since the XMM survey observations were conducted in service mode).
For the present release photometric solutions were attempted for the 37 nights with observation of standard fields, of 41 with science observations. The four nights without standards are: February 2, 3 and 4, 2003 (Public Survey); and November 8, 2002 (contributing program)
A total of roughly 1000 standard OBs, nearly all consisting of one image per OB, have been reduced and measured. While this amounts to slightly more than 7 hours on-target (or 10% of the science observations in terms of time), it exceeds the volume of science frames. Depending on the night the number of measurements can range from a few to over 300, covering from 1 to 3 Landolt fields. As mentioned above, the period of observations extend over one year.
In practice, the photometric pipeline computes photometric parameters for all possible types of fits (one to three-parameters) and assigns the solution with the smallest zeropoint scatter, to be the ``best solution'' for the night. A night is considered photometric if this scatter is less than a pre-defined value, which at the present time is taken to be below 0.1 mag. If none of the solutions satisfy this criteria and/or the solution found yields unrealistic results (e.g. negative extinction) then the night is considered non-photometric and a default value for the zeropoint is adopted and its error set to -1.
For homogeneity, the default value normally adopted is the median of the zeropoints reported in the trend analysis kept by either the telescope team (depending on the instrument) or the internal EIS database. In the case of WFI, only one solution is currently reported in the WFI WEB pages. It is important to emphasize that EIS has and still is conducting several surveys (e.g. Pilot, DPS, Pre-Flames, XMM and Galex) using WFI, covering its entire operational lifetime. Therefore, in the future a trend analysis of the photometry will become naturally available.
The table below summarizes the number of nights with standard observations and type of ``best solution'' obtained by the photometric pipeline for each passband.
For three nights (March 26, 2003; April 2, 2003; August 6, 2003) the solutions obtained in the passbands V, I, R, respectively (either 2- or 3-parameter fits) deviate from the median by -0.26, -0.5, -0.25 mag. Of those only the I-band zeropoint obtained for April 2, 2003 deviates by more than 3 sigma.
The preparation of this release has shown the need for a number of improvements to be made to the photometric pipeline. Among those either under development or being considered are:
It is important to recall that wide-field multi-chip cameras are complex systems and to obtain a proper calibration depends upon a full understanding of the different effects outlined above. This requires extensive tests which can only be effectively done using automatic procedures as well as versioning infrastructure similar to the ones available in the EIS survey system. In order to carry out these tests efficiently and in a systematic way, a re-factoring of the code, and the implementation of suitable workflows and test-environment are currently underway. This study will be of great importance for defining calibration procedures to be adopted not only for WFI but for other wide-field cameras to be used in large imaging surveys
Image Product Calibration
These photometric solutions were used to calibrate the reduced images. The ZP in the header of a reduced image is given by
ZP = ZP' + KX
where ZP' is the zeropoint at zero airmass (determined from the linear fit), K is the extinction coefficient and X is the observed airmass. Therefore, the Vega magnitude is given by
mag (Vega) = -2.5*log(flux) + ZP
where the flux is the number of counts directly measurable on the images (note that the reduced images are normalized to 1 sec).
In the case of a non-photometric night, a default value for the zeropoint is adopted and the error in zeropoint is set to -1. For nights without observation of fields containing standard stars, a default zeropoint and an error of -2 are assigned to the image. Finally, during quality assessment of the data, calibrated images with zeropoints that deviate significantly from a reference value have the zeropoint in the header changed to a default value and its error set to -3. Note that when a default value is assigned all images will have the same zeropoint in the header regardless of the airmass at which they were observed. It is important to stress that the history of the image calibration process can be retrieved independently of the solutions found for a night.
In the case of stacks, the actual value adopted for the default zeropoint of the reduced images is irrelevant, since those images with default zeropoints have to be re-scaled anyway to match the flux-scale of those images properly calibrated. The final photometric accuracy, however, will depend on the type of solutions and number of independent nights contributing to the final stack.
As a preview, the table below shows for each pointing the best type of solution available for each passband.
|#||EIS Field Name||Default||1-par||2-par||3-par|
The following remarks can be made concerning the calibration of the image products being released:
These results imply that currently the stacks for field/filter combination of XMM-01 (R), XMM-02 (R) and XMM-02 (I) cannot be properly calibrated.
For detectors formed by a mosaic of individual CCDs, in order to correct for CCD-to-CCD gain variations, median background values sampled in sub-regions bordering adjacent CCDs are used to harmonize the gain to a value common for all CCDs in the mosaic. This has been applied to both science and standard exposures.
It is also known that large-scale variations due to non-uniform illumination over the field of view of wide-field instruments exist. The significance of this effect is passband-dependent and becomes more pronounced with increasing distance from the optical axis (Manfroid et al. 2001; Koch et al. 2004; Vandame et al. 2004). Automated software to correct for this effect has been developed but due to time constraints it has not yet been applied to these data.
In order to optimize the removal of cosmic rays using a sigma-clipping algorithm, the original Observation Block (OB) has been split into at least three reduction blocks (RB). Tests have shown that by so doing there is a significant decrease of cosmic rays in the general region of inter-chip gaps for the final stacked image, which will become available in a forthcoming release.
It is important to note that although not forming part of this release the methodology adopted for cosmic ray rejection had as its main goal the optimization of the quality of final ``stacked'' images. Therefore, the reduced images (which are in reality intermediate products) being released may still have a high frequency of cosmic rays which do not reflect the quality of the final ``stacked'' image. From visual inspection of the images, the occurrence of cosmic rays is higher in B- and V-bands, being of the order of 2-3 cosmic rays/arcmin square. This is because, typically the total number of exposures in these bands is smaller, thus affecting the cosmic rays rejection algorithm being used.
The performance of the automatic satellite track masking algorithm (using the Hough transform) has been proven to be efficient in removing both bright and faint tracks. The frequency for the appearance of these tracks is about 0.9 satellite-tracks/hour, of which only 1/10 are of the bright type. The most extreme case is 3 satellite tracks of varying intensity in a single exposure.
Those interested in solar system objects should be aware that for the typical integration times used in the present survey minor bodies (e.g. asteroids) should usually exhibit short tracks in individual exposures and should therefore be unaffected by the Hough transform technique. However, visual inspection of the images reveals that in some cases they are rejected by the sigma-clipping rejection technique used for cosmic rays removal.
Before being released the images were examined by eye and graded by the EIS team, with the grade range being from A (best) to D (worst). This grade refers only to the visual aspect of the data (e.g. background, cosmetics). In the future the grade will also include a larger set of quality control parameters (QCP) and will be available for each survey product via a corresponding product log.
Out of 150 reduced images covering the selected XMM fields, 104 were graded A, 35 B, 7 C and 4 D. Note that images with grade D are not being released, since they have no scientific value. In the table below the images with grades C and D are presented. The table, ordered by field and date, lists: in column (1) entry number; in column (2) the EIS field name; in column (3) the passband; in column (4) the date when the night started (YYYY-MM-DD); in column (5) the grade given by the visual inspection; and in column (6) the primary motive for the grade.
|#||EIS Field Name||Passband||Date||Grade||Comment|
|1||XMM-05||R||2002-10-14||D||strong stray light contamination|
|2||XMM-06||I||2003-01-29||D||inadequate fringing correction, probably due to improper dithering|
|3||XMM-12||I||2003-03-29||D||very short integration time|
|5||XMM-01||V||2003-02-01||C||strong shape distortions|
|6||XMM-07||R||2003-08-06||C||stray light contamination|
As can be seen the success rate is better than 95% and most of the lower grades are associated with observational problems rather than inadequate performance of the software operating in an un-supervised mode. An interesting point is that occasionally R-band images are affected by fringing - for instance, in the nights of August 6 and September 23 and 29, 2003, all from the contributing program. The night of August 6 is one of the nights for which the computed R-band zeropoint deviates from the median. This points out the need to consider applying fringing correction also in the R-band, at least in some cases. The R-band fringing problem accounts for five out of seven grade C images. The remaining cases are due to stray-light and strong shape distortions.
The photometric solutions computed automatically by the EIS Survey System were compared with the ``best solution'' recently obtained by the 2p2 Telescope team. The results of this comparison, which agree remarkably well, are presented in the table below for the general case of 3-parameter fits. The table lists: in column (1) the passband; in columns (2)-(4) the offsets (EIS-Telescope Team) of the zeropoint, extinction and color term.
It is worth emphasizing that, for the present release, the periods of observations of standard stars available to the two teams do not coincide.
Not surprisingly, larger offsets are obtained when 2- and 1-parameter fits are included, depending on the passband and estimator used to derive the estimates for extinction and color term, as applicable. Finally, taking into consideration only 3-parameter fit solutions and after rejecting 3 sigma outliers one finds that the scatter of the zeropoints is less than about 0.08 mag. It is important to emphasize that this number is still uncertain given the small number of the 3-parameter fits currently available, especially in the R-band. This value for scatter is a reasonable estimate for the current accuracy of the absolute photometric calibration of the XMM survey data.
An important feature of the EIS survey system is that it attempts to provide a backbone infrastructure to enable the user to have a global view of the dataset being released both in terms of the completeness of the survey as well as the quality of the survey products. This is achieved by the production of plots accessible through the release WEB page and the quantitative information (QCP) available in the product logs, so as to make the selection of products with the desired attributes possible. This infrastructure will be complemented by a proper script that will allow the user to select products based on configurable parameters.
The attributes of the released images are summarized in the following figures, available from the WEB release page:
Comparison with other authors
The reduced data (in the form of stacks) were also compared to results obtained using the GaBoDS pipeline (Schirmer et al. 2003) and Erben et al. (2004) developed by the Bonn group. Extensive and repeated comparisons were made between SExtractor-produced catalogs from the images generated by these two independent systems. Initial discrepancies, resulting from the different techniques used (e.g. cosmic rays removal, gain-harmonization) were resolved, leading at the end to results in excellent agreement.
For some reduced images (e.g. XMM-01, XMM-02 and XMM-12) significant distortions are visible at the corners, especially in the R- and I-bands.
- Raw data for V- and I are not yet available.
- The R-band image of this field shows relatively strong fringing on a large-scale indicating that it may be necessary to apply fringing correction (see above).
- Originally, wrong coordinates were provided to the EIS team which entered them in its supporting database. Since OBs and associated finding charts for execution are automatically generated from these tables, the service mode package submitted for observing period 71 contained this error. This resulted in a series of observation blocks being executed at the incorrect position (20:05:09.8, -02:55:18) for the XMM-10 field executed as part of the large programme 170.A-0789 on the following dates 2003-04-04 (B); 2003-04-07, 2003-04-12, 2003-07-290 (V); 2003-04-08, 2003-04-09, 2003-04-11 (I). As mentioned earlier these data, comprising 53 exposures taken in roughly 5 hours covering seven nights, are not included in this release.
- Data at the correct co-ordinates of (22:05:09.8, -01:55:18) (R- and limited B- passband) were obtained as part of program 71.A-0110 on the following dates: 2003-08-06 (B, R), 2003-09-23, 2003-09-29.
Anticipated future EIS/XMM releases include:
This release is the third of 2004, and number 19 since March 1998 (of which two were software releases). Since May 2001 a total of over 1500 requests (averaging 30 requests/month) have been made for raw (919) and reduced (543) data and of the EIS/MVM image processing software (56) . These requests comprised 68,533 files (4.6 Terabytes; 2.5 Tb compressed) of raw data and 6184 files (470 Gb) of survey products (reduced images from different instruments such as SOFI, ISAAC, and WFI). Note that the last public data release was carried out in May 2004.
Over the past few months significant progress has been made in the development of an entire set of administrative tools enabling users of the EIS Survey System to have easy access to a host of information essential for a controlled data release, ensuring a proper accounting of the destination of the accumulated data and a thorough description of the quality of the survey products, including summary plots now also available to the users of the survey data.. Major improvements were also made to the report-facility responsible for the preparation of the material displayed on the WEB and of the statistics of survey products requests. Most of these improvements are illustrated by the contents of the present release, others will be phased-in in subsequent releases.
Finally, it is worth mentioning that the present release was a necessary intermediate step which will soon be followed by the release of stacks and more advanced products (single-passband and color catalogs).
Arnouts, S. et al. , 2001, A&A, 179, 436
Erben, T. et al. , 2004, in preparation
Koch, A., Grebel, E. K., Odenkirchen, M., Caldwell, J. A. R., 2004, Astronomische Nachrichten, 325, 299
Landolt, A. U., 1992 Astronomical Journal 104, 340
Manfroid, J., Selman, F., Jones, H., 2001 The Messenger 104, 16
Schirmer, M., et al. , 2003, A&A, 207, 869
Vandame, B. et al. , 2004, in preparation
Vandame, B., 2004, PhD thesis, in preparation
This document was generated using the LaTeX2HTML translator Version 2002 (1.62)
Copyright © 1993, 1994, 1995, 1996,
Computer Based Learning Unit, University of Leeds.
Copyright © 1997, 1998, 1999, Ross Moore, Mathematics Department, Macquarie University, Sydney.
The command line arguments were:
latex2html EisReadme10981401.tex -split 1 -no_navigation
The translation was initiated by eis data account on 2004-08-30