The CDS cross-match service

Thomas Boch (CDS, Observatoire de Strasbourg), Pineau (CDS), Derriere (CDS)


Abstract

The CDS has released a cross-match service allowing astronomers to cross-identify sources between very large catalogues (up to 1 billion rows) or between a user-uploaded list of positions and a large catalogue. This service has been built on top of the methods described at ADASS 2010 by Pineau et al., which are based on a dedicated binary table file format and the HEALPix pixelisation associated to specialized KD-trees.
Popular cross-identifications, such as SDSS vs. 2MASS, are pre-computed in order to accelerate these queries.

We will present the architecture of the service, whose core relies on an implementation of the UWS (Universal Worker Service) pattern, a Virtual Observatory standard for asynchronous, stateful services.
Scalability is achieved thanks to a distribution of the different crossmatch jobs on a set of workers, which are also accessed through a UWS interface.
Users interact with the service through a Web interface.
Submission of cross-match jobs through command-line tools are also possible.

Results of the cross-match are stored on a user personal storage space - backed up by iRODS - which is already in production since 2010 for the CDS Portal.

In our poster, we will show the architecture of the service, present the file formats strategy we used to integrate VizieR tables, and discuss our hardware choice, in term of RAM and hard disk drive.

Paper ID: P013

Poster Instructions




Latest News

Quick links

ADASS XXI Conference Poster

Download the Official Conference Flyer:

JPG:   A4  A3

PDF (with printer marks):

8.5in x 11in  11in x 17in  A4  A3  A2