A new development study, led by ASTRON in Dwingeloo (The Netherlands) will investigate the possibility of implementing initial processing steps of ALMA raw data from the correlator in a streaming process of the visibilities. This study builds synergies with the LOFAR and SKA telescopes, where similar processes will be applied by the software engineers at ASTRON. In particular, the Default PreProcessing Pipeline (DP3) is a modular framework that can be adapted to be used on ALMA data.
Development study on streaming visibility processing kicks off
Published: 15 Sep 2023
Participants of the kick-off meeting in front of the historical 25m Dwingeloo radio telescope
The advantages for ALMA to use DP3 include the possibility to significantly reduce the data volume by including a compression step in DP3. This avoids a temporary storage of the raw data during the initial processing steps. Furthermore, this could also enable streaming calibration steps such as the pathlength correction due to water vapour. The reduction of the data volume is particularly relevant in view of the ALMA Wideband Sensitivity Upgrade (WSU), which will increase the data volume by more than an order of magnitude. The study includes an assessment of potential loss of data quality due to such compression procedures on real ALMA data.