Version 3.2.0

Added

  • Added command RetryMergeDaq() which enables the ability to retry failed operations in state DaqState.StateMerging.

  • Added section on Alerts and document all Alerted Conditions.

  • Emit warning alert in DPM, in state Collecting, for each source file that indicates it has been modified after Data Acquisition was stopped. This identifies a data race and could lead to incomplete or corrupted source file being copied.

  • Added OCM configuration parameter cfg/dpm/retry_interval_sec which determines retry interval when DPM is offline. See Configuration File for details.

Changed

  • (build) Use default C++ version from wtools (C++23 in platform:25).

  • (build) Adapted to prefix changes introduced in ifw-core v6.0.0.

  • Moved example configuration from {prefix}/config/daqOcmServer to {prefix}/config/ifw/daq/daqOcmServer.

  • daqOcmServer rejects commands from OcmDaqControl in state NotOperational.

  • Failure to release Data Product is treated as fatal and Data Acquisition will remain in DaqSubState.Releasing. Command RetryMergeDaq() added to allow retries after correction of underlying cause.

  • Register MAL service alias for StdCmds to /StdCmds for compatibility.

  • Modified DaqStatus to include list of active alerts. The derived property error is true if any alert has severity Error.

  • Improved Data Acquisition status recovery after requests to daqDpmServer fails.

Version 3.1.0

Added

Fixed

  • Improve keyword parsing of strings to avoid accidentally parsing string as FITS syntax.

  • Remove limitation where Data Acquisitions without primary or metadata sources could not be started (e.g. merging only existing FITS files).

  • Fix issue where useful information was missing if StartDaqV2 failed, because nested exception messages where not included.

  • Fix issue with boost::future::then() continuation not executed in the main event loop possibly affecting AwaitDaqState().

  • Fix keyword comparison between the string literal form and strongly typed forms which could lead to keywords not being overwritten when merged.

Changed

  • CII v4.0.0 remove use or support for environment variable $CII_LOGS. Logs are instead sent to syslog. To get separate log files for daqOcmServer and daqDpmServer the $DAQ_LOGS environment variable can be set instead. See Logging Configuration.

  • Adapt to fmt v9.1.0, CLI v2.3.2 and boost 1.78 in Fedora 38.

  • Made daqif.DaqStatus.error a derived property which is set if there are are any active issues/alerts for the Data Acquisition. Information about issues is available in daqif.DaqStatus.message.

  • Improved Data Acquisition status propagation between daqOcmServer and daqDpmServer which can be seen mainly with that alerts are now persistent.

  • Modified to support MAL ICD recif v2.0.0. Support for previous versions is removed.

Removed

  • Removed dependency to Guidelines Support Library.

  • The MAL interface daqif was moved to its own separate project and repository with independent versioning at https://gitlab.eso.org/ifw/ifw-daqif/.

Version 3.0.1

Fixed

Version 3.0.0

Added

Changed

  • Updated section Data Acquisition Guide to include step-by-step instructions using data source simulators provided in ifw-daq project and use of daqif.OcmDaqControl.StartDaqV2().

  • Renamed substate Transferring -> Collecting.

  • GetDaqStatus() and AwaitDaqState() will attempt to reply considering a previously archived Data Acquisition.

    For GetDaqStatus() it returns archived status if Data Acquisition is not active. This means that until the same id is reused and archive has not been removed it is possible to query status of old Data Acquisitions.

    For AwaitDaqState() it returns with status if condition is already satisified by an archived Data Acquisition.

  • To support the scenario where daqDpmServer completes a Data Acquisition when daqOcmServer is not running, daqOcmServer will now request DAQ status from daqDpmServer when it is started. If this times out (nominally because daqDpmServer is not running) daqOcmServer will schedule a new attempt after 15s and repeat until successful or fails without timeout.

  • Configuration changed from using yaml-cpp to CII.

  • Online database changed from using redis to CII.

  • Corrected daqOcmCtl JSON serialization of fileId which was using the form file_id.

  • daqOcmServer and daqDpmServer configuration was refactored to use CII and aligned to common schema defined in RAD. The following tables summarize old and new values in shorthand form using map/value where value is a configuration value in YAML map map.

    Table 12 daqOcmServer

    Old

    New

    cfg.instrument_id

    cfg/instrument_id

    cfg.dataroot

    cfg/dataroot

    cfg.workspace

    cfg/daq/workspace

    cfg.daq.stale.acquiring_hours

    cfg/daq/stale_acquiring_hours

    cfg.daq.stale.merging_hours

    cfg/daq/stale_merging_hours

    cfg.log.properties

    cfg/log_properties

    cfg.sm.scxml

    cfg/sm_scxml

    cfg.req.endpoint

    cfg/req_endpoint

    cfg.pub.endpoint

    cfg/pub_endpoint

    cfg.db.endpoint

    removed (CII OLDB is not configurable)

    cfg.db.prefix

    cfg/oldb_uri_prefix

    cfg.db.timeout_sec

    cfg/oldb_conn_timeout

    cfg.dpm.req.endpoint

    cfg/dpm/req_endpoint

    cfg.dpm.pub.endpoint

    cfg/dpm/pub_endpoint

    cfg.dpm.timeout_sec

    cfg/dpm/timeout_sec
    Table 13 daqDpmServer

    Old

    New

    cfg.dataroot

    cfg/dataroot

    cfg.workspace

    cfg/daq/workspace

    cfg.log.properties

    cfg/log_properties

    cfg.req.endpoint

    cfg/req_endpoint

    cfg.pub.endpoint

    cfg/pub_endpoint

    cfg.limit.daq

    cfg/limits/daq

    cfg.limit.merge

    cfg/limits/merge

    cfg.limit.net.receive

    cfg/limits/net_receive

    cfg.limit.net.send

    cfg/limits/net_send

    cfg.bin.merge

    cfg/bin_merge

    cfg.bin.rsync

    cfg/bin_rsync

Fixed

  • If primary data source stops replying this could lead to daqOcmServer wait indefinitely for the completion of the recording. This has been addressed by cancelling the wait process if user manually stops or aborts Data Acquisition.

Version 2.1.0

Added

Changed

Version 2.0.0

This version adds daqDpmServer application together with daqDpmMerge that executes the merge and basic integration with daqOcmServer.

There are open questions regarding Data Product control requirements which will determine the client interface to daqOcmServer. This includes aspects like controlling HDU and keyword order, how to optionally specify the in-place merge target and selecting/transforming keywords, without relying on heuristics. Until those requirements are clarified daqOcmServer makes the following assumptions which has some limitations:

  • If a single primary data source produce a single FITS file this will selected as the in-place merge target.

  • Otherwise data sources produce multi-extension FITS files with no data in primary HDU.

  • Data sources have the following relative priority:

    Note

    Order determines merge order of keywords and extensions.

    1. Primary data sources in the order user specified to StartDaq().

    2. Metadata sources in the order user specified to StartDaq().

  • Each data source produce keywords that will be merged to Data Product primary HDU returned via metadaqif or with FITS file primary HDU.

  • There is currently no way to provide keyword rules that select and transforms keywords. All keywords in user class will be merged.

Added

Changed

  • Application names changed to abide to new naming standards:

    • ocmServer -> daqOcmServer

    • ocmServerCtl -> daqOcmCtl

    • daq-fits-test -> daqFitsTest

  • New mandatory daqOcmServer configuration (see Configuration File).

    • cfg.dpm.req.endpoint

    • cfg.dpm.pub.endpoint

  • New optional daqOcmServer configuration (see Configuration File).

    • cfg.db.prefix

  • OCM MAL ICD changes due to scope changed to include DPM as well.

    • ICD name renamed from ocmif -> daqif.

    • OCM Data Acquisition interface renamed from ocmif.OcmDaq -> OcmDaqControl.

  • Renamed default OCM scxml model config/ocmServer/sm.xml -> config/daqOcmServer/sm.xml.

Version 1.0.0

First release of OCM comes with basic support for performing Data Acquisitions from standard compliant sources (sources implementing metadaqif or recif).

New Features

Known Issues and Limitations

  • Keyword formatting is not configurable and no dictionary validation is performed on FITS keywords (provided with commands StartDaq() and UpdateKeywords() or part of reply from metadaqif).

  • Standard keywords that OCM should provide, are not yet included in the created FITS file from OCM.

  • Data Acquisition state persistence is not implemented yet. If ocmServer is restarted all state is lost.

  • An assertion in the CII Middleware Abstraction Layer used by ocmServerCtl has been observed from time to time. It is easily identified by the following output on stdout:

    ocmServerCtl: ../mal-zpb/cpp/mal-zpb/src/include/Zmq.hpp:453: void zmq::context_t::close():
Assertion `rc == 0' failed.

  • MAL does not handle timeouts correctly. If an endpoint does not reply to a command OCM will wait indefinitely.