Standard Adapters

The CCF shall be considered as an SDK, used to develop DCS solutions. It is therefore considered that for specific CCF instances, it will be necessary to develop one or more adapters of various types. This is typically Communication Adapters, to support a specific camera properties.

However, some adapters considered of common interest, will be provided by the CCF Package. It may be possible to use these, or to use them as base classes for new adapters, adjusted to the specific use cases.

In general it is recommended to attempt to use camera models already in use and to attempt to re-use the Standard Adapter provided by the CCF Package. Developers are encouraged to contact ESO to obtain information about how to reuse existing components rather than implementing new ones.

If developers identify adapters that might be of general interest, they are encouraged to design/develop these in a way that they can be re-used and to propose these for inclusion in the CCF Package.

The Standard Adapters of the three categories 1. Communication Adapters, 2. Processing Recipe Adapters, and 3. Data Publisher Adapters, are described in the following sections.

Standard Communication Adapters

In this release, the following Standard Communication Adapters are provided:

Aravis SDK Communication Adapter - “ifw::ccf::protocols::aravis::ComAdptAravis”: This is based on the Aravis SDK, and implements the GigE Vision protocol.

RTMS Communication Adapter - “ifw::ccf::protocols::rtms::ComAdptRtms”: Communication Adapter capable of receiving image frames via the RTMS protocol. This adapter does not provide features for controlling the image source (camera).

ENVision Communication Adapter - “ifw::ccf::protocols::envision::ComAdptEnvision”: This adapter provides an interface for controlling the camera and retrieving status information, based on OPC UA. The data is received via the RTMS protocol.

Simulation Communication Adapter - “ifw::ccf::control::ComAdptSim”: This provides in-application simulation. For now only play-back of an existing FITS file, typically a FITS cube, is provided.

These adapters are described in more detail in the following sections.

Aravis (GigE Vision) Communication Adapter

This adapter is based on the GenICam standard and provides support for the GigE Vision communication protocol. Even though Aravis is a GenICam implementation, the CCF Aravis Communication Adapter only supports the GigE Vision protocol. Moreover, the present implementation is not a ‘true’ generic GigE Vision adapter; there are a few camera specific features implemented in the adapter and in general, there will usually always be differences between the various cameras to deal with

The Camera Name Mapping feature provided by CCF, may be capable of coping with at least simple differences in the parameter namespaces.

Note: The issue in the context of the communication with COTS cameras is under-going evaluation, with the aim of establishing a better and simpler way of implementing the communication with system specific cameras. It goes without saying that it is not a straightforward task to provide common tools to cover all possible physical interfaces, communication protocols and specific camera SDKs. This raises some concerns in terms of the long-term prospects for ESO, having to handle a plethora of different cameras, interfaces and SDKs. It may be that a solution to mitigate this issue, will be adopted at some point in time. Users of CCF shall therefore be prepared to replace a possible communication solution, put in place, with the new (more generic) solution, provided by ESO.

To use the CCF Aravis Communication Adapter to communicate with a camera with a GigE Vision interface, a first quick test may be carried out:

Connect and configure the camera according to the camera user’s guide.

Launch the example Aravis Communication Adapter executable, “ccfCtrlAravis”. The IP address in the configuration shall be set to the allocated IP address of the camera (refer to “Installation & Deployment” for information about how to execute a CCF Control instance). The “server.simulation” parameter in the configuration should be set to “false”.

Bring CCF Control to “Operational::Idle” by submitting an “stdif::Init” and “stdif::Enable” request.

If this was successful, start the image acquisition by issuing a “dcsif::Start” request.

If executing these commands are successfully executed, it could be attempted to display frames in the DDT Viewer (see “Installation & Deployment”).

If issues are encountered, which seem related to the invalid names, it may be attempted to provide a camera specific Name Mapping; an example can be found within the CCF Package: “config/ccf/protocols/aravis/avt.mapping.yaml”.

If this does not work, it will probably be necessary to implement a dedicated Communication Adapter for the camera in question.

Minor modifications of general interest may be accepted for the CCF Aravis Standard Communication Adapter.

RTMS Communication Adapter

The Communication Adapter can be used to receive RTMS packages from the connected network and have these published, e.g. in FITS files or in the DDT.

The adapter does not provide support for controlling the image source, nor to read the status from it. I.e., it’s a passive subscriber to the data stream.

In order to use this adapter, the specific properties “IpAddress”, “Port”, “NetworkInterface”, “DataType” and “QueueSize”, must be specified in the configuration, e.g.:

devices:                   [
  {
    name:                  'TestSimCamera1',
    <standard parameters removed>
    properties:            [
      {
        name:              'IpAddress',
        value:             '127.0.0.1'
      },
      {
        name:              'Port',
        value:             '59000'
      },
      {
        name:              'NetworkInterface',
        value:             ''
      },
      {
        name:              'DataType',
        value:             'Int16'
      },
      {
        name:              'QueueSize',
        value:             '10'
      }
    ]
  }
]

Please consult the documentation for the RTMS protocol for specific information about this.

ENVision Communication Adapter

The ENVision Communication Adapter uses internally the RTMS Communication Adapter to receive the data.

In addition to receiving the data, it also provides control and status querying of the associated device, via an OPC UA interface.

This RTMS/OPC UA interface is referred to as “ENVision”.

No formal (released) ICD document exists yet, documenting the protocol.

The main goal of defining the ENVision Protocol, is for the “Software Frame Grabber” toolkit. This framework will be released during 2024 (tentatively). It is used to integrate SW frame grabber solutions for specific cameras.

The configuration for the ENVision Communication Adapter is as follows:

devices:                   [
  {
    <standard parameters removed>
    protocol:              'ENVision',
    mapping:               'config/ifw/ccf/protocols/envision/envision.mapping.yaml',
    sdk:                   'OPCUA/RTMS',
    address:               'opc.tcp://127.0.0.1:4840',
    sim_address:           'InternalSimulator',
    properties:            [
      {
        name:              'OpcUaObject',
        value:             'Andor'
      },
      {
        name:              'OpcUaNs',
        value:             '2'
      },
      {
        name:              'IpAddress',
        value:             '127.0.0.1'
      },
      {
        name:              'Port',
        value:             '59000'
      },
      {
        name:              'NetworkInterface',
        value:             ''
      },
      {
        name:              'DataType',
        value:             'Int16'
      },
      {
        name:              'QueueSize',
        value:             '10'
      }
    ]
  }
]

Simulation Communication Adapter

This adapter provides in-application simulation. This means that the Acquisition Thread, executing the Communication Adapter “Receive” method, may have a higher CPU load compared to if the simulation is implemented in an external process.

The simulation provided in this release, is based on playing back the image frame contained in a FITS image cube.

The simulation respects the exposure time, but frame rate and windows parameters are not taken into account in this release. This will be added in a future release as well as other types of simulation, based on generating artificial data. Latter means it is not necessary to provide a FITS file to be used as input for the simulation, but predictable simulated images can be generated for test purposes.

A small tool, “ccfFitsCubeGen”, is provided to generate a simple FITS cube to be used for the simulator.

Standard Processing Recipe Adapters

For this release only one Standard Processing Recipe is provided:

Centroiding Recipe.

The provided Standard Recipes are based on the ICS ODP Component, which again is based on CPL.

Centroiding Recipe

The Centroiding Recipe finds the coordinates of an object, located in the center of the given images, within a certain window. To actually find the centroid, it uses internally the function “clipm_centroiding_gauss()”.

The Configuration Parameters are as follows (example):

proc1.recipe1.adapter:       "ifw::ccf::stdrecipe::RecipeCentroid"
proc1.recipe1.name:          "TestCentroidRecipe"

The Setup Parameters are as follows (example):

ProcThread1:
  enabled:                       true

  recipe_1_1_centroid:
    enabled:                     true
    max_centre_error:            0.5
    max_sigma_error:             3
    robustness:                  7

Parameter	Description
enabled (Boolean)	Enables/disables the recipe.
max_centre_error (Double)	Maximum error from the center to consider the search for the centroid as successful.
max_sigma_error (Double)	Maximum sigma error of the fitting of the Gaussian.
robustness (Integer)	Number of iterations for obtaining the centroid accurately.

Note, the size of the central window in which the location of the centroiding is executed, is computed as (lower left, upper right) corners of window: “(((width / 4), (width / 4) + (width / 2)), (((height / 4), (height / 4) + (height / 2)))”.

The OLDB status keys are of the form: “<server ID>/<CCF instance name>/status/pipelines/<pipeline name>/recipes/<recipe name>.<key>”.

The keys are:

Parameter	Description
sigma_error_x/y (Double)	Output sigma error in X/Y.
centre_error_x/y (Double)	Output centre error X/Y.
centre_intensity (Double)	Intensity (value) of centre pixel.
centre_x/y (Double)	X/Y coordinate of centre pixel.
error (Boolean)	If true, indicates that the centroid searching failed.
error_msg (String)	Error message (if available), in case an error occurred.
max_centre_error (Double)	Maximum centre error specified in the configuration. If the error is above, this will lead to failure of recipe.
max_sigma_error (Double)	Maximum sigma error specified in the configuration. If the error is above, this will lead to failure of recipe.
last_update (Double)	Time since epoch of last update.
probe_window (String)	Status of probe window used ((x1, y1), (x2, y2)).
robustness (Integer)	Maximum number of retries.

Standard Data Publisher Adapters

In this release, three Standard Data Publisher Adapters are provided:

FITS File Publisher.

DDT Publisher.

RTMS Publisher.

These are described in the following.

FITS File Publisher

An example of the parameters for the FITS Publisher is shown in the following:

ProcThread1:
  enabled:                       true

  FitsPublisher:
    enabled:                     true
    basename:                    'PubFitsTest1'
    max_size:                    100      # MB
    nb_of_frames:                7
    format:                      'Single' # Single, Cube, MEF
    overwrite:                   true
    rec_mode:                    'All'    # All, I:<nb>, P:<period> - not supported.

The specific Setup Parameters provided for the adapter are:

Parameter	Description
basename (String)	Basename used for generating the output filenames. The names are of the form: “<proc#.pub#.basename>[_<ISO 8601>].fits”, where the ISO is prepended to make the name unique when “proc#.pub#.overwrite” is false.
max_size (Integer)	Indicates the maximum volume in MB of the output data generated during a Recording Session.
nb_of_frames (Integer)	Indicates the maximum number of image frames to be handled during a Recording Session.
format (String)	Format of the Output Data Product files. Valid options are: Single, Cube, MEF.
overwrite (Boolean)	If set to “true”, the same output filename is used, which is basically the specified “proc#.pub#.basename”. This is used for test purposes to avoid generating a lot of files on disk.
rec_mode (String)	The recording mode. Valid options are: 1: All: All frames are recorded. 2: I:<#>: Every ‘#’th frame is taken. 3: P:<period>: A frame every <period> second is recorded. In this release, only “1: All” is supported.

The target directory for the generated output files is defined as “$DATAROOT/dcs/<cfg:server_id>”.

The FITS Data Publisher is not responsible for any ‘cleaning up’ such that all output files generated, must be removed by the client/user. When ‘overwrite mode’ is used, however, the same file is always written.

DDT Publisher

The DDT Publisher publishes data into the DDT data handling infrastructure to allow for distributed access and for displaying in the DDT image widgets.

When using the CCF DDT Publisher, it is important to ensure that the properly configured instance of the DDT Broker is running, otherwise CCF Control will not start.

The DDT system as such, is not documented here. Refer to the DDT user’s manual for further information.

An example of the parameters for the DDT Publisher is shown in the following:

ProcThread1:
  enabled:                       true

  pub_1_1_ddt:
    enabled:                     true
    ddt_id:                      'CcfTest11'
    ddt_broker:                  'zpb.rr://127.0.0.1:12011/broker/Broker1'
    ddt_max_rate:                1000

The specific Setup Parameters provided for the adapter are:

Parameter	Description
ddt_id (Integer)	Name of the DDT stream used for publishing the data.
ddt_broker (Integer)	The URI of the DDT Broker to use when publishing. Example: “zpb.rr://127.0.0.1:11011/broker/Broker1”.
ddt_max_rate (Double)	Can be used to limit the rate with which data is published in the DDT system, to avoid overloading the DDT and the execution host.

RTMS Publisher

The RTMS Publisher publishes data on the specified network in the MUDPI/RTMS protocol. This is defined for transferring the data (realtime) from the WFS camera to the Realtime Computer, usually via a deterministic network.

The MUDPI and RTMS protocols are documented elsewhere and the format and functioning will not be described in detail in this manual.

An example of the parameters for the RTMS Publisher is shown in the following:

ProcThread1:
  enabled:                       true

  RtmsPublisher:
    enabled:                     true
    id:                          'RtmsPub'
    max_rate:                    220
    bpp:                         2
    address:                     '127.0.0.1'
    port:                        59000
    topic_id:                    10
    sample_id:                   500
    pkt_send_delay:              0.001

The specific Setup Parameters provided for the adapter are:

Parameter	Description
id (String)	ID allocated to the publisher.
max_rate (Double)	Maximum rate, in Hz, with which to publish data. Setting this lower than the image acquisition rate may result in frame skipping.
bpp (Integer)	Bytes per pixel.
address (String)	IP address of network interface on which to broadcast the UDP packets.
port (Integer)	Port used for the UDP socket.
topic_id (Integer)	Used by RTMS to identify the type of data in the packets.
sample_id (Integer)	Used to tag the images (samples) from this source, for identification purposes.
pkt_send_delay (Double)	Delay, in seconds, between sending each UDP packet. May be used if the publisher is ‘too fast’ for the RTMS subscriber to be able to receive all UDP packets.

Note: Using multicast, may saturate the organisation network infrastructure and shall normally only be used on isolated networks. For development purposes, the loopback interface on the development host, can be used to avoid that the packet leave the development machine.