7. Telemetry

Document ID:
Revision:	2.1
Last modification:	July 15, 2025
Status:	DRAFT
Repository:	https://gitlab.eso.org/cii/srv/cii-srv
File:	telemetry.rst
Project:	ELT CII
Owner:	Marcus Schilling

Document History

Revision	Date	Changed/ reviewed	Section(s)	Modification
0.1	09.03.2020	smarincek, jpribosek, dsoklic	All	Document creation.
1.0	06.05.2020	dsoklic/bte rpinc	All	Review updates. Released document.
1.1	17.6.2020	jpribosek	All	Updating manual after receiving support questions.
1.2	7.7.2020	jpribosek	All	Updating manual according to raised LAT issues.
1.3	30.7.2020	jpribosek	All	Updating manual according to raised LAT issues.
1.4	27.8.2020	jpribosek	All	Updating manual according to raised LAT issues.
1.5	3.9.2020	jpribosek	All	Updating manual according to raised LAT issues. Added additional section for setting up OLDB simulator.
1.6	24.9.2020	jpribosek	All	Updating manual according to raised LAT issues.
1.7	06.10.2020	jpribosek	All	Updating manual according to the review comments.
1.8	18.03.2024	mschilli	0	CII v4: public doc
2.0	24.01.2025	mschilli	All	CII v5.0: document restarted
2.1	15.07.2025	mschilli	All	add design description, guidelines

Confidentiality

This document is classified as Public.

Scope

This document is a manual for the Telemetry system of the ELT Core Integration Infrastructure software.

Audience

This document is aimed at Users and Maintainers of the ELT Core Integration Infrastructure software.

Glossary of Terms

API	Application Programming Interface
CII	Core Integration Infrastructure
CLI	Command Line Interface
DP	Data Point
EA	Engineering Archive
OLDB	Online Database

References

1. Logging Manual: https://www.eso.org/projects/elt/develop/cii/srv/latest/manuals/html/log.html

7.1. Overview

xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

NOTE: DRAFT STATUS

THIS DOCUMENT IS UNDERGOING A COMPLETE REWRITE.

MANY SECTIONS ARE INCOMPLETE AT THIS TIME.

xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

This document is a user manual for usage of the CII Telemetry system. It explains how to use the Telemetry API and the available tools.

All examples in this manual will be also presented in the telemetry-examples module.

7.2. Introduction

The purpose of the CII Telemetry system is archiving periodic, event-based and ad-hoc data that will be used for calculation of statistics, and the long-term trends and performance of the ELT telescope.

CII Telemetry consists of:

• a client-side library with APIs for recording, retrieval, management

• a command-line client for bulk recording

• an autonomous daemon for un-supervised recording

7.2.1. Terminology

• (Time)Series - a sequence of measured values over time, plus description

• (Ts)Metadata - descriptive info about a time series

• (Ts)Point - a single measured value with timestamp

• (Telem)Recorder - client-side API for appending points to a series

• (Telem)Archiver - daemon that autonomously measures and records values

• Data Source - where values originate from

• Data Sink - where values are stored to

7.3. Concepts and Design

7.3.1. Telem Library

The CII Telemetry Library separates Frontends from Backends. Additionally it contains the TelemManager.

7.3.1.1. Frontends

Frontends provide high-level APIs for applications/users.

An Application uses a Frontend to interact with the Telemetry Library, e.g.

std::shared_ptr<cii::telem::TelemRecorder> recorder;
[...]
recorder->RecordValue (456);

Frontend availability per language

Frontend	Python	C++
Recorder	yes	yes
Operations	planned	no
Admin	planned	no

All Frontends in Python and C++ share the same abstraction over a time series.

7.3.1.1.1. Time Series

A time series is comprised of:

1. Time Series Metadata

2. Collection of Time Series Points

7.3.1.1.2. Time Series Metadata

Metadata provides a description of a time series. It is crucial for a successful data analysis of series recorded by many applications in an environment that changes over time.

Field	Type	Description	Example
tsid	string	Perpetual ID	abc-xyz-123-127
tsname	string	Time Series Name	/tel/m2/coeff
btype	string	CII Basic Type	UINT16
units	string	SI Unit	m/s
origin	string	producer	m2-sa-est
serial	string	device S/N	inventory-ms123
role	string	purpose	cabinet heat
comment	string	human readable	<text>
json	string	full metadata	<json text>

Code Reference: ciiTelemApi.hpp - TsMetadata

7.3.1.1.3. Time Series Point

A point is one measurement of a value at one moment in time.

Field	Type	Description	Example
value	string|num|bool	measured value	127.0
mtime	string	measure time	2025-07-15T11:11:00+00:00

Code Reference: ciiTelemApi.hpp - TsPoint

Note: the addition of a “trustlevel” (a.k.a. “quality”) field is planned.

7.3.1.2. Backends

A Backend represents a Data Sink (typically a database), e.g.,

• InfluxDB : InfluxDB installation as supported by ELT DevEnv

• FileDB : a CII-internal product based on CSV-files and pandas

• Log-Stream : a write-only data sink based on CII logging

A backend uses datasink-specific data structures and runs on datasink-specific configuration.

An application only deals with a backend directly when configuring datasink-specific options, e.g.

backends.log.config["logger"] = "mylogger"
backends.log.config["level"]  = "INFO"

Internally, the backend implementations share a similar design. A backend usually has:

• Configuration

• Converter

• Backend Data Format

• Connection Manager

The Converter translates between the frontend data format (TsMetadata, TsPoint) and the backend’s own datasink-specific data format (e.g. called DbPoint). During a write operation, the data path usually looks like this:

Frontend Data Format -> Converter -> Backend Data Format -> Connection -> Data Sink

These details are in most cases of no concern to the application using the telemetry library, but in some scenarios, they do become relevant:

• a unit test wants to mock the backend’s database connection

• an application cannot use the telemetry frontend, and has to interact with the backend directly.

7.3.1.3. Frontend x Backend

The following combinations of Frontends x Backends are available. The table also shows availability per language and current implementation status.

Frontend x Backend

Frontend	Backend	Python	C++
Recorder	Log	yes	yes
	Influx	planned	planned
	File	planned	planned
Operations	Log	no	no
	Influx	planned	no
	File	planned	no
Admin	Log	no	no
	Influx	planned	no
	File	planned	no

7.3.1.4. Telem Manager

The TelemManager provides Frontends and Backends, and simplifies the act of wiring a Frontend with a Backend. Also provides some conveniences like factory methods and caching of Frontends.

TelemManager telman{};
telman.CreateTelemRecorder( "reco1", telman.backends.log );

7.3.2. Telem Client

A command-line client for storing points or series. Supports the upload of a previously recorded series, uses the Operations API.

7.3.3. Telem Archiver

A stand-alone, continously running process that listens on a data source (OLDB, DDS, potentially OPC UA), and records points to a data sink, uses the Recorder API.

Configurable options:

• trigger

• threshold

• periodicity

• burstfilter

7.4. Topology

The following topologies are foreseen/possible:

7.4.1. Storage

Under certain circumstances, an application may be unable to use the Frontend APIs. By using (parts of) the backend directly, it can ensure to write the data in the same storage format as the Frontend APIs.

7.4.2. Retrieval

The possible topologies for retrieval depend to a large extent on the actual type of Data Sink, in other words: which database (LogStream, FileDB, InfluxDB, …) was used for storing the series.

7.5. Usage Guidelines

7.5.1. Append/Write

Use of the telemetry frontend APIs shields the application to a high degree from changes in the backends. Should the observatory adopt a different database product, an additional backend implementation would cover this change, and the application would need little to no porting.

Use of the telemetry frontend APIs also ensures that all series stored to a given backend are stored in a consistent manner and follow the same storage model.

As previously described, the data flow of a write operation is:

Frontend Data Format -> Converter -> Backend Data Format -> Connection -> Data Sink

Every backend comes with a Converter, and its purpose is to translate from the (generic, high-level) frontend data format to the (backend-specific, low-level) backend data format. Thereby, it is foremost the Converter that governs the storage format of a backend.

If an application, e.g. an External Archiver, is unable to use the Frontend APIs, it can (must) at least use the Backend directly, so it ends up writing data in the same storage format as the Frontend APIs would.

Correct metadata for series is of utmost importance for long-term data analysis. An application wanting to append/write a series needs to keep the metadata of its series up-to-date, i.e.:

• check for existing metadata

• create metadata

• update metadata

7.5.2. Query/Read

1. via Operations Frontend

The Operations Query API offers some frequently needed queries, abstracting from the concrete backend. Use of it requires no understanding of the backend, and the application needs little to no porting when moving to a different backend. The Operations Read API offers retrieval of series in frontend data format. It additionally offers retrieval of series in pandas dataframe format, a de-facto standard in Python. Note that the Read and Query APIs are available only in Python. Dataframes can be processed further in a variety of ways, e.g. with Python’s matplotlib.

2. via 3rd-party software (e.g. Influx Web UI, Grafana)

If the Functions provided by the Operations Frontend are not enough, data is alternatively queried, retrieved and visualised through 3rd-party software operating directly on the backend’s database. At this time, ELT Control Software is using Grafana as the primary tool for time series analysis and visualisation, other tools (e.g. Tableau, Databricks) are not in use. When using a 3rd-party software, the porting effort for switching to a different backend (thus: underlying database) is on the application/user.

7.6. Dependencies

Use of the …	requires …
Operations frontend	Python
FileDB backend	Python
InfluxDB backend	InfluxDB
Telem Archiver	OLDB or DDS

7.7. API

7.7.1. Import/includes

7.7.1.1. Python

wscript

def configure(cnf):
  cnf.check_python_module('cii.telem')

code

import cii.telem

7.7.1.2. C++

wscript

def configure(cnf):
   cnf.check_wdep(
      wdep_name='cii-services-telemetry.cpp.telem',
      uselib_store='cii-services-telemetry.cpp.telem'
   )

code

#include <ciiTelem.hpp>

7.8. Append API

The Recorder frontend provides convenient access to the Append API from C++ and Python.

7.8.1. Recorder Example

7.8.1.1. Python

# Acquire API
# ----------------
cii.telem.TelemManager.get_instance()
recorder = telman.create_telem_recorder("m2rec", telman.backends.log)

# Specify metadata
# ----------------
tsmeta = tel.TsMetadata()
tsmeta.tsname  = "/elt/tel/m2/lsv/m2saest/mon/sv_force_correction"
tsmeta.comment = "Tel M2 force correction"
tsmeta.origin  = "m2saest"
tsmeta.btype   = "DOUBLE"
recorder.record_metadata (tsmeta)

# Record points
# -------------
recorder.record_point (tel.TsPoint(123))
recorder.record_value (            456 )
recorder.record_point (tel.TsPoint(789))

7.8.1.2. C++

std::shared_ptr<cii::telem::TelemManager> shared_tm = cii::telem::TelemManager::GetInstance();

// Acquire API
// -----------
std::shared_ptr<cii::telem::TelemRecorder> recorder;
recorder = telman.CreateTelemRecorder("m2rec", telman.backends.log);

 // Specify metadata
 // ----------------
 cii::telem::TsMetadata tsmeta {};
 tsmeta.tsname  = "/elt/tel/m2/lsv/m2saest/mon/sv_force_correction";
 tsmeta.comment = "Tel M2 force correction";
 tsmeta.origin  = "m2saest";
 tsmeta.btype   = "DOUBLE";
 recorder->RecordMetadata (tsmeta);

 // Record points
 // -------------
 recorder->RecordPoint (cii::telem::TsPoint(123));
 recorder->RecordValue (                    456 );
 recorder->RecordPoint (cii::telem::TsPoint(789));

7.9. Write API

7.9.1. Store points

PENDING

7.9.2. Store series

PENDING

7.10. Query API

7.10.1. Query by time

PENDING

7.11. Read API

7.11.1. Read series

PENDING

7.12. Management API

7.12.1. Rename series

PENDING

7.12.2. Delete points

PENDING

7.12.3. Delete series

PENDING

7.13. Telem Client

Telem Client is under development, and not part of the current release.

7.14. Telem Archiver

Telem Archiver is under development, and not part of the current release.

7.14.1. Deployment

PENDING

7.14.2. Configuration

PENDING

7.15. Appendix: Design Decisions

7.15.1. Statefulness

The Recorder frontend is stateful. A recorder is bound to one series.

Rationale: The recorder is a convenience feature, with a flat learning curve, limited capabilities, and ease of use. Keeping state allows for that.

The Operations frontend is stateless. The information needed by a function is always fully passed as arguments to the function.

Rationale: The Operations frontend offers many functions, and managing state would introduce massive accidental complexity. But even more importantly, the functions would reflect assumptions about how the data gets stored, and that would strip the backend off its full control over the storage model.

7.15.2. Thread-safety

All APIs are thread-safe.

7.16. Appendix: InfluxDB Backend

7.16.1. Db Point Format

Field	Type	Description	Example
table	string	table of series	cabinet-sensors
mtime	string	measure time	2025-07-15T11:11:00+00:00
meas	string:string	measured values	heat1:1.11
tags	string:string	static labels	role#heat1:cpu1-tempsens

7.16.2. Multi-series measurements

The underlying database supports storage of points from multiple series simultaneously under one common timestamp. This feature comes in very handy when those series are to be time-correlated later. The backend will store multiple series together this way if they participate in a common naming scheme, and points have a common timestamp.

md1 = TsMetadata()
md2 = TsMetadata()
md1.tsname = "cabinet-sensors#heat1"
md2.tsname = "cabinet-sensors#heat2"
md1.role   = "cpu1-tempsens"
md2.role   = "cpu2-tempsens"

ti = "2025-07-15T11:11:00+00:00"
p1 = TsPoint( 1.11, ti )
p2 = TsPoint( 2.22, ti )

pp = [ (md1,p1), (md2,p2) ]
operations.write_points( pp )

7.16.3. Storing Metadata

The underlying database has no support for storing metadata separately. The backend therefore uses the available tag-feature to store metadata together with the series points.

Note: this approach has its disadvantages, and may change in the future.

7.16.4. Supported data types

The underlying database supports neither arrays/sequences nor strings longer than 64 KB. The backend does not make up for, or work around, these limitations.