Tutorial

The purpose of the sequencer is to execute Sequencer scripts, either for engineering or science purposes.

The unit of execution is a single step which is just a normal python function or method with no input parameters.

Sequences are modeled as Directed Acyclic Graph (DAG). Each node in the graph can either be a simple action which just invokes a single sequencer step or a more complex node which contains a complete sequencer script.

This allows sequences to be grouped and nested freely. Ultimately they will execute steps.

The Sequencer API allows the creation of these graphs.

Note: all the examples below can be found in seq/samples/src/seq/samples.

Building Sequences

Sequencer scripts are modeled as DAGs, the Sequencer API allows to create nodes in the DAG. There are different node types that determine the way its children are scheduled for execution (e.g. Parallel, Sequential). The sequencer scripts can use the same or different node types depending on the purpose. select, mix and match the node type(s) that suits better your needs.

The Sequencer Engine expects to find either a module method or a specific class name in a module in order to construct a Sequencer script from it. The conventions are the following.

1. A module level create_sequence() function. See Tutorial 1 (a.py).

2. A class named Tpl which must provide a static method create_sequence. See Parallel tasks sample (b.py).

In either case, the return value is the root node of the graph being implemented.

The first convention is tried first, if no create_sequence() function is found, the second convention is attempted.

For very simple scripts, following the first convention is perfectly fine. For more complex scripts, e.g. a Sequencer script going to be parametrized to control multiple devices at the same time, the class approach is recomended.

Node Dependencies

Regarding the order of execution, the DAG edges allows to represent the nodes dependencies. Each node in the DAG depends, from its parent nodes. Meaning that a node will not be started until every node that precedes it has finished its own execution.

Node dependencies determines the execution order of the sequence steps. A node won’t run until all its dependencies have finished. In the graph, a node dependency is seen as an incoming edge.

Chaining the basic node types Sequence and Parallel defines a dependency hierarchy.

Sequence nodes are executed one after the other, therefore each node depends on its predecessor. On the other hand, Parallel nodes indicates that all nodes it contains shall be executed together. By combining and chaining this two basic node types it is possible to express any dependency graph.

Very simple sequences

As mentioned before the simplest step is a python coroutine with no input parameters which is used to create an Action node.

Note

The no input parameter rule can be bypassed with strategies shows in Passing Arguments to Actions

In this case, we define a sequence that executes two steps, one after the other, namely a() and b(). Source codes is shown below.

Tutorial 1 (a.py)

#!/usr/bin/env python3
"""
Simple example (A)

Executes function a() and b()
"""
import logging
import asyncio
from seq.lib.nodes import Sequence
from seq.lib import getUserLogger


#LOGGER = logging.getLogger(__name__)
LOGGER = getUserLogger()

async def a():
    """Simply a"""
    await asyncio.sleep(1)
    LOGGER.info("Ax")
    print("AAAA")
    return "A"

async def b():
    """b example
    """
    LOGGER.info("Bx")
    print("BBBB")
    return "B"

def create_sequence(*args, **kw):
    """Builds my example sequence"""
    myname = kw.pop('name', "Example A")
    return Sequence.create(a, b, name=myname, **kw)

Important

The Sequencer Engine is based on the asyncio library, therefore it is biased towards coroutines, but they are not mandatory as shown in Loop example (loop1.py).

These are some simple rules to create sequences:

• A step (Action) is a python coroutine with no input parameters. See Passing Arguments to Actions to break this rule.

• In this case, the sequence is created using the Sequence.create (Sequence) constructor, which receives the steps to be executed (in the given order).

• The python module which contains the sequence must define a create_sequence function as shown in the example. It returns a Sequence node that holds the nodes it will execute.

Important

The Sequence.create constructor provides syntax sugar in order to support passing coroutines as the sequence’s graph nodes. In such a case a node of type seq.nodes.Action is automagically created and inserted in the graph.

This simple sequence 01 is graphically shown below. It can be imported from python as:

>>> import seq.samples.a

simple sequence 01

Notice that only two nodes were specified in the create_sequence() function. However the simple sequence 01 figure shows four nodes. The sequencer engine adds a start node (black circle) and end node (double black circle) to every node container type, i.e. those nodes that have children: Parallel, Sequence and Loop.

Note

The start and end node, among other things, makes easy to chain nodes together by linking the end node of a container with the initial node of the next.

Executing Tasks in Parallel

One is not limited to create just linear sequences. Parallel activities (pseudo parallel) can be created using the Parallel.create() constructor. It receives the same parameters as the Sequence node constructor. When executed, the sequencer engine processes the Parallel nodes children in parallel.

Parallel tasks sample (b.py)

"""
Parallel nodes example.
"""
import asyncio
import random
import time
import logging
from seq.lib.nodes import Parallel, ActionInThread
Logger = logging.getLogger(__name__)

class Tpl:
    """A sample Sequence"""

    def a(self):
        """sleeps randomly"""
        t = random.randrange(5)
        time.sleep(t)
        Logger.info("... done A")

    async def b(self):
        """sleeps randomly"""
        t = random.randrange(5)
        await asyncio.sleep(t)
        Logger.info("... done B")

    @staticmethod
    def create(*args, **kw):
        """Builds my sequence"""
        a = Tpl()
        p = Parallel.create( ActionInThread(a.a), a.b, **kw)
        return p

Which is represented graphically as follows.

Parallel Sequence

Points to notice:

1. In this case the Sequencer Engine discover a class named Tpl and calls its create method (@staticmethod as the convention mandates).

2. The example Parallel Sequence also shows that steps are not limited to coroutines. Just wrap it in ActionInThread node.

3. There is no problem mixing normal routines and asynchronous code. The sequencer will send the normal code to a separate thread and execute it there.

In order to avoid normal methods or functions to potentially block the asyncio loop (by holding the CPU) they must be executed on their own Thread.

This is achieved with the ActionInThread node. In the example the a() method is wrapped in such node.

Executing Tasks in a Loop

The Loop node allows to repeat a set of steps while a condition is True.

Loop example (loop1.py)

"""
Implements a loop.
The condition checks Loop's index < 3.
"""
import asyncio
import logging
import random
from seq.lib.nodes import Loop

logger = logging.getLogger(__name__)


class Tpl:  # Mandatory class name
    async def a(self):
        """sleeps up to 1 second"""
        t = random.random()  # 0..1
        await asyncio.sleep(t)
        logger.info(".. done A: %d", Loop.index.get())

    async def b(self):
        """sleeps up to 1 second"""
        t = random.random()  # 0..1
        await asyncio.sleep(t)
        logger.info(" .. done B: %d", Loop.index.get())

    async def c(self):
        pass

    async def check_condition(self):
        """
        The magic of contextvars in asyncio
        Loop.index is local to each asyncio task
        """
        logger.info("Loop index: %d", Loop.index.get())
        return Loop.index.get() < 3

    @staticmethod
    def create(**kw):
        t = Tpl()
        l = Loop.create(t.a, t.b, t.c,
                        condition=t.check_condition, **kw)
        return l

Which is represented graphically as follows.

Loop example

The code in Loop example shows the Loop’s node constructor which takes a variable number of arguments comprising the Loop’s body, i.e. the part that is repeated, a(), b() and c() in this case. The condition node is specified with the condition keyword.

The Loop’s index is kept in the context variable index, meaning it can be accessed as Loop.index.get() as the check_condition() function shows.

In order to separate the index value of the different loops that might be occurring at the same time the Loop’s index is implemented as an asyncio context variable. Therefore to get its value one has to call its get() method as the check_condition() function shows.

Embedding Sequencer Scripts

Sequences can be reused or embedded in order to produce more complex activities. The following example uses the sequences “a” and “b” to create a new sequence that executes them in Parallel and adds a step from a local class.

Important

Embedding a Sequence entails to import the module and instantiate its Sequencer script (either with create_sequence() or by Tpl.create()). The OB object can decide which one should be imported:

from seq.lib.nodes import seq_factory
from seq.samples.a

mynode = seq_factory(a)

Sequence embedding example

#!/usr/bin/env python3
"""
Simple example.

Uses nodes from template defined in module 'a'.
It also uses the 'a' template as a whole.
"""
from seq.lib.nodes import Parallel, seq_factory
from seq.lib.ob import OB

from seq.samples import a
from seq.samples import b

class Tpl:
    async def one(self):
        print("one")
        return 0

    async def two(self):
        print("two")
        return 99

    @staticmethod
    def create():
        aa = OB.create_sequence(a, name="A")
        bb = OB.create_sequence(b, name="B")
        
        #aa = seq_factory(a, name="A")
        #bb = seq_factory(b, name="B")
        s = Tpl()
        return Parallel.create(aa, bb, s.one)

Some points to note:

• Use seq.lib.nodes.seq_factory() to select the right method to instantiate a predefined sequencer script, so it can be reused.

Observation Blocks and Templates

Observation Blocks are defined through JSON files. A JSON file can store simple data structures and objects.

From the point of view of the Sequencer, an Observation Block is a sequence of templates that needs to be executed in the specified order. Therefore the sequencer is only concerned with the “templates” section of the JSON file. The Sample Observation Block file shows a simple Observation Block.

Sample Observation Block file

{
    "obId": 0,
    "itemType": "OB",
    "name": "obex 2",
    "executionTime": 0,
    "runId": "string",
    "instrument": "string",
    "ipVersion": "string",
    "obsDescription": {
        "name": "My humble II OB(A)",
        "userComments": "A",
        "instrumentComments": "AA"
    },
    "constraints": {
        "name": "string",
        "seeing": 0,
        "airmass": 0,
        "moonDistance": 0,
        "waterVapour": 0,
        "atm": "string",
        "fli": 0,
        "strehlRatio": 0,
        "skyTransparency": "string",
        "twilight": 0
    },
    "target": {
        "name": "string",
        "ra": "string",
        "dec": "string",
        "coordinateSystem": "ICRS",
        "comment": "string"
    },
    "templates": [
        {
            "templateName": "seq.samples.tpa",
            "type": "string",
            "parameters": [
                {
                    "name": "par_b",
                    "type": "integer",
                    "value": 0
                },
                {
                    "name": "par_c",
                    "type": "number",
                    "value": 77
                }
            ]
        },
        {
            "templateName": "seq.samples.tpa",
            "type": "string",
            "parameters": [
                {
                    "name": "par_b",
                    "type": "integer",
                    "value": 0
                },
                {
                    "name": "par_c",
                    "type": "number",
                    "value": 10
                }
            ]
        }
    ],
    "pi": {
        "firstName": "I",
        "lastName": "Condor",
        "email": "user@example.com"
    }
}

In the current version, only the templates section is used. The Python modules implementing the desired actions are defined with the templateName keyword. Many templates can be specified this way. Moreover, many instances of the same template can be requested, The samples JSON file shows this case with the seq.samples.tpa Python module.

Note

templateName value must be a valid Python module expressed in dot notation similar to the import performed from the python prompt. Therefore the Python module must be available on the PYTHONPATH.

The parameters section describes the runtime parameters (name, type, value) to be handed over to the Template instance and they can be accessed from the Template’s python code.

Accessing Template Variables

It is possible to access the value of the Template variables inside the Python code implementing a Template.

In order to access to template parameters like the ones defined in Sample Observation Block file one has to use the function get_param(). It knows the current template and request the parameters of interest. In the case of the example, the code to access par_b and par_c is as follows.

Access Template variables in Python

#!/usr/bin/env python3
"""
Shows variables


"""
import sys, inspect
import logging
import asyncio
from seq.lib.nodes import Sequence, Parallel, Template
from seq.lib.nodes import Action as _a
from seq.lib.nodes import ActionInThread as _ait
from seq.lib import logConfig
from seq.lib.nodes import get_param

import tkinter as tk
from tkinter import simpledialog


logger = logging.getLogger(__name__)

__seq_params__ ={"par_b": 11, "par_c":22}

class Tpl:

    async def do_sum(self):
        """Adds two integers"""
        parb = get_param("par_b")
        parc = get_param("par_c")

        logger.debug("Node sum")
        logger.info(
            "xSUM {} + {} = {}".format(parb,parc,parb+parc))
        await asyncio.sleep(1)
                
    async def delay(self):
        """Waits a sec"""
        logger.debug("sleep a bit")
        await asyncio.sleep(1)
            
    @staticmethod
    def create(*args, **kw):
        "Adds two variables"
        a = Tpl()
        return Sequence.create(
            a.do_sum,
            a.delay,
            name="TPL_SUM Example", **kw)

Inserting a basic dialog window

A basic dialog window is displayed when a specified condition is not met. It gives the user the possibility to perform an additional step to overcome the unfulfilled condition and proceed or to stop the sequence execution. An instance of BasicDialog shall be inserted in a sequence as below with a condition and an extra step provided by the user:

Insertion of a basic dialog window in a sequence

#!/usr/bin/env python3
"""Usage of a dialog node node"""
import logging
import asyncio
from seq.lib.nodes import Sequence, BasicDialog
from seq.lib import logConfig


LOGGER = logging.getLogger(__name__)

async def a():
    LOGGER.info("a")
    return "A"

async def b():
    LOGGER.info("b")

async def test_condition():
    await asyncio.sleep(0.5)
    return False

async def extra_step():
    await asyncio.sleep(3)
    print("What an extra step!")
    return True

def create_sequence(*args, **kw):
    """Builds my sequence"""
    logConfig(level=logging.INFO)
    c = BasicDialog(name="some_dialog", description="my first dialog", condition_label = "test condition", check_condition=test_condition, step_label="perform extra step", condition_extra_step=extra_step)
    return Sequence.create(a,c,b, **kw)