Tutorial

The purpose of the sequencer is to execute Sequencer scripts, either for engineering or science purposes.

The unit of execution is a single step which is just a normal python function or method with no input parameters.

Sequences are modeled as Directed Acyclic Graph (DAG). Each node in the graph can either be a simple action which just invokes a single sequencer step or a more complex node which contains a complete sequencer script.

This allows sequences to be grouped and nested freely. Ultimately they will execute steps.

The Sequencer API allows to create these graphs.

Building Sequences

Sequencer scripts are modeled as DAGs, the Sequencer API allows to create nodes in the DAG. There are different node types that determine the way its children are scheduled for execution (e.g. Parallel, Sequential), you are free to select, mix and match the node type(s) that suits better your needs.

The Sequencer Engine expects to find either a module method or a specific class name in a module in order to construct a Sequencer script from it. The conventions are the following.

1. A module level create_sequence() function. See Tutorial 1 (a.py).

2. A class named Tpl which must provide a static method create_sequence. See Parallel tasks sample.

In either case, the return value is the root node of the graph being implemented.

The first convention is tried first, if no create_sequence() function is found, the second convention is attempted.

For very simple scripts, following the first convention is perfectly fine. For more complex scripts, e.g. one is creating a class and methods to control .e.g. a detector, and it is going to be instantiated many times to control many detectors in parallel, the class approach is advised.

Node Dependencies

Regarding the order of execution, the DAG edges allows to represent the nodes dependencies. Each node in the DAG depends, from its parent nodes. Meaning that a node will not be started until every node that precedes it has finished its own execution.

Node dependencies determines the execution order of the sequence steps. A node wont’t run until all its dependencies have finished. In the graph, a node dependency is seen as an icoming edge.

Chaining the basic node types Sequence and Parallel defines a dependency hierarchy.

Sequence nodes are executed one after the other, therefore each node depends on its predecessor. On the other hand, Parallel nodes indicates that all nodes it contains shall be executed together. By combining and chaining this two basic node types it is possible to express any dependency graph.

Very simple sequences

Let’s start with a simple sequence.

As mentioned before the simplest step is a python coroutine with no input parameters which is used to create an Action node.

Note

The no input parameter rule can be bypassed with strategies shows in Passing Arguments to Actions

In this case, we define a sequence that executes two steps, one after the other, namely do_a() and do_b(). Source codes is shown below.

Tutorial 1 (a.py)

#!/usr/bin/env python
"""
Simple example.

Executes function a() and b()
"""
import logging
import asyncio
from seq.lib.nodes import Sequence
from seq.lib import logConfig

LOGGER = logging.getLogger("A")

async def a():
    """Simply do_a"""
    await asyncio.sleep(1)
    LOGGER.info("a")
    return "A"

async def b():
    """do_b example
    """
    LOGGER.info("B")
    return "B"

def create_sequence(*args, **kw):
    """Builds my sequence"""
    #print("logger setup")
    #LOGGER.debug("hola poh")
    return Sequence.create(a, b, **kw)

Important

The Sequencer Engine is based on asyncio library, therefore it is biased towards coroutines, but they are not mandatory as shown in Loop example.

There are some simple rules to create sequences:

• A step (Action) is a python coroutine with no input parameters. See Passing Arguments to Actions to break this rule.

• In this case, the sequence is created using the Sequence.create (Sequence) constructor, which receives the steps to be executed (in the given order).

• The python module which contains the sequence must define a create_sequence function as shown in the example. It returns a Sequence node that holds the nodes it will execute.

Important

The Sequence.create constructor provides syntax sugar in order to support passing coroutines as the sequence’s graph nodes. In such a case a node of type seq.nodes.Action is automagically created and inserted in the graph.

This simple sequence 01 is graphically shown below. It can be imported from python as:

>>> import seq.samples.a

simple sequence 01

Notice that in the create_sequence() function only two nodes were specified, however the simple sequence 01 figure shows 4 nodes. The sequencer engine adds a start node (black circle) and end node (double black circle) to every node container type, i.e. those nodes that have children: Parallel, Sequence and Loop.

Note

The start and end node, among other things, makes easy to chain nodes together by linking the end node of a container with the initial node of the next.

Executing Tasks in Parallel

One is not limited to create just linear sequences. Parallel activities (pseudo parallel) can be created using the Parallel.create() constructor. It receives the same parameters as the Sequence node constructor. When executed, the sequencer engine processes the Parallel nodes children in parallel.

Parallel tasks sample

"""
Parallel nodes example.
"""
import asyncio
import random
import time
from seq.lib.nodes import Parallel, ActionInThread

class Tpl:
    """A sample Sequence"""

    def a(self):
        """sleeps randomly"""
        t = random.randrange(5)
        time.sleep(t)
        print(" .. done A")

    async def b(self):
        """sleeps randomly"""
        t = random.randrange(5)
        await asyncio.sleep(t)
        print(" ... done B")

    @staticmethod
    def create(**kw):
        """Builds my sequence"""
        a = Tpl()
        p = Parallel.create( ActionInThread(a.a), a.b, **kw)
        return p

Which is represented graphically as follows.

Parallel Sequence

Points to notice:

1. In this case the Sequencer Engine discover a class named Tpl and calls its create method (@staticmethod as the convention mandates).

2. The example Parallel Sequence also shows that steps are not limited to coroutines. Just wrap it in ActionInThread node.

3. There is no problem mixing normal routines and asynchronous code. The sequencer will send the normal code to a separate thread and execute it there.

In order to avoid normal methods or functions to potentially block the asyncio loop (by holding th e CPU) they must be executed on their own Thread.

This is achieved with the ActionInThread node. In the example the b() method is wrapped in such node.

Executing Tasks in a Loop

The Loop node allows to repeat a set of steps while a condition is True.

Loop example

"""
Implements a loop.
The condition checks Loop's index < 3.
"""
import asyncio
import logging
import random
from seq.lib.nodes import Loop

logger = logging.getLogger(__name__)


class Tpl:  # Mandatory class name
    async def a(self):
        """sleeps up to 1 second"""
        t = random.random()  # 0..1
        await asyncio.sleep(t)
        logger.info(".. done A: %d", Loop.index.get())

    async def b(self):
        """sleeps up to 1 second"""
        t = random.random()  # 0..1
        await asyncio.sleep(t)
        logger.info(" .. done B: %d", Loop.index.get())

    async def c(self):
        pass

    async def check_condition(self):
        """
        The magic of contextvars in asyncio
        Loop.index is local to each asyncio task
        """
        logger.info("Loop index: %d", Loop.index.get())
        return Loop.index.get() < 3

    @staticmethod
    def create(**kw):
        t = Tpl()
        l = Loop.create(t.a, t.b, t.c,
                        condition=t.check_condition, **kw)
        return l

Which is represented graphically as follows.

Loop example

As the code in Loop example shows, the Loop’s constructor takes a variable number of arguments as the Loop’s body, the part that is repeated, do_a(), do_b() and do_c() in this case. The condition node is specified with the condition keyword.

The Loop’s index is kept in the context variable index, meaning it can be accessed as Loop.index.get() as the my_condition() function shows.

In order to separate the index value of the different loops that might be occurring at the same time the Loop’s index is implemented as an asyncio context variable. Therefore to get its value one has to call its get() method as the my_condition() function shows.

Embedding Sequencer Scripts

Sequences can be reused or embedded in order to produce more complex activities. The following example uses the sequences “Tut_01” and “Tut_02” to create a new sequence that executes them in Parallel and just for the kicks adds an step from a local class.

Important

Embedding a Sequence entails to import the module and instantiate its Sequencer script (either with create_sequence() or by Tpl.create(). But how do you know which one to call? You don’t have to know. You let the OB object do it for you as:

from seqlib.ob import OB
from seq.samples.a

mynode = OB.create_sequence(a)

Sequence embedding example

#!/usr/bin/env python
"""
Simple example.

Uses nodes from template defined in module 'a'.
It also uses the 'a' template as a whole.
"""
from seq.lib.nodes import Parallel
from seq.lib.ob import OB
from seq.samples import a
from seq.samples import b

class Tpl:
    async def one(self):
        print("one")
        return 0

    async def two(self):
        print("two")
        return 99

    @staticmethod
    def create():
        aa = OB.create_sequence(a, name="A")
        bb = OB.create_sequence(b, name="B")
        s = Tpl()
        return Parallel.create(aa, bb, s.one)

Some points to note:

• Use seqlib.ob.OB.create_sequence() to select the right method to instantiate a predefined sequencer script, so it can be reused.

Conclusion

This finishes the basic tutorial. One can create any type of flow using the node types show. Mor details about node’s attributes and context are given in A Deeper Look.