ACS Container

The ACS Container package is responsible for the implementation of the Container part of the Container-Component model of ACS. This includes design patterns and high level services to manage the life cycle of Components and Containers themselves [RD01 - 5.1.2. Procedures]. This high level management part is implemented in a specific sub-package called the Management and Access Control package.

Containers are supervised and managed by an ACS Manager. Manager and Containers cooperate to provide the following functionality:

• Instantiation and de-instantiation of Components.
  Instances of Components are created when needed or when requested and destroyed when not needed any more.

• System startup and shutdown.
  At system startup all needed Components are created and initialized in the proper order.

• Location of Components in the system.
  A client does not need to know where a Component resides and objects can be relocated when necessary, for example to recover from failure, for testing purposes or for load balancing.

• Version control.
  If a newer version of a Component is available, or if its configuration data changes, it must be possible to upgrade or reconfigure the service without interfering with its clients.
  Different versions of the same Component can be loaded or relocated for testing purposes without requiring changes in clients.

• Administration control.
  Administrative users must have an overview and control over the current state of the system, including existing services and clients, and must have the possibility to manually manage services. Administration functionality includes startup and shutdown of the whole ACS, Manager, Services and Containers.

Components will be registered in the CORBA Naming Service. Properties of Characteristic Components are not registered in the naming service, but are accessed by retrieving their CORBA references from the Characteristic Component that contains them.

Access to the CORBA Naming Service and Component's life cycle is handled by a Management and Access Control Interface. A Manager is the communication entry point for clients: it provides access to the CORBA Naming Service (with added security, see below) and delegates to Containers the task to manage the life cycle (code loading, creation and destruction) of Components, based on the request of services from the clients.

The CORBA Naming Service is completely open and anybody can read and write into it. The Manager hides the CORBA NS and takes care of reading and writing into it under stricter control. A non-administrative client cannot get access to all information in the NS, but only to what the Manager provides. Should ACS fully implement the "authorization control" in the future, use of passwords and controlled access to the Manager could be used.

Figure 3.12: Management and Access Control architecture

The basic entities managed by the Management and Access Control interface (MACI) are Components.

To the management system, the Component is an entity with a lifecycle that has to be managed. Components can be instantiated in two different ways:

• regular Components are instantiated on demand when they are first requested.

• startup Components are instantiated when the system is brought online, i.e. when the Container where they are supposed to live in is started.

A Component implements the functionality that allows the interaction between the MACI infrastructure and an underlying object. The underlying object can be a Java Component, a C++ Component or other type of objects, for example CORBA services, which have to be managed by MACI.

Every Component has a unique designation. Well-formed Component designations are Component URLs or CURLs. A CURL is a hierarchical name implemented as a string of arbitrary length that consists of the static prefix curl:, domain identifier and Component name. An example of a CURL might be curl://alma/antenna1/mount, representing the mount Component for ALMA antenna number 1. MACI provides name resolution that from a well-formed Component designation generates the Component reference that can be returned to the client

Regular components are kept active if and only if there are clients referencing them. The Manager keeps track of all references and periodically verifies that the clients are still alive and deactivates unreferenced components.

The Manager provides an API to forcefully deactivate Components even if they are still referenced. This is used when shutting down parts of the system or when replacing at run time the implementation of a Component with anew one.

It is possible to have "non-sticky clients", whose reference to used components is not accounted by the Manager when deciding if a Component needs to be activated/deactivated. This is used, for example, for passive GUIs that monitor Components or interact with them only if the are already active for other reasons. Such "non-sticky clients" do not force the activation of a component if it is not already active when they request the reference from the Manager. In this way starting/stopping a GUI is unrelated with the status of Components and subsystems can be started up and shutdown independently from the GUIs.

It is possible to define a timeout before a regular Component not referenced any more is deactivated. This is useful when there are multiple clients intermittently accessing a component for a long period of time and we do not want the server component to be all the time activated and deactivated. Notice that an "immortal Component", i.e. a component that is never deactivated after a first activation even is all clients release it, is essentially a component with infinite timeout for deactivation.

Manager is the central point of interaction between the Components and the clients requesting their services. 

Manager has the following basic functionality:

• It is the communication entry point. A client requesting a Component service can do so by querying the Manager. Security is tied to this functionality by requiring every client to pass the authorization protocol and subsequently tag every request to the manager with a security token generated by the manager upon successful authentication. Manager serves also as a broker for objects that were activated outside of MACI (non-Components). It provides a mechanism for binding and resolving references to such objects, relying on the CORBA naming service. 

• It performs as a name service, resolving CURLs into object references. If a CURL is passed that is outside the current Manager’s domain, the Manager forwards the request to the Manager closest to the destination domain to resolve. 

• It delegates the Component life cycle management to the Container objects and therefore creates no Components directly. However, it does maintain a list of all available Containers. 

• The Manager uses the configuration database to retrieve relevant configuration for individual Components in the domain, as well as locations of other Managers.

Manager is the only interaction that clients have with MACI. Thus, neither Component implementers nor GUI client developers need concern themselves with aspects of MACI other than the Manager.

Manager implementation is based on CORBA Naming Service and all references to Components are available to clients not aware of MACI functionality through the CORBA Naming Service. A CURL-to-Naming Context mapping allows a one to one correspondence between Components retrieved using MACI services or from the Naming Service.

Manager provides also process control of Containers and CORBA services, i.e. it will be able to start/stop containers and CORBA services upon request.

A demon process runs on every host where Containers or services can be started. The Manager interacts with this demon to instruct it to start Containers. The demon provides also other services (in part still TBD) like collection of statistical data such as CPU , memory and disk load or availability (Partially implemented).

Manager can also deploy "collocated" Components, i.e. it can be requested to "deploy a Component in the same Container of another, given, Component"

• This allows applications to implement deployment recipes, for example for load balancing or for selecting the deployment configuration that satisfies specific application needs, without having to know explicitly the deployment of Containers or, more in general, the topology of the system.

• For example

  • a pipeline application might deploy load balancing Components on all pipeline nodes at system configuration/

  • a load balancing master Component can request information from load balancing Components (disk space, memory, load, performance characteristics....)

  • based on this information can determine the best node where to run a specific pipeline stage Component

  • it can then deploy the pipeline stage "collocated" with the selected load balancing Component

  • this does not require to the application any explicit knowledge of the host and Container where the pipeline stage needs to be deployed.

  • the load balancing Component need to be deployed at configuration time or using administration tools written for that specific purpose, but transparently to applications.

The Pipeline/Offline teams define the requirements for process management by the Manager and described above, like redistribution of processes/containers for load balancing, dynamic allocation of components to container or redundancy and replication of components.

A Container serves as an agent of MACI that is installed on every computer in the control system.

Figure 3.13: Component activation sequence

• Every Container runs in its own process. 

• Manager sends the Container the request to construct/activate a specific Component by passing it the name, type and the path of executable code (depending on the implementation language) of the Component. 

• The Container loads the executable code and begins executing it. Depending on the implementation language, if the dependant executables are not loaded automatically by the operating system (as is the case on the VxWorks platform), Container loads them prior to executing any code. 

• The Container also deactivates Components, when so instructed by the Manager and is able to shutdown by disabling all Components. 

• Container maintains a list of all Components it has activated and is able to return information about an individual Component's implementation (such file path of the loaded code, version and build date). 

• Container implements the ContainerServices interface to allow the Components it hosts to perform their MACI-related tasks, such as issuing requests to the Manager and activating other CORBA objects and to get hold of all the general services they may need.
  Java, C++ and Python Containers will provide slightly different services, since the three languages are supposed to be used in different application domains.
  It is expected that new services will be added to one or the other Container based on the application requests.

• The Java ContainerServices interface defines a number of services. Look at the documentation for more details. The most important are:

  • Logger  getLogger ()
    Retrieves the logging system service object

  • void assignUniqueEntityId (EntityT entity) throws ContainerException
    Get a Unique Entity Id assigned by the archive to entity data

  • org.omg.CORBA.Object getComponent (String componentUrl) throws ContainerException
    Retrieves e reference to another Component. This exists in a number of variants.

  • void releaseComponent (String componentUrl)
    Releases another component

  • DAL getCDB()
    to access the Configuration Database.

  • Object getTransparentXmlComponent (Class transparentXMLInterface, org.omg.CORBA.Object
                       componentReference, Class flatXmlInterface) throws ContainerException
    Converts a "flat-XML" component interface to a "transparent-XML" component interface.

• The C++ ContainerServices is functionally equivalent to the Java one, but does not provide services equivalent to assignUniqueEntityId() and createXmlBindingWrapper(). The getLogger() method is provided directly by the Component base implementation classes and not through the ContainerServices.

• The Python ContainerServices is functionally equivalent to the Java one, but does not provide services equivalent to assignUniqueEntityId() and createXmlBindingWrapper().

There can be two types of Containers:

• Porous Container
  A Porous Container returns to clients directly the CORBA reference to the managed Components. Once they have received the reference, clients will communicate directly with the Component itself and the Container will only be responsible for the lifecycle management and the general services the Container provides to Components, like Logging.

• Tight Container
  A tight Container returns to clients a reference to an internally handled proxy to the managed Components. In this way the communication between client and Component is decoupled allowing us to implement transparently in the proxy layer extra security and optimization functionality, at the expense of an additional layer of indirection, with some
  performance implication.

Figure 3.14: Porous vs. Tight Container access sequence

ACS provides just one implementation for the Manager interface (in Java) and three implementations for the Container:

• C++ Container
  A C++ Container is a “porous” Container to allow better performance avoiding the extra level of indirection introduced by the tight Container wrapper. A C++ Component is assumed to be part of a Dynamically Loaded Library (DLL). Whenever the Manager requests a Component from a C++ Container, it passes the reference to the DLL to be loaded. The DLL is loaded dynamically and the Component instantiated and activated. When the Component is released, the DLL is unloaded.

• Java Container
  A Java Container is a "tight" Container. The Java Class for a Component is dynamically loaded on request by the Container and unloaded when not needed any more. The "tight" implementation allows the transparent implementation of the (de)-serialization of entity (data) objects that appear in the parameter lists of operations. This functionality is very important for high-level Java applications. This means that a Java "binding class" which is a type-safe representation of XML data is transported across processes in the form of an XML string, even though the client and server component only see the real Java classes. This is also true for sequences of entity objects, or structures that contain entity objects.
  There will be one JVM for each Java Container. In each Container we will have many Components, all running in the same JVM. This protects in a better way one Container from another and resource balancing is done re-deploying Components from one Container to another in the same or in another host.
  On the other hand, starting up and keeping alive JVMs is quite expensive. This is particularly important for user interfaces: a user can start at wish many GUIs on the same console, easily overloading the host. Therefore GUIs will have the capability of running inside the same JVM (when a new application/GUI is started, it looks for an existing JVM able to host it. This mechanism is implemented in ABeans 3. GUIs can be just clients for Components or they can be Components themselves.

• Python Container
  A Python Container is a "porous" Container. A Python package for a Component is dynamically loaded on request by the Container and unloaded when not needed any more. Python is used to implement Component that have a strong requirement for flexible and easily editable procedures. Typical examples are pipeline and data reduction recipes. Such recipes are often edited by astronomers or by operators also during operation and are therefore better implemented using a scripting language; the Python components typically use and interact with C++ and or Java Components that implement CPU intensive algorithms, for example interfacing with Fortran legacy procedures..

Specialized Containers

• It is possible to create new Container implementations for specific application purposes.

• ACS Containers are implemented as classes that can be inherited from and extended. For example, specific initialization of legacy systems or data reduction packages to be used together with ACS components can be delegated to specialized containers

• specialized Containers can also be used to implement statically linked ACS executables. The Container will look from the Client and Manager point of view as a standard Container, but the libraries implementing Components are statically linked and instances of Components can be predefined in the linked executable. This was discussed for the implementation of the Offline Data Reduction Framework and ACS can, upon request, provide a standardized implementation for a few specialized containers.

• The Task (see Task section) is a specialized Container that can be run as an executable and does not require Manager or ACS services.

Every client of a Component service that is not itself a Component shall implement an interface called maci::Client.

• The Client interface allows the client to act as a secure party in the communication with the Components, to receive general-purpose string messages from the MACI components and to be notified when any change happens to the Components that the client utilizes. 

• The log in and other requests are issued to the Manager, which serves as a portal to other services.

• Each Client logs in to the MACI system before any other requests are made, and in turn it obtains a security token, which it must use in every subsequent request to the MACI (Partially implemented).

• The authentication can be implemented using a plugin design. If the Archive User Repository is available and if desired, the corresponding plugin will allow to authenticate clients based on the User Repository (Implementation not foreseen for ALMA).

maci::Administrator is a special-purpose client that can monitor the functioning of the domain that it administers. Monitoring includes obtaining the status of the Components as well as notification about the availability of Components.

MACI allows organizing Components hierarchically and handling startup and shutdown dependencies between objects.

• Whenever a client needs a CORBA reference for a Component, a request to Manager is done for the corresponding Component.

• If the object is not already instantiated, the Manager asks the Container to create it.

• When an object contains hierarchical references to contained objects, the dependency is expressed via Component URLs and resolved through requests to the Manager. In this way, the Manager can automatically achieve instantiation of not already active nested objects. This guaranties that all objects are automatically created in the right order and when needed.

• Some objects are needed immediately at bootstrap. They are directly specified in a Manager configuration table (stored in the Configuration Database) and the Manager instantiates them as soon as the Container responsible for their deployment is bootstrapped.

• If there is a root top-level object, just putting this object in the Manager table will trigger a cascade instantiation of all dependent objects.

The Manager is the only responsible for providing references to Components.

• It is not allowed to directly pass a CORBA reference to a Component from one Component (or more in general, client) to another.

• Whenever a Client needs to access a Component, it shall request the reference by name from the Manager, by using the service calls provided by the ContainerServices, or by using directly the Manager's IDL interfaces

• If a Client needs to pass to another client information about a Component to be accessed, this shall be done by passing the Component's name. The Client will then have to get the component reference from the Manager using this name.

• In exceptional situations, Components need to instantiate objects that are accessible remotely, and need to pass them around to other Clients. The ACS::OffShoot interface is a base interface defined for this purpose. ACS::OffShoot objects are instantiated by a Component, who is responsible for their whole lifecycle. Their lifetime is limited to the lifetime of the Component who created and handed them over to clients. Mostly ACS:OffShoots are used internally by ACS, for examples ACS:Callback and ACS:Monitor objects.

• This is because the Manager needs to keep directly track of all the Client-Component associations to be able to handle failures and restarts of Components and to allow administrator Clients to get a realistic picture of the status of the system.
   

An Object Explorer User Interface tool is provided to navigate the hierarchy of Components on the naming hierarchy.[RD01 - 5.1.3 Browser]

• The Object Explorer is equivalent and covers the requirements of the Device Browser Tool described in the [RD45 - EVLA Engineering Software Requirements, 1.1 Accessing a Single Device]. In order to cover all detailed requirements expressed in [RD45], extensions to the Object Explored will need to be implemented.

• All objects in the system can be reached by navigating the hierarchy and all object information can be retrieved and edited, including accessibility for a given user . For example, it is possible to graphically browse the hierarchy of Components in the system, based on the naming hierarchy, reach every single Component and view/edit all values of Properties and Characteristics.

• The Object Explorer allows to browse the interface [RD45 - 1.1-R4] and send any command supported by a Component [RD45 - 1.1-R5]

• It is possible to monitor and draw trend-plots for each property [RD45 - 1.1-R3]

• The Object Explorer uses the CORBA Interface Repository to retrieve information on the interfaces provided by the Components in the system.

• Requirements in [RD45 - 1.1] not covered by the current Object Explorer implementation are:

  • 1.1-R2 - Tabular representation of multiple monitor points

  • 1.1-R3 - Plot multiple monitor points on the same plot (this is possible with the ACS Sampling System)

  • 1.1-R5.4 - Send a command repetitively

  • 1.1-R5.5, 5.7- Save, read and execute sets of commands from file

  • 1.1-R8 - Save tabular data with monitor values (this is possible with the ACS Sampling System)

  • 1.3 Accessing Multiple Devices

An Administrator User Interface (ACS Command Center) tool is provided. The Command Center:

• allows to start/stop ACS Services, Manager, Containers and Components on the local system or remotely

• displays the information about objects in the system. This includes Components, Managers and Containers. Both currently active Components and potentially active Components (i.e. Components that the Manager is able to bring online on request) are displayed. 

• interacts with the Manager through IDL maci::Administrator interface, by receiving notifications about other clients and activators in the system from the Manager.

• acts as a central user interface for starting up administration utilities (logging client, object explorer, event monitor and others), providing a configurable menu structure.

• makes available to developers an API and GUI elements to implement administration applications and GUI.
  The ALMA Executive is an application based on this API and GUI building blocks.