Architecture

Deployment model

ComodIT is built around a client - server - agent architecture. This architecture is often seen in IT Automation tools, having a centralised place to interact with the user, and coordinate actions on multiple hosts, together with an agent running on the managed hosts. The difference between IT Automation tools often lies in the subtle balance between what is done in the central server and what is delegated to the agents.

Within ComodIT, our design philosophy has been to keep all the decision making and orchestration centralized while delegating to the agents the knowledge of their platform, abstracting resources and operational details.

The orchestrator

The ComodIT orchestrator maintains the system state in a database, expose a client API to manipulate the system state, and send commands to machine agents to keep the infrastructure in sync with the state.

While some automation tools rely on protocols like SSH to connect to the managed hosts, we have decided to use an AMQP messaging bus to link the orchestrator to its machines. This provides us with various benefits such as always on connection, realtime notifications and feedback, queuing, auto-discovery of hosts, firewall traversal and more.

Queuing

Messages are routed through a messaging server, being stored in queues while waiting for delivery. This means that if the server sends a message to a machine that is offline or busy, the message won’t be lost, instead they will be queued and delivered when the machine is ready.

Realtime status updates and compliance notifications

The orchestrator also receives status updates, feedback on changes, and compliances notifications from the agents. These messages are available through the API and enable the client to display state information, error messages, or wait for a pending change to be applied.

The compliance alerts signal to the server that a resource on the managed host has changed of state and is no longer in compliance with the system state. In the current version of ComodIT, this is used to display an alert to the user. In the future ComodIT could directly take corrective actions to bring the host back in compliance.

Going behind the firewalls

An interesting aspect of the message bus is that the agent is the one connecting to the infrastructure. This means that the host does not need to be reachable by ComodIT. It can sits behind a firewall, does not even need its SSH port open. As long as it can connects to the ComodIT messaging server, it can be managed. This is why the SaaS version of ComodIT available at my.comodit.com can also be used to manage hosts on the company premises, behind the firewalls.

Autodiscovery

A final benefit of the messaging approach is the host auto-discovery. This feature enable you to add the agent on an existing machine (or a machine provisioned outside of ComodIT) and have it automatically discovered and added in the ComodIT interface. If the agent is configured with the hypervisor management plugin, the platforms managed are also automatically added to ComodIT.

The machine agent

The machine agent is lightweight daemon running on the managed host and connecting to the ComodIT orchestrator over a messaging bus. This agent exposes the various resources of the machine, organized into collections (files, packages, services, users, …) with a RESTFull approach. These resources can therefore be manipulated to change their state on the host, while new resources can be easily added.

A RESTFull API with an abstract data model

The managed resources are described in JSON messages, in a platform agnostic way. This means that they are abstracted from the underlying operational differences between platforms. For example, if you want to add a package on a host, just add a package resource to the package collection, specifying the package name in the JSON payload. The agent will figure out how the package should be installed (e.g. picking between yum, apt, zypper,… depending on the platform).

Having a RESTFull approach coupled to a resource level abstraction as the agent API is what enables the simple descriptive approach used in ComodIT. The architect writing the template can focus on describing the resources required by the application while knowing that the agent will figure out the operational details.

Compliance tracking at the agent level

As we have just seen, the orchestrator can manipulate the resources on the managed host through the agent API by adding or updating resources within collections exposed by the agent. When processing these requests, the agent takes all required operational actions to bring the resource in compliance with its description, but it also stores the resource description in a local storage and monitor the actual resources for compliance with their initial description.

For example, when the orchestrator adds a package (optionally with a given version), on the managed host, the agent will install the package and then monitor the package collection for changes. If the package gets removed by out-of-band means (intentionally or not), the agent will detect the missing resource and trigger a compliance alert to the orchestrator.

Pluggable architecture

The agent is written in Python, a lightweight and powerful scripting language. It is architectured around a plugin engine, making it easy to support additional types of resources, by adding support for new collections. While enabling the proper abstraction by adding platform specific ‘providers’.

For example, one could create a ‘cron jobs’ collection to manage the cron jobs of a host. The first step would be to define a JSON data model that abstract the cron job, and then develop ‘providers’ than can translate the abstract description into operational task to execute on the managed host, depending on its platform.

The clients

The client API exposed by the orchestrator is REST based with a JSON messages. It is well documented and can be used to integrate ComodIT with third party products and to build different clients supporting specific user needs (e.g. a self-service IT portal). It can also be used to orchestrate complex scenarios (either directly invoking the API or using a helper library).

Data model

The system state managed by ComodIT is described in a rich object oriented model. Figure xxx shows the key classes and their relations. This data model is exposed in the API through a set of views described in the API documentation.

Model entities

User

Represent an individual user of ComodIT. The user is identified by a unique username and described by a set of attributes (such as email, name, etc.). A new user can be created through the registration process or added by an administrator through the management dashboard.

Organization

The organization is the container into which the system state reside. A user can create a new organization, or be assigned to an existing one with a role. Organizations are strictly isolated within ComodIT and represent the layer of access control. This enable ComodIT to work in a ‘multi-tenant’ mode, with each customer having one or more organization, to which users are given rights.

In the current version of ComodIT, a user can have two roles within an organization:

  1. User: Can perform any operational tasks within the organization

  2. Administrator: In addition to the user access rights, an administrator can also manage the organizations users and their rights. In the future, new role will be provided such as auditor (read-only access), architect (rights to manage applications), etc.

Environment

Environments are used to segregate hosts within an organization. For example separating the development from the production environment; or organizing hosts across geographies. the environment also act as a container of settings, shared by all hosts within the environment. In the future, environments could be also be used for access control purposes

Host

A host is a definition of a machine that can be provisioned within the infrastructure. It consists of an operating system definition, a set of applications and a target platform. Together with their respective configuration settings. It is important to understand that within ComodIT, a host represent a blueprint to build a machine, not the real underlying machine that has been provisioned using this blueprint.

A host can therefore be in three states:

  1. Defined: No real corresponding machine instance has been provisioned yet, the host is just a blueprint of a machine to be provisioned.

  2. Provisioning: A provisioning process has started.

  3. Provisioned: There is a real machine instance associated with this host.

Instance

An instance represents an actual machine that has been provisioned by ComodIT (or added through the auto-discovery mechanism). The instance describes the state of the underlying machine (e.g. running, stopped, uptime, …) and its properties (e.g. hostname, IP, etc.). In the case where the instance is a virtual machine, it can be controlled through the instance object (e.g. stopped, reboot).

Platforms

Within an organization, one can configure platforms which represent physical environments on which machine can be provisioned (e.g. a hypervisor, a cloud engine, a pxe setup,…). ComodIT proposes various drivers enabling to connect to various kind of platforms.

The configuration of a platform consist of the driver to use and its settings, but can also require templates files that are used by the platform to execute the provisioning operations. For example, the libvirt driver requires two XML files that are used by the driver when provisioning a machine. While the PXE/gPXE driver requires a template of the pxe configuration file to write in the TFTP server.

When a platform is added through the web interface, default values and templates are provided to simplify configuration.

Distribution

A distribution is an operating system that can be installed on a machine, together with all required files to perform the provisioning. In case of Linux distribution, it usually consist of a path to a kernel and initrd.img but also a template for a kickstart, pre-seed or autoyast depending on the flavor being installed. In this case, the distribution can be platform independent. This is the prefered mode of provisioning of Linux systems within ComodIT.

A distribution can also be packaged as a platform dependent ‘gold image’ such as an Amazon AMI, a KVM source image, etc. In this scenario the distribution comprises of a path to the source image and some platform dependent settings. Using a gold image is the only way to deploy a windows based system within ComodIT. In the future, ComodIT will be able to perform unattended install of Microsoft Windows Server 2008, using an ISO and the Automated Installation Kit (AIK).

Application

An application bundles a group of resources that can be installed on a host. These resources can be:

  1. files: describing the path, the permissions, and a template to be rendered

  2. packages: a package to be installed

  3. services: a service and its state (e.g. running at startup)

  4. users and groups: describing attributes of a user or a group to create on the machine. (not yet available in current version).

  5. repositories: additional repositories that should be created on the machine

In addition to these resources, an application can also define a set of ‘handlers’ that should be triggered when a setting of an application is changed.

Parameter

The platforms, distributions and applications can all expose a set of parameters. These are variables that can be changed by the user when associating the platform, distribution or application to a host. Examples of parameters would be the memory size of the virtual machine, the root password of the distribution, the port of a server application, etc.

Parameters can have a default value, which is used when the user does not assign a specific value. Parameters also have a type and an optional schema. These enable to precisely describe the nature of the value (string, boolean, number, array, complex object, etc.).

Putting it all together

When a new host is defined, it is associated to a platform, a distribution, and a set of applications. Since these entities expose some parameters, it is possible for the user to assign custom values to these parameters. We refer to these as settings. These settings are defined with a context.

As an example, if a host is associated to a libvirt based platform, the user has the possibility to assign a value to parameters such as the memory, disk size, architecture, etc. This is done by adding some settings within the context of the platform.

In a similar way, when a platform is created and associated to a driver, the configuration of the driver (e.g. target hostname, API passwords, etc.) are provided through a collection of settings.