# Monq Structure

The document describes the composition of the monq modules, the purpose of each of the modules and the relationship between the modules.

# Modules

Monq software consists of the following components:

  • Module for receiving and preprocessing data from monitoring systems
  • Status calculation module
  • Reaction module
  • Data storage organization module
  • Infrastructure
  • Synthetic monitoring module
  • Service model
  • Public REST API
  • OAuth2.0 + OpenId
  • User interface

Image

# Data receiving and preprocessing module

The module is designed to receive data from information systems of monitoring and logging, with the purpose of their further storage and processing. It allows to receive data in both structured form (JSON), and unstructured (plain text). The module also has capabilities to process the received data using preprocessing scripts to form structured messages.

# Status calculation module

The module is responsible for processing received messages, creating rules for synthetic triggers (ST) using scripts in the Lua language, calculating trigger statuses according to the created rules, and calculating CI statuses according to the statuses of active STs.

# Reaction module

The reaction module consists of the Rules and Actions and My Scripts submodules.

Rules and Actions allows users to set rules for handling events of changes in ST status, and also configure actions as a reaction to validated rules. As actions, you can assign operations to notify users by e-mail and messengers, registration of incidents in ServiceDesk systems and execution of scripts that support launching cURL, SMTP, SNMP, SSH and commands in Lua language.

# Data storage organization module

Data storage organization module – DBMS ClickHouse stores incoming events from external systems, and the connector management component checks the availability of active connectors, cleans the storages from irrelevant data and collects new incoming data.

# Infrastructure

The infrastructure part of monq, along with the description of the system software, includes several submodules:

  • System-wide settings module – contains a set of services that are responsible for the settings of the software instance, which are not related to direct processing of events:
    • Users and workgroups services.
    • Storage and management of monq instance settings – settings of the visual part (icons, labels, styles), system settings (access to the database for microservices, logging level).
  • Module for storing and processing logs of user actions.

# Functional monitoring module (fmonq)

Fmonq contains the API for the receiver of reports on the execution of functional test assemblies, parser for converting reports to fmonq model, API for Synthetic monitoring project management and metadata storage. Denormalized assemblies are stored in the ClickHouse database. Integration in the trigger model part is done using Zabbix.

# CMDB and Service model

The module is designed to store information about the user's IT environment, about IT services and their relationships. It is the control module for the rest. Provides the ability to build relationships between model elements and supports management and configuration functions.

# Public API

Public API allows to interact with the software without a web interface using the software HTTP REST API interface.

# OAuth2.0 + OpenId

Security service for exchange of data between the system components or users.

# User interface

Web interface for user interactions with the software.

# Interaction of modules

In general, the set of modules and the principles of their interaction can be divided into 2 types:

  • Modules that handle data streams, where data processed in one module is passed onto the next one for processing – such modules include Module for receiving and preprocessing data from monitoring systems, Status calculation module, Reaction module, Synthetic monitoring module.
  • Modules that provide support in the form of storing settings, service data links, and service services – such modules include CMDB and Service model, Infrastructure, Data storege organization module.

The source of monq data are systems and tools for monitoring or logging that track necessary metrics and generate primary events. Such metrics can include errors of execution of functional testing scripts of the FMONQ module.

The module for receiving and preprocessing data receives data from sources in JSON format and converts it into a data structure – integral events used internally in monq. The preprocessed events are then placed in the storage based on the ClickHouse DBMS.

Then the data goes to the status calculation module for processing: if among the synthetic triggers there is one under whose prefilter the received event fits, the trigger status is calculated in accordance with the rule implemented inside the trigger in the Lua language and an event is generated inside monq. If the trigger is bound to at least one configuration item, the CI operating status is recalculated according to the ST status.

Each new event is checked by the reaction module for compliance with the conditions (Rules), given by users. If the event meets the necessary conditions, a set of operations of the Action bound to the Rule is launched aimed at eliminating the incident, notifying interested or responsible persons, launching a self-healing script, etc.

Once from the primary monitoring system a message about the completion of the event is received - the event is flagged as completed in monq, which can also be processed by the reaction module. For some types of integrations (Zabbix, SCOM, vCenter) special tasks are developed for receiving events from their APIs or directly from their database and sending the received events to the monq preprocessing module. These tasks are performed with a predetermined interval, and are configured directly in the monq database for the various integrations types.

Configuring relationships of objects with each other, such as CIs, synthetic triggers, data sources, functional testing projects, is done by users in the CMDB and Service model module.

User access to the software is controlled by the security service (OAuth2.0, OpenId), while access to objects and sections of the software is controlled by the System-wide settings module that stores information about users belonging to particular workgroups in particular roles, and what kind of access rights those workgroups and roles have. The module for storing and processing logs also stores the history of user actions.

All user actions are performed in the graphical web interface of the software, which, through the API, sends commands to the server.