Configuration templates for data streams

Predefined configuration templates are available for data streams:

Name	Purpose	Comments
Without template	Receiving primary JSON events without further processing by synthetic triggers (push)	It is possible to add both your own and pre-installed handlers.
AnyStream default	Receiving primary JSON events with further processing by synthetic triggers (push)	It is possible to add both your own and pre-installed handlers.
Zabbix default	Getting alerts from Zabbix system (pull)	Checked compatibility with Zabbix versions 2.4+.
SCOM default	Integration with Microsoft System Center Operations Manager (pull)	Checked compatibility with SCOM v.1806-1809.
Prometheus default	Receiving events from Prometheus system(push)	Receiving data through the public API of the Monq Collector.
ntopng default	Receiving events from ntopng system(push)	Receiving data through the public API of the Monq Collector. Versions 4.0+
Nagios default	Receiving events from Nagios XI system (pull)	Tested for compatibility with Nagios 4.3.x-4.4.x
vCenter default	Getting data about topology changes in vCenter system (pull)	Verified compatibility with vCenter 6.7

Configuration "Without template"

The configuration template "Without template" is available to users to build his own algorithm for processing the incoming stream, by writing his own handlers in Lua, or selecting handlers from the list provided.

Tasks

There are no predefined tasks for this configuration template. Users can add new tasks.

Handlers

There are no predefined handlers for this configuration template. Users can add new handlers.

Configuration "AnyStream default"

The configuration template "AnyStream default" is used for all non-standard source connections. The principle of transferring primary data is based on an authorized POST request to the Monq public API.

Events received by such a stream are sent for processing to the "Synthetic Trigger" prefilter. If this processing is not required, select the configuration template "Without template".

Tasks

There are no predefined tasks for this configuration template. Users can add new tasks.

Handlers

In the template "AnyStream default" there is a predefined handler - Routing for Any stream, which puts the label postRouting: cl.stream-ready-event.new for further event routing. Users can add new handlers to the existing one.

Configuration "Zabbix default"

The configuration template is designed to integrate the Monq system with a Zabbix monitoring system.

Tasks

• "Zabbix - Version Check" - getting the current version of the connected Zabbix server.
  • Agent - SharedAgents - system agent by default, can be changed to an external agent.
  • Default launching interval is 5 min.
• "Zabbix - Api Connection Check" - checking connection status with the Zabbix API.
  • Agent - SharedAgents - system agent by default, can be changed to an external agent.
  • Default launching interval is 30 sec.
• "Zabbix - Events Data Flow" - getting data from Zabbix.
  • Agent - SharedAgents - system agent by default, can be changed to an external agent.
  • Default launching interval is 10 sec.

Users can add new tasks to the preset ones.

Handlers

In the template "Zabbix default" there is a pre-installed handler - Routing for Zabbix stream, which sets the following labels for further event routing:

• type: Zabbix,
• postRouting: cl.stream-ready-event.zabbix.new

labels_add('type', 'Zabbix')
labels_add('postRoutings',{'cl.stream-ready-event.zabbix.new'})
next_step(source)

Users can add new handlers to the existing one.

Configuration "SCOM default"

The configuration template is designed to integrate the Monq system with an SCOM monitoring system.

Tasks

• "Scom - Events Data Flow" - getting data from SCOM.
  • Agent - SharedAgents - system agent by default, can be changed to an external agent.
  • Default launching interval is 10 sec.
• "Scom - Database Connection Check" - checking connection with the SCOM database.
  • Agent - SharedAgents - system agent by default, can be changed to an external agent.
  • Default launching interval is 30 sec.

Users can add new tasks to the preset ones.

Handlers

In the template "SCOM default" there is a pre-installed handler - Routing for Scom stream, which sets the following labels for further event routing:

• type: SCOM,
• postRouting: cl.stream-ready-event.new.

labels_add('type', 'SCOM')
labels_add('postRoutings',{'cl.stream-ready-event.new'})
next_step(source)

Users can add new handlers to the existing one.

Configuration "Prometheus default"

The configuration template is designed to integrate the Monq system with a Prometheus-AlertsManager monitoring system.

Tasks

There are no predefined tasks for this configuration template. Users can add new tasks.

Handlers

The following handlers are available in the configuration template "Prometheus default":

• "Parser for Prometheus stream" for parsing the incoming stream in a format convenient for Monq.

  function is_array(t)
     local i = 0
     for _ in pairs(t) do
        i = i + 1
        if t[i] == nil then return false end
     end
     return true
  end
  function convert_date_time(date_string)
     local pattern = "(%d+)%-(%d+)%-(%d+)(%a)(%d+)%:(%d+)%:([%d%.]+)([Z%p])(%d*)%:?(%d*)";
     local xyear, xmonth, xday, xdelimit, xhour, xminute, xseconds, xoffset, xoffsethour, xoffsetmin
     xyear, xmonth, xday, xdelimit, xhour, xminute, xseconds, xoffset, xoffsethour, xoffsetmin = string.match(date_string,pattern)
     return string.format("%s-%s-%s%s%s:%s:%s%s", xyear, xmonth, xday, xdelimit, xhour, xminute, string.sub(xseconds, 1, 8), xoffset)
  end
  function alerts_parse(result_alerts, source_json)
     for key, alert in pairs(source_json.alerts) do 
        alert["startsAt"]=convert_date_time(alert["startsAt"])
        alert["endsAt"]=convert_date_time(alert["endsAt"])
        result_alerts[#result_alerts+1]=alert
     end   
  end 
  local sources_json = json.decode(source)
  result_alerts = {};
  if (is_array(sources_json)) then
     for key, source_json in pairs(sources_json) do 
        alerts_parse(result_alerts, source_json)
     end
  else
     alerts_parse(result_alerts, sources_json)
  end
  next_step(json.encode(result_alerts))
  

• "Routing for Prometheus stream" for setting up further routing of the data by adding proper labels.

  labels_add('type', 'Prometheus')
  labels_add('postRoutings',{'cl.stream-ready-event.new'})
  next_step(json.encode(source))
  

Configuration "ntopng default"

The configuration template is designed to integrate the Monq system with an ntopng monitoring system.

Tasks

There are no predefined tasks for this configuration template. Users can add new tasks.

Handlers

In the template "ntopng default" there is a pre-installed handler - Routing for Ntop stream, which sets the following labels for further event routing:

• type:Ntop,
• postRouting:cl.stream-ready-event.new.

labels_add('type', 'Ntop')
labels_add('postRoutings',{'cl.stream-ready-event.new'})
next_step(source)

Users can add new handlers to the existing one.

Configuration "Nagios default"

The configuration template is designed to integrate the Monq system with a Nagios XI monitoring system.

Tasks

• "Nagios - Events Data Flow" - getting data from Nagios XI.
  • Agent - SharedAgents - system agent by default, can be changed to an external agent.
  • Default launching interval is 10 sec.
• "Nagios - Api Connection Check" - checking connection to the Nagios database.
  • Agent - SharedAgents - system agent by default, can be changed to an external agent.
  • Default launching interval is 30 sec.

Users can add new tasks to the preset ones.

Handlers

In the template "Nagios default" there is a pre-installed handler - Routing for Nagios stream, which sets the following labels for further event routing:

• type:Nagios,
• postRouting:cl.stream-ready-event.new.

Users can add new handlers to the existing one.

Configuration "vCenter default"

The configuration template is designed to integrate the Monq system with the virtualization management system VMWare vCenter.

Tasks

• "vCenter - TopologySync" - provides the transfer of data about the entire vCenter topology available to the user.
  • Agent - SharedAgents - system agent by default, can be changed to an external agent.
  • Default launching interval is 24 hours.
• "vCenter - EventsDataFlow" - provides transmission of events about changes in the vCenter topology.
  • Agent - SharedAgents - system agent by default, can be changed to an external agent.
  • Default launching interval is 30 sec.

Users can add new tasks to the preset ones.

Handlers

In the template "vCenter default" there is a pre-installed handler - Routing for vCenter stream, which sets the following labels for further event routing:

• type:vcenter,
• postRouting:cl.stream-ready-event.new.

Users can add new handlers to the existing one.