Zabbix - manual openstack integration

TAGS: zabbix openstack

Zabbix openstack manual installation/integration.

Intro

  • use standart zabbix fuel plugin or you will regret
  • look after zabbix server/client version accordance (zabbix 3, zabbix-agent 1.3, for example)

What we need to do:

  • install agents on nodes
  • install zabbix server on one node
  • configure all agent scripts and configs in /etc/zabbix
  • configure server configs
  • configure mariadb/mysql serverside
  • imort templates in zabbix
  • configure templates, triggers, alerts in webui Not a rocket science but sometimes it’s very hard to find any logic in zabbix ui tabs/panes/menu.

Zabbix agent installation

wget http://repo.zabbix.com/zabbix/3.0/ubuntu/pool/main/z/zabbix/zabbix-agent_3.0.0-1+trusty_amd64.deb && dpkg -i zabbix-agent_3.0.0-1+trusty_amd64.deb

vi /etc/zabbix/zabbix_agentd.conf

  • Server=serverIp(or fqdn)
  • ServerAcive=serverIp(or fqdn)
  • #Hostname
  • UserParameter=rabbitmq.queue.items,/etc/zabbix/scripts/check_rabbit.py queues-items (example)
    cp -r scripts /etc/zabbix/ get scripts from this repo: github/zabbix-fuel-plugin/scripts
    service zabbix-agent restart

UserParameter scripts configuration

UserParameter consists from key reference rabbitmq.queue.items path to script /etc/zabbix/scripts/check_rabbit.py and arguments queues-items for it execution

All we need is to write them down carefully. After that we can use them for monitoring of our infrastructure.
You can test your UserParameter with this command:
zabbix_agentd -t key[argument]
Example: zabbix_agentd -t proc.num[] выведет кол-во активных процессов.

Example of a UserParamter configuraion:
github/fuel-zabix-plugin/monitoring

Zabbix Server installation

Offical doc:
zabbix.com

At first you need to gather some basic information about your future zabbix installation such as:

  • Pollers count
  • Timeout for zabbix agents(VERY important parameter)
  • Acceptable openconnections to db count

When you gathered all information about your configuration edit server config:
vi /etc/zabbix/zabbix_server.conf Timeout=30
StartPollers=1000
vi /etc/systemd/system/mariadb.service (create new file)
.include /lib/systemd/system/mariadb.service
[Service]
LimitNOFILE=10000
LimitMEMLOCK=infinitysystemctl restart mariadb.service
Also you’ll need to change max_connections variable in DB via /etc/my.cnf.d/server.conf file
check your settings:
mysql -u USER -p PASS;SHOW VARIABLES LIKE 'open%';show variables like 'max_connections';

Template import

Login in zabbix webui, go to Configuration -> Templates
Press import, choose the file, press import
None of the import rules should be changed

If you get some errors with importing your handy templates - you should check your template’s structure with the referenced one:

  • name of template
  • group
  • application
  • items
  • triggers

Aplication and group are needed for template and item grouping in zabbix. And furthermore, for every item we need to point on its own application.

Items and triggers are the key elements of our template.
When you work with Items pay attention to <key> and <application> sections.
<key> is a key that referenced on test that will be excuted by zabbix agent.
Example:
 <key>net.tcp.service[service,0.0.0.0,8779]</key>.
This key tells zabbix that it should execute a standart zabbix test ‘net.tcp.service’ and check service availability on the 0.0.0.0:8779 address.

Trigger is a conditional statement for item checking (placed in section <expression>).
Example:
<expression>{Template App OpenStack Tesora Trove:net.tcp.service[service,0.0.0.0,8779].last(0)}=0</expression>
The first part of trigger (Template App ….) is a host OR template reference. Zabbix will use our conditional check on them.
Your can use either hostname or tempate name.
The second part of trigger is a key which we can get from our item.
And the last part of trigger is a conditional itself.
In our case .last(0)}=0 checks if exit code equal 0 or not and if exit code=0 then trigger runs.
Also you can set alert level for trigger on scale from 0 to 5 (Informatio/Disaster).

Host groups creation and Template binding

From UI:

  1. Host discovering - enable autodiscovering rules
  2. Assert hosts to specific groups for your own convenience - controllers, compute and etc
  3. Go to Configuration -> Templates and add hosts to each your template
    Example:
    Hosts/Template - Other goups - controllers - drag and drop all controllers to the left.

Few tips for web UI

Don’t use Monitoring tab! Instead use Configuration tab. They both have the same list of menus but Monitoring tab doesn’t allow you to change configuration.
In UI there are a lot of different filters so be carefull with them or you will not find anything. These filters are evrywhere! In Monitoring tab you can find them on the top right (group and trigger filters), in the center (status, name filters and others).
You can disable triggers items and other stuff for each host individually. You should go to Configuration - Hosts - triggers tab if this is what you want. When the trigger is disabled it can be deleted from Overview menu in trigger tab - left click on trigger - configuration - delete.

Written on October 12, 2017

Archive

linux openstack kubernetes gluster ceph ansible docker systemctl python openshift log centos sed registry kolla kibana keepalived elasticsearch deploy cloud-init auth HA zabbix vsphere vmware venv tools swift ssl ssh scm ruby rsyslog rhel rbac rabbitmq prometheus postgres policy pgpool2 patrole pacemaker ntp nfs net mq monitoring mongo mbr lvm logstash kubespray kubeadm kolla-ansible jenkins jekyII java ironic helm heketi harbor grub github ganesha fluentd fluent-bit fio drbd diskimage-builder devstack debug deb db cisco cicd ceph-ansible calico build bitbucket benchmarking apparmor api NUMA Linux Golang Go ELK 3par