ZCS5 Day 2 Lab Manual

The Enterprise class Monitoring Solution for Everyone
www.zabbix.com
Zabbix 5.0
Certified Specialist course
LAB Manual Day 2

LAB 5
Zabbix Agent installation
ZABBIX AGENT INSTALLATION
Go to Monitoring -> Hosts
Availability icon for Zabbix passive agent will be red, because we do not have Zabbix
agent on the host
Click on the red ZBX icon to see the error message
Hosts Training-VM-XX and Training Resources does not have any passive agent items, the Zabbix
passive agent availability icon is gray
Zabbix server internal items are working, they do not depend on Zabbix agent !
ZABBIX 5.0 Lab Manual 3

Install Zabbix agent from official Zabbix repository
dnf -y install zabbix-agent
======================================================================================================
Package Architecture Version
======================================================================================================
Installing:
zabbix-agent x86_64 5.0.0.el8
Transaction Summary
======================================================================================================
Install 1 Package
...
Installed:
zabbix-agent-5.0.0.el8.x86_64
Complete!
You can install Zabbix agent with a simple command because Zabbix official repository was
installed on day one, when Zabbix server was installed
If you do not have official Zabbix repository installed on monitored servers, it is possible to use
rpm files to install Zabbix agent locally, but in this case, Zabbix agent will be not auto updated !

Enable Zabbix agent auto-start
systemctl enable zabbix-agent
Created symlink /etc/systemd/system/multi-user.target.wants/zabbix-agent.service →

/usr/lib/systemd/system/zabbix-agent.service.
Start Zabbix-agent
systemctl start zabbix-agent
Check Zabbix agent status

systemctl status zabbix-agent
zabbix-agent.service - Zabbix Agent

Loaded: loaded (/usr/lib/systemd/system/zabbix-agent.service; enabled; vendor preset: disabled)
Active: active (running) since Mon 2020-04-20 08:23:10 EEST; 1 day 8h ago
Main PID: 1624 (zabbix_agentd)
CGroup: /system.slice/zabbix-agent.service
├─1624 /usr/sbin/zabbix_agentd -c /etc/zabbix/zabbix_agentd.conf
├─1625 /usr/sbin/zabbix_agentd: collector [idle 1 sec]
├─1628 /usr/sbin/zabbix_agentd: listener #1 [waiting for connection]
├─1627 /usr/sbin/zabbix_agentd: listener #2 [waiting for connection]
student-XX systemd[1]: Started Zabbix Agent.

Wait until availability icon for Zabbix passive agent turns green (it can take a minute)
Click on Latest data for Zabbix server host
The Latest data frontend section will open

Open the filter, check the Show details checkbox and press Apply
The items now shows more details, like item key and type
• Look for items which have type "Zabbix agent"
Check that items of type Zabbix agent are reporting data !

Now Press Select button right to the Application input field

In the Application form choose CPU

Press Apply to apply the new filter
Click on Name header to sort items by name (ascending)

Scroll down until you find Load average items and select all of them
Press Display graph

The graph is built dynamically on the fly from your selected items
You can use the time selector to select different time scale
The most part of graph is empty, because it displays last hour of collected data, and we have just
started to collect some metrics a few minutes ago
CONGRATULATIONS!
Zabbix Agent installation lab is finished
LAB 6
Zabbix Agent passive checks
LAB6 - ZABBIX AGENT PASSIVE CHECKS
Go to Configuration -> Hosts
Enter Training-VM into Name field and press Apply to filter hosts
Click on Items for Training-VM-XX
The availability icon is currently gray because we do not have any Zabbix passive agent
items created on this host.

Press Create item
Fill the required fields

• Name: CPU load
• Type: Zabbix agent
• Key: system.cpu.load
• Type: Numeric (float)
• Update int: 10s
• New application: CPU
10s is a very short update interval,

we will use it for training purposes
Application is optional, but it

helps to filter out data
Press Test

Press Get value and test
Look at the error message - agent dropped connection because of access permissions !?
Where are these access permissions ?
Switch to SSH console and open Zabbix agent configuration file
# vi /etc/zabbix/zabbix_agentd.conf
Search for server parameter and append your DNS name (or IP address) to the Server list
### Option: Server
# List of comma delimited IP addresses or DNS names of Zabbix servers and Zabbix proxies.
# Incoming connections will be accepted only from the hosts listed here.
# If IPv6 support is enabled then '127.0.0.1', '::127.0.0.1', '::ffff:127.0.0.1' are treated equally
# and '::/0' will allow any IPv4 or IPv6 address.
# '0.0.0.0/0' can be used to allow any IPv4 address.
# Example: Server=127.0.0.1,192.168.1.0/24,::1,2001:db8::/32,zabbix.example.com
#
# Mandatory: yes, if StartAgents is not explicitly set to 0
# Default:
Server=127.0.0.1,student-XX
Save the configuration file and restart Zabbix agent

# systemctl restart zabbix-agent
remember to always check that agent started successfully after configuration changes !

Press Get value and test again, and look at the Result field
The item is working now ! Press Cancel to close the test form
Press Add at the bottom of item configuration form to create the item

You have first Zabbix passive agent item created !
The availability icon is still gray, because we have just created the item. Zabbix server needs to
collect the value of the item at least one time before it detects that agent is up
You can wait for a minute and press Apply on filter section, the ZBX will turn green

Next, you need to monitor network traffic. Take a closer look at the item key:
https://www.zabbix.com/documentation/5.0/manual/config/items/itemtypes/zabbix_agent
This item requires network interface name, you can find interfaces with following command
# ip link show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT......
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP mode .......
The eth0 interface will be used for traffic monitoring

Create a new item by pressing Create item

• Name - Interface eth0: Incoming traffic
• Type - Zabbix agent
• Key - net.if.in[eth0]
• Type of information - numeric (unsigned)
• Units - Bps
• Update interval - 10s
• New application - Network
Press Test

Press Get value and test
The item is getting back data

Press Cancel to close item test form and then press Add on item form to create the item

You need to wait up to 1 minute until the new item is checked, but you can also force the
Zabbix configuration cache reload
Open SSH console and force reload Zabbix server configuration cache
# zabbix_server -R config_cache_reload
zabbix_server [22727]: command sent successfully

Go to Monitoring -> Hosts, click on Latest data for Training-VM-XX host
The item is reporting back data, it's working !

But wait, why you are seeing such a heavy traffic on a small host ? Is Zabbix generating so
much traffic !?
If you will look at the graph for some time, you will find quite a heavy traffic, which
constantly rises, and never falls.
Something feels wrong here... It does not look like a typical network traffic graph...
Because this item is a counter, it just counts outgoing traffic bytes. To display speed per
second, we need to process the collected data on Zabbix server !

Go back to Configuration -> Hosts and click on Items for Training-VM-XX host
Click on Interface eth0: Incoming traffic

There is another tab called Preprocessing, where data manipulation can be done
Select Preprocessing tab and click Add to create a new preprocessing step
Choose Change per second preprocessing step in the Name field

Now click on Test
Test item form will open. The values are empty, check the Get value from host checkbox

Press Get value and test 2 times with a few seconds interval
You need to press the button 2 times because this preprocessing step needs 2 values
Now in the Result field you can see the actual speed per second
Zabbix will collect 2 values, calculate the difference and divide by seconds between checks

Close the test form and press Update on item form to save changes
Go back to Monitoring - > Hosts and look at Latest data for Training-VM-XX host

You need to wait up to 1 minute or reload configuration cache for changes to take effect
After that it will be clearly visible on the graph that now it reports correct data
Now here comes another problem - the old data does not allow to see any updated traffic
data because scaling is too different

Open Filter, click Show details checkbox if it is not checked and press Apply
Click on item key net.if.in[eth0] for Interface eth0: Incoming traffic

The item configuration page will open
Press Clear history and trends at the bottom of the item configuration page
The historical data are now cleared

Open Latest data for host Training-VM-XX and look at the graph again
Set the resolution to last 5 minutes
Now the item reports correct data, and the graphs look nice

You still need to create another item for outgoing network item
This time just clone the existing incoming traffic item to save time
Open configuration page for the items and click on the incoming traffic item
The item configuration page will open

At the bottom of the item configuration form is a Clone button, press it

The clone button clones all item properties
You need to change only 2 fields
• Name: Interface eth0: Outgoing traffic
• Key: net.if.out[eth0]
Press Add
The second item is created just like the previous one, just with different name and key
Item cloning is a very effective way to create items fast
Be careful when cloning, preprocessing steps are hidden under Preprocessing tab, but they are also
cloned !

Now you have items for both incoming and outgoing network traffic
Go to Monitoring -> Hosts and open Latest data for Training-VM-XX host
Select both items and press Display graph to show them on graph

Now both items are reporting data and are displayed correctly on the graph

While Zabbix 5.0 allows to test items from the graphical user interface, sometimes it is
easier to test them from the command line
Install Zabbix-get command line utility
# dnf -y install zabbix-get
Running transaction
Preparing :
Installing : zabbix-get-5.0.0-1.el8.x86_64
Verifying : zabbix-get-5.0.0-1.el8.x86_64
Installed:
zabbix-get-5.0.0-1.el8.x86_64
Zabbix-get is a standalone command-line utility, it does not require Zabbix server or agent to be
installed on the host
This utility is for troubleshooting purposes, or can be used in your scripts to collect data from
passive Zabbix agents
This utility still requires the "Server=" parameter to be correctly set in agent configuration file,
otherwise agent will refuse connection from host which executed zabbix_get command !

Use zabbix_get utility to get current swap size in bytes from local agent
# zabbix_get -s 127.0.0.1 -k system.swap.size
999542784
The reported data is a raw value in bytes just like it is specified in documentation
Use zabbix_get utility to get current swap size in percentage from local agent
# zabbix_get -s 127.0.0.1 -k system.swap.size[,pfree]
93.090031
The reported data is percentage, because pfree parameter was specified
You can get data from any remote agent if it will accept connection from your host!
You could also use Zabbix agent binary to test your items from command line
Type following command to show all available keys with values
# zabbix_agentd -p
agent.hostname [s|Zabbix server]

agent.ping [u|1]
agent.version [s|5.0.0beta1]
system.localtime[utc] [u|1587648377]
...
net.tcp.service[ssh,127.0.0.1,22] [u|1]
net.tcp.service.perf[ssh,127.0.0.1,22] [d|0.019896745681762695]
net.udp.service[ntp,127.0.0.1,123] [u|0]
net.udp.service.perf[ntp,127.0.0.1,123] [d|0]
system.hostname [s|student-xx]
You can test any of the item keys locally to see if they work

Let's troubleshoot a simple issue - try to get contents of /etc/sudoers file with zabbix_get
# zabbix_get -s 127.0.0.1 -k vfs.file.contents[/etc/sudoers]
ZBX_NOTSUPPORTED: Cannot open file: [13] Permission denied
The same item check with zabbix_agentd

# zabbix_agentd -t vfs.file.contents[/etc/sudoers]
vfs.file.contents[/etc/sudoers] [t|## Sudoers allows particular users to run various

commands as
## the root user, without needing the root password.
##
## Examples are provided at the bottom of the file for collections
## of related commands, which can then be delegated out to particular
## users or groups.
Why the same key is working from agent command line but not working with zabbix_get ?

CONGRATULATIONS!
Zabbix Agent passive checks lab is finished
LAB 7
Zabbix Agent active checks
ZABBIX AGENT ACTIVE CHECKS
Create a host for Zabbix agent active checks. You can create a new host, but it can be
done faster - just clone the existing host with all items and change the item type
Go to Configuration - Hosts and click on Training-VM-XX host

At the bottom of the host configuration form are two buttons:
• Clone
• Full clone
Press Full clone to clone host with all items

All the host properties are already copied, we just need to change the Host name
Enter student-XX into Host name field (replace XX with your student number !)
Press Add

Host is created.
Remember to check your filters on the Hosts page if you can not see the host !
Click on items for student-XX host

All the items are copied together with host because we pressed Full clone button.
The only problem is that we need to change Type to Zabbix agent (active)
Select all items in this host and press Mass update
To quickly select all items at once, just check the upper checkbox
You can update all items one-by-one, but mass update is more efficient way

Select Type and choose Zabbix agent (active) from the drop-down box
Scroll down and press Update at the bottom of the mass update form

All items are changed to Zabbix agent (active)
Is this enough to complete the task ?

Go to Monitoring -> Hosts and open Latest data for student-XX host
It seems everything is working
But wait - are the data changing ? Look at the Last check column, the time is in a past !
When working with latest data, always check that values are current !
Click on the graph for CPU load item

When you first cloned the host, it still had items of type Zabbix agent
That is why you can see some data collected on the graph and in latest data
After you changed the item type to Zabbix agent (active) the data collection stopped
Our items are correct, because we just cloned the working ones !
It is not possible to Execute now or Test active agent type items

Look at Zabbix agent configuration file - search for parameters related to active checks
• ServerActive
• Hostname
### Option: ServerActive

# List of comma delimited IP:port (or DNS name:port) pairs of Zabbix for active checks.
#
# ServerActive=
ServerActive=127.0.0.1
### Option: Hostname

# Unique, case sensitive hostname.
# Hostname=
Hostname=Zabbix server
The ServerActive is 127.0.0.1, which is correct (we have Zabbix server on the same host)
The Hostname is Zabbix server !!! (it is by default when you install a new Zabbix agent)
What Hostname should be used?
Change the Hostname to student-XX (replace XX with your student number)
alternatively, you can just comment out this line, the system hostname will be used
### Option: Hostname

# Unique, case sensitive hostname.
# Hostname=
Hostname=student-XX
Save the file and restart Zabbix agent

Check zabbix agent status (always check the status after configuration changes)
# systemctl status zabbix-agent
● zabbix-agent.service - Zabbix Agent

Loaded: loaded (/usr/lib/systemd/system/zabbix-agent.service; enabled; vendor preset: disabled)
Active: active (running) since Thu 2020-04-23 17:39:52 EEST; 27s ago
Process: 6234 ExecStop=/bin/kill -SIGTERM $MAINPID (code=exited, status=0/SUCCESS)
Process: 6236 ExecStart=/usr/sbin/zabbix_agentd -c $CONFFILE (code=exited, status=0/SUCCESS)

Go to Monitoring -> Hosts and open Latest data for student-XX host
Look at Last check - this field must now constantly update
The graph is displaying real time data again

Go to Configuration -> Hosts and click on student-xx host
Enter Training-VM-XX active checks into Visible name field
Only Host name field must match active agent configuration !

Press Update

One last thing - Look at the Availability icons on Monitoring -> Hosts section
The active checks are working now, but the availability icon stays gray for student-XX
Zabbix availability icons are displayed for Zabbix passive agent only, you do not have any passive
agent items on this host

CONGRATULATIONS!
Zabbix Agent active checks lab is finished
LAB 8
Problem detection
PROBLEM DETECTION
Go to Configuration -> Hosts and click on Triggers for Training Resources server

PROBLEM DETECTION
Press Create trigger
When you are creating items and collecting data, problems are not detected. You need to
define triggers with thresholds to start getting alerts

PROBLEM DETECTION
First, we will check the NTP service status on Training Resources server
• Name: NTP server is down on Training Resources server
• Severity: choose Warning
• Expression: Press Add to open the popup form

PROBLEM DETECTION
Now create a trigger condition
• Item: Press Select and choose Training Resources: Service NTP availability
• Function: last()
• Result: =0
• Press Insert on Condition form

PROBLEM DETECTION
The trigger configuration will look like this
Press Add to create trigger

PROBLEM DETECTION
The trigger is created
The trigger has Value OK, because NTP service is up on remote host

PROBLEM DETECTION
Press Create trigger to create another trigger for ICMP Ping
Name the trigger : ICMP Ping is down on Training Resources server
Fill out required fields
• Name: ICMP Ping is down on Training Resources server
• Severity: Average
• Expression: Press Add to the right of the expression field
• Item: Select Training Resources: ICMP Ping
• Function: max()
• Last of (T): 3 Count
• Result: =0
• Press Insert

PROBLEM DETECTION
The trigger will look like this
Press Add on Trigger configuration form to create trigger

This more advanced trigger will ignore two consequent 0 values to avoid problem flapping
max from 1|0|0 =1
max from 0|0|0 =0
With such trigger you must miss 3 pings in a row to rise problem

PROBLEM DETECTION
Press Create trigger to create the third trigger - this will be more advanced trigger setup

PROBLEM DETECTION
This trigger will detect poor web service performance or that service is down
Use expression constructor to build the problem expression
Set the initial properties of trigger
• Name: Web service has problems on {HOST.NAME}
• Operational data: Web performance: {ITEM.LASTVALUE}
• Severity: High
Click on Expression constructor

PROBLEM DETECTION
Press Edit
• Item: Select Training Resources: Service WEB performance
• Function: max()
• Last of (T): 3 Count
• Result: =0
Press Insert

PROBLEM DETECTION
The trigger function is inserted into Expression field
Press Add below the expression form
Expression is added to expression constructor

PROBLEM DETECTION
Press Edit and enter settings for second part of trigger
• Item: Select Training Resources: Service WEB performance
• Function: min()
• Last of (T): 5m Time
• Result: >5
Press Insert

PROBLEM DETECTION
Expression is created
Press Or to add the expression with logical OR condition

PROBLEM DETECTION
Because we have created complex trigger with min ,max and logical functions, it is a good
idea to test the behavior of our expression
Click on Test

PROBLEM DETECTION
A new form will open, where you can insert item values and test the trigger
Two test examples are provided, which one will fire the trigger ?
Entire trigger expression must become TRUE for trigger to fire !
When the tests are finished, press Cancel to close trigger test form
Press Add on trigger configuration form to create trigger

PROBLEM DETECTION
Trigger is created
If you do not need to see filters, you can press Filter to hide them
Remember, even hidden filters are still active, so look at them if you are not seeing data
If problem will be detected, trigger value will be shown as PROBLEM here

PROBLEM DETECTION
Go to Monitoring - Hosts
One problem is detected on this host, click on Problems to see what is happening
Trigger detected problem with web service

PROBLEM DETECTION
Press Filter to show filters and switch operational data to be shown separately
The Operational data column will be displayed, which contains last value received

PROBLEM DETECTION
You can also see graph showing historical data for the problem
• By clicking on problem
• By clicking on operational data
The graph will display history and you will clearly see when problem started

CONGRATULATIONS!
The Triggers lab is finished
LAB 9
Advanced Problem detection
ADVANCED PROBLEM DETECTION
Go to Configuration -> Hosts and click on the Training-VM-XX host name

Click on Macros tab
Add two macros for CPU load (click on Add to add more macros)
• Macro: {$CPU.LOAD.WARNING} Value: 1 Description: High CPU load
• Macro: {$CPU.LOAD.HIGH} Value: 2 Description: Very high CPU load
Press Update

Click on Triggers for host Training-VM-XX
Press Create trigger

Create a new trigger
• Name: CPU Load is high on {HOST.NAME}
• Operational data: {ITEM.LASTVALUE}
• Severity: Warning
• Expression: {Training-VM-XX:system.cpu.load.min(#3)}>{$CPU.LOAD.WARNING}
• Allow manual close: Yes
You can use expression builder, or enter the expression directly into Expression field
Press Add

The trigger is created. Click on the trigger name
The trigger configuration page will open for current trigger
Press Clone at the bottom of the page

You need to change just a few parameters
• Name: CPU Load is very high on {HOST.NAME}
• Severity: High
• Macro in the expression field: {Training-VM-XX:system.cpu.load.min(#3)}>{$CPU.LOAD.HIGH}
Add recovery expression by switching OK event generation

• {Training-VM-XX:system.cpu.load.min(#3)}<={$CPU.LOAD.WARNING}
Press Add
Two Triggers are created
Now open SSH console and enter command

# cat /dev/urandom | md5sum
Go to Monitoring -> Problems page and wait a few minutes until problem will appear

The problem about high CPU load will appear
And in a few minutes a second problem with high severity will appear
If you will look at the graphs, you will see trigger lines

Now you have two problems which informs about the same problem
Define trigger dependencies to show only the most actual problem

Click on Problem with warning severity and select Configuration from menu

Click on Dependencies tab
Click Add to create dependency

Select the trigger with High severity (just click on trigger name)
Press Update
The trigger dependency is created

Only the high severity trigger is displayed now
Click on the up arrow to display dependent triggers
Try to solve the problem by simply closing it - click on No in the Ack column, check Close
problem and press Update in the Update problem form

Yes ! The problem is resolved
Oh no, its detected once again....
Closing the problem will not help, because the problem is detected again with next item
update

It is possible to lower severity for a problem only this time. Click on No in the Ack column
On Update problem form check Change severity and set it to Information
Press Update

The severity is changed to Information
All actions are logged, click on the arrow -> to see the list of actions
Cancel CPU load simulator (press Ctrl+C in SSH console)

Next time the problem will be detected with High severity again !

CONGRATULATIONS!
Advanced triggers lab is complete !
LAB 10
Templates
TEMPLATES
Go to Configuration -> Templates and press Create template
Enter Template name: Template Basic

In Groups field enter: Training/Templates and click on the suggested name (new)
It is possible to create Host groups directly from Hosts or Templates configuration screen
TEMPLATES
Switch to Tags tab
• Enter Name: Environment Value: Training
• Press Add
The template is created. Enter Basic in the filter field to show our new template only
Use filters to quickly find hosts or templates !

TEMPLATES
Template Basic currently is empty without any items or triggers
Open Configuration -> Hosts and click on Items for Training-VM-XX host

TEMPLATES
Select all items and press Copy
Select Target type: Templates and enter target: Template Basic
Press Copy

TEMPLATES
All selected items are copied
Repeat the same steps with Triggers (copy from Training-VM-XX host to Template Basic)

TEMPLATES
Go to Configuration -> Templates and look at Template basic
It has 3 items and 2 triggers now, but no Applications
It is not possible to copy Applications, you need to recreate them.

Create applications CPU and Network

TEMPLATES
Assign CPU application to CPU load item (by editing item)
Assign Network application to network traffic items (by using mass update)
You can update items one by one by editing them directly or using mass update

TEMPLATES
Now items have their applications
You also need to create Macros on template.

Click on Template Basic

TEMPLATES
Open Macros tab and enter Macro names and values (the same as on host)
Press Update to update Template Basic

Assign Template to the host. Go to Configuration -> Hosts and click on Training-VM-XX

TEMPLATES
Click on Templates tab,
Enter Basic into Link new templates field and select Template Basic
Press Update

TEMPLATES
Host is updated, Template Basic is linked now
Click on the Items on the host
All items are the same, but now they are linked to template

TEMPLATES
Click on Training-VM-XX and select Macros tab
Switch to Inherited and host macros
Click on Remove to remove host macros (They are duplicated by Template)

TEMPLATES
Now Macros are greyed out, which means you are using template values
It is the best practice to use templated macros and only re-define them per host if you need to
adjust some macros on this particular host level
Press Update

TEMPLATES
Now click on host name Training-VM-XX again
Press Clone (It will clone host with template linkage

TEMPLATES
Replace student number XX with XY in host name field
Replace student number xx with xy in DNS name field
XY means student number for one of your neighbors (like 01,02 etc...)
Press Add

TEMPLATES
Press on Training-VM-XX and clone it one more time
Name the new host Training-VM-XZ with DNS name student-xz

(XZ means student number of another neighbor, like 01,02 etc.)
You have created two more hosts. But why they are red ? You have cloned a working host...
TEMPLATES
Click on the red ZBX icon, the error will be displayed
Your neighbors are not allowing incoming connections to their agents from your host !
They must fix it by adding your DNS name to zabbix_agentd.conf
The following configuration changes needs to be done on your neighbor's machines !
### Option: Server

Server=127.0.0.1,student-xx,student-xy,student-xz,trainer
Zabbix agent restart is always required after configuration file changes !!!

TEMPLATES
Go to Monitoring -> Hosts. In a few minutes, your neighbor hosts must become green
Click on Latest data, you will see that data are collected from the remote hosts

CONGRATULATIONS!
Templates lab is complete !
LAB 11
User Parameters
USER PARAMETERS
First, we will check that our command is working at all
# mysqladmin -uzabbix -pP455w0RD status
Uptime: 409981 Threads: 26 Questions: 5641477 Slow queries: 32 Opens: 130 Flush tables: 2 Open
tables: 156 Queries per second avg: 13.760
It is also recommended to test that Zabbix user can execute this command
# sudo -u zabbix mysqladmin -uzabbix -pP455w0RD status
Uptime: 409981 Threads: 26 Questions: 5641477 Slow queries: 32 Opens: 130 Flush tables: 2 Open
tables: 156 Queries per second avg: 13.760
Create file userparameter_training.conf in /etc/zabbix/zabbix_agentd.d

# vi /etc/zabbix/zabbix_agentd.d/userparameter_training.conf
UserParameter=mysql.uptime.s,mysqladmin -uzabbix -pP455w0RD status 2>/dev/null
The 2>/dev/null command removes stderr(2) output

Restart Zabbix agent

USER PARAMETERS
Test the new key using zabbix_agentd command line utility
# zabbix_agentd -t mysql.uptime.s
mysql.uptime.s [t|Uptime: 410215 Threads: 26 Questions: 5644878 Slow queries: 32

Opens: 130 Flush tables: 2 Open tables: 156 Queries per second avg: 13.760]
The data are reported back successfully. [t| means this is a textual information
Can you use this output to monitor Uptime in seconds? Yes, use preprocessing !
Go to Configuration -> Templates and click on Items for Template Basic
From this moment all our items will be created using templates

USER PARAMETERS
Press Create item
Fill required fields

• Name: Application MySQL uptime simple
• Type: Zabbix agent
• Key: mysql.uptime.s
• Type of information: Numeric(unsigned)
• Units: s
• New application: MySQL

USER PARAMETERS
Select Preprocessing tab and click Add
Fill required information and after that press Test:

• Name: Regular expression
• Parameters: Uptime:\s(\d+) \1
• This is regular expression, which means search for Uptime: followed by single space \s and then
capture one or multiple digits (\d+) into first capturing group \1

USER PARAMETERS
This item is created on Template Basic; it is not possible to get value from a host
Uncheck the Get value from host, and copy output from command line into Value field
mysql.uptime.s, [t| and closing bracket ] are not part of the value!
by example: Uptime: 410215 Threads: 26 Questions: 5644878 Slow queries: 32 Opens: 130 Flush
tables: 2 Open tables: 156 Queries per second avg: 13.760
Press Test
The item must return Result (the value is returned is seconds)

Press Cancel to close Test item form
Press Add on the item configuration form to create item
USER PARAMETERS
On previous item username and password are typed into plain text configuration file,
which is not the best practice
Click on Template Basic
Select Macros tab
This section already contains macros from previous tasks

USER PARAMETERS
Add two additional macros
• {$MYSQL.USERNAME} zabbix MySQL user name
• {$MYSQL.PASSWORD} P455w0RD MySQL user password
• It is recommended to add description to your macros to describe them to other users
But this is not secure ! They are visible to any user with Zabbix admin access rights to this host !
Can you make this more secure ?

USER PARAMETERS
Click on T icon and select Secret text for both sensitive macros
Press Update
Now your passwords are stored in a safe way !

USER PARAMETERS
Now create another UserParameter which will accept custom username and password
This time use Linux utilities to extract information
# mysqladmin -uzabbix -pP455w0RD status 2>/dev/null | grep Uptime | awk {'print $2'}
413033
If you have good command line knowledge, you can use more complex commands for user
parameters, otherwise, you can use preprocessing with regex
Create user parameter
# vi /etc/zabbix/zabbix_agentd.d/userparameter_training.conf
UserParameter=mysql.uptime.f[*],mysqladmin -u$1 -p$2 status 2>/dev/null | awk '{print $$2}'

restart Zabbix agent
Test new key from command line

# zabbix_agentd -t mysql.uptime.f[zabbix,P455w0RD]
mysql.uptime.f[zabbix,P455w0RD] [t|441258]

USER PARAMETERS
Go to Configuration -> Templates and click on Items for Template Basic
Click on Application MySQL Uptime simple item

USER PARAMETERS
Press Clone at the bottom of the item configuration page
Change important fields:
• Name: Application MySQL Uptime flexible
• Key: mysql.uptime.f[{$MYSQL.USERNAME},{$MYSQL.PASSWORD}]
• Units: uptime
Use different units (unixtime) this time to show the difference how Zabbix can display the same
information using different units
Notice that username and password are specified using macros, which are protected by Zabbix
Press Test at the bottom of the page
This time use get value from host and specify host address
USER PARAMETERS
You are testing item from template, which does not have any physical IP address
It is possible to specify host from where to test the value
• Check Get value from host
• Enter Host address: student-xx Port: 10050
• Enter P455w0RD into password Macro field
• Press Get value
Zabbix currently cannot test items with

protected passwords
The value is received, which means username and password are accepted
Now press Get value and test
Something went wrong.... Why the test failed ?

USER PARAMETERS
Because just a numerical value is reported back and preprocessing is failing
Press Cancel to close testing form and click on Preprocessing tab
Click Remove to remove unneeded step
Press Test to test the item again

USER PARAMETERS
The test must succeed now, press Cancel to close testing form
Press Add on item configuration form to create item

Now you have two different items, with simple and flexible user parameters

USER PARAMETERS
Go to Monitoring -> Latest data to check results
Clear all filters by pressing Reset and enter MySQL into Application filter field
Check both Show items without data and show details checkboxes
Press Apply
Some of the item keys may be unsupported on other hosts. Why ?

USER PARAMETERS
You have already created both user parameters successfully, but other students may be
still creating them
Wait for other students to complete the lab, all items must become supported
Userparameters must be configured on every agent to work !
Last value is displayed differently for both items. Why ?

CONGRATULATIONS!
UserParameters lab is complete !
The Enterprise class Monitoring Solution for Everyone
www.zabbix.com
Zabbix 5.0
Certified Specialist course
End of Day 2

ZCS5 Day 2 Lab Manual

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

ZCS5 Day 2 Lab Manual

Uploaded by

Copyright:

Available Formats

The Enterprise class Monitoring Solution for Everyone

LAB Manual Day 2

ZABBIX 5.0 Lab Manual 3

ZABBIX 5.0 Lab Manual 4

Created symlink /etc/systemd/system/multi-user.target.wants/zabbix-agent.service →

Check Zabbix agent status

zabbix-agent.service - Zabbix Agent

ZABBIX 5.0 Lab Manual 5

The Latest data frontend section will open

ZABBIX 5.0 Lab Manual 6

Check that items of type Zabbix agent are reporting data !

ZABBIX 5.0 Lab Manual 7

ZABBIX 5.0 Lab Manual 8

ZABBIX 5.0 Lab Manual 9

ZABBIX 5.0 Lab Manual 10

ZABBIX 5.0 Lab Manual 11

ZABBIX 5.0 Lab Manual 15

Fill the required fields

10s is a very short update interval,

Application is optional, but it

ZABBIX 5.0 Lab Manual 16

Save the configuration file and restart Zabbix agent

ZABBIX 5.0 Lab Manual 18

ZABBIX 5.0 Lab Manual 19

ZABBIX 5.0 Lab Manual 20

The eth0 interface will be used for traffic monitoring

ZABBIX 5.0 Lab Manual 21

Fill the required fields

ZABBIX 5.0 Lab Manual 22

The item is getting back data

ZABBIX 5.0 Lab Manual 23

zabbix_server [22727]: command sent successfully

The item is reporting back data, it's working !

ZABBIX 5.0 Lab Manual 25

Click on Interface eth0: Incoming traffic

ZABBIX 5.0 Lab Manual 26

Choose Change per second preprocessing step in the Name field

ZABBIX 5.0 Lab Manual 27

ZABBIX 5.0 Lab Manual 28

ZABBIX 5.0 Lab Manual 29

ZABBIX 5.0 Lab Manual 30

ZABBIX 5.0 Lab Manual 31

Click on item key net.if.in[eth0] for Interface eth0: Incoming traffic

ZABBIX 5.0 Lab Manual 32

The historical data are now cleared

ZABBIX 5.0 Lab Manual 33

ZABBIX 5.0 Lab Manual 34

The item configuration page will open

ZABBIX 5.0 Lab Manual 35

Item cloning is a very effective way to create items fast

ZABBIX 5.0 Lab Manual 36

ZABBIX 5.0 Lab Manual 37

ZABBIX 5.0 Lab Manual 38

ZABBIX 5.0 Lab Manual 39

# zabbix_get -s 127.0.0.1 -k system.swap.size

The reported data is percentage, because pfree parameter was specified

agent.hostname [s|Zabbix server]

ZABBIX 5.0 Lab Manual 41

ZBX_NOTSUPPORTED: Cannot open file: [13] Permission denied

The same item check with zabbix_agentd

vfs.file.contents[/etc/sudoers] [t|## Sudoers allows particular users to run various

ZABBIX 5.0 Lab Manual 42

ZABBIX 5.0 Lab Manual 45

ZABBIX 5.0 Lab Manual 46