ZCS5 Day 4 Lab Manual

The Enterprise class Monitoring Solution for Everyone
www.zabbix.com
Zabbix 5.0
Certified Specialist course
LAB Manual Day 4

LAB 21
Inventory collection
INVENTORY COLLECTION
Go to Configuration -> Templates
Create a new template
• Name: Template Basic module system info
• Group: Training/Templates
Create new item on the template
Fill required fields:
• Name: System OS
• Type: Zabbix agent
• Key: system.sw.os[name]
• Type of information: Character
• Update interval: 1h
• New application: Operational System
• Populates host invent...:OS
Press Add to create item
ZABBIX 5.0 Lab Manual 3

Create another item on Template Basic module system info
• Name: System hostname
• Type: Zabbix agent
• Key: system.hostname
• Type of information: Character
• Update interval: 1h
• New application: Operational System
• Populates host invent...:Name
Press Add to create item
Link Template Basic module system info

to Template Basic

Select all Training-VM hosts and press Mass update
Set Inventory mode to Automatic and change location field
Press Update

Click on Items for one of the Training-VM hosts
Adjust filters
• Host groups: Training/Servers
• Application: Operational System
Select all items and press Execute now

Go to Monitoring -> Hosts
Click on the host name for Training-VM-XX host and select Inventory
Select Details tab

CONGRATULATIONS!
Inventory Lab is complete !
LAB 22
Actions
ACTIONS
Go to Administration -> Media Types and Press Create media type
It is possible to customize existing Email media, but for training purposes a new media will
be created
Zabbix 5.0 supports a lot of media types out of box.

ACTIONS
Fill required fields
• Name: Training email
• Type: Email
• SMTP server: training.lan
• SMTP server port: 25
• SMTP helo: training.lan
• SMTP email: trainingXX@training.lan
• Message format: HTML
Switch to Message templates tab

ACTIONS
Click Add to add message type
• Message type: Problem
• Subject: Problem: {EVENT.NAME}
• Message
Problem started at {EVENT.TIME} on {EVENT.DATE} 
Problem name: {EVENT.NAME} 
Host: {HOST.NAME} 
Severity: {EVENT.SEVERITY} 
Operational data: {EVENT.OPDATA} 
Original problem ID: {EVENT.ID} 
Location: {INVENTORY.LOCATION} 
Operational System: {INVENTORY.OS}
In Zabbix 5.0 you can define your own message formats using frontend
HTML or plain text format can be used for messages
Press Add on Message template form

ACTIONS
Click Add to add another message type
• Message type: Problem recovery
• Subject: Resolved: {EVENT.NAME}
• Message
Problem has been resolved
at {EVENT.RECOVERY.TIME} on {EVENT.RECOVERY.DATE} 
Problem name: {EVENT.NAME} 
Problem duration: {EVENT.DURATION} 
Host: {HOST.NAME} 
Severity: {EVENT.SEVERITY} 
Location: {INVENTORY.LOCATION} 
Operational System: {INVENTORY.OS}
Press Add on Message template form

You must define message templates for every message type you will be using !
Press Add on Media types form

13
ZABBIX 5.0 Lab Manual
ACTIONS
Go to User settings
Select Media tab and click on Add

• Type: Training email
• Send to: your email address
• When active: 1-5,09:00-18:00
• Use if severity: Uncheck Informational severity
Press Add on Media form
Press Update on User profile form

ACTIONS
Go to Configuration -> Actions and press Create Action
Trigger actions needs to be selected as action type !

Type Report Training hosts problems in the Name field
Click on Add to add new condition

ACTIONS
Create condition by clicking on Add
• Type: Host group
• Operator: Equals
• Host groups: Training/Servers
• Press Add on New condition form
Create another condition

• Type: Trigger severity
• Operator: is greater than or equals
• Severity: Warning
Two conditions are now defined

• Type of calculation must be A and B

ACTIONS
Switch to Operations tab
Click on Add for Operations

• Operator type: Send message
• Steps: 1-1
• Step duration: 1m
• Send to Users
• Click on Add and select student-XX
• Send only to: Training email
Press Add on Operation details form

ACTIONS
Click on Add for to add another step

• Operator type: Remote Command
• Steps: 2-2
• Step duration: 0
• Target list: Check current host
• Type: Custom script
• Execute on: Zabbix server
• Commands: echo {EVENT.TIME} {HOST.NAME} {EVENT.NAME} >> /var/log/zabbix/zabbix_problems.log
• Conditions: Click Add and fill form
• Type: Event acknowledged
• Operator: Equals
• Acknowledged: No
Press Add on Operation details form

ACTIONS
Click on Add on Recover operations section to add Recovery operation

• Operation type: Notify all involved
• Custom message: Unchecked
• Press Add on Operation details form
It is recommended to create recovery operations on actions to inform that problem is solved

ACTIONS
Now you have Action for Training/Servers host group with 2 steps and recovery operation
Click Add to create action

ACTIONS
Open SSH console and generate high CPU load
# cat /dev/urandom | md5sum
Go to Monitoring -> Problems

Wait for the problem to appear and look at the Actions column
In a few minutes two actions will be executed
• Training email
• Remote command
Press Ctrl + C in SSH console to stop CPU load simulation

ACTIONS
Go to Reports -> Action log
All Actions are displayed here
Remote command created log file, you can check it

# cat /var/log/zabbix/zabbix_problems.log
12:08:49 Training-VM-XX CPU Load is very high on Training-VM-XX

12:22:19 Training-VM-XX CPU Load is high on Training-VM-XX

CONGRATULATIONS!
Actions Lab is complete!
LAB 23
Internal Actions
INTERNAL ACTIONS
First you need to define message template for internal actions
Go to Administration -> Media types
You can use filters to find media types faster

Click on Training email

INTERNAL ACTIONS
Click on Add to create new message template

• Message type: Internal problem
• Subject: {ITEM.STATE}: {HOST.NAME}:{ITEM.NAME}
• Message:
Host: {HOST.NAME}
Item: {ITEM.NAME}
Key: {ITEM.KEY}
State: {ITEM.STATE}
Press Add

INTERNAL ACTIONS
Add another template for recovery
• Message type: Internal problem recovery
• Subject: {ITEM.STATE}: {HOST.NAME}:{ITEM.NAME}
• Message:
Host: {HOST.NAME}
Item: {ITEM.NAME}
Key: {ITEM.KEY}
State: {ITEM.STATE}
Press Add
Press Update on Media types page to update media types

INTERNAL ACTIONS
Go to Configuration -> Actions and choose Internal actions
It is very important to choose right action type when creating action !

Press Create action

INTERNAL ACTIONS
Fill the Name field: Report not supported SSH items
Click Add to add condition

• Type: Application
• Operator: equals
• Value: SSH checks
Click Add to add second condition
• Type: Event type
• Operator: equals
• Value: Item in "not supported" state

INTERNAL ACTIONS
Switch to Operations tab
Add Operation
• Send to Users: student-XX
• Send only to: Training email
Add Recovery Operation

• Operation type: Notify all involved

INTERNAL ACTIONS
Press Add to create action
Internal action is created

INTERNAL ACTIONS
Open SSH console and change password used for SSH checks to NEWremoteXX
# passwd monitor
Changing password for user monitor.

New password: NEWremoteXX
Retype new password: NEWremoteXX
passwd: all authentication tokens updated successfully.
Go to Configuration -> Hosts and look at ssh items for Training-VM-XX server
Both SSH items will turn to unsupported state in 1 minute (click on i to see details)
You can select item and press Execute now to do it faster

INTERNAL ACTIONS
Emails are sent for both SSH items

Open SSH console and restore password
# passwd monitor
Changing password for user monitor.

New password: sshremoteXX
Retype new password: sshremoteXX
passwd: all authentication tokens updated successfully.

INTERNAL ACTIONS
Go to Configuration -> Hosts and open Items for Training-VM-XX host
Select both SSH items and press Execute now
The items will become supported

INTERNAL ACTIONS
Go to Configuration -> Hosts and open Items for Training-VM-XX hosts
Emails are also sent about items returning to normal status

CONGRATULATIONS!
Internal actions Lab is complete!
LAB 24
Custom media types
INTERNAL ACTIONS
Open SSH console and create a new alert script
# vi /usr/lib/zabbix/alertscripts/logfile.sh
filename=$1
message=$2
now=$(date +"%D %T")
echo "$now $message" >> "/var/log/zabbix/$filename.log"
Make the file executable

# chmod +x /usr/lib/zabbix/alertscripts/logfile.sh
Go to Administration -> Media types and create a new media type

• Name: Write to log file
• Type: Script
• Script name: logfile.sh
• Script parameters:
• {ALERT.SENDTO}
• {ALERT.MESSAGE}

INTERNAL ACTIONS
Switch to Message templates tab
Add problem message type
• Message type: Problem
• Message: Host: {HOST.NAME} Problem: {EVENT.NAME}
Add problem recovery message type

• Message type: Problem recovery
• Message: Host: {HOST.NAME} Resolved: {EVENT.NAME}
Press Add to create new media type

INTERNAL ACTIONS
Go to User profile and switch to Media tab
Click Add to add another media

• Type: Write to log file
• Send to: student-xx
Press Add on Media form
Press Update on User profile form

INTERNAL ACTIONS
Go to Configuration -> Actions and click on Report training host problems action
Switch to Operations tab and edit Run remote commands on current host step

INTERNAL ACTIONS
Change Operation type to Send message for step 2
• Click Edit
• Change Operation type to Send message
• Send to users: student-xx
• Send only to: Write to log file
Press Update on Operation details form
Press Update on Action form

Open SSH console and generate CPU load

INTERNAL ACTIONS
After one minute Action with type Write to log file must succeed
Stop generating CPU load (press CTRL+C on SSH console}

Look at log file
# cat /var/log/zabbix/student-xx.log
06/02/20 15:44:22 Host: Training-VM-XX Problem: CPU Load is high on Training-VM-XX

06/02/20 15:45:52 Host: Training-VM-XX Resolved: CPU Load is high on Training-VM-XX
The recovery message may appear after some time, recheck the log file

CONGRATULATIONS!
Custom media types Lab is complete
LAB 25
Maintenance
MAINTENANCE
Go to Configuration -> Maintenance and press Create maintenance period

• Name: Training maintenance
• Maintenance type: With data collection
• Active since: Today at 00:00
• Active till: 2025-01-01 at 00:00
Switch to Periods tab

MAINTENANCE
Click on Add next to period type
Fill required fields in popup form
• Period type: One time only
• Date: Now + 5 minutes
• Maintenance period length: 0 Days 1 Hours 0 Minutes
Press Add on Maintenance period popup form

Switch to Hosts and groups tab
MAINTENANCE
Enter Training-VM-XX into Hosts field
You can click select and choose host from the list, or input host name directly in the field
Press Add to create maintenance period

Maintenance periods must be created before problem actually happens !

MAINTENANCE
Maintenance period is created
Open SSH console and generate a problem by using Zabbix sender

Zabbix sender is another simple way how to test trigger behavior
# zabbix_sender -z student-xx -s Training-VM-XX -k persons -o 1
Response from "student-xx:10051": "processed: 1; failed: 0; total: 1; seconds spent: 0.000135"

sent: 1; skipped: 0; total: 1
Go to Monitoring -> Problems
The problem is not displayed ! Is it now not detected at all ?

MAINTENANCE
The problem is hidden because it is suppressed !
Click on filter for Problems page and check Show suppressed problems
Press Apply
Notice that yellow wrench icon is displayed next to the host name. This means that this
host is in maintenance and problem is suppressed

MAINTENANCE
Ask on of your neighbors to send value 1 to his host name on your server
# zabbix_sender -z student-xx -s Training-VM-XY -k persons -o 1
Response from "student-xy:10051": "processed: 1; failed: 0; total: 1; seconds spent: 0.000135"

sent: 1; skipped: 0; total: 1
This command must be executed from host student-xy and sent to your server to host named
Training-VM-XY, otherwise it will be not accepted
The problem on another host is not in maintenance mode
Go to Monitoring -> Hosts

The maintenance symbol when host is in
maintenance mode is displayed in multiple places in
Zabbix frontend

CONGRATULATIONS!
Maintenance Lab is complete!
LAB 26
Business level monitoring
BUSINESS LEVEL MONITORING
Go to Configuration -> Services
Click Add child
Enter required information

• Name: Zabbix training SLA
• Status calculation algorithm: Problem, if at least one child has a problem
• Calculate SLA, acceptable SLA (in %): Check 95

Switch to Time tab
Enter New service time
• Period type: Uptime
• From: Monday Time 10:00
• Till: Monday Time 18:00
Define times for other working days the same way

• From: <day> Time 09:00
• Till: <day> Time 18:00
Repeat for every working day

The working time for training is defined
Now define lunch breaks for every day (replace <day> with working days
• Period type: Downtime
• From: <day> Time 12:00
• Till <day> Time 13:00
Repeat for every working day

The final schedule will look like this:
Press Add

Click Add child for Zabbix training SLA

• Name: Student-XX
• Trigger: Press Select
Press Add
Select correct Host - Training-VM-XX
Select CPU Load is very high on Training-VM-XX
Press Add

Click on small triangle to expand list
Repeat the same steps for Training-VM-XY and Training-VM-XZ hosts

• remember to choose correct host in trigger selection form !
Zabbix training SLA will fall down, if any of the SLA for student hosts will fall down

Go to Monitoring -> Services
Generate high CPU load on your machine
When CPU usage will be detected on one of the hosts, SLA for Zabbix training will start to
fall down

CONGRATULATIONS!
Business level monitoring Lab is complete !
LAB 27
Low Level Discovery
LOW LEVEL DISCOVERY
Link Template Module Linux filesystems by Zabbix agent to Training-VM-XX host
Press Update
Click on Discovery for Training-VM-XX host

LOW LEVEL DISCOVERY
Select Mounted filesystem discovery and press Execute now
Wait a few seconds and click on Items
Items and triggers for / filesystem are created automatically

Do not forget about configuration cache ! Items are created in database, and you need to reload
configuration cache or wait up to 1 minute before they will start collecting data

LOW LEVEL DISCOVERY
Data are also displayed in Latest data section for Training-VM-XX host
Now you have only one filesystem, but it is possible to add more
Open SSH console and issue command
# mkdir /zabbixlogs
# mount --bind /var/log/zabbix/ /zabbixlogs/
# mount -l | grep zabbix
/dev/vda1 on /zabbixlogs type xfs (rw,relatime,attr2,inode64,noquota)
This is a fake filesystem mounted only for training purposes !

LOW LEVEL DISCOVERY
Click on Discovery for Training-VM-XX host again
Select Mounted filesystem discovery and press Execute now
New filesystem will be discovered automatically in 1 hour, but we do not have so much time
Wait a few seconds and click on Items
New file system is discovered and monitored!

CONGRATULATIONS!
Low Level Discovery Lab is complete !
LAB 28
XML import / export
LOW LEVEL DISCOVERY
Go to Configuration -> Hosts
Select Training-VM-XX host and press Export
Host will be saved in downloads folder for your web browser

The file name will be zbx_export_hosts.xml
The file name is always zbx_export_hosts, hostname is defined inside XML file !

LOW LEVEL DISCOVERY
Open the file using text editor (like notepad)
Do not use MS WORD or other advanced text editors, they may add they own tags to the file and
break XML syntax
Change values for <host> and <name> fields
Change value for <dns> field
Save the zbx_export_hosts.xml file

LOW LEVEL DISCOVERY
Press Import on Configuration -> Hosts page
Press Choose file and browse the file
Press Import

LOW LEVEL DISCOVERY
Go to Configuration -> Hosts and search for the new host
Open configuration for Trainers Name VM host
Check that all hosts details including tags and macros are imported
CONGRATULATIONS!
XML import / export is complete !
The Enterprise class Monitoring Solution for Everyone
www.zabbix.com
Zabbix 5.0
Certified Specialist course
End of Day 4

ZCS5 Day 4 Lab Manual

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

ZCS5 Day 4 Lab Manual

Uploaded by

Copyright:

Available Formats

The Enterprise class Monitoring Solution for Everyone

LAB Manual Day 4

Press Add to create item

ZABBIX 5.0 Lab Manual 3

Press Add to create item

Link Template Basic module system info

ZABBIX 5.0 Lab Manual 4

Set Inventory mode to Automatic and change location field

ZABBIX 5.0 Lab Manual 5

ZABBIX 5.0 Lab Manual 6

Select Details tab

ZABBIX 5.0 Lab Manual 7

ZABBIX 5.0 Lab Manual 10

Switch to Message templates tab

ZABBIX 5.0 Lab Manual 11

Press Add on Message template form

ZABBIX 5.0 Lab Manual 12

Press Add on Message template form

Press Add on Media types form

Fill required fields:

Press Update on User profile form

ZABBIX 5.0 Lab Manual 14

Trigger actions needs to be selected as action type !

Click on Add to add new condition

ZABBIX 5.0 Lab Manual 15

Create another condition

Two conditions are now defined

ZABBIX 5.0 Lab Manual 16

Fill required fields

Press Add on Operation details form

ZABBIX 5.0 Lab Manual 17

Fill required fields

Press Add on Operation details form

Fill required fields

It is recommended to create recovery operations on actions to inform that problem is solved

ZABBIX 5.0 Lab Manual 19

Click Add to create action

ZABBIX 5.0 Lab Manual 20

# cat /dev/urandom | md5sum

Go to Monitoring -> Problems

Press Ctrl + C in SSH console to stop CPU load simulation

ZABBIX 5.0 Lab Manual 21

Remote command created log file, you can check it

12:08:49 Training-VM-XX CPU Load is very high on Training-VM-XX

ZABBIX 5.0 Lab Manual 22

You can use filters to find media types faster

ZABBIX 5.0 Lab Manual 25

Fill required fields

ZABBIX 5.0 Lab Manual 26

Press Update on Media types page to update media types

ZABBIX 5.0 Lab Manual 27

It is very important to choose right action type when creating action !

ZABBIX 5.0 Lab Manual 28

Click Add to add condition

ZABBIX 5.0 Lab Manual 29

Add Recovery Operation

ZABBIX 5.0 Lab Manual 30

Internal action is created

ZABBIX 5.0 Lab Manual 31

Changing password for user monitor.

ZABBIX 5.0 Lab Manual 32

Emails are sent for both SSH items

Changing password for user monitor.

ZABBIX 5.0 Lab Manual 33