Raymond Kuiper - Working the API like a Unix Pro

Communicating with the Zabbix API can be quite cumbersome, especially if you don't have a background as a programmer. For a sysadmin, it would be very nice if one could just run some CLI commands to control Zabbix behavior.

Wouldn't it be wonderful if you could fetch a list of active triggers and parse it with grep or sed to find the specific triggers you are looking for? Or perhaps you need a list of historic values that you can parse in a custom script? How about a cronjob that downloads and emails all the graphs in the system matching a certain regex?

In this presentation Raymond Kuiper will talk about some of these possibilities and show you how he achieved these things in his Zabbix setup.

Zabbix Conference 2015

1. L
a
y
o
u
t

b
y

o
r
n
g
j
c
e
2
2
3
,

C
C
-
B
Y
ZabConf2015 – Sept 11 2015
---
Raymond Kuiper
Working the API like a Unix Pro

2. L
a
y
o
u
t

b
y

o
r
n
g
j
c
e
2
2
3
,

C
C
-
B
Y
Quick introduction
●@RaymondKuiper
●Zabbix user since 2006, never looked back
●q1x on Freenode (fnd me in #zabbix)
●Proud to work for these guys:

3. L
a
y
o
u
t

b
y

o
r
n
g
j
c
e
2
2
3
,

C
C
-
B
Y
Sysadmins <3 Automation

4. L
a
y
o
u
t

b
y

o
r
n
g
j
c
e
2
2
3
,

C
C
-
B
Y
Automation in Zabbix

5. L
a
y
o
u
t

b
y

o
r
n
g
j
c
e
2
2
3
,

C
C
-
B
Y
How to automate via the API?

6. L
a
y
o
u
t

b
y

o
r
n
g
j
c
e
2
2
3
,

C
C
-
B
Y
API bash script using Curl
Pros:
●Seems nice and easy at frst
●You can show of your mad curl skills

7. L
a
y
o
u
t

b
y

o
r
n
g
j
c
e
2
2
3
,

C
C
-
B
Y
API bash script using Curl
Cons:
●No object awareness
●Honestly, it's a PITA

8. L
a
y
o
u
t

b
y

o
r
n
g
j
c
e
2
2
3
,

C
C
-
B
Y
Develop an API based tool
Pros:
●Perfect solution to your problem
●Highly efcient code

9. L
a
y
o
u
t

b
y

o
r
n
g
j
c
e
2
2
3
,

C
C
-
B
Y
Develop an API based tool
Cons:
●Takes time and efort to develop (and
debug) a ftting solution
●Not very fexible unless you invest heavily
in development (monolithic)
●Gets very complex, very fast. You might
need a developer to maintain it

10. L
a
y
o
u
t

b
y

o
r
n
g
j
c
e
2
2
3
,

C
C
-
B
Y
Use a CLI tool
(Zabcon and friends)
Pros:
●Can be used in shell scripts
●Can be used quickly, no need for direct
API code

11. L
a
y
o
u
t

b
y

o
r
n
g
j
c
e
2
2
3
,

C
C
-
B
Y
Use a CLI tool
(Zabcon and friends)
Cons:
●Learning a new CLI language
●Limited to the functionality the developer
thought was useful

12. L
a
y
o
u
t

b
y

o
r
n
g
j
c
e
2
2
3
,

C
C
-
B
Y
My solution?

13. L
a
y
o
u
t

b
y

o
r
n
g
j
c
e
2
2
3
,

C
C
-
B
Y
*nix
The IT swiss army knife

14. L
a
y
o
u
t

b
y

o
r
n
g
j
c
e
2
2
3
,

C
C
-
B
Y
*nix
The IT swiss army knife
“The whole is greater than the sum of its
parts.”
― Aristotle

15. L
a
y
o
u
t

b
y

o
r
n
g
j
c
e
2
2
3
,

C
C
-
B
Y
Introducing:
the Zabbix Gnomes

16. L
a
y
o
u
t

b
y

o
r
n
g
j
c
e
2
2
3
,

C
C
-
B
Y
Zabbix Gnomes
https://github.com/q1x/zabbix-gnomes

17. L
a
y
o
u
t

b
y

o
r
n
g
j
c
e
2
2
3
,

C
C
-
B
Y
Zabbix Gnomes
~/.zbx.conf
[Zabbix API]
username=johndoe
password=verysecretpassword
api=https://zabbix.mycompany.com/path/to/zabbix/frontend/
no_verify=true

18. L
a
y
o
u
t

b
y

o
r
n
g
j
c
e
2
2
3
,

C
C
-
B
Y
Zabbix Gnomes
Making them work together

19. L
a
y
o
u
t

b
y

o
r
n
g
j
c
e
2
2
3
,

C
C
-
B
Y
Ex. 1: Hostgroup Templates
Scenario:
●Hosts in the host group 'App Servers' should
be linked to 'Template App Customapp'
●$someone keeps forgetting to link that
template to newly deployed App servers
●Post-it notes about this don't seem to work

20. L
a
y
o
u
t

b
y

o
r
n
g
j
c
e
2
2
3
,

C
C
-
B
Y
Ex. 1: Hostgroup Templates
user@localhost$ ./zghostfinder.py 'App Servers'
app-server-001
app-server-002
...
app-server-042
app-server-043
(Finds hosts in a host group)

21. L
a
y
o
u
t

b
y

o
r
n
g
j
c
e
2
2
3
,

C
C
-
B
Y
Ex. 1: Hostgroup Templates
user@localhost$ ./zhtmplfinder.py 'app-server-001'
Template OS Linux
Template App Customapp
user@localhost$ ./zhtmplfinder.py 'app-server-042'
Template OS Linux
(Finds templates linked to a host)

22. L
a
y
o
u
t

b
y

o
r
n
g
j
c
e
2
2
3
,

C
C
-
B
Y
Ex. 1: Hostgroup Templates
user@localhost$ ./zhtmpllinker.py
usage: zhtmpllinker.py [-h]
(-H HOSTNAMES [HOSTNAMES ...] | -G
HOSTGROUPS [HOSTGROUPS ...])
-t TEMPLATES [TEMPLATES ...]
user@localhost$ ./zhtmpllinker.py -G 'App Servers' \
-t 'Template App Customapp'
(Links templates to hosts)

23. L
a
y
o
u
t

b
y

o
r
n
g
j
c
e
2
2
3
,

C
C
-
B
Y
Ex. 1: Hostgroup Templates
user@localhost$ ./zhtmpllinker.py -G 'App Servers' \
-t 'Template App Customapp'
*Pro-tip: Stick it in a crontab!*

24. L
a
y
o
u
t

b
y

o
r
n
g
j
c
e
2
2
3
,

C
C
-
B
Y
Ex. 1: Hostgroup Templates
user@localhost$ for host in $(./zghostfinder.py 'Netco Test');
do echo "$host : $(./zhtmplfinder.py "$host" | tr '\n' ',')";
done
app-server-001 : Template OS Linux,Template App
app-server-002 : Template OS Linux,Template App
...
app-server-042 : Template OS Linux,Template App
app-server-043 : Template OS Linux,Template App

25. L
a
y
o
u
t

b
y

o
r
n
g
j
c
e
2
2
3
,

C
C
-
B
Y
Ex. 2: Host Inventory
Scenario:
●$bossman has negotiated a new hardware
support contract with $vendor
●Demands serial/os/type of all Cisco 800s
●Your template stores this info in the Inventory
●You are a happy user of network discovery

26. L
a
y
o
u
t

b
y

o
r
n
g
j
c
e
2
2
3
,

C
C
-
B
Y
Ex. 2: Host Inventory
user@localhost$ ./zhinvswitcher.py
usage: zhinvswitcher.py [-h] (-H HOSTNAMES [HOSTNAMES ...]
| -G HOSTGROUPS [HOSTGROUPS ...]
| --all-hosts) … [-m MODE] …
(Switches Inv. Mode on hosts)
user@localhost$ ./zhinvswitcher.py -G “Cisco Devices” -m autouser@localhost$ ./zhinvswitcher.py -G “Cisco Devices” -m auto
(See also ZBXNEXT-1241)

27. L
a
y
o
u
t

b
y

o
r
n
g
j
c
e
2
2
3
,

C
C
-
B
Y
Ex. 2: Host Inventory
user@localhost$ ./zgetinventory.py
usage: zgetinventory.py [-h] (-H HOSTNAMES [HOSTNAMES ...]
| -G HOSTGROUPS [HOSTGROUPS …]) …
[-m] [-i] … (-A | -F FIELDS [FIELDS ...])
(Gets Zabbix inventory information)

28. L
a
y
o
u
t

b
y

o
r
n
g
j
c
e
2
2
3
,

C
C
-
B
Y
Ex. 2: Host Inventory
user@localhost$ ./zgetinventory.py -G 'Cisco Devices ' \
-e -m -i -F serialno_a type os | \
sed -n -e 1p -e '/C8../'p | \
cut -d ',' -f 2-
"host","serialno_a","type","os"
"RT01","FCZ999999A","C819G-4G-G-K9","15.4(3)M1, RELEASE SOFTWARE
(fc1)"
"RT02","FCZ999999B","C887VA-K9","15.4(1)T1, RELEASE SOFTWARE
(fc2)"
"RT03","FCZ999999C","C887VA-K9","15.4(3)M1, RELEASE SOFTWARE
(fc1)"
...

29. L
a
y
o
u
t

b
y

o
r
n
g
j
c
e
2
2
3
,

C
C
-
B
Y
Ex. 3: Emailing Graphs
Scenario:
●$colleague needs daily CPU graphs
●Can't be bothered with browsing the GUI
every morning before getting cofee
●You'd like to help him reach his mailbox quota

30. L
a
y
o
u
t

b
y

o
r
n
g
j
c
e
2
2
3
,

C
C
-
B
Y
Ex. 3: Emailing Graphs
user@localhost$ ./zghostfinder.py 'Linux Servers'
Wakizashi
Kodachi
Katana

31. L
a
y
o
u
t

b
y

o
r
n
g
j
c
e
2
2
3
,

C
C
-
B
Y
Ex. 3: Emailing Graphs
user@localhost$ ./zhgraphfinder.py -e Kodachi
568:Network traffic on eth0
555:CPU jumps
556:CPU load
557:CPU utilization
558:Swap usage
569:Disk space usage /
560:Memory usage
(Finds graphs confgured on a host)

32. L
a
y
o
u
t

b
y

o
r
n
g
j
c
e
2
2
3
,

C
C
-
B
Y
Ex. 3: Emailing Graphs
user@localhost$ ./zgetgraph.py -f graph.png 557
(Downloads a graph from the frontend)

33. L
a
y
o
u
t

b
y

o
r
n
g
j
c
e
2
2
3
,

C
C
-
B
Y
#!/bin/bash
graphs=""
filelist=""
# find graphs on hosts in group 'Linux Servers'
for host in $(zghostfinder.py "Linux Servers"); do
hgraphs=$(zhgraphfinder.py -e $host | grep "CPU utilization" \
| cut -d ':' -f 1 )
graphs="$graphs $hgraphs"
done
# download graphs
for graph in $graphs; do
file="/tmp/$graph.png"
filelist="$filelist -a $file"
zgetgraph.py -t 86400 -f "$file" "$graph"
done
# Mail the graphs and cleanup
echo "See attached." | mail $filelist -s "Daily graphs"
"[email protected]"
rm -r $(echo "$filelist" | sed 's/-a\ //g')
Ex. 3: Emailing Graphs

34. L
a
y
o
u
t

b
y

o
r
n
g
j
c
e
2
2
3
,

C
C
-
B
Y
Things to think about

35. L
a
y
o
u
t

b
y

o
r
n
g
j
c
e
2
2
3
,

C
C
-
B
Y
Questions?