Difference between revisions of "Monitoring openvz resources using rrdtool"
(→Collect script) |
(Fixed headline) |
||
| (6 intermediate revisions by 2 users not shown) | |||
| Line 1: | Line 1: | ||
[[Category: Monitoring]] | [[Category: Monitoring]] | ||
| − | In this setup | + | In this setup we're using [http://oss.oetiker.ch/rrdtool/ rrdtool] to collect data from several openvz hardware nodes to a central [http://oss.oetiker.ch/rrdtool/doc/rrdtool.en.html#IRRD_Server rrd server]. |
=== Setting up rrd server === | === Setting up rrd server === | ||
| Line 23: | Line 23: | ||
} | } | ||
| − | Add | + | Add 'rrdsrv' to /etc/services |
rrdsrv 13900/tcp | rrdsrv 13900/tcp | ||
| Line 29: | Line 29: | ||
=== Collect script === | === Collect script === | ||
| − | Put the following script on the openvz hardware nodes: | + | Put the following script on the openvz hardware nodes (ubc_rrd_collect.sh): |
#!/bin/sh | #!/bin/sh | ||
| Line 65: | Line 65: | ||
done | done | ||
| − | VPSIDS=$(vzlist | + | VPSIDS=$(vzlist -H -o veid) |
PARAMS=$(cat /proc/bc/0/resources | awk '{ print $1 }'| sort -u | xargs) | PARAMS=$(cat /proc/bc/0/resources | awk '{ print $1 }'| sort -u | xargs) | ||
| Line 123: | Line 123: | ||
fi | fi | ||
done | done | ||
| + | |||
| + | === Usage === | ||
| + | |||
| + | ubc_rrd_collect.sh [update] [init] [send] | ||
| + | |||
| + | update - run this to update rrd files | ||
| + | init - scan system and create rrd files | ||
| + | send - use this to send commands to rddtool | ||
| + | |||
| + | === Migration and new container === | ||
| + | |||
| + | When you have this script running in cron on every hardware node and you're migrating a container from one node to the other the script will automatically detect this and pick it up on the other node. | ||
| + | |||
| + | When you create a new container you have to run <code>ubc_rrd_collect.sh init send</code> and it will ping rddtool to create a new rrd database. It won't overwrite existing databases because it first checks if a database already exists. | ||
| + | |||
| + | === Setting up cron === | ||
| + | |||
| + | Run update every 5 minutes: | ||
| + | |||
| + | */5 * * * * /path/to/ubc_rrd_collect.sh update send | grep -v '^OK' | ||
| + | |||
| + | Run init every hour (or run it manually when needed): | ||
| + | 0 * * * * /path/to/ubc_rrd_collect.sh init send | grep -v '^OK' | ||
Latest revision as of 11:38, 11 July 2008
In this setup we're using rrdtool to collect data from several openvz hardware nodes to a central rrd server.
Contents
Setting up rrd server[edit]
Create a directory /var/rrd on central server.
mkdir /var/rrd
In /etc/xinetd.d create a file rrdsrv and adjust addresses and paths according to local settings.
service rrdsrv
{
disable = no
socket_type = stream
protocol = tcp
wait = no
user = root
only_from = 192.168.2.0/24
server = /usr/bin/rrdtool
server_args = - /var/rrd
}
Add 'rrdsrv' to /etc/services
rrdsrv 13900/tcp
Collect script[edit]
Put the following script on the openvz hardware nodes (ubc_rrd_collect.sh):
#!/bin/sh
PATH="/bin:/sbin:/usr/bin:/usr/sbin"
SENDCOMMAND='netcat 192.168.2.1 13900'
HEARTBEAT=600
function error {
echo $1
exit 1
}
test -r /proc/bc/0/resources || error "Not openvz kernel?"
command='cat'
init=0
update=0
send=0
while [ ! -z "$1" ]; do
case "$1" in
send)
send=1
command=$SENDCOMMAND
;;
init)
init=1
;;
update)
update=1
;;
esac
shift
done
VPSIDS=$(vzlist -H -o veid)
PARAMS=$(cat /proc/bc/0/resources | awk '{ print $1 }'| sort -u | xargs)
if [ "$send" -eq 1 ]; then
# to spread the load we sleep for a random while (between 0 and 60 sec)
number=$RANDOM
let "number %= 60"
sleep $number
fi
for VPSID in $VPSIDS; do
RRDFILENAME="ubc_${VPSID}.rrd"
if [ "$init" -eq 1 ]; then
exists=0
if [ "$send" -eq 1 ]; then
echo "info $RRDFILENAME" | $command | grep -q "No such file or directory" || exists=1
fi
if [ "$exists" -eq 0 ]; then
(
echo -n "create ${RRDFILENAME} "
for param in $PARAMS; do
echo -n "DS:${param}_f:DERIVE:$HEARTBEAT:0:U "
echo -n "DS:${param}_h:GAUGE:$HEARTBEAT:0:U "
echo -n "DS:${param}_b:GAUGE:$HEARTBEAT:0:U "
done
echo -n "RRA:AVERAGE:0.5:1:600 "
echo -n "RRA:AVERAGE:0.5:6:700 "
echo -n "RRA:AVERAGE:0.5:24:775 "
echo -n "RRA:AVERAGE:0.5:288:797 "
echo -n "RRA:MAX:0.5:1:600 "
echo -n "RRA:MAX:0.5:6:700 "
echo -n "RRA:MAX:0.5:24:775 "
echo "RRA:MAX:0.5:288:797"
) | $command
fi
fi
if [ "$update" -eq 1 ]; then
(
cat /proc/bc/$VPSID/resources | awk -v rrdfile=$RRDFILENAME '
BEGIN {
ORS="";
getline
keys=$1 "_f"; values=$6;
keys=keys ":" $1 "_h"; values=values ":" $2;
keys=keys ":" $1 "_b"; values=values ":" $4;
}
{
keys=keys ":" $1 "_f"; values=values ":" $6;
keys=keys ":" $1 "_h"; values=values ":" $2;
keys=keys ":" $1 "_b"; values=values ":" $4;
}
END { print "update " rrdfile " -t " keys " N:" values "\n"; } '
) | $command
fi
done
Usage[edit]
ubc_rrd_collect.sh [update] [init] [send] update - run this to update rrd files init - scan system and create rrd files send - use this to send commands to rddtool
Migration and new container[edit]
When you have this script running in cron on every hardware node and you're migrating a container from one node to the other the script will automatically detect this and pick it up on the other node.
When you create a new container you have to run ubc_rrd_collect.sh init send and it will ping rddtool to create a new rrd database. It won't overwrite existing databases because it first checks if a database already exists.
Setting up cron[edit]
Run update every 5 minutes:
*/5 * * * * /path/to/ubc_rrd_collect.sh update send | grep -v '^OK'
Run init every hour (or run it manually when needed):
0 * * * * /path/to/ubc_rrd_collect.sh init send | grep -v '^OK'
