Monitoring openvz resources using rrdtool
In this setup we're using rrdtool to collect data from several openvz hardware nodes to a central rrd server.
Contents
Setting up rrd server
Create a directory /var/rrd on central server.
mkdir /var/rrd
In /etc/xinetd.d create a file rrdsrv and adjust addresses and paths according to local settings.
service rrdsrv { disable = no socket_type = stream protocol = tcp wait = no user = root only_from = 192.168.2.0/24 server = /usr/bin/rrdtool server_args = - /var/rrd }
Add 'rrdsrv' to /etc/services
rrdsrv 13900/tcp
Collect script
Put the following script on the openvz hardware nodes (ubc_rrd_collect.sh):
#!/bin/sh PATH="/bin:/sbin:/usr/bin:/usr/sbin" SENDCOMMAND='netcat 192.168.2.1 13900' HEARTBEAT=600 function error { echo $1 exit 1 } test -r /proc/bc/0/resources || error "Not openvz kernel?" command='cat' init=0 update=0 send=0 while [ ! -z "$1" ]; do case "$1" in send) send=1 command=$SENDCOMMAND ;; init) init=1 ;; update) update=1 ;; esac shift done VPSIDS=$(vzlist -H -o veid) PARAMS=$(cat /proc/bc/0/resources | awk '{ print $1 }'| sort -u | xargs) if [ "$send" -eq 1 ]; then # to spread the load we sleep for a random while (between 0 and 60 sec) number=$RANDOM let "number %= 60" sleep $number fi for VPSID in $VPSIDS; do RRDFILENAME="ubc_${VPSID}.rrd" if [ "$init" -eq 1 ]; then exists=0 if [ "$send" -eq 1 ]; then echo "info $RRDFILENAME" | $command | grep -q "No such file or directory" || exists=1 fi if [ "$exists" -eq 0 ]; then ( echo -n "create ${RRDFILENAME} " for param in $PARAMS; do echo -n "DS:${param}_f:DERIVE:$HEARTBEAT:0:U " echo -n "DS:${param}_h:GAUGE:$HEARTBEAT:0:U " echo -n "DS:${param}_b:GAUGE:$HEARTBEAT:0:U " done echo -n "RRA:AVERAGE:0.5:1:600 " echo -n "RRA:AVERAGE:0.5:6:700 " echo -n "RRA:AVERAGE:0.5:24:775 " echo -n "RRA:AVERAGE:0.5:288:797 " echo -n "RRA:MAX:0.5:1:600 " echo -n "RRA:MAX:0.5:6:700 " echo -n "RRA:MAX:0.5:24:775 " echo "RRA:MAX:0.5:288:797" ) | $command fi fi if [ "$update" -eq 1 ]; then ( cat /proc/bc/$VPSID/resources | awk -v rrdfile=$RRDFILENAME ' BEGIN { ORS=""; getline keys=$1 "_f"; values=$6; keys=keys ":" $1 "_h"; values=values ":" $2; keys=keys ":" $1 "_b"; values=values ":" $4; } { keys=keys ":" $1 "_f"; values=values ":" $6; keys=keys ":" $1 "_h"; values=values ":" $2; keys=keys ":" $1 "_b"; values=values ":" $4; } END { print "update " rrdfile " -t " keys " N:" values "\n"; } ' ) | $command fi done
Usage
ubc_rrd_collect.sh [update] [init] [send] update - run this to update rrd files init - scan system and create rrd files send - use this to send commands to rddtool
Migration and new container
When you have this script running in cron on every hardware node and you're migrating a container from one node to the other the script will automatically detect this and pick it up on the other node.
When you create a new container you have to run ubc_rrd_collect.sh init send
and it will ping rddtool to create a new rrd database. It won't overwrite existing databases because it first checks if a database already exists.
Setting up cron
Run update every 5 minutes:
*/5 * * * * /path/to/ubc_rrd_collect.sh update send | grep -v '^OK'
Run init every hour (or run it manually when needed):
0 * * * * /path/to/ubc_rrd_collect.sh init send | grep -v '^OK'