Editing Monitoring openvz resources using rrdtool

[[Category: Monitoring]]

In this setup there is a central rrd server that collects data from several openvz hardware nodes

=== Setting up rrd server ===

Create a directory /var/rrd on central server.

 mkdir /var/rrd

In /etc/xinetd.d create a file rrdsrv and adjust addresses and paths according to local settings.

 service rrdsrv
 {
   disable         = no
   socket_type     = stream
   protocol        = tcp
   wait            = no
   user            = root
   only_from       = 192.168.2.0/24
   server          = /usr/bin/rrdtool
   server_args     = - /var/rrd
 }

Add service in /etc/services
 rrdsrv                13900/tcp


=== Collect script ===

Put the following script on the openvz hardware nodes:

 #!/bin/sh
 
 PATH="/bin:/sbin:/usr/bin:/usr/sbin"
 SENDCOMMAND='netcat 192.168.2.1 13900'
 HEARTBEAT=600
 
 function error {
         echo $1
         exit 1
 }
 
 test -r /proc/bc/0/resources || error "Not openvz kernel?"
 
 command='cat'
 init=0
 update=0
 send=0
 
 while [ ! -z "$1" ]; do
         case "$1" in
                 send)
                         send=1
                         command=$SENDCOMMAND
                         ;;
                 init)
                         init=1
                         ;;
                 update)
                         update=1
                         ;;
         esac
         shift
 done
 
 VPSIDS=$(vzlist | awk ' BEGIN { ORS=" "; getline } { print $1 }')
 
 PARAMS=$(cat /proc/bc/0/resources | awk '{ print $1 }'| sort -u | xargs)
 if [ "$send" -eq 1 ]; then
         # to spread the load we sleep for a random while (between 0 and 60 sec)
         number=$RANDOM
         let "number %= 60"
         sleep $number
 fi
 
 for VPSID in $VPSIDS; do
        RRDFILENAME="ubc_${VPSID}.rrd" 
 
         if [ "$init" -eq 1 ]; then
                 exists=0
                 if [ "$send" -eq 1 ]; then
                         echo "info $RRDFILENAME" | $command | grep -q "No such file or directory" || exists=1
                 fi      
                 
                 if [ "$exists" -eq 0 ]; then
                 (
                         echo -n "create ${RRDFILENAME} "
                         for param in $PARAMS; do
                                 echo -n "DS:${param}_f:DERIVE:$HEARTBEAT:0:U "
                                 echo -n "DS:${param}_h:GAUGE:$HEARTBEAT:0:U "
                                 echo -n "DS:${param}_b:GAUGE:$HEARTBEAT:0:U "
                         done
                         echo -n "RRA:AVERAGE:0.5:1:600 "
                         echo -n "RRA:AVERAGE:0.5:6:700 "
                         echo -n "RRA:AVERAGE:0.5:24:775 "
                         echo -n "RRA:AVERAGE:0.5:288:797 "
                         echo -n "RRA:MAX:0.5:1:600 "
                         echo -n "RRA:MAX:0.5:6:700 "
                         echo -n "RRA:MAX:0.5:24:775 "
                         echo "RRA:MAX:0.5:288:797"
                 ) | $command
                 fi
         fi
@@ Line 1: / Line 1: @@
 [[Category: Monitoring]]
-In this setup we're using [http://oss.oetiker.ch/rrdtool/ rrdtool] to collect data from several openvz hardware nodes to a central [http://oss.oetiker.ch/rrdtool/doc/rrdtool.en.html#IRRD_Server rrd server].
+In this setup there is a central rrd server that collects data from several openvz hardware nodes
 === Setting up rrd server ===
@@ Line 23: / Line 23: @@
   }
-Add 'rrdsrv' to /etc/services
+Add service in /etc/services
   rrdsrv                13900/tcp
@@ Line 29: / Line 29: @@
 === Collect script ===
-Put the following script on the openvz hardware nodes (ubc_rrd_collect.sh):
+Put the following script on the openvz hardware nodes:
   #!/bin/sh
@@ Line 65: / Line 65: @@
   done
-  VPSIDS=$(vzlist -H -o veid)
+  VPSIDS=$(vzlist | awk ' BEGIN { ORS=" "; getline } { print $1 }')
   PARAMS=$(cat /proc/bc/0/resources | awk '{ print $1 }'| sort -u | xargs)
@@ Line 103: / Line 103: @@
                   fi
           fi
-         if [ "$update" -eq 1 ]; then
-                 (
-                         cat /proc/bc/$VPSID/resources | awk -v rrdfile=$RRDFILENAME '
-                         BEGIN {
-                                 ORS="";
-                                 getline
-                                 keys=$1 "_f"; values=$6;
-                                 keys=keys ":" $1 "_h"; values=values ":" $2;
-                                 keys=keys ":" $1 "_b"; values=values ":" $4;
-                         }
-                         {
-                                 keys=keys ":" $1 "_f"; values=values ":" $6;
-                                 keys=keys ":" $1 "_h"; values=values ":" $2;
-                                 keys=keys ":" $1 "_b"; values=values ":" $4;
-                         }
-                         END { print "update " rrdfile " -t " keys " N:" values "\n"; } '
-                 ) | $command
-         fi
- done
-=== Usage ===
- ubc_rrd_collect.sh [update] [init] [send]
-   update - run this to update rrd files
-   init   - scan system and create rrd files
-   send   - use this to send commands to rddtool
-=== Migration and new container ===
-When you have this script running in cron on every hardware node and you're migrating a container from one node to the other the script will automatically detect this and pick it up on the other node.
-When you create a new container you have to run <code>ubc_rrd_collect.sh init send</code> and it will ping rddtool to create a new rrd database. It won't overwrite existing databases because it first checks if a database already exists.
-=== Setting up cron ===
-Run update every 5 minutes:
- */5     *       *       *       *       /path/to/ubc_rrd_collect.sh update send | grep -v '^OK'
-Run init every hour (or run it manually when needed):
-       *       *       *       *       /path/to/ubc_rrd_collect.sh init send | grep -v '^OK'