Editing Backup a running container over the network with ezvzdump

Jump to: navigation, search

Warning: You are not logged in. Your IP address will be publicly visible if you make any edits. If you log in or create an account, your edits will be attributed to your username, along with other benefits.

The edit can be undone. Please check the comparison below to verify that this is what you want to do, and then save the changes below to finish undoing the edit.
Latest revision Your text
Line 1: Line 1:
 
== Ezvzdump ==
 
== Ezvzdump ==
  
−
Ezvzdump (not to be confused with [[vzdump]]) is a shell script that also allows  
+
Ezvzdump (not to be confused with vzdump) is a shell script that also allows  
 
you to dump out a running VE container. The key differences are that ezvzdump  
 
you to dump out a running VE container. The key differences are that ezvzdump  
 
allows you to utilize past dumps to speed things up, and that a remote host can  
 
allows you to utilize past dumps to speed things up, and that a remote host can  
Line 7: Line 7:
  
 
The dump files that are created are compatible with those that vzdump creates,  
 
The dump files that are created are compatible with those that vzdump creates,  
−
so you must still use <code>vzdump --restore</code> to restore them.
+
so you must still use `vzdump --restore` to restore them.
  
−
Dependencies: sendEmail package for emailing of statistics
 
−
 
−
== Cron ==
 
−
I recommend using <code>nice</code><code></code> and <code>ionice</code> to run this script so it doesn't bring the host to it's knees. Place this example in <code>/etc/cron.d/ezvzdump</code> and the backups will run at 3:30 AM every day.
 
−
 
−
<pre>
 
−
# /etc/cron.d/ezvzdump
 
−
SHELL=/bin/bash  #This is necessary on Ubuntu where dash is the default shell and is thus used by cron.
 
−
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
 
−
 
−
30 3 * * * root /usr/bin/nice -n19 /usr/bin/ionice -c3 /usr/local/scripts/ezvzdump > /dev/null 2>&1
 
−
</pre>
 
−
 
−
== Changes ==
 
−
 
−
2009/7/24 - JBiel - Added support for GZip compression and backup
 
−
retention/rotation. Also added support for emailing of statistics.
 
−
 
−
2015-03-18 - OSiUX - fork script in https://github.com/gcoop-libre/plan-b
 
  
 
== Download ==
 
== Download ==
Line 34: Line 15:
  
 
#!/bin/bash
 
#!/bin/bash
−
+
#
 
# ezvzdump  
 
# ezvzdump  
−
 
+
#
 
# Copyright (C) 2008 Alex Lance (alla at cyber.com.au)
 
# Copyright (C) 2008 Alex Lance (alla at cyber.com.au)
 
# Sponsored by Silverband Pty. Ltd.  
 
# Sponsored by Silverband Pty. Ltd.  
−
+
#
 
# This program is free software: you can redistribute it and/or modify
 
# This program is free software: you can redistribute it and/or modify
 
# it under the terms of the GNU General Public License as published by the Free
 
# it under the terms of the GNU General Public License as published by the Free
 
# Software Foundation, either version 3 of the License, or (at your option) any
 
# Software Foundation, either version 3 of the License, or (at your option) any
 
# later version.
 
# later version.
−
+
#
 
# This program is distributed in the hope that it will be useful, but
 
# This program is distributed in the hope that it will be useful, but
 
# WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
 
# WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
 
# FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
 
# FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
 
# more details: http://www.gnu.org/licenses/gpl.txt
 
# more details: http://www.gnu.org/licenses/gpl.txt
−
+
#
−
 
+
#
 
# Instructions  
 
# Instructions  
 
# ------------
 
# ------------
−
 
+
#
 
# This script rsyncs a VE to a specified local directory, suspends the running
 
# This script rsyncs a VE to a specified local directory, suspends the running
 
# VE, rsyncs again, and then resumes the VE. This creates a stable snapshot of
 
# VE, rsyncs again, and then resumes the VE. This creates a stable snapshot of
 
# the VE directories with minimal downtime.
 
# the VE directories with minimal downtime.
−
 
+
#
 
# Once the VE has been dumped out locally, it is rsynced to a remote host. When
 
# Once the VE has been dumped out locally, it is rsynced to a remote host. When
 
# the rsync has completed, a tar archive is created on the remote host. The tar
 
# the rsync has completed, a tar archive is created on the remote host. The tar
 
# archive is compatible with the vzdump tar format, so the VE tar file may be
 
# archive is compatible with the vzdump tar format, so the VE tar file may be
 
# later restored with the `vzdump --restore` command.
 
# later restored with the `vzdump --restore` command.
−
 
+
#
 
# By tarring the files together on the remote host, the burden of creating
 
# By tarring the files together on the remote host, the burden of creating
 
# the tar archive is taken away from the hardware node, and given to the remote
 
# the tar archive is taken away from the hardware node, and given to the remote
 
# host / backup server. This ensures that minimal additional CPU/disk resources
 
# host / backup server. This ensures that minimal additional CPU/disk resources
 
# are used on the machine that is running the VE's.
 
# are used on the machine that is running the VE's.
−
 
+
#
 
# This script runs slow the first time you use it, but from then on it utilizes
 
# This script runs slow the first time you use it, but from then on it utilizes
 
# the local and remote stored snapshot directories so that the rsyncs happen
 
# the local and remote stored snapshot directories so that the rsyncs happen
 
# expediently. The script does not wait for the remote tar process to complete,
 
# expediently. The script does not wait for the remote tar process to complete,
 
# it simply kicks off the tar archive creation and then immediately continues.
 
# it simply kicks off the tar archive creation and then immediately continues.
−
 
+
#
 
# This script uses rsync and ssh and assumes that you already have ssh keys
 
# This script uses rsync and ssh and assumes that you already have ssh keys
 
# set up between your hosts. This script was written because vzdump takes too
 
# set up between your hosts. This script was written because vzdump takes too
Line 78: Line 59:
 
# backups. This script also puts less strain on the hardware node by finishing
 
# backups. This script also puts less strain on the hardware node by finishing
 
# backups more quickly, and making the remote host do the heavy lifting.
 
# backups more quickly, and making the remote host do the heavy lifting.
 +
#
  
−
# NASTY BUG! Make sure you're not "vzctl enter"ing a VE while it's being
 
−
# suspended. On my machine (Ubuntu 2.6.24-24-openvz) it caused the VE to
 
−
# zombifiy and I had to reboot the host (including killing the VZ shutdown
 
−
# script because it was unable to shutdown the VE/processes.) Just don't do it!
 
  
 
# This section contains variables that require user customisation.  
 
# This section contains variables that require user customisation.  
Line 89: Line 67:
 
# This variable contains a space delimited list of VEID's to be backed up.
 
# This variable contains a space delimited list of VEID's to be backed up.
 
# You can use VEIDS="$(cd /vz/private/ && echo *)"  to backup all your VE's.
 
# You can use VEIDS="$(cd /vz/private/ && echo *)"  to backup all your VE's.
−
VEIDS="101 102 103 104"
+
VEIDS="1000 2000 3000"            
  
 
VZ_CONF="/etc/vz/conf/"            # the path to your openvz $VEID.conf files
 
VZ_CONF="/etc/vz/conf/"            # the path to your openvz $VEID.conf files
 
VZ_PRIVATE="/vz/private/"          # the path to the running VE's
 
VZ_PRIVATE="/vz/private/"          # the path to the running VE's
−
LOCAL_DIR="/vz/ezvzdump/"          # the local rsync cache / destination directory
+
LOCAL_DIR="/vz/ezvzdump/"          # the local rsync cache / destination directory  
−
 
+
 
# The remote host and path that this script will rsync the VE's to.
 
# The remote host and path that this script will rsync the VE's to.
 
REMOTE_HOST="somehost.example.com"
 
REMOTE_HOST="somehost.example.com"
 
REMOTE_DIR="/backup/"
 
REMOTE_DIR="/backup/"
−
 
+
 
# Default rsync flags (please note the potentially unsafe delete flags).
 
# Default rsync flags (please note the potentially unsafe delete flags).
 
# You can also remove the v flag to get less verbose logging.
 
# You can also remove the v flag to get less verbose logging.
−
RSYNC_DEFAULT="rsync -ravH --delete-after --delete-excluded"
+
RSYNC_DEFAULT="rsync -ravH --delete-after --delete-excluded"  
−
 
+
RSYNC_EXCLUDE="/usr/portage /var/log"
−
# Exclude these directories from backup (space delimited).
 
−
# I left /var/log in the backup because when doing a full restore
 
−
# it's necessary that this directory structure is present.
 
−
RSYNC_EXCLUDE="/usr/portage"
 
  
 
# Path to vzctl executable
 
# Path to vzctl executable
 
VZCTL="vzctl"
 
VZCTL="vzctl"
 +
  
 
# Nice debugging messages...
 
# Nice debugging messages...
−
function e {
+
function e {  
−
   echo -e $(date "+%F %T"):  $1
+
   echo -e $(date "+%F %T"):  $1  
 
}
 
}
 
function die {
 
function die {
Line 125: Line 100:
 
[ ! -d "${LOCAL_DIR}" ]  && die "\$LOCAL_DIR directory doesn't exist. ($LOCAL_DIR)"
 
[ ! -d "${LOCAL_DIR}" ]  && die "\$LOCAL_DIR directory doesn't exist. ($LOCAL_DIR)"
  
−
e "`hostname` - VZ backup for containers $VEIDS started." > /tmp/vzbackuptimes
+
 
 
# Loop through each VEID
 
# Loop through each VEID
 
for VEID in $VEIDS; do
 
for VEID in $VEIDS; do
  
−
  VEHOSTNAME=`vzlist -o hostname $VEID -H`
 
 
   echo ""
 
   echo ""
 
   e "Beginning backup of VEID: $VEID";
 
   e "Beginning backup of VEID: $VEID";
Line 138: Line 112:
 
     RSYNC+=" --exclude=${VEID}${path}"
 
     RSYNC+=" --exclude=${VEID}${path}"
 
   done;
 
   done;
 +
 
  
 
   e "Commencing initial ${RSYNC} ${VZ_PRIVATE}${VEID} ${LOCAL_DIR}"
 
   e "Commencing initial ${RSYNC} ${VZ_PRIVATE}${VEID} ${LOCAL_DIR}"
Line 147: Line 122:
  
 
     e "Suspending VEID: $VEID"
 
     e "Suspending VEID: $VEID"
−
    before="$(date +%s)"
 
 
     ${VZCTL} chkpnt $VEID --suspend
 
     ${VZCTL} chkpnt $VEID --suspend
−
 
+
 
 
     e "Commencing second pass rsync ..."
 
     e "Commencing second pass rsync ..."
 
     ${RSYNC} ${VZ_PRIVATE}${VEID} ${LOCAL_DIR}
 
     ${RSYNC} ${VZ_PRIVATE}${VEID} ${LOCAL_DIR}
−
 
+
 
 
     e "Resuming VEID: $VEID"
 
     e "Resuming VEID: $VEID"
 
     ${VZCTL} chkpnt $VEID --resume
 
     ${VZCTL} chkpnt $VEID --resume
−
    after="$(date +%s)"
 
−
    elapsed_seconds="$(expr $after - $before)"
 
  
 
     e "Done."
 
     e "Done."
−
    e "Container ${VEID} ($VEHOSTNAME) was down $elapsed_seconds seconds during backup process." >> /tmp/vzbackuptimes
 
  
 
   else
 
   else
Line 165: Line 136:
 
   fi
 
   fi
  
 +
 
   # Copy VE config files over into the VE storage/cache area
 
   # Copy VE config files over into the VE storage/cache area
−
   if [ ! -d "${LOCAL_DIR}${VEID}/etc/vzdump" ]; then
+
   if [ ! -d "${LOCAL_DIR}${VEID}/etc/vzdump" ]; then  
 
     e "Creating directory for openvz config files: mkdir ${LOCAL_DIR}${VEID}/etc/vzdump"
 
     e "Creating directory for openvz config files: mkdir ${LOCAL_DIR}${VEID}/etc/vzdump"
 
     mkdir ${LOCAL_DIR}${VEID}/etc/vzdump
 
     mkdir ${LOCAL_DIR}${VEID}/etc/vzdump
Line 187: Line 159:
 
     ${RSYNC} ${LOCAL_DIR}${VEID}/ ${REMOTE_HOST}:${REMOTE_DIR}${VEID}/
 
     ${RSYNC} ${LOCAL_DIR}${VEID}/ ${REMOTE_HOST}:${REMOTE_DIR}${VEID}/
  
−
     # Rotate older tar.gz backups if they exist. You can comment out these lines if you wish to have only one copy.
+
     # Move old $VEID.tar archive to $VEID.tar.backup
−
     e "Checking for existing files ${REMOTE_HOST}:${REMOTE_DIR}${VEID}.X.tar and rotating them"
+
     e "Checking for existing file ${REMOTE_HOST}:${REMOTE_DIR}${VEID}.tar (and moving it to .backup if exists)"
−
    ssh ${REMOTE_HOST} "[ -f ${REMOTE_DIR}${VEID}.6.tar.gz ] && mv -f ${REMOTE_DIR}${VEID}.6.tar.gz ${REMOTE_DIR}${VEID}.7.tar.gz"
+
     ssh ${REMOTE_HOST} "[ -f ${REMOTE_DIR}${VEID}.tar ] && mv -f ${REMOTE_DIR}${VEID}.tar ${REMOTE_DIR}${VEID}.tar.backup"
−
    ssh ${REMOTE_HOST} "[ -f ${REMOTE_DIR}${VEID}.5.tar.gz ] && mv -f ${REMOTE_DIR}${VEID}.5.tar.gz ${REMOTE_DIR}${VEID}.6.tar.gz"
 
−
    ssh ${REMOTE_HOST} "[ -f ${REMOTE_DIR}${VEID}.4.tar.gz ] && mv -f ${REMOTE_DIR}${VEID}.4.tar.gz ${REMOTE_DIR}${VEID}.5.tar.gz"
 
−
    ssh ${REMOTE_HOST} "[ -f ${REMOTE_DIR}${VEID}.3.tar.gz ] && mv -f ${REMOTE_DIR}${VEID}.3.tar.gz ${REMOTE_DIR}${VEID}.4.tar.gz"
 
−
     ssh ${REMOTE_HOST} "[ -f ${REMOTE_DIR}${VEID}.2.tar.gz ] && mv -f ${REMOTE_DIR}${VEID}.2.tar.gz ${REMOTE_DIR}${VEID}.3.tar.gz"
 
−
    ssh ${REMOTE_HOST} "[ -f ${REMOTE_DIR}${VEID}.1.tar.gz ] && mv -f ${REMOTE_DIR}${VEID}.1.tar.gz ${REMOTE_DIR}${VEID}.2.tar.gz"
 
−
    ssh ${REMOTE_HOST} "[ -f ${REMOTE_DIR}${VEID}.0.tar.gz ] && mv -f ${REMOTE_DIR}${VEID}.0.tar.gz ${REMOTE_DIR}${VEID}.1.tar.gz"
 
  
 
     # Create a remote tar archive - note you can remove the ampersand from the end if you
 
     # Create a remote tar archive - note you can remove the ampersand from the end if you
 
     # don't want multiple tar processes running on the remote host simultaneously.
 
     # don't want multiple tar processes running on the remote host simultaneously.
−
     e "Making a g-zip compresssed tar archive on remote host (this process will run in the background on the remote host)."
+
     e "Making a tar archive on remote host (this process will run in the background on the remote host)."
−
     ssh ${REMOTE_HOST} "tar czf ${REMOTE_DIR}${VEID}.0.tar.gz --numeric-owner -C ${REMOTE_DIR}${VEID} ./ 2>/dev/null " &
+
     ssh ${REMOTE_HOST} "tar cf ${REMOTE_DIR}${VEID}.tar -C ${REMOTE_DIR}${VEID} ./ 2>/dev/null " &
 
   fi
 
   fi
  
 
   e "Done."
 
   e "Done."
 +
 
done;
 
done;
−
e "`hostname` - VZ backup for containers $VEIDS complete!" >> /tmp/vzbackuptimes
+
 
−
# Email a log of the backup process to some email address. Can be modified slightly to use native "mail" command
+
 
−
# if sendmail is installed and configured locally.
+
 
−
cat /tmp/vzbackuptimes | sendEmail -f root@`hostname` -t someuser@example.com -u "`hostname` VZ backup statistics." -s mail.example.com #(put your open relay here)
+
 
−
echo
 
−
cat /tmp/vzbackuptimes
 
−
rm /tmp/vzbackuptimes
 
 
</pre>
 
</pre>
  

Please note that all contributions to OpenVZ Virtuozzo Containers Wiki may be edited, altered, or removed by other contributors. If you don't want your writing to be edited mercilessly, then don't submit it here.
 If you are going to add external links to an article, read the External links policy first!

To edit this page, please answer the question that appears below (more info):

Cancel Editing help (opens in new window)
Retrieved from "https://wiki.openvz.org/Backup_a_running_container_over_the_network_with_ezvzdump"