A managed OpenVZ installation

This article will show a managed installation of OpenVZ based on Debian Lenny (Debian 5.0) and some other interesting software pieces like PuppetMaster. It is mainly written to show how to manage more than one OpenVZ server in a production environment.

Pre-Requirements

You should know this software, because it is used during the setup.

If you have any further questions, please feel free to contact us.

Software list:

• Debian Installer and preseed
• Apt-Proxy
• Private Debian repository
• Puppet Master

The Debian preseed setup

At first some basics. To use a so called "preseed" file it is good to know what it is. The Debian installation is done with the "Debian Installer", short d-i. This d-i normally is a ncurses based console application that asks you some questions about your timezone, your partition setup, your network and so on. All this questions could be answered trough a preseed file. In this file some or all questions could be answered.

Now it is good to know that if you do not answer a question or if a question pops up which is not in the preseed file, the d-i waits until you give a right answer. So it is a really good thing because you could leave something open and use the same preseed file for different hardware boxes e.g. one hardware has one network interface, another hardware has four network interfaces.

Also it is good to know that you can use the preseed file to answer questions from any Debian package which is using the debconf interface, for example postfix.

How to start the preseed setup

Simply download the Netinstall CD from a Debian mirror put it in your cdrom-drive and boot the server. After booting from the CD you should see the Debian Grub Boot menu. In this menu you should select the "Advanced Option" and in the following menu place your cursor on "Automated install" - but do not press enter!

You have to edit this boot menu entry by pressing the TAB key. Now you can append the URL option to the end of this boot line. Now press enter and continue the setup. Here is an example and a screen shot.

URL=http://yourserver:yourport/yourpreseedfile

Grub menu screen

As an example here is our preseed file:

During the preseed setup

If the d-i faces a question where it finds no answer in the preseed file it will hold until you answer the question. The d-i will not cancel the installation process.

This is an interesting feature because you could use one file for different hardware setups. For example, if you have not configured that the d-i always uses the eth0 device during setup and there is more then one network card in your hardware, the d-i will wait until you choose one device.

d-i wait for answer

Additional informations

At our site we use different preseed files for different purposes. As example one preseed for VMWare (Guest) servers etc... All our preseed files are placed on a webserver but you should know that it is also possible to integrate the preseed process into a self made Debian installation medium.

The apt-proxy

The Debian Netinstall CD covers only a minimal system. So if you need more packages like rsync or others you have to connect to the internet. Without choosing a Debian mirror during the setup, the setup could not continue. If you use Debian OpenVZ Servers and Debian based Virtual Private Server at your site, you should use an apt-proxy to cache the downloaded packages. After the first setup this will speed up your installation time by factors!

The apt-proxy setup is as easy as 1-2-3 go.

Here is the preseed file, please note that I have removed the comments but I make some explanations for clearer understanding:

d-i debian-installer/locale string en_US.UTF8
d-i console-keymaps-at/keymap select de-latin1-nodeadkeys
d-i netcfg/choose_interface select auto
d-i netcfg/dhcp_timeout string 60
d-i netcfg/get_hostname string unassigned-hostname
d-i netcfg/get_domain string unassigned-domain
d-i netcfg/wireless_wep string
d-i mirror/country string enter information manually

#This is our APT-PROXY address
d-i mirror/http/hostname string youraptproxy:9999

d-i mirror/http/directory string /debian
d-i mirror/suite string stable
d-i clock-setup/ntp boolean true

#NTP Server
d-i clock-setup/ntp-server string yourtimeserver

#The partition setup, be careful!
#By default the OpenVZ debian packages are using /var/lib/vz
d-i partman-auto/method string lvm
d-i partman-auto/purge_lvm_from_device boolean true
d-i partman-lvm/confirm boolean true
d-i partman-auto/expert_recipe string                         \
      boot-root ::                                            \
              150 150 150 ext3                                \
                      $primary{ } $bootable{ }                \
                      method{ format } format{ }              \
                      use_filesystem{ } filesystem{ ext3 }    \
                      mountpoint{ /boot }                     \
              .                                               \
              5120 5120 5120 ext3                             \
                      $primary{ } method{ format } format{ }  \
                      use_filesystem{ } filesystem{ ext3 }    \
                      mountpoint{ / }                         \
              .                                               \
              500 10000 1000000000 ext3                       \
                      method{ format } format{ } $lvmok{ }    \
                      use_filesystem{ } filesystem{ ext3 }    \
                      mountpoint{ /var }                      \
              .                                               \
              4096 4096 4096 linux-swap                       \
                      method{ swap } format{ } $lvmok { }     \
              .
d-i partman/confirm_write_new_label boolean true
d-i partman/choose_partition \
       select Finish partitioning and write changes to disk
d-i partman/confirm boolean true
d-i clock-setup/utc boolean true
d-i time/zone string Europe/Vienna
d-i apt-setup/non-free boolean true
d-i apt-setup/contrib boolean true
d-i apt-setup/security_host youraptproxy:9999

#Our private internal repository for some packages
d-i apt-setup/local0/repository string \
       http://yourprivaterepository:10000/debian hns stable
d-i apt-setup/local0/key string http://yourprivaterepository:10000/PublicKey
d-i passwd/make-user boolean false

#The encrypted root password (this is only an example :) )
d-i passwd/root-password-crypted password $1$v4rfe7wv$gEkbCLxCPhKaj92s.uJbD1
d-i grub-installer/only_debian boolean true
d-i grub-installer/with_other_os boolean true
tasksel tasksel/first multiselect standard

#Additional base packages
d-i pkgsel/include string openssh-server build-essential vim snmpd lib32z1-dev rsync ntp ntpdate
d-i finish-install/reboot_in_progress note
d-i cdrom-detect/eject boolean false
xserver-xorg xserver-xorg/autodetect_monitor boolean true
xserver-xorg xserver-xorg/config/monitor/selection-method \
       select medium
xserver-xorg xserver-xorg/config/monitor/mode-list \
       select 1024x768 @ 60 Hz

#Packages that gets installed AFTER the base installation
#hns* packages are build by our own to fasten the setup and management
d-i preseed/late_command string apt-install hp-health hp-snmp-agents hpsmh \
  hp-smh-templates hpacucli cpqacuxe ethtool linux-headers-2.6-openvz-amd64 \
  linux-image-2.6-openvz-amd64 vzctl vzquota hns-zabbix-agentd hns-openvz-common \
  hns-openvz-ubuntu-hosting hns-puppet firmware-qlogic

The private Debian repository

In our case the private repository is an essential factor because it will provide you with a fast software roll out, replicable software and consistent system state trough a large number of setups.

It is quite easy to setup your own repository with reprepro and it is really useful.

As you can see in the preseed file we have a lot of this packages in our repository. Some examples:

• hns-zabbix-agentd

is used for the automated installation of the Zabbix System Monitoring Agent.

• hns-openvz-common

creates a backup cronjob and installs a backup script

installs a set of scripts for Zabbix monitoring and therefore depends on the hns-zabbix-agentd package

• hns-openvz-ubuntu-hosting

installs our pre-created Ubuntu template for automated guest-installation

• hns-puppet

installs the PuppetMaster client

The Puppet Master configuration management

If your preseeded setup was successful you have to configure your fresh server to fit your expectations. At our site we have ten OpenVZ servers in different countries and it is really important that they have the same configuration.

To make life easier we decided to use a configuration management like cfengine but not as complex in setup and management. We found our solution in Puppet Master.

The only dependency Puppet Master has is Ruby. This should be installable on every distribution in minutes.

After the installation you have to configure the Puppet Master to do things. Typically this means accepting the public private key pair and doing some configuration stuff. At our site we use Puppet Master to manage an installation of about 100 servers.

One thing it does is to change the sysctl.conf file.

To get a feeling for it here is an configuration example (only OpenVZ server hosts):

# ALL----- OPENVZ
node "yourserver1.yourdns.com", "yourserver2.yourdns.com", "yourserver3.yourdns.com", "yourserver4.yourdns.com", "yourserver5.yourdns.com", "yourserver6.yourdns.com", "yourserver7.yourdns.com"{
    file { "/etc/aliases":
        mode => 644,
        owner => root,
        group => root,
        source => "puppet://yourpuppetmaster/files/openvz-hosts/etc/aliases",
    }
    exec { "subscribe-newaliases":
        command => "/usr/bin/newaliases && /bin/echo NEALIASSES",
        subscribe => File["/etc/aliases"],
        refreshonly => true,
        logoutput => true
    }

    file { "/etc/apt/sources.list":
        mode => 644,
        owner => root,
        group => root,
        source => "puppet://yourpuppetmaster/files/openvz-hosts/etc/apt/sources.list"
    }
    file { "/root/.ssh/authorized_keys2":
        mode => 644,
        owner => root,
        group => root,
        source => "puppet://yourpuppetmaster/files/openvz-hosts/root/authorized_keys2"
    }
    file { "/etc/vim/vimrc":
        mode => 644,
        owner => root,
        group => root,
        source => "puppet://yourpuppetmaster/files/openvz-hosts/etc/vim/vimrc"
    }
    file { "/etc/vz/conf/ve-vps.10.conf-sample":
        mode => 644,
        owner => root,
        group => root,
        source => "puppet://yourpuppetmaster/files/openvz-hosts/etc/vz/conf/ve-vps.10.conf-sample"
    }
    file { "/etc/sysctl.conf":
        mode => 644,
        owner => root,
        group => root,
        source => "puppet://yourpuppetmaster/files/openvz-hosts/etc/sysctl.conf"
    }
    exec { subscribe-sysctl:
        command => "/sbin/sysctl -p && /bin/echo SYSCTL EXECUTED",
        logoutput => true,
        refreshonly => true,
        subscribe => file["/etc/sysctl.conf"]
    }
    file { "/etc/ntp.conf":
        mode => 644,
        owner => root,
        group => root,
        source => "puppet://yourpuppetmaster/files/openvz-hosts/etc/ntp.conf"
    }
    file { "/root/.bashrc":
        mode => 644,
        owner => root,
        group => root,
        source => "puppet://yourpuppetmaster/files/openvz-hosts/root/bashrc"
    }
    file { "/srv/exim_config.sh":
        mode => 744,
        owner => root,
        group => root,
        source => "puppet://yourpuppetmaster/files/openvz-hosts/srv/exim_config.sh"
    }
    exec { subscribe-exim_config:
        command => "/srv/exim_config.sh && /usr/sbin/update-exim4.conf && /etc/init.d/exim4 restart && /bin/echo EXIM_CONFIG",
        logoutput => true,
        refreshonly => true,
        subscribe => file["/srv/exim_config.sh"]
    }
    file { "/etc/default/snmpd":
        mode => 644,
        owner => root,
        group => root,
        source => "puppet://yourpuppetmaster/files/openvz-hosts/etc/default/snmpd"
    }
    exec { subscribe-snmpd_config:
        command => "/etc/init.d/snmpd restart && /bin/echo SNMPD_CONFIG",
        logoutput => true,
        refreshonly => true,
        subscribe => file["/etc/default/snmpd"]
    }
}

Further information

For further information on the particular tools like apt-proxy/Debian repository/... we will add some documentation links! Write us if you have questions or if you want to see practical documentations on some of the tools.