This article will show a managed installation of OpenVZ based on Debian Lenny (Debian 5.0) and some other interesting software pices like PuppetMaster. It is mainly written to show how to manage more than one OpenVZ server in a productive environment.
Contents
Pre-Requierements
You should know this software, because it is used during the setup.
If you have any further questions, please feel free to contact us.
Software list:
The Debian preseed setup
At first some basics. To use a so called "preseed" file it is good to know what it is. The Debian installation is done with the "Debian Installer", short d-i. This d-i normally is a ncurses based console application that asks you some questions about your timezone, your partition setup, your network and so on. All this questions could be answered trough a preseed file. In this file some or all questions could be answerd.
Now it is good to know that if you do not answer a question or if a question pops up which is not in the preseed file, the d-i wait until you give a right answer. So it is a really good thing because you could leave some thing open and use the same preseed file for different hardware boxes e.g. one hardware have one network interface, another hardware has four network interfaces.
Also it is good to know that you can use the preseed file to answer questions from any Debian package which is using the debconf interface, for example postfix.
How to start the preseed setup
Simply download the Netinstall CD from a Debain mirror put it in your drive an boot the computer. After booting from the CD you should see the Debian Grub Boot menu. In this menu you should select the "Advanced Option" and in the follwing menu place your coursor on "Automated install" - but do not press enter!
You have to edit this boot menu entry by pressing the TAB key. Now you can append the URL option to the end of this boot line. Now press enter and continue the setup. Here is an example and a screenshot.
URL=http://yourserver:yourport/yourpreseedfile
As example here is our preseed file:
During the preseed setup
If the d-i faces a question where it founds no answer in the preseed file it will hold on until you answer the question. The d-i will not cancel the installation process.
This is an interesting feature because you could use one file for different hardware setups. For example, if you have not configured that the d-i to always use the eth0 device during setup and there is more then one network card in your hardware, the d-i will wait until you choose one device.
Additional informations
At our site we use different preseed files for different purposes. As example one preseed for VMWare servers etc... All our preseed files are placed on a webserver but you should know that it is also possible to integrate the preseed process into a self made Debian installation medium.
The apt-proxy
The Debian Netinstall CD covers only a minimal system. So if you need more packages like rsync or others you have to connect to the internet. Without choosing a Debian mirror during the setup, the setup could not continue. If you use Debian OpenVZ Servers and Debian based Virtual Private Server at your site, you should use an apt-proxy to cache the downloaded packages. After the first setup this will speed up your installation time by factors!
The apt-proxy setup is as easy as 1-2-3 go.
Here the preseed file, please note the I have removed the comments but I make some explanations for clearer understanding:
d-i debian-installer/locale string en_US.UTF8
d-i console-keymaps-at/keymap select de-latin1-nodeadkeys
d-i netcfg/choose_interface select auto
d-i netcfg/dhcp_timeout string 60
d-i netcfg/get_hostname string unassigned-hostname
d-i netcfg/get_domain string unassigned-domain
d-i netcfg/wireless_wep string
d-i mirror/country string enter information manually
#This is our APT-PROXY address
d-i mirror/http/hostname string youraptproxy:9999
d-i mirror/http/directory string /debian
d-i mirror/suite string stable
d-i clock-setup/ntp boolean true
#NTP Server
d-i clock-setup/ntp-server string yourtimeserver
#The partition setup – be careful!
#By default the OpenVZ debian packages are using /var/lib/vz
d-i partman-auto/method string lvm
d-i partman-auto/purge_lvm_from_device boolean true
d-i partman-lvm/confirm boolean true
d-i partman-auto/expert_recipe string \
boot-root :: \
150 150 150 ext3 \
$primary{ } $bootable{ } \
method{ format } format{ } \
use_filesystem{ } filesystem{ ext3 } \
mountpoint{ /boot } \
. \
5120 5120 5120 ext3 \
$primary{ } method{ format } format{ } \
use_filesystem{ } filesystem{ ext3 } \
mountpoint{ / } \
. \
500 10000 1000000000 ext3 \
method{ format } format{ } $lvmok{ } \
use_filesystem{ } filesystem{ ext3 } \
mountpoint{ /var } \
. \
4096 4096 4096 linux-swap \
method{ swap } format{ } $lvmok { } \
.
d-i partman/confirm_write_new_label boolean true
d-i partman/choose_partition \
select Finish partitioning and write changes to disk
d-i partman/confirm boolean true
d-i clock-setup/utc boolean true
d-i time/zone string Europe/Vienna
d-i apt-setup/non-free boolean true
d-i apt-setup/contrib boolean true
d-i apt-setup/security_host youraptproxy:9999
#Our private internal repository for some packages
d-i apt-setup/local0/repository string \
http://yourprivaterepository:10000/debian hns stable
d-i apt-setup/local0/key string http://yourprivaterepository:10000/PublicKey
d-i passwd/make-user boolean false
#The encrypted root password (this is only an example :) )
d-i passwd/root-password-crypted password $1$v4rfe7wv$gEkbCLxCPhKaj92s.uJbD1
d-i grub-installer/only_debian boolean true
d-i grub-installer/with_other_os boolean true
tasksel tasksel/first multiselect standard
#Additional base packages
d-i pkgsel/include string openssh-server build-essential vim snmpd lib32z1-dev rsync ntp ntpdate
d-i finish-install/reboot_in_progress note
d-i cdrom-detect/eject boolean false
xserver-xorg xserver-xorg/autodetect_monitor boolean true
xserver-xorg xserver-xorg/config/monitor/selection-method \
select medium
xserver-xorg xserver-xorg/config/monitor/mode-list \
select 1024x768 @ 60 Hz
#Packages that gets installated AFTER the base installation
#hns* packages are build by our own to fasten the setup and management
d-i preseed/late_command string apt-install hp-health hp-snmp-agents hpsmh hp-smh-templates hpacucli cpqacuxe ethtool linux-headers-2.6-openvz-amd64 linux-image-2.6-openvz-amd64 vzctl vzquota hns-zabbix-agentd hns-openvz-common hns-openvz-ubuntu-hosting hns-puppet firmware-qlogic
The private Debian repository
In our case the private repository is an essential factor because it will provide you with a fast software roll out, replicable software and consistent system state trough a large number of setups.
It is quite easy to setup your own repositor with reprepro [5] and it is really useful.
As you can see in the preseed file we have a lot of this packages in our repository. Some examples:
- hns-zabbix-agentd
- is used for the automated installation of the Zabbix System Monitoring Agent.[6]
- hns-openvz-common
- creates a backup cronjob and installs a backup script
- installs a set of scripts for Zabbix monitoring
- hns-openvz-ubuntu-hosting
- installs our pre-created Ubuntu[7] template
- hns-puppet
- installs the PuppetMaster client
The Puppet Master configuration management
If your preseeded setup was successful you have to configure your fresh server to fit your expectations. At our site we have ten OpenVZ servers in different countries and is really important that they have the same configuration.
To make life easier we decided to use a configuration management like cfengine[8] but not as complex in setup and management. We found our solution in Puppet Master.
The only dependency Puppet Master has is Ruby. This should be installalbe on every ditribution in minutes.
After the installation you have to configure the Puppet Master to do things. Typically this means accepting the public private key pair and doing some configuration stuff. At our site use Puppet Master to manage an installation of about 100 servers.
One thing it does is to change the sysctl.conf file.
To get a feeling for it here is an configuration example (only OpenVZ server hosts):
# ALL----- OPENVZ
node "yourserver1.yourdns.com", "yourserver2.yourdns.com", "yourserver3.yourdns.com", "yourserver4.yourdns.com", "yourserver5.yourdns.com", "yourserver6.yourdns.com", "yourserver7.yourdns.com"{
file { "/etc/aliases":
mode => 644,
owner => root,
group => root,
source => "puppet://yourpuppetmaster/files/openvz-hosts/etc/aliases",
}
exec { "subscribe-newaliases":
command => "/usr/bin/newaliases && /bin/echo NEALIASSES",
subscribe => File["/etc/aliases"],
refreshonly => true,
logoutput => true
}
file { "/etc/apt/sources.list":
mode => 644,
owner => root,
group => root,
source => "puppet://yourpuppetmaster/files/openvz-hosts/etc/apt/sources.list"
}
file { "/root/.ssh/authorized_keys2":
mode => 644,
owner => root,
group => root,
source => "puppet://yourpuppetmaster/files/openvz-hosts/root/authorized_keys2"
}
file { "/etc/vim/vimrc":
mode => 644,
owner => root,
group => root,
source => "puppet://yourpuppetmaster/files/openvz-hosts/etc/vim/vimrc"
}
file { "/etc/vz/conf/ve-vps.10.conf-sample":
mode => 644,
owner => root,
group => root,
source => "puppet://yourpuppetmaster/files/openvz-hosts/etc/vz/conf/ve-vps.10.conf-sample"
}
file { "/etc/sysctl.conf":
mode => 644,
owner => root,
group => root,
source => "puppet://yourpuppetmaster/files/openvz-hosts/etc/sysctl.conf"
}
exec { subscribe-sysctl:
command => "/sbin/sysctl -p && /bin/echo SYSCTL EXECUTED",
logoutput => true,
refreshonly => true,
subscribe => file["/etc/sysctl.conf"]
}
file { "/etc/ntp.conf":
mode => 644,
owner => root,
group => root,
source => "puppet://yourpuppetmaster/files/openvz-hosts/etc/ntp.conf"
}
file { "/root/.bashrc":
mode => 644,
owner => root,
group => root,
source => "puppet://yourpuppetmaster/files/openvz-hosts/root/bashrc"
}
file { "/srv/exim_config.sh":
mode => 744,
owner => root,
group => root,
source => "puppet://yourpuppetmaster/files/openvz-hosts/srv/exim_config.sh"
}
exec { subscribe-exim_config:
command => "/srv/exim_config.sh && /usr/sbin/update-exim4.conf && /etc/init.d/exim4 restart && /bin/echo EXIM_CONFIG",
logoutput => true,
refreshonly => true,
subscribe => file["/srv/exim_config.sh"]
}
file { "/etc/default/snmpd":
mode => 644,
owner => root,
group => root,
source => "puppet://yourpuppetmaster/files/openvz-hosts/etc/default/snmpd"
}
exec { subscribe-snmpd_config:
command => "/etc/init.d/snmpd restart && /bin/echo SNMPD_CONFIG",
logoutput => true,
refreshonly => true,
subscribe => file["/etc/default/snmpd"]
}
}