Difference between revisions of "A managed OpenVZ installation"

From OpenVZ Virtuozzo Containers Wiki
Jump to: navigation, search
m (Reverted edits by StevenCook (talk) to last revision by Kir)
 
(15 intermediate revisions by 4 users not shown)
Line 1: Line 1:
This article will show a managed installation of OpenVZ based on Debian Lenny (Debian 5.0) and some other interesting software pices like PuppetMaster. It is mainly written to show how to manage more than one OpenVZ server in a productive environment.
+
This article will show a managed installation of OpenVZ based on Debian Lenny (Debian 5.0) and some other interesting software pieces like PuppetMaster. It is mainly written to show how to manage more than one OpenVZ server in a production environment.
  
==Pre-Requierements==
+
==Pre-Requirements==
  
 
You should know this software, because it is used during the setup.
 
You should know this software, because it is used during the setup.
Line 15: Line 15:
 
==The Debian preseed setup==
 
==The Debian preseed setup==
  
At first some basics. To use a so called "preseed" file it is good to know what it is. The Debian installation is done with the "Debian Installer", short d-i. This d-i normally is a ncurses based console application that asks you some questions about your timezone, your partition setup, your network and so on. All this questions could be answered trough a preseed file. In this file some or all questions could be answerd.
+
At first some basics. To use a so called "preseed" file it is good to know what it is. The Debian installation is done with the "Debian Installer", short d-i. This d-i normally is a ncurses based console application that asks you some questions about your timezone, your partition setup, your network and so on. All this questions could be answered trough a preseed file. In this file some or all questions could be answered.
  
Now it is good to know that if you do not answer a question or if a question pops up which is not in the preseed file, the d-i wait until you give a right answer. So it is a really good thing because you could leave some thing open and use the same preseed file for different hardware boxes e.g. one hardware have one network interface, another hardware has four network interfaces.
+
Now it is good to know that if you do not answer a question or if a question pops up which is not in the preseed file, the d-i waits until you give a right answer. So it is a really good thing because you could leave something open and use the same preseed file for different hardware boxes e.g. one hardware has one network interface, another hardware has four network interfaces.
  
 
Also it is good to know that you can use the preseed file to answer questions from any Debian package which is using the debconf interface, for example postfix.
 
Also it is good to know that you can use the preseed file to answer questions from any Debian package which is using the debconf interface, for example postfix.
Line 23: Line 23:
 
===How to start the preseed setup===
 
===How to start the preseed setup===
  
Simply download the Netinstall CD from a Debain mirror put it in your drive an boot the computer. After booting from the CD you should see the Debian Grub Boot menu. In this menu you should select the "Advanced Option" and in the follwing menu place your coursor on "Automated install" - '''but do not press enter!'''
+
Simply download the Netinstall CD from a Debian mirror put it in your cdrom-drive and boot the server. After booting from the CD you should see the Debian Grub Boot menu. In this menu you should select the "Advanced Option" and in the following menu place your cursor on "Automated install" - '''but do not press enter!'''
  
You have to edit this boot menu entry by pressing the '''TAB''' key. Now you can append the '''URL''' option to the end of this boot line. Now press enter and continue the setup. Here is an example and a screenshot.
+
You have to edit this boot menu entry by pressing the '''TAB''' key. Now you can append the '''URL''' option to the end of this boot line. Now press enter and continue the setup. Here is an example and a screen shot.
  
<pre>URL=http://yourserver:yourport/yourpreseedfile</pre>
+
URL=http://yourserver:yourport/yourpreseedfile
  
 
[[Image:Notbuu_1_preseed.png|200px|thumb|Grub menu screen]]
 
[[Image:Notbuu_1_preseed.png|200px|thumb|Grub menu screen]]
  
As example here is our preseed file:
+
As an example here is our preseed file:
  
 
===During the preseed setup===
 
===During the preseed setup===
  
If the d-i faces a question where it founds no answer in the preseed file it will hold on until you answer the question. The d-i will not cancel the installation process.
+
If the d-i faces a question where it finds no answer in the preseed file it will hold until you answer the question. The d-i will not cancel the installation process.
  
This is an interesting feature because you could use one file for different hardware setups. For example, if you have not configured that the d-i to always use the eth0 device during setup and there is more then one network card in your hardware, the d-i will wait until you choose one device.  
+
This is an interesting feature because you could use one file for different hardware setups. For example, if you have not configured that the d-i always uses the eth0 device during setup and there is more then one network card in your hardware, the d-i will wait until you choose one device.  
  
 
[[Image:Notbuu_2_preseed.png|200px|thumb|d-i wait for answer]]
 
[[Image:Notbuu_2_preseed.png|200px|thumb|d-i wait for answer]]
Line 43: Line 43:
 
====Additional informations====
 
====Additional informations====
  
At our site we use different preseed files for different purposes. As example one preseed for VMWare servers etc...
+
At our site we use different preseed files for different purposes. As example one preseed for VMWare (Guest) servers etc...
 
All our preseed files are placed on a webserver but you should know that it is also possible to integrate the preseed process into a self made Debian installation medium.
 
All our preseed files are placed on a webserver but you should know that it is also possible to integrate the preseed process into a self made Debian installation medium.
  
Line 52: Line 52:
 
The apt-proxy setup is as easy as 1-2-3 go.
 
The apt-proxy setup is as easy as 1-2-3 go.
  
Here the preseed file, please note the I have removed the comments but I make some explanations for clearer understanding:
+
Here is the preseed file, please note that I have removed the comments but I make some explanations for clearer understanding:
  
 
<pre>
 
<pre>
Line 63: Line 63:
 
d-i netcfg/wireless_wep string
 
d-i netcfg/wireless_wep string
 
d-i mirror/country string enter information manually
 
d-i mirror/country string enter information manually
 +
 
#This is our APT-PROXY address
 
#This is our APT-PROXY address
 
d-i mirror/http/hostname string youraptproxy:9999
 
d-i mirror/http/hostname string youraptproxy:9999
 +
 
d-i mirror/http/directory string /debian
 
d-i mirror/http/directory string /debian
 
d-i mirror/suite string stable
 
d-i mirror/suite string stable
 
d-i clock-setup/ntp boolean true
 
d-i clock-setup/ntp boolean true
 +
 
#NTP Server
 
#NTP Server
 
d-i clock-setup/ntp-server string yourtimeserver
 
d-i clock-setup/ntp-server string yourtimeserver
#The partition setup be careful!  
+
 
 +
#The partition setup, be careful!
 
#By default the OpenVZ debian packages are using /var/lib/vz
 
#By default the OpenVZ debian packages are using /var/lib/vz
 
d-i partman-auto/method string lvm
 
d-i partman-auto/method string lvm
Line 91: Line 95:
 
                       method{ format } format{ } $lvmok{ }    \
 
                       method{ format } format{ } $lvmok{ }    \
 
                       use_filesystem{ } filesystem{ ext3 }    \
 
                       use_filesystem{ } filesystem{ ext3 }    \
                       mountpoint{ /var }                       \
+
                       mountpoint{ /var }                     \
 
               .                                              \
 
               .                                              \
 
               4096 4096 4096 linux-swap                      \
 
               4096 4096 4096 linux-swap                      \
Line 105: Line 109:
 
d-i apt-setup/contrib boolean true
 
d-i apt-setup/contrib boolean true
 
d-i apt-setup/security_host youraptproxy:9999
 
d-i apt-setup/security_host youraptproxy:9999
 +
 
#Our private internal repository for some packages
 
#Our private internal repository for some packages
 
d-i apt-setup/local0/repository string \
 
d-i apt-setup/local0/repository string \
Line 110: Line 115:
 
d-i apt-setup/local0/key string http://yourprivaterepository:10000/PublicKey
 
d-i apt-setup/local0/key string http://yourprivaterepository:10000/PublicKey
 
d-i passwd/make-user boolean false
 
d-i passwd/make-user boolean false
 +
 
#The encrypted root password (this is only an example :) )
 
#The encrypted root password (this is only an example :) )
 
d-i passwd/root-password-crypted password $1$v4rfe7wv$gEkbCLxCPhKaj92s.uJbD1
 
d-i passwd/root-password-crypted password $1$v4rfe7wv$gEkbCLxCPhKaj92s.uJbD1
Line 115: Line 121:
 
d-i grub-installer/with_other_os boolean true
 
d-i grub-installer/with_other_os boolean true
 
tasksel tasksel/first multiselect standard
 
tasksel tasksel/first multiselect standard
 +
 
#Additional base packages
 
#Additional base packages
 
d-i pkgsel/include string openssh-server build-essential vim snmpd lib32z1-dev rsync ntp ntpdate
 
d-i pkgsel/include string openssh-server build-essential vim snmpd lib32z1-dev rsync ntp ntpdate
Line 124: Line 131:
 
xserver-xorg xserver-xorg/config/monitor/mode-list \
 
xserver-xorg xserver-xorg/config/monitor/mode-list \
 
       select 1024x768 @ 60 Hz
 
       select 1024x768 @ 60 Hz
#Packages that gets installated AFTER the base installation
+
 
 +
#Packages that gets installed AFTER the base installation
 
#hns* packages are build by our own to fasten the setup and management
 
#hns* packages are build by our own to fasten the setup and management
d-i preseed/late_command string apt-install hp-health hp-snmp-agents hpsmh hp-smh-templates hpacucli cpqacuxe ethtool linux-headers-2.6-openvz-amd64 linux-image-2.6-openvz-amd64 vzctl vzquota hns-zabbix-agentd hns-openvz-common hns-openvz-ubuntu-hosting hns-puppet firmware-qlogic
+
d-i preseed/late_command string apt-install hp-health hp-snmp-agents hpsmh \
 +
  hp-smh-templates hpacucli cpqacuxe ethtool linux-headers-2.6-openvz-amd64 \
 +
  linux-image-2.6-openvz-amd64 vzctl vzquota hns-zabbix-agentd hns-openvz-common \
 +
  hns-openvz-ubuntu-hosting hns-puppet firmware-qlogic
 
</pre>
 
</pre>
  
Line 133: Line 144:
 
In our case the private repository is an essential factor because it will provide you with a fast software roll out, replicable software and consistent system state trough a large number of setups.
 
In our case the private repository is an essential factor because it will provide you with a fast software roll out, replicable software and consistent system state trough a large number of setups.
  
It is quite easy to setup your own repositor with reprepro <ref>http://mirrorer.alioth.debian.org/</ref> and it is really useful.
+
It is quite easy to setup your own repository with reprepro <ref>http://mirrorer.alioth.debian.org/</ref> and it is really useful.
  
 
As you can see in the preseed file we have a lot of this packages in our repository. Some examples:
 
As you can see in the preseed file we have a lot of this packages in our repository. Some examples:
Line 141: Line 152:
 
*hns-openvz-common
 
*hns-openvz-common
 
:creates a backup cronjob and installs a backup script
 
:creates a backup cronjob and installs a backup script
:installs a set of scripts for Zabbix monitoring
+
:installs a set of scripts for Zabbix monitoring and therefore depends on the hns-zabbix-agentd package
 
*hns-openvz-ubuntu-hosting
 
*hns-openvz-ubuntu-hosting
:installs our pre-created Ubuntu<ref>http://www.ubuntu.com/</ref> template
+
:installs our pre-created Ubuntu<ref>http://www.ubuntu.com/</ref> template for automated guest-installation
 
*hns-puppet
 
*hns-puppet
 
:installs the PuppetMaster client
 
:installs the PuppetMaster client
Line 149: Line 160:
 
==The Puppet Master configuration management==
 
==The Puppet Master configuration management==
  
If your preseeded setup was successful you have to configure your fresh server to fit your expectations. At our site we have ten OpenVZ servers in different countries and is really important that they have the same configuration.
+
If your preseeded setup was successful you have to configure your fresh server to fit your expectations. At our site we have ten OpenVZ servers in different countries and it is really important that they have the same configuration.
  
 
To make life easier we decided to use a configuration management like cfengine<ref>http://www.cfengine.org/</ref> but not as complex in setup and management. We found our solution in Puppet Master.
 
To make life easier we decided to use a configuration management like cfengine<ref>http://www.cfengine.org/</ref> but not as complex in setup and management. We found our solution in Puppet Master.
  
The only dependency Puppet Master has is Ruby. This should be installalbe on every ditribution in minutes.
+
The only dependency Puppet Master has is Ruby. This should be installable on every distribution in minutes.
 
 
After the installation you have to configure the Puppet Master to do things. Typically this means accepting the public private key pair and doing some configuration stuff. At our site use Puppet Master to manage an installation of about 100 servers.
 
  
 +
After the installation you have to configure the Puppet Master to do things. Typically this means accepting the public private key pair and doing some configuration stuff. At our site we use Puppet Master to manage an installation of about 100 servers.
  
 +
One thing it does is to change the sysctl.conf file.
  
 
To get a feeling for it here is an configuration example (only OpenVZ server hosts):
 
To get a feeling for it here is an configuration example (only OpenVZ server hosts):
Line 252: Line 263:
 
}
 
}
 
</pre>
 
</pre>
 +
 +
==Further information==
 +
 +
For further information on the particular tools like apt-proxy/Debian repository/... we will add some documentation links!
 +
Write us if you have questions or if you want to see practical documentations on some of the tools.
  
 
==References==
 
==References==
 
<references/>
 
<references/>
 +
 +
[[Category: Installation]]
 +
[[Category: Debian]]
 +
[[Category: HOWTO]]

Latest revision as of 20:46, 4 August 2012

This article will show a managed installation of OpenVZ based on Debian Lenny (Debian 5.0) and some other interesting software pieces like PuppetMaster. It is mainly written to show how to manage more than one OpenVZ server in a production environment.

Pre-Requirements

You should know this software, because it is used during the setup.

If you have any further questions, please feel free to contact us.

Software list:

  • Debian Installer and preseed [1]
  • Apt-Proxy [2]
  • Private Debian repository [3]
  • Puppet Master [4]

The Debian preseed setup

At first some basics. To use a so called "preseed" file it is good to know what it is. The Debian installation is done with the "Debian Installer", short d-i. This d-i normally is a ncurses based console application that asks you some questions about your timezone, your partition setup, your network and so on. All this questions could be answered trough a preseed file. In this file some or all questions could be answered.

Now it is good to know that if you do not answer a question or if a question pops up which is not in the preseed file, the d-i waits until you give a right answer. So it is a really good thing because you could leave something open and use the same preseed file for different hardware boxes e.g. one hardware has one network interface, another hardware has four network interfaces.

Also it is good to know that you can use the preseed file to answer questions from any Debian package which is using the debconf interface, for example postfix.

How to start the preseed setup

Simply download the Netinstall CD from a Debian mirror put it in your cdrom-drive and boot the server. After booting from the CD you should see the Debian Grub Boot menu. In this menu you should select the "Advanced Option" and in the following menu place your cursor on "Automated install" - but do not press enter!

You have to edit this boot menu entry by pressing the TAB key. Now you can append the URL option to the end of this boot line. Now press enter and continue the setup. Here is an example and a screen shot.

URL=http://yourserver:yourport/yourpreseedfile
Grub menu screen

As an example here is our preseed file:

During the preseed setup

If the d-i faces a question where it finds no answer in the preseed file it will hold until you answer the question. The d-i will not cancel the installation process.

This is an interesting feature because you could use one file for different hardware setups. For example, if you have not configured that the d-i always uses the eth0 device during setup and there is more then one network card in your hardware, the d-i will wait until you choose one device.

d-i wait for answer

Additional informations

At our site we use different preseed files for different purposes. As example one preseed for VMWare (Guest) servers etc... All our preseed files are placed on a webserver but you should know that it is also possible to integrate the preseed process into a self made Debian installation medium.

The apt-proxy

The Debian Netinstall CD covers only a minimal system. So if you need more packages like rsync or others you have to connect to the internet. Without choosing a Debian mirror during the setup, the setup could not continue. If you use Debian OpenVZ Servers and Debian based Virtual Private Server at your site, you should use an apt-proxy to cache the downloaded packages. After the first setup this will speed up your installation time by factors!

The apt-proxy setup is as easy as 1-2-3 go.

Here is the preseed file, please note that I have removed the comments but I make some explanations for clearer understanding:

d-i debian-installer/locale string en_US.UTF8
d-i console-keymaps-at/keymap select de-latin1-nodeadkeys
d-i netcfg/choose_interface select auto
d-i netcfg/dhcp_timeout string 60
d-i netcfg/get_hostname string unassigned-hostname
d-i netcfg/get_domain string unassigned-domain
d-i netcfg/wireless_wep string
d-i mirror/country string enter information manually

#This is our APT-PROXY address
d-i mirror/http/hostname string youraptproxy:9999

d-i mirror/http/directory string /debian
d-i mirror/suite string stable
d-i clock-setup/ntp boolean true

#NTP Server
d-i clock-setup/ntp-server string yourtimeserver

#The partition setup, be careful!
#By default the OpenVZ debian packages are using /var/lib/vz
d-i partman-auto/method string lvm
d-i partman-auto/purge_lvm_from_device boolean true
d-i partman-lvm/confirm boolean true
d-i partman-auto/expert_recipe string                         \
      boot-root ::                                            \
              150 150 150 ext3                                \
                      $primary{ } $bootable{ }                \
                      method{ format } format{ }              \
                      use_filesystem{ } filesystem{ ext3 }    \
                      mountpoint{ /boot }                     \
              .                                               \
              5120 5120 5120 ext3                             \
                      $primary{ } method{ format } format{ }  \
                      use_filesystem{ } filesystem{ ext3 }    \
                      mountpoint{ / }                         \
              .                                               \
              500 10000 1000000000 ext3                       \
                      method{ format } format{ } $lvmok{ }    \
                      use_filesystem{ } filesystem{ ext3 }    \
                      mountpoint{ /var }                      \
              .                                               \
              4096 4096 4096 linux-swap                       \
                      method{ swap } format{ } $lvmok { }     \
              .
d-i partman/confirm_write_new_label boolean true
d-i partman/choose_partition \
       select Finish partitioning and write changes to disk
d-i partman/confirm boolean true
d-i clock-setup/utc boolean true
d-i time/zone string Europe/Vienna
d-i apt-setup/non-free boolean true
d-i apt-setup/contrib boolean true
d-i apt-setup/security_host youraptproxy:9999

#Our private internal repository for some packages
d-i apt-setup/local0/repository string \
       http://yourprivaterepository:10000/debian hns stable
d-i apt-setup/local0/key string http://yourprivaterepository:10000/PublicKey
d-i passwd/make-user boolean false

#The encrypted root password (this is only an example :) )
d-i passwd/root-password-crypted password $1$v4rfe7wv$gEkbCLxCPhKaj92s.uJbD1
d-i grub-installer/only_debian boolean true
d-i grub-installer/with_other_os boolean true
tasksel tasksel/first multiselect standard

#Additional base packages
d-i pkgsel/include string openssh-server build-essential vim snmpd lib32z1-dev rsync ntp ntpdate
d-i finish-install/reboot_in_progress note
d-i cdrom-detect/eject boolean false
xserver-xorg xserver-xorg/autodetect_monitor boolean true
xserver-xorg xserver-xorg/config/monitor/selection-method \
       select medium
xserver-xorg xserver-xorg/config/monitor/mode-list \
       select 1024x768 @ 60 Hz

#Packages that gets installed AFTER the base installation
#hns* packages are build by our own to fasten the setup and management
d-i preseed/late_command string apt-install hp-health hp-snmp-agents hpsmh \
  hp-smh-templates hpacucli cpqacuxe ethtool linux-headers-2.6-openvz-amd64 \
  linux-image-2.6-openvz-amd64 vzctl vzquota hns-zabbix-agentd hns-openvz-common \
  hns-openvz-ubuntu-hosting hns-puppet firmware-qlogic

The private Debian repository

In our case the private repository is an essential factor because it will provide you with a fast software roll out, replicable software and consistent system state trough a large number of setups.

It is quite easy to setup your own repository with reprepro [5] and it is really useful.

As you can see in the preseed file we have a lot of this packages in our repository. Some examples:

  • hns-zabbix-agentd
is used for the automated installation of the Zabbix System Monitoring Agent.[6]
  • hns-openvz-common
creates a backup cronjob and installs a backup script
installs a set of scripts for Zabbix monitoring and therefore depends on the hns-zabbix-agentd package
  • hns-openvz-ubuntu-hosting
installs our pre-created Ubuntu[7] template for automated guest-installation
  • hns-puppet
installs the PuppetMaster client

The Puppet Master configuration management

If your preseeded setup was successful you have to configure your fresh server to fit your expectations. At our site we have ten OpenVZ servers in different countries and it is really important that they have the same configuration.

To make life easier we decided to use a configuration management like cfengine[8] but not as complex in setup and management. We found our solution in Puppet Master.

The only dependency Puppet Master has is Ruby. This should be installable on every distribution in minutes.

After the installation you have to configure the Puppet Master to do things. Typically this means accepting the public private key pair and doing some configuration stuff. At our site we use Puppet Master to manage an installation of about 100 servers.

One thing it does is to change the sysctl.conf file.

To get a feeling for it here is an configuration example (only OpenVZ server hosts):


# ALL----- OPENVZ
node "yourserver1.yourdns.com", "yourserver2.yourdns.com", "yourserver3.yourdns.com", "yourserver4.yourdns.com", "yourserver5.yourdns.com", "yourserver6.yourdns.com", "yourserver7.yourdns.com"{
    file { "/etc/aliases":
        mode => 644,
        owner => root,
        group => root,
        source => "puppet://yourpuppetmaster/files/openvz-hosts/etc/aliases",
    }
    exec { "subscribe-newaliases":
        command => "/usr/bin/newaliases && /bin/echo NEALIASSES",
        subscribe => File["/etc/aliases"],
        refreshonly => true,
        logoutput => true
    }

    file { "/etc/apt/sources.list":
        mode => 644,
        owner => root,
        group => root,
        source => "puppet://yourpuppetmaster/files/openvz-hosts/etc/apt/sources.list"
    }
    file { "/root/.ssh/authorized_keys2":
        mode => 644,
        owner => root,
        group => root,
        source => "puppet://yourpuppetmaster/files/openvz-hosts/root/authorized_keys2"
    }
    file { "/etc/vim/vimrc":
        mode => 644,
        owner => root,
        group => root,
        source => "puppet://yourpuppetmaster/files/openvz-hosts/etc/vim/vimrc"
    }
    file { "/etc/vz/conf/ve-vps.10.conf-sample":
        mode => 644,
        owner => root,
        group => root,
        source => "puppet://yourpuppetmaster/files/openvz-hosts/etc/vz/conf/ve-vps.10.conf-sample"
    }
    file { "/etc/sysctl.conf":
        mode => 644,
        owner => root,
        group => root,
        source => "puppet://yourpuppetmaster/files/openvz-hosts/etc/sysctl.conf"
    }
    exec { subscribe-sysctl:
        command => "/sbin/sysctl -p && /bin/echo SYSCTL EXECUTED",
        logoutput => true,
        refreshonly => true,
        subscribe => file["/etc/sysctl.conf"]
    }
    file { "/etc/ntp.conf":
        mode => 644,
        owner => root,
        group => root,
        source => "puppet://yourpuppetmaster/files/openvz-hosts/etc/ntp.conf"
    }
    file { "/root/.bashrc":
        mode => 644,
        owner => root,
        group => root,
        source => "puppet://yourpuppetmaster/files/openvz-hosts/root/bashrc"
    }
    file { "/srv/exim_config.sh":
        mode => 744,
        owner => root,
        group => root,
        source => "puppet://yourpuppetmaster/files/openvz-hosts/srv/exim_config.sh"
    }
    exec { subscribe-exim_config:
        command => "/srv/exim_config.sh && /usr/sbin/update-exim4.conf && /etc/init.d/exim4 restart && /bin/echo EXIM_CONFIG",
        logoutput => true,
        refreshonly => true,
        subscribe => file["/srv/exim_config.sh"]
    }
    file { "/etc/default/snmpd":
        mode => 644,
        owner => root,
        group => root,
        source => "puppet://yourpuppetmaster/files/openvz-hosts/etc/default/snmpd"
    }
    exec { subscribe-snmpd_config:
        command => "/etc/init.d/snmpd restart && /bin/echo SNMPD_CONFIG",
        logoutput => true,
        refreshonly => true,
        subscribe => file["/etc/default/snmpd"]
    }
}

Further information

For further information on the particular tools like apt-proxy/Debian repository/... we will add some documentation links! Write us if you have questions or if you want to see practical documentations on some of the tools.

References