Editing HA cluster with DRBD and Heartbeat
Warning: You are not logged in. Your IP address will be publicly visible if you make any edits. If you log in or create an account, your edits will be attributed to your username, along with other benefits.
The edit can be undone.
Please check the comparison below to verify that this is what you want to do, and then save the changes below to finish undoing the edit.
Latest revision | Your text | ||
Line 1: | Line 1: | ||
− | This article shows how to setup a OpenVZ high availability (HA) cluster using the data replication software DRBD and the cluster manager Heartbeat. In this example the two machines | + | This article shows how to setup a OpenVZ high availability (HA) cluster using the data replication software DRBD and the cluster manager Heartbeat. In this example the two machines builing the cluster run on CentOS 4.3. The article also shows how to do kernel updates in the cluster, including necessary steps like recompiling of new DRBD userspace tools. For this purpose, kernel 2.6.8-022stab078.10 (containing DRBD module 0.7.17) is used as initial kernel version, and kernel 2.6.8-022stab078.14 (containing DRBD module 0.7.20) as updated kernel version. |
− | + | Additional information about clustering of virtual machines can be found in the following paper: http://www.linuxtag.org/2006/fileadmin/linuxtag/dvd/12080-paper.pdf | |
− | |||
− | Additional information about clustering of virtual machines can be found in the following paper: | ||
− | |||
− | |||
− | |||
− | |||
+ | Some other additional information can be found in the documentation of the Thomas-Krenn.AG cluster (The author of this howto is working in the cluster development there, that is the reason why he was able to write this howto :-). The full documentation with interesting illustrations is currently only available in German: | ||
+ | http://my.thomas-krenn.com/service_support/index.php/page.242 | ||
== Prerequisites == | == Prerequisites == | ||
The OpenVZ kernel already includes the DRBD module. The DRBD userspace tools and the cluster manager Heartbeat must be provided seperately. As the API version of the DRBD userspace tools must exactly match the API version of the module, compile them yourself. Also compile Heartbeat yourself, as at the time of this writing the CentOS extras repository only contained an old CVS version of Heartbeat. | The OpenVZ kernel already includes the DRBD module. The DRBD userspace tools and the cluster manager Heartbeat must be provided seperately. As the API version of the DRBD userspace tools must exactly match the API version of the module, compile them yourself. Also compile Heartbeat yourself, as at the time of this writing the CentOS extras repository only contained an old CVS version of Heartbeat. | ||
− | On a hardware node for production use there should not be any | + | On a hardware node for production use there should not be any applications that are not really needed for running OpenVZ (any things which are not needed by OpenVZ should run in a VE for security reasons). As a result, compile DRBD and Heartbeat on another machine running CentOS 4.3 (in this example I used a virtual machine on a VMware Server). |
=== Compiling Heartbeat === | === Compiling Heartbeat === | ||
Line 85: | Line 81: | ||
! other install options | ! other install options | ||
| no firewall, no SELinux | | no firewall, no SELinux | ||
− | | no firewall, no | + | | no firewall, no SELinux |
+ | |- | ||
+ | ! package groups | ||
+ | | deactivated everything, only kept vim-enhanced | ||
+ | | deactivated everything, only kept vim-enhanced | ||
|} | |} | ||
Line 92: | Line 92: | ||
Get the OpenVZ kernel and utilities and install them on both nodes, as described in [[quick installation]]. Update grub configuration to use the OpenVZ kernel by default. Disable starting of OpenVZ on system boot on both nodes (OpenVZ will be started and stopped by Heartbeat): | Get the OpenVZ kernel and utilities and install them on both nodes, as described in [[quick installation]]. Update grub configuration to use the OpenVZ kernel by default. Disable starting of OpenVZ on system boot on both nodes (OpenVZ will be started and stopped by Heartbeat): | ||
<pre> | <pre> | ||
− | [root@ovz-node1 ~]# chkconfig vz | + | [root@ovz-node1 ~]# chkconfig --del vz |
[root@ovz-node1 ~]# | [root@ovz-node1 ~]# | ||
</pre> | </pre> | ||
Line 99: | Line 99: | ||
== Setting up DRBD == | == Setting up DRBD == | ||
− | + | On each of the two nodes create a partition that acts as underlying DRBD device. The partitions should have exectly the same size (I created a 10 GB partition hda3 using fdisk on each node for this example). Note that it might be necessary to reboot the machines to re-read the partition table. | |
Install the rpm of the DRBD userspace tools on both nodes: | Install the rpm of the DRBD userspace tools on both nodes: | ||
Line 213: | Line 213: | ||
[root@ovz-node1 ~]# mkdir -p /vz/cluster/etc/sysconfig | [root@ovz-node1 ~]# mkdir -p /vz/cluster/etc/sysconfig | ||
[root@ovz-node1 ~]# mkdir -p /vz/cluster/var | [root@ovz-node1 ~]# mkdir -p /vz/cluster/var | ||
− | [root@ovz-node1 ~]# cp -a /etc/vz | + | [root@ovz-node1 ~]# cp -a /etc/vz /vz/cluster/etc/ |
− | [root@ovz-node1 ~]# cp -a /etc/sysconfig/vz-scripts | + | [root@ovz-node1 ~]# cp -a /etc/sysconfig/vz-scripts /vz/cluster/etc/sysconfig/ |
− | [root@ovz-node1 ~]# cp -a /var/vzquota | + | [root@ovz-node1 ~]# cp -a /var/vzquota /vz/cluster/var/ |
[root@ovz-node1 ~]# umount /dev/drbd0 | [root@ovz-node1 ~]# umount /dev/drbd0 | ||
[root@ovz-node1 ~]# | [root@ovz-node1 ~]# | ||
Line 231: | Line 231: | ||
[root@ovz-node1 ~]# | [root@ovz-node1 ~]# | ||
</pre> | </pre> | ||
− | Create the Heartbeat configuration file ha.cf and copy it to | + | Create the Heartbeat configuration file ha.cf and copy it to /etc/ha.d/ha.cf on both nodes. Details about this file can be found at http://www.linux-ha.org/ha.cf. Below is an example configuration which uses the two network connections and also a serial connection for heartbeat packets: |
<pre> | <pre> | ||
# Heartbeat logging configuration | # Heartbeat logging configuration | ||
Line 262: | Line 262: | ||
respawn hacluster /usr/lib64/heartbeat/ipfail | respawn hacluster /usr/lib64/heartbeat/ipfail | ||
</pre> | </pre> | ||
− | Create the Heartbeat configuration file authkeys and copy it to | + | Create the Heartbeat configuration file authkeys and copy it to /etc/ha.d/authkeys on both nodes. Set the permissions of this file to 600. Details about this file can be found at http://www.linux-ha.org/authkeys. Below is an example: |
<pre> | <pre> | ||
auth 1 | auth 1 | ||
1 sha1 PutYourSuperSecretKeyHere | 1 sha1 PutYourSuperSecretKeyHere | ||
</pre> | </pre> | ||
− | Create the Heartbeat configuration file haresources and copy it to | + | Create the Heartbeat configuration file haresources and copy it to /etc/ha.d/haresources on both nodes. Details about this file can be found at http://www.linux-ha.org/haresources. Below is an example: |
<pre> | <pre> | ||
− | ovz-node1 | + | ovz-node1 datadisk::r0 Filesystem::/dev/drbd0::/vz::ext3 vz MailTo::youremail@yourdomain.tld |
</pre> | </pre> | ||
Finally, you can now start heartbeat on both nodes: | Finally, you can now start heartbeat on both nodes: | ||
Line 279: | Line 279: | ||
</pre> | </pre> | ||
− | == | + | == How to do OpenVZ kernel updates when it contains a new DRBD version == |
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
As mentioned above, it is important to use the correct version of the DRBD userspace tools. When an OpenVZ kernel contains a new DRBD version, it is important that the DRBD API version of the userspace tools matches the API version of the DRBD module that is included in the OpenVZ kernel. The API versions can be found at http://svn.drbd.org/drbd/branches/drbd-0.7/ChangeLog. The best way is to always use the version of the DRBD userspace tools that matches the version of the DRBD module that is included in the OpenVZ kernel. | As mentioned above, it is important to use the correct version of the DRBD userspace tools. When an OpenVZ kernel contains a new DRBD version, it is important that the DRBD API version of the userspace tools matches the API version of the DRBD module that is included in the OpenVZ kernel. The API versions can be found at http://svn.drbd.org/drbd/branches/drbd-0.7/ChangeLog. The best way is to always use the version of the DRBD userspace tools that matches the version of the DRBD module that is included in the OpenVZ kernel. | ||
Line 373: | Line 344: | ||
Ensure after every update of OpenVZ tools that OpenVZ is not started on system boot. To disable starting of OpenVZ on system boot execute on both nodes: | Ensure after every update of OpenVZ tools that OpenVZ is not started on system boot. To disable starting of OpenVZ on system boot execute on both nodes: | ||
<pre> | <pre> | ||
− | [root@ovz-node1 ~]# chkconfig vz | + | [root@ovz-node1 ~]# chkconfig --del vz |
[root@ovz-node1 ~]# | [root@ovz-node1 ~]# | ||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
</pre> | </pre> | ||
[[Category: HOWTO]] | [[Category: HOWTO]] |