OpenVZ Virtuozzo Containers Wiki - User contributions [en]

Quick installation (legacy)

2012-03-19T19:46:06Z

Iandunn: /* Rebooting into OpenVZ kernel */ changed x86_64 note to indicate that the problem is now obsolete and users don't need the to follow special build instructions

This document briefly describes the steps needed to install OpenVZ on your (RPM based) machine. For '''Debian''' based systems see [[Installation on Debian]].

This document is also available in the following languages: [http://forum.openvz.org/index.php?t=tree&goto=35&#msg_35 French], [http://forum.openvz.org/index.php?t=tree&goto=1805&#msg_1805 German],
[http://wiki.openvz.jp Japanese],
[[Quick_installation_(Spanish)|Spanish]].

OpenVZ consists of a kernel, user-level tools, and container templates. This guide tells how to install the kernel and the tools.

== Requirements ==
This guide assumes you are running recent release of Fedora Core (like FC5) or RHEL/CentOS 4. Currently, OpenVZ kernel tries to support the same hardware that Red Hat kernels support. For full hardware compatibility list, see [http://www.swsoft.com/en/products/virtuozzo/hcl/ Virtuozzo HCL].

=== Filesystems ===
It is recommended to use a separate partition for container's private directories (by default /vz/private/<veid>). The reason why you should do so is that if you wish to use OpenVZ per-container disk quota, you won't be able to use usual Linux disk quotas on the same partition. Bear in mind, that per-container quota in this context includes not only pure per-container quota, but also usual Linux disk quota used in containers, not on [[HN]].

At least try to avoid using the root partition for containers, because the root user of a container will be able to overcome the 5% disk space barrier in some situations. This way the HN root partition can be completely filled and it will break the system.

OpenVZ per-container disk quota is supported only for ext2/ext3 filesystems. So use one of these filesystems (ext3 is recommended) if you need per-container disk quota.

=== rpm or yum? ===

In case you have yum utility available on your system, you may want to use it effectively to install and update OpenVZ packages. In case you don't have yum, or don't want to use it, you can use plain old rpm. Instructions for both rpm and yum are provided below.

=== yum pre-setup ===
If you want to use yum, you should set up OpenVZ yum repository first.

Download [http://download.openvz.org/openvz.repo openvz.repo] file and put it to your <code>/etc/yum.repos.d/</code> repository. This can be achieved by the following commands, as root:
<pre>
# cd /etc/yum.repos.d
# wget http://download.openvz.org/openvz.repo
# rpm --import http://download.openvz.org/RPM-GPG-Key-OpenVZ
</pre>

In case you can not cd to /etc/yum.repos.d, it means either yum is not installed on your system, or yum version is too old. In that case, just stick to rpm installation method.

== Kernel installation ==

{{Note|In case you want to recompile the kernel yourself rather than use the one provided by OpenVZ, see [[kernel build]].}}

First, you need to choose what “flavor” of the kernel you want to install. Please refer to [[Kernel flavors]] for more information.

=== Using yum ===
Run the following command
<pre>
# yum install [o]vzkernel[-flavor]
</pre>

For RHEL6 kernel branch, use '''vzkernel''', for other branches use '''ovzkernel'''.

Note that <code>[-flavor]</code> is optional, and can be <code>-smp</code> or <code>-enterprise</code>. Refer to [[kernel flavors]] for more info.

{{Note|if you need to install <code>x86_64</code> kernel and yum offers to install both <code>x86_64</code> and <code>i686</code> kernels, answer <code>No</code> and specify architecture manually, like this: <code>yum install ovzkernel.x86_64</code>. This is fixed in newer yum versions.}}

=== Using rpm ===
Get the kernel binary RPM from the [[Download/kernel]] page. You only need one kernel RPM so please [[Kernel flavors|choose the appropriate one]] depending on your hardware.

Next, install the kernel RPM you chose:

<pre>
# rpm -ihv ovzkernel[-flavor]*.rpm
</pre>

Here <code>[-flavor]</code> is optional, and can be <code>-smp</code> or <code>-enterprise</code>. Refer to [[kernel flavors]] for more info.

{{Note|<tt>rpm -U</tt> (where <tt>-U</tt> stands for ''upgrade'') should '''not''' be used, otherwise all currently installed kernels will be uninstalled.}}

== Configuring the bootloader ==

In case GRUB is used as the boot loader, it will be configured automatically: lines similar to these will be added to the <tt>/boot/grub/grub.conf</tt> file:

<pre>
title Fedora Core (2.6.8-022stab029.1)
root (hd0,0)
kernel /vmlinuz-2.6.8-022stab029.1 ro root=/dev/sda5 quiet rhgb vga=0x31B
initrd /initrd-2.6.8-022stab029.1.img
</pre>
Change <tt>Fedora Core</tt> to <tt>OpenVZ</tt> (just for clarity reasons, so the OpenVZ kernels will not be mixed up with non-OpenVZ ones). Remove extra arguments from the kernel line, leaving only the <tt>root=...</tt> parameter. The modifed portion of <tt>/etc/grub.conf</tt> should look like this:

<pre>
title OpenVZ (2.6.8-022stab029.1)
root (hd0,0)
kernel /vmlinuz-2.6.8-022stab029.1 ro root=/dev/sda5
initrd /initrd-2.6.8-022stab029.1.img
</pre>

== Configuring ==

Please make sure the following steps are performed before rebooting into OpenVZ kernel.

=== sysctl ===

There are a number of kernel parameters that should be set for OpenVZ to work correctly. These parameters are stored in <tt>/etc/sysctl.conf</tt> file. Here are the relevant portions of the file; please edit accordingly.

<pre>
# On Hardware Node we generally need
# packet forwarding enabled and proxy arp disabled
net.ipv4.ip_forward = 1
net.ipv6.conf.default.forwarding = 1
net.ipv6.conf.all.forwarding = 1
net.ipv4.conf.default.proxy_arp = 0

# Enables source route verification
net.ipv4.conf.all.rp_filter = 1

# Enables the magic-sysrq key
kernel.sysrq = 1

# We do not want all our interfaces to send redirects
net.ipv4.conf.default.send_redirects = 1
net.ipv4.conf.all.send_redirects = 0
</pre>

=== SELinux ===

SELinux should be disabled. To that effect, put the following line to <code>/etc/sysconfig/selinux</code>:
<pre>
SELINUX=disabled
</pre>

=== Conntracks ===

{{Note|In OpenVZ kernels later than 2.6.8, connection tracking is enabled by default so skip this section.}}

In the old (2.6.8-based) OpenVZ kernels netfilter connection tracking for [[CT0]] is disabled by default. If you have a stateful firewall enabled on the host node (it is there by default on some distributions) you should either disable it, or enable connection tracking for [[CT0]].

To enable conntracks for CT0, add the following line to <code>/etc/modprobe.conf</code> file:
<pre>
options ip_conntrack ip_conntrack_enable_ve0=1
</pre>

== Rebooting into OpenVZ kernel ==

Now reboot the machine and choose "OpenVZ" on the boot loader menu. If the OpenVZ kernel has been booted successfully, proceed to installing the user-level tools for OpenVZ.

{{ Note |At one time there were problems with vzpkg and vzyum in x86_64 systems that required [[Install_OpenVZ_on_a_x86_64_system_Centos-Fedora|special installation procedures]], but these appear to be resolved in current builds.}}

== Installing the utilities ==

OpenVZ needs some user-level tools installed. Those are:

; vzctl
: A utility to control OpenVZ containers (create, destroy, start, stop, set parameters etc.)
; vzquota
: A utility to manage quotas for containers. Mostly used indirectly (by vzctl).

=== Using yum ===

<pre>
# yum install vzctl vzquota
</pre>

If on the x86_64 platform you would probably want to:

<pre>
# yum install vzctl.x86_64 vzquota.x86_64
</pre>

=== Using rpm ===

Download the binary RPMs of these utilities from [[Download/utils]]. Install them:

<pre>
# rpm -Uhv vzctl*.rpm vzquota*.rpm
</pre>

If rpm complains about unresolved dependencies, you'll have to satisfy them first, then repeat the installation.

When all the tools are installed, start the OpenVZ subsystem.

== Starting OpenVZ ==

As root, execute the following command:

<pre>
# /sbin/service vz start
</pre>

This will load all the needed OpenVZ kernel modules. This script should also start all the containers marked to be auto-started on machine boot (there aren't any yet).

During the next reboot, this script should be executed automatically.

== Installing OS template caches ==

An OS template cache is a Linux distribution installed into a container
and then packed into a gzipped tarball. Using such a cache, a new container
can be created in a matter of minutes.

Download precreated template caches from [http://openvz.org/download/template/cache Downloads » Templates » Precreated], or directly from [http://download.openvz.org/template/precreated/ download.openvz.org/template/precreated], or from one of the [[Download mirrors|mirrors]]. Put those tarballs '''as-is (no unpacking needed)''' to the <tt>/vz/template/cache/</tt> directory
(for Debain, this is <tt>/var/lib/vz/template/cache/</tt>).

== Next steps ==

OpenVZ is now set up on your machine. To load OpenVZ kernel by default, edit the default line in the /boot/grub/grub.conf file to point to the OpenVZ kernel. For example, if the OpenVZ kernel is the first kernel mentioned in the file, put it as default 0. See man grub.conf for more details.

Follow on to [[basic operations in OpenVZ environment]] document.
[[Category: Installation]]
[[Category: HOWTO]]

Setting up an iptables firewall

2012-03-19T18:19:19Z

Iandunn: /* Setting up a HN-based firewall */ added the URL to the wiki page to the script so that people will know where it came from

This document consists of two parts. The first is setting up a firewall (using iptables) on the HN, which will restrict traffic to the containers. The effect would emulate, as far as the containers and their customers are concerned, an external hardware firewall controlled by the sysadmin. The second is setting up a firewall that protects the [[HN]] itself but still allows traffic to the containers, thus allowing individual containers to define their own iptables.

While the firewalls shown here can be accomplished using iptables manually (or using Fedora core's iptables service), the methods presented here are especially modular and easy to modify. This is important when you have 20+ containers and a lot of other things to be doing...

The scripts and pathnames given here are for Fedora Core 6, though they can probably be applied to most similar SysV-like systems with little modification.

== A little background ==

On our systems, we use the HN to provide privileged services which are not appropriate for access by the containers. For example, the HN acts as a backup server, runs Nagios for health monitoring, has a webserver for managing the 3ware RAID controller, etc. The containers are leased to customers, who can't entirely be trusted, especially if they get hacked. As such, our scenario is one in which the HN must be protected from all access (even from the containers) except for a few trusted hosts (e.g. my home-office).

The exception to this is the nameserver, which we want open to the world. We use it as a caching nameserver for our containers and also to host DNS for a few customer domain.

== Simple firewall configuration independent to IP addresses: vzfirewall ==

<code>Vzfirewall</code> tool allows you to open/close ports for incoming connections with no dependencies to foreign IP addresses. E.g. you may allow a hostname <code>release.prod.example.com</code> to connect to port 5432 of VE 1234 and leave all other ports closed by modifying <code>1234.conf</code> file adding multiline <code>FIREWALL</code> directive into it:

<pre>
...
PRIVVMPAGES="300000:300000"
HOSTNAME="example.com"
...
FIREWALL="
...
# Allow access to PostgreSQL port only from release.prod
# and release.test machines. You may use domain names here.
[5432]
release.prod.example.com
release.test.example.com
...
"
</pre>

You must then run <code>vzfirewall -a</code> on your hardware node to apply changes made in <code>*.conf</code>.

Note that it is recommended to use hostnames instead of IP addresses here, so the configuration is persistent for VE movements to different IP-address: you just need to run <code>vzfirewall -a</code> again after movement. It is also reboot-safe, because applied to <code>/etc/sysconfig/iptables</code> (at RHEL systems).

Vzfirewall and its documentation are available at [http://en.dklab.ru/lib/dklab_vzfirewall/ http://en.dklab.ru/lib/dklab_vzfirewall/].

== An alternative from the author of Shorewall ==

For those who might find the solution provided in this wiki article unsatisfactory (for whatever reason), the creator of Shorewall (Tom Eastep) has written a nice article explaining how to use Shorewall on an OpenVZ host node to manage the host node, containers, and more... with quite a complex setup as an example. The article IS NOT an introduction to Shorewall for beginners, so some pre-existing knowledge and understanding of Shorewall may be required.

Shorewall and OpenVZ by Tom Eastep - http://www.shorewall.net/OpenVZ.html

See also this OpenVZ Forum posting - http://forum.openvz.org/index.php?t=msg&goto=16406&

== Setting up a HN-based firewall ==

This setup emulates (to the containers anyway) an external hardware firewall. It protects the HN from any access and then defines what services and ports are allowed/banned for individual containers. This leaves the firewall controlled by the site administrator, not be individual containers and the hackers who've gotten into them. ;)

First off, let's disable Fedora's existing <code>iptables</code> service:
<pre>
service iptables stop
chkconfig iptables off
</pre>

Now create the new <code>firewall</code> service. This code should be <code>/etc/init.d/firewall</code> and then should be chmod'd 755.
<pre>
#!/bin/sh
# firewall Start iptables firewall
# chkconfig: 2345 97 87
# description: Starts, stops and saves iptables firewall
# This script sets up the firewall for the INPUT chain (which is for
# the HN itself) and then processes the config files under
# /etc/firewall.d to set up additional rules in the FORWARD chain
# to allow access to containers' services.
# http://wiki.openvz.org/Setting_up_an_iptables_firewall

. /etc/init.d/functions

# the IP block allocated to this server
SEGMENT="192.168.0.0/24"
# the IP used by the hosting server itself
THISHOST="192.168.0.1"
# services that should be allowed to the HN;
# services for containers are configured in /etc/firewall.d/*
OKPORTS="53"
# hosts allowed full access through the firewall,
# to all containers and to this server
DMZS="12.34.56.78 90.123.45.67"

purge() {
echo -n "Firewall: Purging and allowing all traffic"
iptables -P OUTPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P INPUT ACCEPT
iptables -F
success ; echo
}

setup() {
echo -n "Firewall: Setting default policies to DROP"
iptables -P INPUT DROP
iptables -P FORWARD DROP
iptables -I INPUT -j ACCEPT -m state --state ESTABLISHED,RELATED
iptables -I FORWARD -j ACCEPT -m state --state ESTABLISHED,RELATED
iptables -I INPUT -j ACCEPT -i lo
iptables -I FORWARD -j ACCEPT --source $SEGMENT
success ; echo

echo "Firewall: Allowing access to HN"
for port in $OKPORTS ; do
echo -n " port $port"
iptables -I INPUT -j ACCEPT -s $SEGMENT -d $THISHOST --protocol tcp --destination-port $port
iptables -I INPUT -j ACCEPT -s $SEGMENT -d $THISHOST --protocol udp --destination-port $port
success ; echo
done
for ip in $DMZS ; do
echo -n " DMZ $ip"
iptables -I INPUT -i eth0 -j ACCEPT -s $ip
iptables -I FORWARD -i eth0 -j ACCEPT -s $ip
success ; echo
done

CTSETUPS=`echo /etc/firewall.d/*`
if [ "$CTSETUPS" != "/etc/firewall.d/*" ] ; then
echo "Firewall: Setting up container firewalls"
for i in $CTSETUPS ; do
. $i
echo -n " $CTNAME CT$CTID"
if [ -n "$BANNED" ]; then
for source in $BANNED ; do iptables -I FORWARD -j DROP --destination $CTIP --source $source ; done
fi
if [ -n "$OPENPORTS" ]; then
for port in $OPENPORTS ; do iptables -I FORWARD -j ACCEPT --protocol tcp --destination $CTIP --destination-port $port ; done
for port in $OPENPORTS ; do iptables -I FORWARD -j ACCEPT --protocol udp --destination $CTIP --destination-port $port ; done
fi
if [ -n "$DMZS" ]; then
for source in $DMZS ; do iptables -I FORWARD -j ACCEPT --protocol tcp --destination $CTIP --source $source ; done
for source in $DMZS ; do iptables -I FORWARD -j ACCEPT --protocol udp --destination $CTIP --source $source ; done
fi
[ $? -eq 0 ] && success || failure
echo
done
fi
}

case "$1" in
start)
echo "Starting firewall..."
purge
setup
;;
stop)
echo "Stopping firewall..."
purge
;;
restart)
$0 stop
$0 start
;;
status)
iptables -n -L
;;
*)
echo "Usage: $0 <start|stop|restart|status>"
;;
esac
</pre>

Note: This will only allow access to the HN from the hosts/networks defined in SEGMENT. If you'd like to open up the OKPORTS on the HN to everybody, you can remove the ''-s $SEGMENT'' parameters from the iptables commands under the "Firewall: Allowing access to HN" section. The modified lines would look like this:

<pre>
iptables -I INPUT -j ACCEPT -d $THISHOST --protocol tcp --destination-port $port
iptables -I INPUT -j ACCEPT -d $THISHOST --protocol udp --destination-port $port
</pre>

The above script can be called like this:
<pre>
service firewall start
service firewall stop
service firewall restart
service firewall status
</pre>

It will set up the firewall for the HN according to the parameters you specified for OKPORTS, DMZs, etc. and then it will call each file under /etc/firewall.d and process its configuration.

So create a file under /etc/firewall.d The exact filename isn't important, as long as it's meaningful to you, e.g. <code>ExampleCompany</code> or <code>ve12</code> and give it content like this:

<pre>
# This file is processed by /etc/init.d/firewall
CTID="1" # the container's ID#
CTNAME="Customer1" # A human-friendly label for the container
CTIP="192.168.1.34" # the IP address for this container
OPENPORTS="80 443" # ports that should be universally opened
# to the entire Internet
DMZS="1.2.3.0/24 5.6.7.8/32" # IPs and blocks that should have full access
# to the container's services
BANNED="" # IPs and blocks that should be entirely
# blocked from the container's services
</pre>

And there you go.

Note: You can only put one IP address inside the CTIP variable, but if your container has multiple IP addresses you can create a copy of the file for each IP address.

Go ahead and start the firewall and check its status:
<pre>
service firewall restart
service firewall status
</pre>

As you can see, you can now add and edit the configurations for individual containers very easily. This method proves a lot easier to manage than Fedora's iptables-config mechamism!

To make the firewall service automatically start when the HN boots, run
<pre>
chkconfig --add firewall
</pre>

=== Debian Notes ===

The setup above works fine for Debian as well, however /etc/init.d/functions is missing. Here is a very simple version that you can use:

# /etc/init.d/functions

success() {
echo -n "...success"
}
failure() {
echo -n "...failure"
}

== Setting up a firewall that allows per-container configuration ==

This setup configures iptables on the HN to disallow access to all hosts, including the containers. However, it allows all traffic into the containers so they may define their own iptables rules and therefore manage their own firewall.

<pre>
iptables -P FORWARD ACCEPT
iptables -F FORWARD
</pre>

This will remove all rules for the FORWARD chain so all packets can pass back and forth between containers and the outside world.

If you want to use a firewall inside a container, please load these modules BEFORE starting the container:

<pre>
modprobe xt_tcpudp
modprobe ip_conntrack
</pre>

If you do not, you will get an error like this: "iptables: No chain/target/match by that name"

If you want to use stateful firewall rules (and you should!) you will also need to make sure that 'ipt_state' is in the 'IPTABLES' option in your vz.conf file:

<pre>
IPTABLES="ipt_REJECT ipt_tos ipt_limit ipt_multiport iptable_filter iptable_mangle ipt_TCPMSS ipt_tcpmss ipt_ttl ipt_length ipt_state"
</pre>

Also make sure the 'xt_state' module is loaded on the host:

<pre>
modprobe xt_state
</pre>

== See also ==

* [[Traffic accounting with iptables]]

[[Category: Networking]]

Setting up an iptables firewall

2011-12-27T19:23:43Z

Iandunn: /* Setting up a HN-based firewall */ Added note about creating a copy of the container config file for each IP the container has assigned

This document consists of two parts. The first is setting up a firewall (using iptables) on the HN, which will restrict traffic to the containers. The effect would emulate, as far as the containers and their customers are concerned, an external hardware firewall controlled by the sysadmin. The second is setting up a firewall that protects the [[HN]] itself but still allows traffic to the containers, thus allowing individual containers to define their own iptables.

While the firewalls shown here can be accomplished using iptables manually (or using Fedora core's iptables service), the methods presented here are especially modular and easy to modify. This is important when you have 20+ containers and a lot of other things to be doing...

The scripts and pathnames given here are for Fedora Core 6, though they can probably be applied to most similar SysV-like systems with little modification.

== A little background ==

On our systems, we use the HN to provide privileged services which are not appropriate for access by the containers. For example, the HN acts as a backup server, runs Nagios for health monitoring, has a webserver for managing the 3ware RAID controller, etc. The containers are leased to customers, who can't entirely be trusted, especially if they get hacked. As such, our scenario is one in which the HN must be protected from all access (even from the containers) except for a few trusted hosts (e.g. my home-office).

The exception to this is the nameserver, which we want open to the world. We use it as a caching nameserver for our containers and also to host DNS for a few customer domain.

== Simple firewall configuration independent to IP addresses: vzfirewall ==

<code>Vzfirewall</code> tool allows you to open/close ports for incoming connections with no dependencies to foreign IP addresses. E.g. you may allow a hostname <code>release.prod.example.com</code> to connect to port 5432 of VE 1234 and leave all other ports closed by modifying <code>1234.conf</code> file adding multiline <code>FIREWALL</code> directive into it:

<pre>
...
PRIVVMPAGES="300000:300000"
HOSTNAME="example.com"
...
FIREWALL="
...
# Allow access to PostgreSQL port only from release.prod
# and release.test machines. You may use domain names here.
[5432]
release.prod.example.com
release.test.example.com
...
"
</pre>

You must then run <code>vzfirewall -a</code> on your hardware node to apply changes made in <code>*.conf</code>.

Note that it is recommended to use hostnames instead of IP addresses here, so the configuration is persistent for VE movements to different IP-address: you just need to run <code>vzfirewall -a</code> again after movement. It is also reboot-safe, because applied to <code>/etc/sysconfig/iptables</code> (at RHEL systems).

Vzfirewall and its documentation are available at [http://en.dklab.ru/lib/dklab_vzfirewall/ http://en.dklab.ru/lib/dklab_vzfirewall/].

== An alternative from the author of Shorewall ==

For those who might find the solution provided in this wiki article unsatisfactory (for whatever reason), the creator of Shorewall (Tom Eastep) has written a nice article explaining how to use Shorewall on an OpenVZ host node to manage the host node, containers, and more... with quite a complex setup as an example. The article IS NOT an introduction to Shorewall for beginners, so some pre-existing knowledge and understanding of Shorewall may be required.

Shorewall and OpenVZ by Tom Eastep - http://www.shorewall.net/OpenVZ.html

See also this OpenVZ Forum posting - http://forum.openvz.org/index.php?t=msg&goto=16406&

== Setting up a HN-based firewall ==

This setup emulates (to the containers anyway) an external hardware firewall. It protects the HN from any access and then defines what services and ports are allowed/banned for individual containers. This leaves the firewall controlled by the site administrator, not be individual containers and the hackers who've gotten into them. ;)

First off, let's disable Fedora's existing <code>iptables</code> service:
<pre>
service iptables stop
chkconfig iptables off
</pre>

Now create the new <code>firewall</code> service. This code should be <code>/etc/init.d/firewall</code> and then should be chmod'd 755.
<pre>
#!/bin/sh
# firewall Start iptables firewall
# chkconfig: 2345 97 87
# description: Starts, stops and saves iptables firewall
# This script sets up the firewall for the INPUT chain (which is for
# the HN itself) and then processes the config files under
# /etc/firewall.d to set up additional rules in the FORWARD chain
# to allow access to containers' services.

. /etc/init.d/functions

# the IP block allocated to this server
SEGMENT="192.168.0.0/24"
# the IP used by the hosting server itself
THISHOST="192.168.0.1"
# services that should be allowed to the HN;
# services for containers are configured in /etc/firewall.d/*
OKPORTS="53"
# hosts allowed full access through the firewall,
# to all containers and to this server
DMZS="12.34.56.78 90.123.45.67"

purge() {
echo -n "Firewall: Purging and allowing all traffic"
iptables -P OUTPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P INPUT ACCEPT
iptables -F
success ; echo
}

setup() {
echo -n "Firewall: Setting default policies to DROP"
iptables -P INPUT DROP
iptables -P FORWARD DROP
iptables -I INPUT -j ACCEPT -m state --state ESTABLISHED,RELATED
iptables -I FORWARD -j ACCEPT -m state --state ESTABLISHED,RELATED
iptables -I INPUT -j ACCEPT -i lo
iptables -I FORWARD -j ACCEPT --source $SEGMENT
success ; echo

echo "Firewall: Allowing access to HN"
for port in $OKPORTS ; do
echo -n " port $port"
iptables -I INPUT -j ACCEPT -s $SEGMENT -d $THISHOST --protocol tcp --destination-port $port
iptables -I INPUT -j ACCEPT -s $SEGMENT -d $THISHOST --protocol udp --destination-port $port
success ; echo
done
for ip in $DMZS ; do
echo -n " DMZ $ip"
iptables -I INPUT -i eth0 -j ACCEPT -s $ip
iptables -I FORWARD -i eth0 -j ACCEPT -s $ip
success ; echo
done

CTSETUPS=`echo /etc/firewall.d/*`
if [ "$CTSETUPS" != "/etc/firewall.d/*" ] ; then
echo "Firewall: Setting up container firewalls"
for i in $CTSETUPS ; do
. $i
echo -n " $CTNAME CT$CTID"
if [ -n "$BANNED" ]; then
for source in $BANNED ; do iptables -I FORWARD -j DROP --destination $CTIP --source $source ; done
fi
if [ -n "$OPENPORTS" ]; then
for port in $OPENPORTS ; do iptables -I FORWARD -j ACCEPT --protocol tcp --destination $CTIP --destination-port $port ; done
for port in $OPENPORTS ; do iptables -I FORWARD -j ACCEPT --protocol udp --destination $CTIP --destination-port $port ; done
fi
if [ -n "$DMZS" ]; then
for source in $DMZS ; do iptables -I FORWARD -j ACCEPT --protocol tcp --destination $CTIP --source $source ; done
for source in $DMZS ; do iptables -I FORWARD -j ACCEPT --protocol udp --destination $CTIP --source $source ; done
fi
[ $? -eq 0 ] && success || failure
echo
done
fi
}

case "$1" in
start)
echo "Starting firewall..."
purge
setup
;;
stop)
echo "Stopping firewall..."
purge
;;
restart)
$0 stop
$0 start
;;
status)
iptables -n -L
;;
*)
echo "Usage: $0 <start|stop|restart|status>"
;;
esac
</pre>

Note: This will only allow access to the HN from the hosts/networks defined in SEGMENT. If you'd like to open up the OKPORTS on the HN to everybody, you can remove the ''-s $SEGMENT'' parameters from the iptables commands under the "Firewall: Allowing access to HN" section. The modified lines would look like this:

<pre>
iptables -I INPUT -j ACCEPT -d $THISHOST --protocol tcp --destination-port $port
iptables -I INPUT -j ACCEPT -d $THISHOST --protocol udp --destination-port $port
</pre>

The above script can be called like this:
<pre>
service firewall start
service firewall stop
service firewall restart
service firewall status
</pre>

It will set up the firewall for the HN according to the parameters you specified for OKPORTS, DMZs, etc. and then it will call each file under /etc/firewall.d and process its configuration.

So create a file under /etc/firewall.d The exact filename isn't important, as long as it's meaningful to you, e.g. <code>ExampleCompany</code> or <code>ve12</code> and give it content like this:

<pre>
# This file is processed by /etc/init.d/firewall
CTID="1" # the container's ID#
CTNAME="Customer1" # A human-friendly label for the container
CTIP="192.168.1.34" # the IP address for this container
OPENPORTS="80 443" # ports that should be universally opened
# to the entire Internet
DMZS="1.2.3.0/24 5.6.7.8/32" # IPs and blocks that should have full access
# to the container's services
BANNED="" # IPs and blocks that should be entirely
# blocked from the container's services
</pre>

And there you go.

Note: You can only put one IP address inside the CTIP variable, but if your container has multiple IP addresses you can create a copy of the file for each IP address.

Go ahead and start the firewall and check its status:
<pre>
service firewall restart
service firewall status
</pre>

As you can see, you can now add and edit the configurations for individual containers very easily. This method proves a lot easier to manage than Fedora's iptables-config mechamism!

To make the firewall service automatically start when the HN boots, run
<pre>
chkconfig --add firewall
</pre>

=== Debian Notes ===

The setup above works fine for Debian as well, however /etc/init.d/functions is missing. Here is a very simple version that you can use:

# /etc/init.d/functions

success() {
echo -n "...success"
}
failure() {
echo -n "...failure"
}

== Setting up a firewall that allows per-container configuration ==

This setup configures iptables on the HN to disallow access to all hosts, including the containers. However, it allows all traffic into the containers so they may define their own iptables rules and therefore manage their own firewall.

<pre>
iptables -P FORWARD ACCEPT
iptables -F FORWARD
</pre>

This will remove all rules for the FORWARD chain so all packets can pass back and forth between containers and the outside world.

If you want to use a firewall inside a container, please load these modules BEFORE starting the container:

<pre>
modprobe xt_tcpudp
modprobe ip_conntrack
</pre>

If you do not, you will get an error like this: "iptables: No chain/target/match by that name"

If you want to use stateful firewall rules (and you should!) you will also need to make sure that 'ipt_state' is in the 'IPTABLES' option in your vz.conf file:

<pre>
IPTABLES="ipt_REJECT ipt_tos ipt_limit ipt_multiport iptable_filter iptable_mangle ipt_TCPMSS ipt_tcpmss ipt_ttl ipt_length ipt_state"
</pre>

Also make sure the 'xt_state' module is loaded on the host:

<pre>
modprobe xt_state
</pre>

== See also ==

* [[Traffic accounting with iptables]]

[[Category: Networking]]

Setting up an iptables firewall

2010-04-22T22:42:13Z

Iandunn: /* Added chkconfig command to make service automatically start on boot */

This document consists of two parts. The first is setting up a firewall (using iptables) on the HN, which will restrict traffic to the containers. The effect would emulate, as far as the containers and their customers are concerned, an external hardware firewall controlled by the sysadmin. The second is setting up a firewall that protects the [[HN]] itself but still allows traffic to the containers, thus allowing individual containers to define their own iptables.

While the firewalls shown here can be accomplished using iptables manually (or using Fedora core's iptables service), the methods presented here are especially modular and easy to modify. This is important when you have 20+ containers and a lot of other things to be doing...

The scripts and pathnames given here are for Fedora Core 6, though they can probably be applied to most similar SysV-like systems with little modification.

== A little background ==

On our systems, we use the HN to provide privileged services which are not appropriate for access by the containers. For example, the HN acts as a backup server, runs Nagios for health monitoring, has a webserver for managing the 3ware RAID controller, etc. The containers are leased to customers, who can't entirely be trusted, especially if they get hacked. As such, our scenario is one in which the HN must be protected from all access (even from the containers) except for a few trusted hosts (e.g. my home-office).

The exception to this is the nameserver, which we want open to the world. We use it as a caching nameserver for our containers and also to host DNS for a few customer domain.

== An alternative from the author of Shorewall ==

For those who might find the solution provided in this wiki article unsatisfactory (for whatever reason), the creator of Shorewall (Tom Eastep) has written a nice article explaining how to use Shorewall on an OpenVZ host node to manage the host node, containers, and more... with quite a complex setup as an example. The article IS NOT an introduction to Shorewall for beginners, so some pre-existing knowledge and understanding of Shorewall may be required.

Shorewall and OpenVZ by Tom Eastep - http://www.shorewall.net/OpenVZ.html

See also this OpenVZ Forum posting - http://forum.openvz.org/index.php?t=msg&goto=16406&

== Setting up a HN-based firewall ==

This setup emulates (to the containers anyway) an external hardware firewall. It protects the HN from any access and then defines what services and ports are allowed/banned for individual containers. This leaves the firewall controlled by the site administrator, not be individual containers and the hackers who've gotten into them. ;)

First off, let's disable Fedora's existing <code>iptables</code> service:
<pre>
service iptables stop
chkconfig iptables off
</pre>

Now create the new <code>firewall</code> service. This code should be <code>/etc/init.d/firewall</code> and then should be chmod'd 755.
<pre>
#!/bin/sh
# firewall Start iptables firewall
# chkconfig: 2345 08 92
# description: Starts, stops and saves iptables firewall
# This script sets up the firewall for the INPUT chain (which is for
# the HN itself) and then processes the config files under
# /etc/firewall.d to set up additional rules in the FORWARD chain
# to allow access to containers' services.

. /etc/init.d/functions

# the IP block allocated to this server
SEGMENT="192.168.0.0/24"
# the IP used by the hosting server itself
THISHOST="192.168.0.1"
# services that should be allowed to the HN;
# services for containers are configured in /etc/firewall.d/*
OKPORTS="53"
# hosts allowed full access through the firewall,
# to all containers and to this server
DMZS="12.34.56.78 90.123.45.67"

purge() {
echo -n "Firewall: Purging and allowing all traffic"
iptables -P OUTPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P INPUT ACCEPT
iptables -F
success ; echo
}

setup() {
echo -n "Firewall: Setting default policies to DROP"
iptables -P INPUT DROP
iptables -P FORWARD DROP
iptables -I INPUT -j ACCEPT -m state --state ESTABLISHED,RELATED
iptables -I FORWARD -j ACCEPT -m state --state ESTABLISHED,RELATED
iptables -I INPUT -j ACCEPT -i lo
iptables -I FORWARD -j ACCEPT --source $SEGMENT
success ; echo

echo "Firewall: Allowing access to HN"
for port in $OKPORTS ; do
echo -n " port $port"
iptables -I INPUT -j ACCEPT -s $SEGMENT -d $THISHOST --protocol tcp --destination-port $port
iptables -I INPUT -j ACCEPT -s $SEGMENT -d $THISHOST --protocol udp --destination-port $port
success ; echo
done
for ip in $DMZS ; do
echo -n " DMZ $ip"
iptables -I INPUT -i eth0 -j ACCEPT -s $ip
iptables -I FORWARD -i eth0 -j ACCEPT -s $ip
success ; echo
done

CTSETUPS=`echo /etc/firewall.d/*`
if [ "$CTSETUPS" != "/etc/firewall.d/*" ] ; then
echo "Firewall: Setting up container firewalls"
for i in $CTSETUPS ; do
. $i
echo -n " $CTNAME CT$CTID"
if [ -n "$BANNED" ]; then
for source in $BANNED ; do iptables -I FORWARD -j DROP --destination $CTIP --source $source ; done
fi
if [ -n "$OPENPORTS" ]; then
for port in $OPENPORTS ; do iptables -I FORWARD -j ACCEPT --protocol tcp --destination $CTIP --destination-port $port ; done
for port in $OPENPORTS ; do iptables -I FORWARD -j ACCEPT --protocol udp --destination $CTIP --destination-port $port ; done
fi
if [ -n "$DMZS" ]; then
for source in $DMZS ; do iptables -I FORWARD -j ACCEPT --protocol tcp --destination $CTIP --source $source ; done
for source in $DMZS ; do iptables -I FORWARD -j ACCEPT --protocol udp --destination $CTIP --source $source ; done
fi
[ $? -eq 0 ] && success || failure
echo
done
fi
}

case "$1" in
start)
echo "Starting firewall..."
purge
setup
;;
stop)
echo "Stopping firewall..."
purge
;;
restart)
$0 stop
$0 start
;;
status)
iptables -n -L
;;
*)
echo "Usage: $0 <start|stop|restart|status>"
;;
esac
</pre>

The above script can be called like this:
<pre>
service firewall start
service firewall stop
service firewall restart
service firewall status
</pre>

It will set up the firewall for the HN according to the parameters you specified for OKPORTS, DMZs, etc. and then it will call each file under /etc/firewall.d and process its configuration.

So create a file under /etc/firewall.d The exact filename isn't important, as long as it's meaningful to you, e.g. <code>ExampleCompany</code> or <code>ve12</code> and give it content like this:

<pre>
# This file is processed by /etc/init.d/firewall
CTID="1" # the container's ID#
CTNAME="Customer1" # A human-friendly label for the container
CTIP="192.168.1.34" # the IP address for this container
OPENPORTS="80 443" # ports that should be universally opened
# to the entire Internet
DMZS="1.2.3.0/24 5.6.7.8/32" # IPs and blocks that should have full access
# to the container's services
BANNED="" # IPs and blocks that should be entirely
# blocked from the container's services
</pre>

And there you go. Go ahead and start the firewall and check its status:
<pre>
service firewall restart
service firewall status
</pre>

As you can see, you can now add and edit the configurations for individual containers very easily. This method proves a lot easier to manage than Fedora's iptables-config mechamism!

To make the firewall service automatically start when the HN boots, run
<pre>
chkconfig --add firewall
</pre>

=== Debian Notes ===

The setup above works fine for Debian as well, however /etc/init.d/functions is missing. Here is a very simple version that you can use:

# /etc/init.d/functions

success() {
echo -n "...success"
}
failure() {
echo -n "...failure"
}

== Setting up a firewall that allows per-container configuration ==

This setup configures iptables on the HN to disallow access to all hosts, including the containers. However, it allows all traffic into the containers so they may define their own iptables rules and therefore manage their own firewall.

<pre>
iptables -P FORWARD ACCEPT
iptables -F FORWARD
</pre>

This will remove all rules for the FORWARD chain so all packets can pass back and forth between containers and the outside world.

If you want to use a firewall inside a container, please load these modules BEFORE starting the container:

<pre>
modprobe xt_tcpudp
modprobe ip_conntrack
</pre>

If you do not, you will get an error like this: "iptables: No chain/target/match by that name"

If you want to use stateful firewall rules (and you should!) you will also need to make sure that 'ipt_state' is in the 'IPTABLES' option in your vz.conf file:

<pre>
IPTABLES="ipt_REJECT ipt_tos ipt_limit ipt_multiport iptable_filter iptable_mangle ipt_TCPMSS ipt_tcpmss ipt_ttl ipt_length ipt_state"
</pre>

Also make sure the 'xt_state' module is loaded on the host:

<pre>
modprobe xt_state
</pre>

== See also ==

* [[Traffic accounting with iptables]]

[[Category: Networking]]