OpenVZ Virtuozzo Containers Wiki - User contributions [en]

Setting up an iptables firewall

2010-11-11T23:37:11Z

DmitryKoterov: /* Simple firewall configuration independent to IP addresses: vzfirewall */

This document consists of two parts. The first is setting up a firewall (using iptables) on the HN, which will restrict traffic to the containers. The effect would emulate, as far as the containers and their customers are concerned, an external hardware firewall controlled by the sysadmin. The second is setting up a firewall that protects the [[HN]] itself but still allows traffic to the containers, thus allowing individual containers to define their own iptables.

While the firewalls shown here can be accomplished using iptables manually (or using Fedora core's iptables service), the methods presented here are especially modular and easy to modify. This is important when you have 20+ containers and a lot of other things to be doing...

The scripts and pathnames given here are for Fedora Core 6, though they can probably be applied to most similar SysV-like systems with little modification.

== A little background ==

On our systems, we use the HN to provide privileged services which are not appropriate for access by the containers. For example, the HN acts as a backup server, runs Nagios for health monitoring, has a webserver for managing the 3ware RAID controller, etc. The containers are leased to customers, who can't entirely be trusted, especially if they get hacked. As such, our scenario is one in which the HN must be protected from all access (even from the containers) except for a few trusted hosts (e.g. my home-office).

The exception to this is the nameserver, which we want open to the world. We use it as a caching nameserver for our containers and also to host DNS for a few customer domain.

== Simple firewall configuration independent to IP addresses: vzfirewall ==

<code>Vzfirewall</code> tool allows you to open/close ports for incoming connections with no dependencies to foreign IP addresses. E.g. you may allow a hostname <code>release.prod.example.com</code> to connect to port 5432 of VE 1234 and leave all other ports closed by modifying <code>1234.conf</code> file adding multiline <code>FIREWALL</code> directive into it:

<pre>
...
PRIVVMPAGES="300000:300000"
HOSTNAME="example.com"
...
FIREWALL="
...
# Allow access to PostgreSQL port only from release.prod
# and release.test machines. You may use domain names here.
[5432]
release.prod.example.com
release.test.example.com
...
"
</pre>

You must then run <code>vzfirewall -a</code> on your hardware node to apply changes made in <code>*.conf</code>.

Note that it is recommended to use hostnames instead of IP addresses here, so the configuration is persistent for VE movements to different IP-address: you just need to run <code>vzfirewall -a</code> again after movement. It is also reboot-safe, because applied to <code>/etc/sysconfig/iptables</code> (at RHEL systems).

Vzfirewall and its documentation are available at [http://en.dklab.ru/lib/dklab_vzfirewall/ http://en.dklab.ru/lib/dklab_vzfirewall/].

== An alternative from the author of Shorewall ==

For those who might find the solution provided in this wiki article unsatisfactory (for whatever reason), the creator of Shorewall (Tom Eastep) has written a nice article explaining how to use Shorewall on an OpenVZ host node to manage the host node, containers, and more... with quite a complex setup as an example. The article IS NOT an introduction to Shorewall for beginners, so some pre-existing knowledge and understanding of Shorewall may be required.

Shorewall and OpenVZ by Tom Eastep - http://www.shorewall.net/OpenVZ.html

See also this OpenVZ Forum posting - http://forum.openvz.org/index.php?t=msg&goto=16406&

== Setting up a HN-based firewall ==

This setup emulates (to the containers anyway) an external hardware firewall. It protects the HN from any access and then defines what services and ports are allowed/banned for individual containers. This leaves the firewall controlled by the site administrator, not be individual containers and the hackers who've gotten into them. ;)

First off, let's disable Fedora's existing <code>iptables</code> service:
<pre>
service iptables stop
chkconfig iptables off
</pre>

Now create the new <code>firewall</code> service. This code should be <code>/etc/init.d/firewall</code> and then should be chmod'd 755.
<pre>
#!/bin/sh
# firewall Start iptables firewall
# chkconfig: 2345 08 92
# description: Starts, stops and saves iptables firewall
# This script sets up the firewall for the INPUT chain (which is for
# the HN itself) and then processes the config files under
# /etc/firewall.d to set up additional rules in the FORWARD chain
# to allow access to containers' services.

. /etc/init.d/functions

# the IP block allocated to this server
SEGMENT="192.168.0.0/24"
# the IP used by the hosting server itself
THISHOST="192.168.0.1"
# services that should be allowed to the HN;
# services for containers are configured in /etc/firewall.d/*
OKPORTS="53"
# hosts allowed full access through the firewall,
# to all containers and to this server
DMZS="12.34.56.78 90.123.45.67"

purge() {
echo -n "Firewall: Purging and allowing all traffic"
iptables -P OUTPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P INPUT ACCEPT
iptables -F
success ; echo
}

setup() {
echo -n "Firewall: Setting default policies to DROP"
iptables -P INPUT DROP
iptables -P FORWARD DROP
iptables -I INPUT -j ACCEPT -m state --state ESTABLISHED,RELATED
iptables -I FORWARD -j ACCEPT -m state --state ESTABLISHED,RELATED
iptables -I INPUT -j ACCEPT -i lo
iptables -I FORWARD -j ACCEPT --source $SEGMENT
success ; echo

echo "Firewall: Allowing access to HN"
for port in $OKPORTS ; do
echo -n " port $port"
iptables -I INPUT -j ACCEPT -s $SEGMENT -d $THISHOST --protocol tcp --destination-port $port
iptables -I INPUT -j ACCEPT -s $SEGMENT -d $THISHOST --protocol udp --destination-port $port
success ; echo
done
for ip in $DMZS ; do
echo -n " DMZ $ip"
iptables -I INPUT -i eth0 -j ACCEPT -s $ip
iptables -I FORWARD -i eth0 -j ACCEPT -s $ip
success ; echo
done

CTSETUPS=`echo /etc/firewall.d/*`
if [ "$CTSETUPS" != "/etc/firewall.d/*" ] ; then
echo "Firewall: Setting up container firewalls"
for i in $CTSETUPS ; do
. $i
echo -n " $CTNAME CT$CTID"
if [ -n "$BANNED" ]; then
for source in $BANNED ; do iptables -I FORWARD -j DROP --destination $CTIP --source $source ; done
fi
if [ -n "$OPENPORTS" ]; then
for port in $OPENPORTS ; do iptables -I FORWARD -j ACCEPT --protocol tcp --destination $CTIP --destination-port $port ; done
for port in $OPENPORTS ; do iptables -I FORWARD -j ACCEPT --protocol udp --destination $CTIP --destination-port $port ; done
fi
if [ -n "$DMZS" ]; then
for source in $DMZS ; do iptables -I FORWARD -j ACCEPT --protocol tcp --destination $CTIP --source $source ; done
for source in $DMZS ; do iptables -I FORWARD -j ACCEPT --protocol udp --destination $CTIP --source $source ; done
fi
[ $? -eq 0 ] && success || failure
echo
done
fi
}

case "$1" in
start)
echo "Starting firewall..."
purge
setup
;;
stop)
echo "Stopping firewall..."
purge
;;
restart)
$0 stop
$0 start
;;
status)
iptables -n -L
;;
*)
echo "Usage: $0 <start|stop|restart|status>"
;;
esac
</pre>

Note: This will only allow access to the HN from the hosts/networks defined in SEGMENT. If you'd like to open up the OKPORTS on the HN to everybody, you can remove the ''-s $SEGMENT'' parameters from the iptables commands under the "Firewall: Allowing access to HN" section. The modified lines would look like this:

<pre>
iptables -I INPUT -j ACCEPT -d $THISHOST --protocol tcp --destination-port $port
iptables -I INPUT -j ACCEPT -d $THISHOST --protocol udp --destination-port $port
</pre>

The above script can be called like this:
<pre>
service firewall start
service firewall stop
service firewall restart
service firewall status
</pre>

It will set up the firewall for the HN according to the parameters you specified for OKPORTS, DMZs, etc. and then it will call each file under /etc/firewall.d and process its configuration.

So create a file under /etc/firewall.d The exact filename isn't important, as long as it's meaningful to you, e.g. <code>ExampleCompany</code> or <code>ve12</code> and give it content like this:

<pre>
# This file is processed by /etc/init.d/firewall
CTID="1" # the container's ID#
CTNAME="Customer1" # A human-friendly label for the container
CTIP="192.168.1.34" # the IP address for this container
OPENPORTS="80 443" # ports that should be universally opened
# to the entire Internet
DMZS="1.2.3.0/24 5.6.7.8/32" # IPs and blocks that should have full access
# to the container's services
BANNED="" # IPs and blocks that should be entirely
# blocked from the container's services
</pre>

And there you go. Go ahead and start the firewall and check its status:
<pre>
service firewall restart
service firewall status
</pre>

As you can see, you can now add and edit the configurations for individual containers very easily. This method proves a lot easier to manage than Fedora's iptables-config mechamism!

To make the firewall service automatically start when the HN boots, run
<pre>
chkconfig --add firewall
</pre>

=== Debian Notes ===

The setup above works fine for Debian as well, however /etc/init.d/functions is missing. Here is a very simple version that you can use:

# /etc/init.d/functions

success() {
echo -n "...success"
}
failure() {
echo -n "...failure"
}

== Setting up a firewall that allows per-container configuration ==

This setup configures iptables on the HN to disallow access to all hosts, including the containers. However, it allows all traffic into the containers so they may define their own iptables rules and therefore manage their own firewall.

<pre>
iptables -P FORWARD ACCEPT
iptables -F FORWARD
</pre>

This will remove all rules for the FORWARD chain so all packets can pass back and forth between containers and the outside world.

If you want to use a firewall inside a container, please load these modules BEFORE starting the container:

<pre>
modprobe xt_tcpudp
modprobe ip_conntrack
</pre>

If you do not, you will get an error like this: "iptables: No chain/target/match by that name"

If you want to use stateful firewall rules (and you should!) you will also need to make sure that 'ipt_state' is in the 'IPTABLES' option in your vz.conf file:

<pre>
IPTABLES="ipt_REJECT ipt_tos ipt_limit ipt_multiport iptable_filter iptable_mangle ipt_TCPMSS ipt_tcpmss ipt_ttl ipt_length ipt_state"
</pre>

Also make sure the 'xt_state' module is loaded on the host:

<pre>
modprobe xt_state
</pre>

== See also ==

* [[Traffic accounting with iptables]]

[[Category: Networking]]

UBC primary parameters

2010-11-11T23:33:18Z

DmitryKoterov: /* vmguarpages */

{{UBC toc}}

The most important parameters determining the resources available to
container are explained below. The meaning of the parameters
is illustrated assuming that the container runs some network
server applications.

== numproc ==
Maximum number of processes and kernel-level threads allowed for this container.

Many server applications (like Apache Web server, FTP and mail servers)
spawn a process to handle each client, so the limit on number of processes
defines how many clients the application will be able to handle in parallel.
However, the number of processes doesn't limit how “heavy” the application is
and whether the server will be able to serve heavy requests from clients.

Configuring resource control system, it is important to estimate both
the maximum number of processes and the average number of processes
(referred to as <code>avnumproc</code> later). Other (dependent) resource
control parameters may depend both on the limit and the average number (see [[UBC consistency check]]).

The <code>barrier</code> of <code>numproc</code> doesn't provide additional control and should be set equal to the <code>limit</code>.

There is a restriction on the total number of processes in the system.
More than about 16000 processes start to cause poor responsiveness of
the system, worsening when the number grows. Total number of processes
exceeding 32000 is very likely to cause hang of the system.

Note that in practice the number of processes is usually less. Each
process consumes some memory, and the available memory and the
"low memory" (see [[UBC systemwide configuration#“Low memory”|“Low memory”]]) limit the number of processes to lower
values. With typical processes, it is normal to be able to run only up
to 8000 processes in a system.

== numtcpsock ==
Maximum number of TCP sockets.

This parameter limits the
number of TCP connections and, thus, the number of clients the server
application can handle in parallel.

The <code>barrier</code> of this parameter should be set equal to the <code>limit</code>.

If each container has it's own set of IP addresses (which is
the only way a OpenVZ system can be configured), there are no direct
limits on the total number of TCP sockets in the system. The number
of sockets needs to be controlled because each socket needs certain
amount of memory for receive and transmit buffers (see descriptions
of <code>[[tcpsndbuf]]</code> and <code>[[tcprcvbuf]]</code>), and
the memory is a limited resource.

== numothersock ==
Maximum number of non-TCP sockets (local sockets, UDP and other types of sockets).

Local (UNIX-domain) sockets are used for communications inside the
system. Multi-tier applications (for example, a Web application with a
database server as a back-end) may need one or multiple local sockets
to serve each client. Straightforward applications (for example, most
mail servers, with the exception of Postfix) do not use local sockets.

UDP sockets are used for Domain Name Service (DNS) queries, but
the number of such sockets opened simultaneously is low. UDP and
other sockets may also be used in some very special applications
(SNMP agents and others).

The <code>barrier</code> of this parameter should be set equal to the <code>limit</code>.
The number of local sockets in a system is not limited. The number of
UDP sockets in a system, similarly to TCP sockets, is not limited in
OpenVZ systems.

Similarly to <code>numtcpsock</code> parameter discussed above, the number of
non-TCP sockets needs to be controlled because each socket needs certain
amount of memory for its buffers, and the memory is a limited
resource.

== vmguarpages ==
Memory allocation guarantee.

This parameter controls how much memory is available to the Virtual
Environment (i.e. how much memory its applications can allocate by
<code>malloc(3)</code> or other standard Linux memory allocation mechanisms).
The more clients are served or the more “heavy” the application is, the
more memory it needs.

The amount of memory that container's applications are
guaranteed to be able to allocate is specified as the <code>barrier</code> of
<code>vmguarpages</code> parameter. The current amount of allocated memory space
is accounted into <code>privvmpages</code> parameter, and <code>vmguarpages</code>
parameter does not have its own accounting. The <code>barrier</code> and the <code>limit</code> of
<code>privvmpages</code> parameter impose an upper limit on the memory allocations
(see [[privvmpages]]). The meaning of the <code>limit</code> for the
<code>vmguarpages</code> parameter is unspecified in the current version and should be set
to the maximal allowed value ([[MAX_ULONG]]).

If the current amount of allocated memory space does not exceed the
guaranteed amount (the <code>barrier</code> of <code>vmguarpages</code>),
memory allocations of container's applications always succeed.
If the current amount of allocated memory space exceeds the guarantee but below
the <code>barrier</code> of <code>privvmpages</code>, allocations may or may not succeed,
depending on the total amount of available memory in the system.

Starting from the <code>barrier</code> of <code>privvmpages</code>,
normal priority allocations and, starting from the <code>limit</code>
of <code>privvmpages</code>, all memory allocations
made by the applications fail.
The memory allocation guarantee (<code>vmguarpages</code>) is a primary tool for
controlling the memory available to containers, because
it allows administrators to provide Service Level Agreements — agreements
guaranteeing certain quality of service, certain amount of resources
and general availability of the service. The unit of measurement
of vmguarpages values is memory pages (4KB on x86 and x86_64 processors).
The total memory allocation guarantees given to containers
are limited by the physical resources of the computer — the size of RAM
and the swap space — as discussed in [[UBC systemwide configuration]].

There is a ''pseudo-graphical'' tool - <code>[http://en.dklab.ru/lib/dklab_vzmem/ vzmem]</code> - which allows you to distribute physical memory among all VEs consistently. It shows all physical memory blocks graphically in <code>/etc/vz/conf/MEM-MAP</code> text file and lets you to move these blocks from one VE to another to redistribute the memory. Also you may specify "additional" memory personally for each VE: such memory will be obtained from system's free memory or swap (it is reflected as modifying of <code>privvmpages</code> parameter).

UBC primary parameters

2010-11-11T23:32:36Z

DmitryKoterov: /* vmguarpages */

{{UBC toc}}

The most important parameters determining the resources available to
container are explained below. The meaning of the parameters
is illustrated assuming that the container runs some network
server applications.

== numproc ==
Maximum number of processes and kernel-level threads allowed for this container.

Many server applications (like Apache Web server, FTP and mail servers)
spawn a process to handle each client, so the limit on number of processes
defines how many clients the application will be able to handle in parallel.
However, the number of processes doesn't limit how “heavy” the application is
and whether the server will be able to serve heavy requests from clients.

Configuring resource control system, it is important to estimate both
the maximum number of processes and the average number of processes
(referred to as <code>avnumproc</code> later). Other (dependent) resource
control parameters may depend both on the limit and the average number (see [[UBC consistency check]]).

The <code>barrier</code> of <code>numproc</code> doesn't provide additional control and should be set equal to the <code>limit</code>.

There is a restriction on the total number of processes in the system.
More than about 16000 processes start to cause poor responsiveness of
the system, worsening when the number grows. Total number of processes
exceeding 32000 is very likely to cause hang of the system.

Note that in practice the number of processes is usually less. Each
process consumes some memory, and the available memory and the
"low memory" (see [[UBC systemwide configuration#“Low memory”|“Low memory”]]) limit the number of processes to lower
values. With typical processes, it is normal to be able to run only up
to 8000 processes in a system.

== numtcpsock ==
Maximum number of TCP sockets.

This parameter limits the
number of TCP connections and, thus, the number of clients the server
application can handle in parallel.

The <code>barrier</code> of this parameter should be set equal to the <code>limit</code>.

If each container has it's own set of IP addresses (which is
the only way a OpenVZ system can be configured), there are no direct
limits on the total number of TCP sockets in the system. The number
of sockets needs to be controlled because each socket needs certain
amount of memory for receive and transmit buffers (see descriptions
of <code>[[tcpsndbuf]]</code> and <code>[[tcprcvbuf]]</code>), and
the memory is a limited resource.

== numothersock ==
Maximum number of non-TCP sockets (local sockets, UDP and other types of sockets).

Local (UNIX-domain) sockets are used for communications inside the
system. Multi-tier applications (for example, a Web application with a
database server as a back-end) may need one or multiple local sockets
to serve each client. Straightforward applications (for example, most
mail servers, with the exception of Postfix) do not use local sockets.

UDP sockets are used for Domain Name Service (DNS) queries, but
the number of such sockets opened simultaneously is low. UDP and
other sockets may also be used in some very special applications
(SNMP agents and others).

The <code>barrier</code> of this parameter should be set equal to the <code>limit</code>.
The number of local sockets in a system is not limited. The number of
UDP sockets in a system, similarly to TCP sockets, is not limited in
OpenVZ systems.

Similarly to <code>numtcpsock</code> parameter discussed above, the number of
non-TCP sockets needs to be controlled because each socket needs certain
amount of memory for its buffers, and the memory is a limited
resource.

== vmguarpages ==
Memory allocation guarantee.

This parameter controls how much memory is available to the Virtual
Environment (i.e. how much memory its applications can allocate by
<code>malloc(3)</code> or other standard Linux memory allocation mechanisms).
The more clients are served or the more “heavy” the application is, the
more memory it needs.

The amount of memory that container's applications are
guaranteed to be able to allocate is specified as the <code>barrier</code> of
<code>vmguarpages</code> parameter. The current amount of allocated memory space
is accounted into <code>privvmpages</code> parameter, and <code>vmguarpages</code>
parameter does not have its own accounting. The <code>barrier</code> and the <code>limit</code> of
<code>privvmpages</code> parameter impose an upper limit on the memory allocations
(see [[privvmpages]]). The meaning of the <code>limit</code> for the
<code>vmguarpages</code> parameter is unspecified in the current version and should be set
to the maximal allowed value ([[MAX_ULONG]]).

If the current amount of allocated memory space does not exceed the
guaranteed amount (the <code>barrier</code> of <code>vmguarpages</code>),
memory allocations of container's applications always succeed.
If the current amount of allocated memory space exceeds the guarantee but below
the <code>barrier</code> of <code>privvmpages</code>, allocations may or may not succeed,
depending on the total amount of available memory in the system.

Starting from the <code>barrier</code> of <code>privvmpages</code>,
normal priority allocations and, starting from the <code>limit</code>
of <code>privvmpages</code>, all memory allocations
made by the applications fail.
The memory allocation guarantee (<code>vmguarpages</code>) is a primary tool for
controlling the memory available to containers, because
it allows administrators to provide Service Level Agreements — agreements
guaranteeing certain quality of service, certain amount of resources
and general availability of the service. The unit of measurement
of vmguarpages values is memory pages (4KB on x86 and x86_64 processors).
The total memory allocation guarantees given to containers
are limited by the physical resources of the computer — the size of RAM
and the swap space — as discussed in [[UBC systemwide configuration]].

There is a tool - <code>[http://en.dklab.ru/lib/dklab_vzmem/ vzmem]</code> - which allows you to distribute physical memory among all VEs consistently. It shows all physical memory blocks graphically in <code>/etc/vz/conf/MEM-MAP</code> text file and lets you to move these blocks from one VE to another to redistribute the memory. Also you may specify "additional" memory personally for each VE: such memory will be obtained from system's free memory or swap (it is reflected as modifying of <code>privvmpages</code> parameter).

UBC primary parameters

2010-11-11T23:31:38Z

DmitryKoterov: /* vmguarpages */

{{UBC toc}}

The most important parameters determining the resources available to
container are explained below. The meaning of the parameters
is illustrated assuming that the container runs some network
server applications.

== numproc ==
Maximum number of processes and kernel-level threads allowed for this container.

Many server applications (like Apache Web server, FTP and mail servers)
spawn a process to handle each client, so the limit on number of processes
defines how many clients the application will be able to handle in parallel.
However, the number of processes doesn't limit how “heavy” the application is
and whether the server will be able to serve heavy requests from clients.

Configuring resource control system, it is important to estimate both
the maximum number of processes and the average number of processes
(referred to as <code>avnumproc</code> later). Other (dependent) resource
control parameters may depend both on the limit and the average number (see [[UBC consistency check]]).

The <code>barrier</code> of <code>numproc</code> doesn't provide additional control and should be set equal to the <code>limit</code>.

There is a restriction on the total number of processes in the system.
More than about 16000 processes start to cause poor responsiveness of
the system, worsening when the number grows. Total number of processes
exceeding 32000 is very likely to cause hang of the system.

Note that in practice the number of processes is usually less. Each
process consumes some memory, and the available memory and the
"low memory" (see [[UBC systemwide configuration#“Low memory”|“Low memory”]]) limit the number of processes to lower
values. With typical processes, it is normal to be able to run only up
to 8000 processes in a system.

== numtcpsock ==
Maximum number of TCP sockets.

This parameter limits the
number of TCP connections and, thus, the number of clients the server
application can handle in parallel.

The <code>barrier</code> of this parameter should be set equal to the <code>limit</code>.

If each container has it's own set of IP addresses (which is
the only way a OpenVZ system can be configured), there are no direct
limits on the total number of TCP sockets in the system. The number
of sockets needs to be controlled because each socket needs certain
amount of memory for receive and transmit buffers (see descriptions
of <code>[[tcpsndbuf]]</code> and <code>[[tcprcvbuf]]</code>), and
the memory is a limited resource.

== numothersock ==
Maximum number of non-TCP sockets (local sockets, UDP and other types of sockets).

Local (UNIX-domain) sockets are used for communications inside the
system. Multi-tier applications (for example, a Web application with a
database server as a back-end) may need one or multiple local sockets
to serve each client. Straightforward applications (for example, most
mail servers, with the exception of Postfix) do not use local sockets.

UDP sockets are used for Domain Name Service (DNS) queries, but
the number of such sockets opened simultaneously is low. UDP and
other sockets may also be used in some very special applications
(SNMP agents and others).

The <code>barrier</code> of this parameter should be set equal to the <code>limit</code>.
The number of local sockets in a system is not limited. The number of
UDP sockets in a system, similarly to TCP sockets, is not limited in
OpenVZ systems.

Similarly to <code>numtcpsock</code> parameter discussed above, the number of
non-TCP sockets needs to be controlled because each socket needs certain
amount of memory for its buffers, and the memory is a limited
resource.

== vmguarpages ==
Memory allocation guarantee.

This parameter controls how much memory is available to the Virtual
Environment (i.e. how much memory its applications can allocate by
<code>malloc(3)</code> or other standard Linux memory allocation mechanisms).
The more clients are served or the more “heavy” the application is, the
more memory it needs.

The amount of memory that container's applications are
guaranteed to be able to allocate is specified as the <code>barrier</code> of
<code>vmguarpages</code> parameter. The current amount of allocated memory space
is accounted into <code>privvmpages</code> parameter, and <code>vmguarpages</code>
parameter does not have its own accounting. The <code>barrier</code> and the <code>limit</code> of
<code>privvmpages</code> parameter impose an upper limit on the memory allocations
(see [[privvmpages]]). The meaning of the <code>limit</code> for the
<code>vmguarpages</code> parameter is unspecified in the current version and should be set
to the maximal allowed value ([[MAX_ULONG]]).

If the current amount of allocated memory space does not exceed the
guaranteed amount (the <code>barrier</code> of <code>vmguarpages</code>),
memory allocations of container's applications always succeed.
If the current amount of allocated memory space exceeds the guarantee but below
the <code>barrier</code> of <code>privvmpages</code>, allocations may or may not succeed,
depending on the total amount of available memory in the system.

Starting from the <code>barrier</code> of <code>privvmpages</code>,
normal priority allocations and, starting from the <code>limit</code>
of <code>privvmpages</code>, all memory allocations
made by the applications fail.
The memory allocation guarantee (<code>vmguarpages</code>) is a primary tool for
controlling the memory available to containers, because
it allows administrators to provide Service Level Agreements — agreements
guaranteeing certain quality of service, certain amount of resources
and general availability of the service. The unit of measurement
of vmguarpages values is memory pages (4KB on x86 and x86_64 processors).
The total memory allocation guarantees given to containers
are limited by the physical resources of the computer — the size of RAM
and the swap space — as discussed in [[UBC systemwide configuration]].

There is a tool - <code>[http://en.dklab.ru/lib/dklab_vzmem/ vzmem]</code> - which allows you to distribute physical memory among all VEs consistently. It shows all the physical memory blocks graphically in /etc/vz/conf/MEM-MAP text file and lets you to move these blocks from one VE to another to redistribute the memory. Also you may specify "additional" memory personally for each VE: such memory will be obtained from system's free memory or swap (it is reflected as modifying of <code>privvmpages</code> parameter).

Setting up an iptables firewall

2010-11-11T23:08:21Z

DmitryKoterov: /* Simple firewall configuration independent to IP addresses: vzfirewall */

This document consists of two parts. The first is setting up a firewall (using iptables) on the HN, which will restrict traffic to the containers. The effect would emulate, as far as the containers and their customers are concerned, an external hardware firewall controlled by the sysadmin. The second is setting up a firewall that protects the [[HN]] itself but still allows traffic to the containers, thus allowing individual containers to define their own iptables.

While the firewalls shown here can be accomplished using iptables manually (or using Fedora core's iptables service), the methods presented here are especially modular and easy to modify. This is important when you have 20+ containers and a lot of other things to be doing...

The scripts and pathnames given here are for Fedora Core 6, though they can probably be applied to most similar SysV-like systems with little modification.

== A little background ==

On our systems, we use the HN to provide privileged services which are not appropriate for access by the containers. For example, the HN acts as a backup server, runs Nagios for health monitoring, has a webserver for managing the 3ware RAID controller, etc. The containers are leased to customers, who can't entirely be trusted, especially if they get hacked. As such, our scenario is one in which the HN must be protected from all access (even from the containers) except for a few trusted hosts (e.g. my home-office).

The exception to this is the nameserver, which we want open to the world. We use it as a caching nameserver for our containers and also to host DNS for a few customer domain.

== Simple firewall configuration independent to IP addresses: vzfirewall ==

''Vzfirewall'' tool allows you to open/close ports for incoming connections with no dependencies to foreign IP addresses. E.g. you may allow a hostname ''release.prod.example.com'' to connect to port 5432 of VE 1234 and leave all other ports closed by modifying 1234.conf file adding multiline ''FIREWALL'' directive into it:

<pre>
...
PRIVVMPAGES="300000:300000"
HOSTNAME="example.com"
...
FIREWALL="
...
# Allow access to PostgreSQL port only from release.prod
# and release.test machines. You may use domain names here.
[5432]
release.prod.example.com
release.test.example.com
...
"
</pre>

You must then run ''vzfirewall -a'' on your hardware node to apply changes made in *.conf.

Note that it is recommended to use hostnames instead of IP addresses here, so the configuration is persistent for VE movements to different IP-address: you just need to run ''vzfirewall -a'' again after movement. It is also reboot-safe, because applied to /etc/sysconfig/iptables (at RHEL systems).

Vzfirewall and its documentation are available at [http://en.dklab.ru/lib/dklab_vzfirewall/ http://en.dklab.ru/lib/dklab_vzfirewall/].

== An alternative from the author of Shorewall ==

For those who might find the solution provided in this wiki article unsatisfactory (for whatever reason), the creator of Shorewall (Tom Eastep) has written a nice article explaining how to use Shorewall on an OpenVZ host node to manage the host node, containers, and more... with quite a complex setup as an example. The article IS NOT an introduction to Shorewall for beginners, so some pre-existing knowledge and understanding of Shorewall may be required.

Shorewall and OpenVZ by Tom Eastep - http://www.shorewall.net/OpenVZ.html

See also this OpenVZ Forum posting - http://forum.openvz.org/index.php?t=msg&goto=16406&

== Setting up a HN-based firewall ==

This setup emulates (to the containers anyway) an external hardware firewall. It protects the HN from any access and then defines what services and ports are allowed/banned for individual containers. This leaves the firewall controlled by the site administrator, not be individual containers and the hackers who've gotten into them. ;)

First off, let's disable Fedora's existing <code>iptables</code> service:
<pre>
service iptables stop
chkconfig iptables off
</pre>

Now create the new <code>firewall</code> service. This code should be <code>/etc/init.d/firewall</code> and then should be chmod'd 755.
<pre>
#!/bin/sh
# firewall Start iptables firewall
# chkconfig: 2345 08 92
# description: Starts, stops and saves iptables firewall
# This script sets up the firewall for the INPUT chain (which is for
# the HN itself) and then processes the config files under
# /etc/firewall.d to set up additional rules in the FORWARD chain
# to allow access to containers' services.

. /etc/init.d/functions

# the IP block allocated to this server
SEGMENT="192.168.0.0/24"
# the IP used by the hosting server itself
THISHOST="192.168.0.1"
# services that should be allowed to the HN;
# services for containers are configured in /etc/firewall.d/*
OKPORTS="53"
# hosts allowed full access through the firewall,
# to all containers and to this server
DMZS="12.34.56.78 90.123.45.67"

purge() {
echo -n "Firewall: Purging and allowing all traffic"
iptables -P OUTPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P INPUT ACCEPT
iptables -F
success ; echo
}

setup() {
echo -n "Firewall: Setting default policies to DROP"
iptables -P INPUT DROP
iptables -P FORWARD DROP
iptables -I INPUT -j ACCEPT -m state --state ESTABLISHED,RELATED
iptables -I FORWARD -j ACCEPT -m state --state ESTABLISHED,RELATED
iptables -I INPUT -j ACCEPT -i lo
iptables -I FORWARD -j ACCEPT --source $SEGMENT
success ; echo

echo "Firewall: Allowing access to HN"
for port in $OKPORTS ; do
echo -n " port $port"
iptables -I INPUT -j ACCEPT -s $SEGMENT -d $THISHOST --protocol tcp --destination-port $port
iptables -I INPUT -j ACCEPT -s $SEGMENT -d $THISHOST --protocol udp --destination-port $port
success ; echo
done
for ip in $DMZS ; do
echo -n " DMZ $ip"
iptables -I INPUT -i eth0 -j ACCEPT -s $ip
iptables -I FORWARD -i eth0 -j ACCEPT -s $ip
success ; echo
done

CTSETUPS=`echo /etc/firewall.d/*`
if [ "$CTSETUPS" != "/etc/firewall.d/*" ] ; then
echo "Firewall: Setting up container firewalls"
for i in $CTSETUPS ; do
. $i
echo -n " $CTNAME CT$CTID"
if [ -n "$BANNED" ]; then
for source in $BANNED ; do iptables -I FORWARD -j DROP --destination $CTIP --source $source ; done
fi
if [ -n "$OPENPORTS" ]; then
for port in $OPENPORTS ; do iptables -I FORWARD -j ACCEPT --protocol tcp --destination $CTIP --destination-port $port ; done
for port in $OPENPORTS ; do iptables -I FORWARD -j ACCEPT --protocol udp --destination $CTIP --destination-port $port ; done
fi
if [ -n "$DMZS" ]; then
for source in $DMZS ; do iptables -I FORWARD -j ACCEPT --protocol tcp --destination $CTIP --source $source ; done
for source in $DMZS ; do iptables -I FORWARD -j ACCEPT --protocol udp --destination $CTIP --source $source ; done
fi
[ $? -eq 0 ] && success || failure
echo
done
fi
}

case "$1" in
start)
echo "Starting firewall..."
purge
setup
;;
stop)
echo "Stopping firewall..."
purge
;;
restart)
$0 stop
$0 start
;;
status)
iptables -n -L
;;
*)
echo "Usage: $0 <start|stop|restart|status>"
;;
esac
</pre>

Note: This will only allow access to the HN from the hosts/networks defined in SEGMENT. If you'd like to open up the OKPORTS on the HN to everybody, you can remove the ''-s $SEGMENT'' parameters from the iptables commands under the "Firewall: Allowing access to HN" section. The modified lines would look like this:

<pre>
iptables -I INPUT -j ACCEPT -d $THISHOST --protocol tcp --destination-port $port
iptables -I INPUT -j ACCEPT -d $THISHOST --protocol udp --destination-port $port
</pre>

The above script can be called like this:
<pre>
service firewall start
service firewall stop
service firewall restart
service firewall status
</pre>

It will set up the firewall for the HN according to the parameters you specified for OKPORTS, DMZs, etc. and then it will call each file under /etc/firewall.d and process its configuration.

So create a file under /etc/firewall.d The exact filename isn't important, as long as it's meaningful to you, e.g. <code>ExampleCompany</code> or <code>ve12</code> and give it content like this:

<pre>
# This file is processed by /etc/init.d/firewall
CTID="1" # the container's ID#
CTNAME="Customer1" # A human-friendly label for the container
CTIP="192.168.1.34" # the IP address for this container
OPENPORTS="80 443" # ports that should be universally opened
# to the entire Internet
DMZS="1.2.3.0/24 5.6.7.8/32" # IPs and blocks that should have full access
# to the container's services
BANNED="" # IPs and blocks that should be entirely
# blocked from the container's services
</pre>

And there you go. Go ahead and start the firewall and check its status:
<pre>
service firewall restart
service firewall status
</pre>

As you can see, you can now add and edit the configurations for individual containers very easily. This method proves a lot easier to manage than Fedora's iptables-config mechamism!

To make the firewall service automatically start when the HN boots, run
<pre>
chkconfig --add firewall
</pre>

=== Debian Notes ===

The setup above works fine for Debian as well, however /etc/init.d/functions is missing. Here is a very simple version that you can use:

# /etc/init.d/functions

success() {
echo -n "...success"
}
failure() {
echo -n "...failure"
}

== Setting up a firewall that allows per-container configuration ==

This setup configures iptables on the HN to disallow access to all hosts, including the containers. However, it allows all traffic into the containers so they may define their own iptables rules and therefore manage their own firewall.

<pre>
iptables -P FORWARD ACCEPT
iptables -F FORWARD
</pre>

This will remove all rules for the FORWARD chain so all packets can pass back and forth between containers and the outside world.

If you want to use a firewall inside a container, please load these modules BEFORE starting the container:

<pre>
modprobe xt_tcpudp
modprobe ip_conntrack
</pre>

If you do not, you will get an error like this: "iptables: No chain/target/match by that name"

If you want to use stateful firewall rules (and you should!) you will also need to make sure that 'ipt_state' is in the 'IPTABLES' option in your vz.conf file:

<pre>
IPTABLES="ipt_REJECT ipt_tos ipt_limit ipt_multiport iptable_filter iptable_mangle ipt_TCPMSS ipt_tcpmss ipt_ttl ipt_length ipt_state"
</pre>

Also make sure the 'xt_state' module is loaded on the host:

<pre>
modprobe xt_state
</pre>

== See also ==

* [[Traffic accounting with iptables]]

[[Category: Networking]]

Setting up an iptables firewall

2010-11-11T23:06:56Z

DmitryKoterov: /* Simple firewall configuration independent to IP addresses: vzfirewall */

This document consists of two parts. The first is setting up a firewall (using iptables) on the HN, which will restrict traffic to the containers. The effect would emulate, as far as the containers and their customers are concerned, an external hardware firewall controlled by the sysadmin. The second is setting up a firewall that protects the [[HN]] itself but still allows traffic to the containers, thus allowing individual containers to define their own iptables.

While the firewalls shown here can be accomplished using iptables manually (or using Fedora core's iptables service), the methods presented here are especially modular and easy to modify. This is important when you have 20+ containers and a lot of other things to be doing...

The scripts and pathnames given here are for Fedora Core 6, though they can probably be applied to most similar SysV-like systems with little modification.

== A little background ==

On our systems, we use the HN to provide privileged services which are not appropriate for access by the containers. For example, the HN acts as a backup server, runs Nagios for health monitoring, has a webserver for managing the 3ware RAID controller, etc. The containers are leased to customers, who can't entirely be trusted, especially if they get hacked. As such, our scenario is one in which the HN must be protected from all access (even from the containers) except for a few trusted hosts (e.g. my home-office).

The exception to this is the nameserver, which we want open to the world. We use it as a caching nameserver for our containers and also to host DNS for a few customer domain.

== Simple firewall configuration independent to IP addresses: vzfirewall ==

''Vzfirewall'' tool allows you to open/close ports for incoming connections with no dependencies to foreign IP addresses. E.g. you may allow a hostname ''release.prod.example.com'' to connect to port 5432 of VE 1234 and leave all other ports closed by modifying 1234.conf file adding multiline ''FIREWALL'' directive into it:

<pre>
...
PRIVVMPAGES="300000:300000"
HOSTNAME="example.com"
...
FIREWALL="
...
# Allow access to PostgreSQL port only from release.prod
# and release.test machines. You may use domain names here.
[5432]
release.prod.example.com
release.test.example.com
...
"
</pre>

You must then run ''vzfirewall -a'' on your hardware node to apply changes made in *.conf.

Note that it is recommended to use hostnames instead of IP addresses here, so the configuration is persistent fore VE movements to different IP-address.

Vzfirewall and its documentation are available at [http://en.dklab.ru/lib/dklab_vzfirewall/ http://en.dklab.ru/lib/dklab_vzfirewall/].

== An alternative from the author of Shorewall ==

For those who might find the solution provided in this wiki article unsatisfactory (for whatever reason), the creator of Shorewall (Tom Eastep) has written a nice article explaining how to use Shorewall on an OpenVZ host node to manage the host node, containers, and more... with quite a complex setup as an example. The article IS NOT an introduction to Shorewall for beginners, so some pre-existing knowledge and understanding of Shorewall may be required.

Shorewall and OpenVZ by Tom Eastep - http://www.shorewall.net/OpenVZ.html

See also this OpenVZ Forum posting - http://forum.openvz.org/index.php?t=msg&goto=16406&

== Setting up a HN-based firewall ==

This setup emulates (to the containers anyway) an external hardware firewall. It protects the HN from any access and then defines what services and ports are allowed/banned for individual containers. This leaves the firewall controlled by the site administrator, not be individual containers and the hackers who've gotten into them. ;)

First off, let's disable Fedora's existing <code>iptables</code> service:
<pre>
service iptables stop
chkconfig iptables off
</pre>

Now create the new <code>firewall</code> service. This code should be <code>/etc/init.d/firewall</code> and then should be chmod'd 755.
<pre>
#!/bin/sh
# firewall Start iptables firewall
# chkconfig: 2345 08 92
# description: Starts, stops and saves iptables firewall
# This script sets up the firewall for the INPUT chain (which is for
# the HN itself) and then processes the config files under
# /etc/firewall.d to set up additional rules in the FORWARD chain
# to allow access to containers' services.

. /etc/init.d/functions

# the IP block allocated to this server
SEGMENT="192.168.0.0/24"
# the IP used by the hosting server itself
THISHOST="192.168.0.1"
# services that should be allowed to the HN;
# services for containers are configured in /etc/firewall.d/*
OKPORTS="53"
# hosts allowed full access through the firewall,
# to all containers and to this server
DMZS="12.34.56.78 90.123.45.67"

purge() {
echo -n "Firewall: Purging and allowing all traffic"
iptables -P OUTPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P INPUT ACCEPT
iptables -F
success ; echo
}

setup() {
echo -n "Firewall: Setting default policies to DROP"
iptables -P INPUT DROP
iptables -P FORWARD DROP
iptables -I INPUT -j ACCEPT -m state --state ESTABLISHED,RELATED
iptables -I FORWARD -j ACCEPT -m state --state ESTABLISHED,RELATED
iptables -I INPUT -j ACCEPT -i lo
iptables -I FORWARD -j ACCEPT --source $SEGMENT
success ; echo

echo "Firewall: Allowing access to HN"
for port in $OKPORTS ; do
echo -n " port $port"
iptables -I INPUT -j ACCEPT -s $SEGMENT -d $THISHOST --protocol tcp --destination-port $port
iptables -I INPUT -j ACCEPT -s $SEGMENT -d $THISHOST --protocol udp --destination-port $port
success ; echo
done
for ip in $DMZS ; do
echo -n " DMZ $ip"
iptables -I INPUT -i eth0 -j ACCEPT -s $ip
iptables -I FORWARD -i eth0 -j ACCEPT -s $ip
success ; echo
done

CTSETUPS=`echo /etc/firewall.d/*`
if [ "$CTSETUPS" != "/etc/firewall.d/*" ] ; then
echo "Firewall: Setting up container firewalls"
for i in $CTSETUPS ; do
. $i
echo -n " $CTNAME CT$CTID"
if [ -n "$BANNED" ]; then
for source in $BANNED ; do iptables -I FORWARD -j DROP --destination $CTIP --source $source ; done
fi
if [ -n "$OPENPORTS" ]; then
for port in $OPENPORTS ; do iptables -I FORWARD -j ACCEPT --protocol tcp --destination $CTIP --destination-port $port ; done
for port in $OPENPORTS ; do iptables -I FORWARD -j ACCEPT --protocol udp --destination $CTIP --destination-port $port ; done
fi
if [ -n "$DMZS" ]; then
for source in $DMZS ; do iptables -I FORWARD -j ACCEPT --protocol tcp --destination $CTIP --source $source ; done
for source in $DMZS ; do iptables -I FORWARD -j ACCEPT --protocol udp --destination $CTIP --source $source ; done
fi
[ $? -eq 0 ] && success || failure
echo
done
fi
}

case "$1" in
start)
echo "Starting firewall..."
purge
setup
;;
stop)
echo "Stopping firewall..."
purge
;;
restart)
$0 stop
$0 start
;;
status)
iptables -n -L
;;
*)
echo "Usage: $0 <start|stop|restart|status>"
;;
esac
</pre>

Note: This will only allow access to the HN from the hosts/networks defined in SEGMENT. If you'd like to open up the OKPORTS on the HN to everybody, you can remove the ''-s $SEGMENT'' parameters from the iptables commands under the "Firewall: Allowing access to HN" section. The modified lines would look like this:

<pre>
iptables -I INPUT -j ACCEPT -d $THISHOST --protocol tcp --destination-port $port
iptables -I INPUT -j ACCEPT -d $THISHOST --protocol udp --destination-port $port
</pre>

The above script can be called like this:
<pre>
service firewall start
service firewall stop
service firewall restart
service firewall status
</pre>

It will set up the firewall for the HN according to the parameters you specified for OKPORTS, DMZs, etc. and then it will call each file under /etc/firewall.d and process its configuration.

So create a file under /etc/firewall.d The exact filename isn't important, as long as it's meaningful to you, e.g. <code>ExampleCompany</code> or <code>ve12</code> and give it content like this:

<pre>
# This file is processed by /etc/init.d/firewall
CTID="1" # the container's ID#
CTNAME="Customer1" # A human-friendly label for the container
CTIP="192.168.1.34" # the IP address for this container
OPENPORTS="80 443" # ports that should be universally opened
# to the entire Internet
DMZS="1.2.3.0/24 5.6.7.8/32" # IPs and blocks that should have full access
# to the container's services
BANNED="" # IPs and blocks that should be entirely
# blocked from the container's services
</pre>

And there you go. Go ahead and start the firewall and check its status:
<pre>
service firewall restart
service firewall status
</pre>

As you can see, you can now add and edit the configurations for individual containers very easily. This method proves a lot easier to manage than Fedora's iptables-config mechamism!

To make the firewall service automatically start when the HN boots, run
<pre>
chkconfig --add firewall
</pre>

=== Debian Notes ===

The setup above works fine for Debian as well, however /etc/init.d/functions is missing. Here is a very simple version that you can use:

# /etc/init.d/functions

success() {
echo -n "...success"
}
failure() {
echo -n "...failure"
}

== Setting up a firewall that allows per-container configuration ==

This setup configures iptables on the HN to disallow access to all hosts, including the containers. However, it allows all traffic into the containers so they may define their own iptables rules and therefore manage their own firewall.

<pre>
iptables -P FORWARD ACCEPT
iptables -F FORWARD
</pre>

This will remove all rules for the FORWARD chain so all packets can pass back and forth between containers and the outside world.

If you want to use a firewall inside a container, please load these modules BEFORE starting the container:

<pre>
modprobe xt_tcpudp
modprobe ip_conntrack
</pre>

If you do not, you will get an error like this: "iptables: No chain/target/match by that name"

If you want to use stateful firewall rules (and you should!) you will also need to make sure that 'ipt_state' is in the 'IPTABLES' option in your vz.conf file:

<pre>
IPTABLES="ipt_REJECT ipt_tos ipt_limit ipt_multiport iptable_filter iptable_mangle ipt_TCPMSS ipt_tcpmss ipt_ttl ipt_length ipt_state"
</pre>

Also make sure the 'xt_state' module is loaded on the host:

<pre>
modprobe xt_state
</pre>

== See also ==

* [[Traffic accounting with iptables]]

[[Category: Networking]]

Setting up an iptables firewall

2010-11-11T23:05:38Z

DmitryKoterov: /* Simple firewall configuration independent to IP addresses: vzfirewall */

This document consists of two parts. The first is setting up a firewall (using iptables) on the HN, which will restrict traffic to the containers. The effect would emulate, as far as the containers and their customers are concerned, an external hardware firewall controlled by the sysadmin. The second is setting up a firewall that protects the [[HN]] itself but still allows traffic to the containers, thus allowing individual containers to define their own iptables.

While the firewalls shown here can be accomplished using iptables manually (or using Fedora core's iptables service), the methods presented here are especially modular and easy to modify. This is important when you have 20+ containers and a lot of other things to be doing...

The scripts and pathnames given here are for Fedora Core 6, though they can probably be applied to most similar SysV-like systems with little modification.

== A little background ==

On our systems, we use the HN to provide privileged services which are not appropriate for access by the containers. For example, the HN acts as a backup server, runs Nagios for health monitoring, has a webserver for managing the 3ware RAID controller, etc. The containers are leased to customers, who can't entirely be trusted, especially if they get hacked. As such, our scenario is one in which the HN must be protected from all access (even from the containers) except for a few trusted hosts (e.g. my home-office).

The exception to this is the nameserver, which we want open to the world. We use it as a caching nameserver for our containers and also to host DNS for a few customer domain.

== Simple firewall configuration independent to IP addresses: vzfirewall ==

''Vzfirewall'' tool allows you to open/close ports for incoming connections with no dependencies to foreign IP addresses. E.g. you may allow a hostname ''release.prod.example.com'' to connect to port 5432 of VE 1234 and leave all other ports closed by modifying 1234.conf file adding multiline ''FIREWALL'' directive into it:

<pre>
...
PRIVVMPAGES="300000:300000"
HOSTNAME="example.com"
...
FIREWALL="
...
# Allow access to PostgreSQL port only from release.prod machine.
# Note that you may use domain names here.
[5432]
release.prod.example.com
release.test.example.com
...
"
</pre>

You must then run ''vzfirewall -a'' on your hardware node to apply changes made in *.conf.

Note that it is recommended to use hostnames instead of IP addresses here, so the configuration is persistent fore VE movements to different IP-address.

Vzfirewall and its documentation are available at [http://en.dklab.ru/lib/dklab_vzfirewall/ http://en.dklab.ru/lib/dklab_vzfirewall/].

== An alternative from the author of Shorewall ==

For those who might find the solution provided in this wiki article unsatisfactory (for whatever reason), the creator of Shorewall (Tom Eastep) has written a nice article explaining how to use Shorewall on an OpenVZ host node to manage the host node, containers, and more... with quite a complex setup as an example. The article IS NOT an introduction to Shorewall for beginners, so some pre-existing knowledge and understanding of Shorewall may be required.

Shorewall and OpenVZ by Tom Eastep - http://www.shorewall.net/OpenVZ.html

See also this OpenVZ Forum posting - http://forum.openvz.org/index.php?t=msg&goto=16406&

== Setting up a HN-based firewall ==

This setup emulates (to the containers anyway) an external hardware firewall. It protects the HN from any access and then defines what services and ports are allowed/banned for individual containers. This leaves the firewall controlled by the site administrator, not be individual containers and the hackers who've gotten into them. ;)

First off, let's disable Fedora's existing <code>iptables</code> service:
<pre>
service iptables stop
chkconfig iptables off
</pre>

Now create the new <code>firewall</code> service. This code should be <code>/etc/init.d/firewall</code> and then should be chmod'd 755.
<pre>
#!/bin/sh
# firewall Start iptables firewall
# chkconfig: 2345 08 92
# description: Starts, stops and saves iptables firewall
# This script sets up the firewall for the INPUT chain (which is for
# the HN itself) and then processes the config files under
# /etc/firewall.d to set up additional rules in the FORWARD chain
# to allow access to containers' services.

. /etc/init.d/functions

# the IP block allocated to this server
SEGMENT="192.168.0.0/24"
# the IP used by the hosting server itself
THISHOST="192.168.0.1"
# services that should be allowed to the HN;
# services for containers are configured in /etc/firewall.d/*
OKPORTS="53"
# hosts allowed full access through the firewall,
# to all containers and to this server
DMZS="12.34.56.78 90.123.45.67"

purge() {
echo -n "Firewall: Purging and allowing all traffic"
iptables -P OUTPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P INPUT ACCEPT
iptables -F
success ; echo
}

setup() {
echo -n "Firewall: Setting default policies to DROP"
iptables -P INPUT DROP
iptables -P FORWARD DROP
iptables -I INPUT -j ACCEPT -m state --state ESTABLISHED,RELATED
iptables -I FORWARD -j ACCEPT -m state --state ESTABLISHED,RELATED
iptables -I INPUT -j ACCEPT -i lo
iptables -I FORWARD -j ACCEPT --source $SEGMENT
success ; echo

echo "Firewall: Allowing access to HN"
for port in $OKPORTS ; do
echo -n " port $port"
iptables -I INPUT -j ACCEPT -s $SEGMENT -d $THISHOST --protocol tcp --destination-port $port
iptables -I INPUT -j ACCEPT -s $SEGMENT -d $THISHOST --protocol udp --destination-port $port
success ; echo
done
for ip in $DMZS ; do
echo -n " DMZ $ip"
iptables -I INPUT -i eth0 -j ACCEPT -s $ip
iptables -I FORWARD -i eth0 -j ACCEPT -s $ip
success ; echo
done

CTSETUPS=`echo /etc/firewall.d/*`
if [ "$CTSETUPS" != "/etc/firewall.d/*" ] ; then
echo "Firewall: Setting up container firewalls"
for i in $CTSETUPS ; do
. $i
echo -n " $CTNAME CT$CTID"
if [ -n "$BANNED" ]; then
for source in $BANNED ; do iptables -I FORWARD -j DROP --destination $CTIP --source $source ; done
fi
if [ -n "$OPENPORTS" ]; then
for port in $OPENPORTS ; do iptables -I FORWARD -j ACCEPT --protocol tcp --destination $CTIP --destination-port $port ; done
for port in $OPENPORTS ; do iptables -I FORWARD -j ACCEPT --protocol udp --destination $CTIP --destination-port $port ; done
fi
if [ -n "$DMZS" ]; then
for source in $DMZS ; do iptables -I FORWARD -j ACCEPT --protocol tcp --destination $CTIP --source $source ; done
for source in $DMZS ; do iptables -I FORWARD -j ACCEPT --protocol udp --destination $CTIP --source $source ; done
fi
[ $? -eq 0 ] && success || failure
echo
done
fi
}

case "$1" in
start)
echo "Starting firewall..."
purge
setup
;;
stop)
echo "Stopping firewall..."
purge
;;
restart)
$0 stop
$0 start
;;
status)
iptables -n -L
;;
*)
echo "Usage: $0 <start|stop|restart|status>"
;;
esac
</pre>

Note: This will only allow access to the HN from the hosts/networks defined in SEGMENT. If you'd like to open up the OKPORTS on the HN to everybody, you can remove the ''-s $SEGMENT'' parameters from the iptables commands under the "Firewall: Allowing access to HN" section. The modified lines would look like this:

<pre>
iptables -I INPUT -j ACCEPT -d $THISHOST --protocol tcp --destination-port $port
iptables -I INPUT -j ACCEPT -d $THISHOST --protocol udp --destination-port $port
</pre>

The above script can be called like this:
<pre>
service firewall start
service firewall stop
service firewall restart
service firewall status
</pre>

It will set up the firewall for the HN according to the parameters you specified for OKPORTS, DMZs, etc. and then it will call each file under /etc/firewall.d and process its configuration.

So create a file under /etc/firewall.d The exact filename isn't important, as long as it's meaningful to you, e.g. <code>ExampleCompany</code> or <code>ve12</code> and give it content like this:

<pre>
# This file is processed by /etc/init.d/firewall
CTID="1" # the container's ID#
CTNAME="Customer1" # A human-friendly label for the container
CTIP="192.168.1.34" # the IP address for this container
OPENPORTS="80 443" # ports that should be universally opened
# to the entire Internet
DMZS="1.2.3.0/24 5.6.7.8/32" # IPs and blocks that should have full access
# to the container's services
BANNED="" # IPs and blocks that should be entirely
# blocked from the container's services
</pre>

And there you go. Go ahead and start the firewall and check its status:
<pre>
service firewall restart
service firewall status
</pre>

As you can see, you can now add and edit the configurations for individual containers very easily. This method proves a lot easier to manage than Fedora's iptables-config mechamism!

To make the firewall service automatically start when the HN boots, run
<pre>
chkconfig --add firewall
</pre>

=== Debian Notes ===

The setup above works fine for Debian as well, however /etc/init.d/functions is missing. Here is a very simple version that you can use:

# /etc/init.d/functions

success() {
echo -n "...success"
}
failure() {
echo -n "...failure"
}

== Setting up a firewall that allows per-container configuration ==

This setup configures iptables on the HN to disallow access to all hosts, including the containers. However, it allows all traffic into the containers so they may define their own iptables rules and therefore manage their own firewall.

<pre>
iptables -P FORWARD ACCEPT
iptables -F FORWARD
</pre>

This will remove all rules for the FORWARD chain so all packets can pass back and forth between containers and the outside world.

If you want to use a firewall inside a container, please load these modules BEFORE starting the container:

<pre>
modprobe xt_tcpudp
modprobe ip_conntrack
</pre>

If you do not, you will get an error like this: "iptables: No chain/target/match by that name"

If you want to use stateful firewall rules (and you should!) you will also need to make sure that 'ipt_state' is in the 'IPTABLES' option in your vz.conf file:

<pre>
IPTABLES="ipt_REJECT ipt_tos ipt_limit ipt_multiport iptable_filter iptable_mangle ipt_TCPMSS ipt_tcpmss ipt_ttl ipt_length ipt_state"
</pre>

Also make sure the 'xt_state' module is loaded on the host:

<pre>
modprobe xt_state
</pre>

== See also ==

* [[Traffic accounting with iptables]]

[[Category: Networking]]

Setting up an iptables firewall

2010-11-11T23:04:47Z

DmitryKoterov: /* Simple firewall configuration independent to IP addresses: vzfirewall */

This document consists of two parts. The first is setting up a firewall (using iptables) on the HN, which will restrict traffic to the containers. The effect would emulate, as far as the containers and their customers are concerned, an external hardware firewall controlled by the sysadmin. The second is setting up a firewall that protects the [[HN]] itself but still allows traffic to the containers, thus allowing individual containers to define their own iptables.

While the firewalls shown here can be accomplished using iptables manually (or using Fedora core's iptables service), the methods presented here are especially modular and easy to modify. This is important when you have 20+ containers and a lot of other things to be doing...

The scripts and pathnames given here are for Fedora Core 6, though they can probably be applied to most similar SysV-like systems with little modification.

== A little background ==

On our systems, we use the HN to provide privileged services which are not appropriate for access by the containers. For example, the HN acts as a backup server, runs Nagios for health monitoring, has a webserver for managing the 3ware RAID controller, etc. The containers are leased to customers, who can't entirely be trusted, especially if they get hacked. As such, our scenario is one in which the HN must be protected from all access (even from the containers) except for a few trusted hosts (e.g. my home-office).

The exception to this is the nameserver, which we want open to the world. We use it as a caching nameserver for our containers and also to host DNS for a few customer domain.

== Simple firewall configuration independent to IP addresses: vzfirewall ==

''Vzfirewall'' tool allows you to open/close ports for incoming connections with no dependencies to foreign IP addresses. E.g. you may allow a hostname ''release.prod.example.com'' to connect to port 5432 of VE 1234 and leave all other ports closed by modifying 1234.conf file adding multiline ''FIREWALL'' directive into it:

<pre>
...
PRIVVMPAGES="300000:300000"
HOSTNAME="example.com"
...
FIREWALL="
...
# Allow access to PostgreSQL port only from release.prod machine.
# Note that you may use domain names here.
[5432]
release.prod.example.com
release.test.example.com
...
"
</pre>

You must then run ''vzfirewall -a'' on your hardware node to apply changes made in *.conf.

Note that it is recommended to use hostnames instead of IP addresses here, so the configuration is persistent fore VE movements to different IP-address.

Vzfirewall and its documentation tool is available at [http://en.dklab.ru/lib/dklab_vzfirewall/ http://en.dklab.ru/lib/dklab_vzfirewall/].

== An alternative from the author of Shorewall ==

For those who might find the solution provided in this wiki article unsatisfactory (for whatever reason), the creator of Shorewall (Tom Eastep) has written a nice article explaining how to use Shorewall on an OpenVZ host node to manage the host node, containers, and more... with quite a complex setup as an example. The article IS NOT an introduction to Shorewall for beginners, so some pre-existing knowledge and understanding of Shorewall may be required.

Shorewall and OpenVZ by Tom Eastep - http://www.shorewall.net/OpenVZ.html

See also this OpenVZ Forum posting - http://forum.openvz.org/index.php?t=msg&goto=16406&

== Setting up a HN-based firewall ==

This setup emulates (to the containers anyway) an external hardware firewall. It protects the HN from any access and then defines what services and ports are allowed/banned for individual containers. This leaves the firewall controlled by the site administrator, not be individual containers and the hackers who've gotten into them. ;)

First off, let's disable Fedora's existing <code>iptables</code> service:
<pre>
service iptables stop
chkconfig iptables off
</pre>

Now create the new <code>firewall</code> service. This code should be <code>/etc/init.d/firewall</code> and then should be chmod'd 755.
<pre>
#!/bin/sh
# firewall Start iptables firewall
# chkconfig: 2345 08 92
# description: Starts, stops and saves iptables firewall
# This script sets up the firewall for the INPUT chain (which is for
# the HN itself) and then processes the config files under
# /etc/firewall.d to set up additional rules in the FORWARD chain
# to allow access to containers' services.

. /etc/init.d/functions

# the IP block allocated to this server
SEGMENT="192.168.0.0/24"
# the IP used by the hosting server itself
THISHOST="192.168.0.1"
# services that should be allowed to the HN;
# services for containers are configured in /etc/firewall.d/*
OKPORTS="53"
# hosts allowed full access through the firewall,
# to all containers and to this server
DMZS="12.34.56.78 90.123.45.67"

purge() {
echo -n "Firewall: Purging and allowing all traffic"
iptables -P OUTPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P INPUT ACCEPT
iptables -F
success ; echo
}

setup() {
echo -n "Firewall: Setting default policies to DROP"
iptables -P INPUT DROP
iptables -P FORWARD DROP
iptables -I INPUT -j ACCEPT -m state --state ESTABLISHED,RELATED
iptables -I FORWARD -j ACCEPT -m state --state ESTABLISHED,RELATED
iptables -I INPUT -j ACCEPT -i lo
iptables -I FORWARD -j ACCEPT --source $SEGMENT
success ; echo

echo "Firewall: Allowing access to HN"
for port in $OKPORTS ; do
echo -n " port $port"
iptables -I INPUT -j ACCEPT -s $SEGMENT -d $THISHOST --protocol tcp --destination-port $port
iptables -I INPUT -j ACCEPT -s $SEGMENT -d $THISHOST --protocol udp --destination-port $port
success ; echo
done
for ip in $DMZS ; do
echo -n " DMZ $ip"
iptables -I INPUT -i eth0 -j ACCEPT -s $ip
iptables -I FORWARD -i eth0 -j ACCEPT -s $ip
success ; echo
done

CTSETUPS=`echo /etc/firewall.d/*`
if [ "$CTSETUPS" != "/etc/firewall.d/*" ] ; then
echo "Firewall: Setting up container firewalls"
for i in $CTSETUPS ; do
. $i
echo -n " $CTNAME CT$CTID"
if [ -n "$BANNED" ]; then
for source in $BANNED ; do iptables -I FORWARD -j DROP --destination $CTIP --source $source ; done
fi
if [ -n "$OPENPORTS" ]; then
for port in $OPENPORTS ; do iptables -I FORWARD -j ACCEPT --protocol tcp --destination $CTIP --destination-port $port ; done
for port in $OPENPORTS ; do iptables -I FORWARD -j ACCEPT --protocol udp --destination $CTIP --destination-port $port ; done
fi
if [ -n "$DMZS" ]; then
for source in $DMZS ; do iptables -I FORWARD -j ACCEPT --protocol tcp --destination $CTIP --source $source ; done
for source in $DMZS ; do iptables -I FORWARD -j ACCEPT --protocol udp --destination $CTIP --source $source ; done
fi
[ $? -eq 0 ] && success || failure
echo
done
fi
}

case "$1" in
start)
echo "Starting firewall..."
purge
setup
;;
stop)
echo "Stopping firewall..."
purge
;;
restart)
$0 stop
$0 start
;;
status)
iptables -n -L
;;
*)
echo "Usage: $0 <start|stop|restart|status>"
;;
esac
</pre>

Note: This will only allow access to the HN from the hosts/networks defined in SEGMENT. If you'd like to open up the OKPORTS on the HN to everybody, you can remove the ''-s $SEGMENT'' parameters from the iptables commands under the "Firewall: Allowing access to HN" section. The modified lines would look like this:

<pre>
iptables -I INPUT -j ACCEPT -d $THISHOST --protocol tcp --destination-port $port
iptables -I INPUT -j ACCEPT -d $THISHOST --protocol udp --destination-port $port
</pre>

The above script can be called like this:
<pre>
service firewall start
service firewall stop
service firewall restart
service firewall status
</pre>

It will set up the firewall for the HN according to the parameters you specified for OKPORTS, DMZs, etc. and then it will call each file under /etc/firewall.d and process its configuration.

So create a file under /etc/firewall.d The exact filename isn't important, as long as it's meaningful to you, e.g. <code>ExampleCompany</code> or <code>ve12</code> and give it content like this:

<pre>
# This file is processed by /etc/init.d/firewall
CTID="1" # the container's ID#
CTNAME="Customer1" # A human-friendly label for the container
CTIP="192.168.1.34" # the IP address for this container
OPENPORTS="80 443" # ports that should be universally opened
# to the entire Internet
DMZS="1.2.3.0/24 5.6.7.8/32" # IPs and blocks that should have full access
# to the container's services
BANNED="" # IPs and blocks that should be entirely
# blocked from the container's services
</pre>

And there you go. Go ahead and start the firewall and check its status:
<pre>
service firewall restart
service firewall status
</pre>

As you can see, you can now add and edit the configurations for individual containers very easily. This method proves a lot easier to manage than Fedora's iptables-config mechamism!

To make the firewall service automatically start when the HN boots, run
<pre>
chkconfig --add firewall
</pre>

=== Debian Notes ===

The setup above works fine for Debian as well, however /etc/init.d/functions is missing. Here is a very simple version that you can use:

# /etc/init.d/functions

success() {
echo -n "...success"
}
failure() {
echo -n "...failure"
}

== Setting up a firewall that allows per-container configuration ==

This setup configures iptables on the HN to disallow access to all hosts, including the containers. However, it allows all traffic into the containers so they may define their own iptables rules and therefore manage their own firewall.

<pre>
iptables -P FORWARD ACCEPT
iptables -F FORWARD
</pre>

This will remove all rules for the FORWARD chain so all packets can pass back and forth between containers and the outside world.

If you want to use a firewall inside a container, please load these modules BEFORE starting the container:

<pre>
modprobe xt_tcpudp
modprobe ip_conntrack
</pre>

If you do not, you will get an error like this: "iptables: No chain/target/match by that name"

If you want to use stateful firewall rules (and you should!) you will also need to make sure that 'ipt_state' is in the 'IPTABLES' option in your vz.conf file:

<pre>
IPTABLES="ipt_REJECT ipt_tos ipt_limit ipt_multiport iptable_filter iptable_mangle ipt_TCPMSS ipt_tcpmss ipt_ttl ipt_length ipt_state"
</pre>

Also make sure the 'xt_state' module is loaded on the host:

<pre>
modprobe xt_state
</pre>

== See also ==

* [[Traffic accounting with iptables]]

[[Category: Networking]]

Setting up an iptables firewall

2010-11-11T23:03:33Z

DmitryKoterov: /* Simple firewall configuration independent to IP addresses */

This document consists of two parts. The first is setting up a firewall (using iptables) on the HN, which will restrict traffic to the containers. The effect would emulate, as far as the containers and their customers are concerned, an external hardware firewall controlled by the sysadmin. The second is setting up a firewall that protects the [[HN]] itself but still allows traffic to the containers, thus allowing individual containers to define their own iptables.

While the firewalls shown here can be accomplished using iptables manually (or using Fedora core's iptables service), the methods presented here are especially modular and easy to modify. This is important when you have 20+ containers and a lot of other things to be doing...

The scripts and pathnames given here are for Fedora Core 6, though they can probably be applied to most similar SysV-like systems with little modification.

== A little background ==

On our systems, we use the HN to provide privileged services which are not appropriate for access by the containers. For example, the HN acts as a backup server, runs Nagios for health monitoring, has a webserver for managing the 3ware RAID controller, etc. The containers are leased to customers, who can't entirely be trusted, especially if they get hacked. As such, our scenario is one in which the HN must be protected from all access (even from the containers) except for a few trusted hosts (e.g. my home-office).

The exception to this is the nameserver, which we want open to the world. We use it as a caching nameserver for our containers and also to host DNS for a few customer domain.

== Simple firewall configuration independent to IP addresses: vzfirewall ==

''Vzfirewall'' tool allows you to open/close ports for incoming connections with no dependencies to foreign IP addresses. E.g. you may allow a hostname ''release.prod.example.com'' to connect to port 5432 of VE 1234 by modifying 1234.conf file adding multiline ''FIREWALL'' directive into it:

<pre>
...
PRIVVMPAGES="300000:300000"
HOSTNAME="example.com"
...
FIREWALL="
...
# Allow access to PostgreSQL port only from release.prod machine.
# Note that you may use domain names here.
[5432]
release.prod.example.com
release.test.example.com
...
"
</pre>

You must then run ''vzfirewall -a'' on your hardware node to apply changes made in *.conf.

Note that it is recommended to use hostnames instead of IP addresses here, so the configuration is persistent fore VE movements to different IP-address.

Vzfirewall and its documentation tool is available at [http://en.dklab.ru/lib/dklab_vzfirewall/].

== An alternative from the author of Shorewall ==

For those who might find the solution provided in this wiki article unsatisfactory (for whatever reason), the creator of Shorewall (Tom Eastep) has written a nice article explaining how to use Shorewall on an OpenVZ host node to manage the host node, containers, and more... with quite a complex setup as an example. The article IS NOT an introduction to Shorewall for beginners, so some pre-existing knowledge and understanding of Shorewall may be required.

Shorewall and OpenVZ by Tom Eastep - http://www.shorewall.net/OpenVZ.html

See also this OpenVZ Forum posting - http://forum.openvz.org/index.php?t=msg&goto=16406&

== Setting up a HN-based firewall ==

This setup emulates (to the containers anyway) an external hardware firewall. It protects the HN from any access and then defines what services and ports are allowed/banned for individual containers. This leaves the firewall controlled by the site administrator, not be individual containers and the hackers who've gotten into them. ;)

First off, let's disable Fedora's existing <code>iptables</code> service:
<pre>
service iptables stop
chkconfig iptables off
</pre>

Now create the new <code>firewall</code> service. This code should be <code>/etc/init.d/firewall</code> and then should be chmod'd 755.
<pre>
#!/bin/sh
# firewall Start iptables firewall
# chkconfig: 2345 08 92
# description: Starts, stops and saves iptables firewall
# This script sets up the firewall for the INPUT chain (which is for
# the HN itself) and then processes the config files under
# /etc/firewall.d to set up additional rules in the FORWARD chain
# to allow access to containers' services.

. /etc/init.d/functions

# the IP block allocated to this server
SEGMENT="192.168.0.0/24"
# the IP used by the hosting server itself
THISHOST="192.168.0.1"
# services that should be allowed to the HN;
# services for containers are configured in /etc/firewall.d/*
OKPORTS="53"
# hosts allowed full access through the firewall,
# to all containers and to this server
DMZS="12.34.56.78 90.123.45.67"

purge() {
echo -n "Firewall: Purging and allowing all traffic"
iptables -P OUTPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P INPUT ACCEPT
iptables -F
success ; echo
}

setup() {
echo -n "Firewall: Setting default policies to DROP"
iptables -P INPUT DROP
iptables -P FORWARD DROP
iptables -I INPUT -j ACCEPT -m state --state ESTABLISHED,RELATED
iptables -I FORWARD -j ACCEPT -m state --state ESTABLISHED,RELATED
iptables -I INPUT -j ACCEPT -i lo
iptables -I FORWARD -j ACCEPT --source $SEGMENT
success ; echo

echo "Firewall: Allowing access to HN"
for port in $OKPORTS ; do
echo -n " port $port"
iptables -I INPUT -j ACCEPT -s $SEGMENT -d $THISHOST --protocol tcp --destination-port $port
iptables -I INPUT -j ACCEPT -s $SEGMENT -d $THISHOST --protocol udp --destination-port $port
success ; echo
done
for ip in $DMZS ; do
echo -n " DMZ $ip"
iptables -I INPUT -i eth0 -j ACCEPT -s $ip
iptables -I FORWARD -i eth0 -j ACCEPT -s $ip
success ; echo
done

CTSETUPS=`echo /etc/firewall.d/*`
if [ "$CTSETUPS" != "/etc/firewall.d/*" ] ; then
echo "Firewall: Setting up container firewalls"
for i in $CTSETUPS ; do
. $i
echo -n " $CTNAME CT$CTID"
if [ -n "$BANNED" ]; then
for source in $BANNED ; do iptables -I FORWARD -j DROP --destination $CTIP --source $source ; done
fi
if [ -n "$OPENPORTS" ]; then
for port in $OPENPORTS ; do iptables -I FORWARD -j ACCEPT --protocol tcp --destination $CTIP --destination-port $port ; done
for port in $OPENPORTS ; do iptables -I FORWARD -j ACCEPT --protocol udp --destination $CTIP --destination-port $port ; done
fi
if [ -n "$DMZS" ]; then
for source in $DMZS ; do iptables -I FORWARD -j ACCEPT --protocol tcp --destination $CTIP --source $source ; done
for source in $DMZS ; do iptables -I FORWARD -j ACCEPT --protocol udp --destination $CTIP --source $source ; done
fi
[ $? -eq 0 ] && success || failure
echo
done
fi
}

case "$1" in
start)
echo "Starting firewall..."
purge
setup
;;
stop)
echo "Stopping firewall..."
purge
;;
restart)
$0 stop
$0 start
;;
status)
iptables -n -L
;;
*)
echo "Usage: $0 <start|stop|restart|status>"
;;
esac
</pre>

Note: This will only allow access to the HN from the hosts/networks defined in SEGMENT. If you'd like to open up the OKPORTS on the HN to everybody, you can remove the ''-s $SEGMENT'' parameters from the iptables commands under the "Firewall: Allowing access to HN" section. The modified lines would look like this:

<pre>
iptables -I INPUT -j ACCEPT -d $THISHOST --protocol tcp --destination-port $port
iptables -I INPUT -j ACCEPT -d $THISHOST --protocol udp --destination-port $port
</pre>

The above script can be called like this:
<pre>
service firewall start
service firewall stop
service firewall restart
service firewall status
</pre>

It will set up the firewall for the HN according to the parameters you specified for OKPORTS, DMZs, etc. and then it will call each file under /etc/firewall.d and process its configuration.

So create a file under /etc/firewall.d The exact filename isn't important, as long as it's meaningful to you, e.g. <code>ExampleCompany</code> or <code>ve12</code> and give it content like this:

<pre>
# This file is processed by /etc/init.d/firewall
CTID="1" # the container's ID#
CTNAME="Customer1" # A human-friendly label for the container
CTIP="192.168.1.34" # the IP address for this container
OPENPORTS="80 443" # ports that should be universally opened
# to the entire Internet
DMZS="1.2.3.0/24 5.6.7.8/32" # IPs and blocks that should have full access
# to the container's services
BANNED="" # IPs and blocks that should be entirely
# blocked from the container's services
</pre>

And there you go. Go ahead and start the firewall and check its status:
<pre>
service firewall restart
service firewall status
</pre>

As you can see, you can now add and edit the configurations for individual containers very easily. This method proves a lot easier to manage than Fedora's iptables-config mechamism!

To make the firewall service automatically start when the HN boots, run
<pre>
chkconfig --add firewall
</pre>

=== Debian Notes ===

The setup above works fine for Debian as well, however /etc/init.d/functions is missing. Here is a very simple version that you can use:

# /etc/init.d/functions

success() {
echo -n "...success"
}
failure() {
echo -n "...failure"
}

== Setting up a firewall that allows per-container configuration ==

This setup configures iptables on the HN to disallow access to all hosts, including the containers. However, it allows all traffic into the containers so they may define their own iptables rules and therefore manage their own firewall.

<pre>
iptables -P FORWARD ACCEPT
iptables -F FORWARD
</pre>

This will remove all rules for the FORWARD chain so all packets can pass back and forth between containers and the outside world.

If you want to use a firewall inside a container, please load these modules BEFORE starting the container:

<pre>
modprobe xt_tcpudp
modprobe ip_conntrack
</pre>

If you do not, you will get an error like this: "iptables: No chain/target/match by that name"

If you want to use stateful firewall rules (and you should!) you will also need to make sure that 'ipt_state' is in the 'IPTABLES' option in your vz.conf file:

<pre>
IPTABLES="ipt_REJECT ipt_tos ipt_limit ipt_multiport iptable_filter iptable_mangle ipt_TCPMSS ipt_tcpmss ipt_ttl ipt_length ipt_state"
</pre>

Also make sure the 'xt_state' module is loaded on the host:

<pre>
modprobe xt_state
</pre>

== See also ==

* [[Traffic accounting with iptables]]

[[Category: Networking]]

Setting up an iptables firewall

2010-11-11T23:01:03Z

DmitryKoterov:

This document consists of two parts. The first is setting up a firewall (using iptables) on the HN, which will restrict traffic to the containers. The effect would emulate, as far as the containers and their customers are concerned, an external hardware firewall controlled by the sysadmin. The second is setting up a firewall that protects the [[HN]] itself but still allows traffic to the containers, thus allowing individual containers to define their own iptables.

While the firewalls shown here can be accomplished using iptables manually (or using Fedora core's iptables service), the methods presented here are especially modular and easy to modify. This is important when you have 20+ containers and a lot of other things to be doing...

The scripts and pathnames given here are for Fedora Core 6, though they can probably be applied to most similar SysV-like systems with little modification.

== A little background ==

On our systems, we use the HN to provide privileged services which are not appropriate for access by the containers. For example, the HN acts as a backup server, runs Nagios for health monitoring, has a webserver for managing the 3ware RAID controller, etc. The containers are leased to customers, who can't entirely be trusted, especially if they get hacked. As such, our scenario is one in which the HN must be protected from all access (even from the containers) except for a few trusted hosts (e.g. my home-office).

The exception to this is the nameserver, which we want open to the world. We use it as a caching nameserver for our containers and also to host DNS for a few customer domain.

== Simple firewall configuration independent to IP addresses ==

''Vzfirewall'' tool allows you to open/close ports for incoming connections with no dependencies to foreign IP addresses. E.g. you may allow a hostname ''release.prod.example.com'' to connect to VE 1234 by modifying 1234.conf file adding multiline ''FIREWALL'' directive into it:

<pre>
...
PRIVVMPAGES="300000:300000"
HOSTNAME="example.com"
...
FIREWALL="
...
# Allow access to PostgreSQL port only from release.prod machine.
# Note that you may use domain names here.
[5432]
release.prod.example.com
release.test.example.com
...
"
</pre>

Note that you may use hostnames instead of IP addresses, so the configuration is persistent fore VE movements to different IP-address.

Vzfirewall and its documentation tool is available at [http://en.dklab.ru/lib/dklab_vzfirewall/].

== An alternative from the author of Shorewall ==

For those who might find the solution provided in this wiki article unsatisfactory (for whatever reason), the creator of Shorewall (Tom Eastep) has written a nice article explaining how to use Shorewall on an OpenVZ host node to manage the host node, containers, and more... with quite a complex setup as an example. The article IS NOT an introduction to Shorewall for beginners, so some pre-existing knowledge and understanding of Shorewall may be required.

Shorewall and OpenVZ by Tom Eastep - http://www.shorewall.net/OpenVZ.html

See also this OpenVZ Forum posting - http://forum.openvz.org/index.php?t=msg&goto=16406&

== Setting up a HN-based firewall ==

This setup emulates (to the containers anyway) an external hardware firewall. It protects the HN from any access and then defines what services and ports are allowed/banned for individual containers. This leaves the firewall controlled by the site administrator, not be individual containers and the hackers who've gotten into them. ;)

First off, let's disable Fedora's existing <code>iptables</code> service:
<pre>
service iptables stop
chkconfig iptables off
</pre>

Now create the new <code>firewall</code> service. This code should be <code>/etc/init.d/firewall</code> and then should be chmod'd 755.
<pre>
#!/bin/sh
# firewall Start iptables firewall
# chkconfig: 2345 08 92
# description: Starts, stops and saves iptables firewall
# This script sets up the firewall for the INPUT chain (which is for
# the HN itself) and then processes the config files under
# /etc/firewall.d to set up additional rules in the FORWARD chain
# to allow access to containers' services.

. /etc/init.d/functions

# the IP block allocated to this server
SEGMENT="192.168.0.0/24"
# the IP used by the hosting server itself
THISHOST="192.168.0.1"
# services that should be allowed to the HN;
# services for containers are configured in /etc/firewall.d/*
OKPORTS="53"
# hosts allowed full access through the firewall,
# to all containers and to this server
DMZS="12.34.56.78 90.123.45.67"

purge() {
echo -n "Firewall: Purging and allowing all traffic"
iptables -P OUTPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P INPUT ACCEPT
iptables -F
success ; echo
}

setup() {
echo -n "Firewall: Setting default policies to DROP"
iptables -P INPUT DROP
iptables -P FORWARD DROP
iptables -I INPUT -j ACCEPT -m state --state ESTABLISHED,RELATED
iptables -I FORWARD -j ACCEPT -m state --state ESTABLISHED,RELATED
iptables -I INPUT -j ACCEPT -i lo
iptables -I FORWARD -j ACCEPT --source $SEGMENT
success ; echo

echo "Firewall: Allowing access to HN"
for port in $OKPORTS ; do
echo -n " port $port"
iptables -I INPUT -j ACCEPT -s $SEGMENT -d $THISHOST --protocol tcp --destination-port $port
iptables -I INPUT -j ACCEPT -s $SEGMENT -d $THISHOST --protocol udp --destination-port $port
success ; echo
done
for ip in $DMZS ; do
echo -n " DMZ $ip"
iptables -I INPUT -i eth0 -j ACCEPT -s $ip
iptables -I FORWARD -i eth0 -j ACCEPT -s $ip
success ; echo
done

CTSETUPS=`echo /etc/firewall.d/*`
if [ "$CTSETUPS" != "/etc/firewall.d/*" ] ; then
echo "Firewall: Setting up container firewalls"
for i in $CTSETUPS ; do
. $i
echo -n " $CTNAME CT$CTID"
if [ -n "$BANNED" ]; then
for source in $BANNED ; do iptables -I FORWARD -j DROP --destination $CTIP --source $source ; done
fi
if [ -n "$OPENPORTS" ]; then
for port in $OPENPORTS ; do iptables -I FORWARD -j ACCEPT --protocol tcp --destination $CTIP --destination-port $port ; done
for port in $OPENPORTS ; do iptables -I FORWARD -j ACCEPT --protocol udp --destination $CTIP --destination-port $port ; done
fi
if [ -n "$DMZS" ]; then
for source in $DMZS ; do iptables -I FORWARD -j ACCEPT --protocol tcp --destination $CTIP --source $source ; done
for source in $DMZS ; do iptables -I FORWARD -j ACCEPT --protocol udp --destination $CTIP --source $source ; done
fi
[ $? -eq 0 ] && success || failure
echo
done
fi
}

case "$1" in
start)
echo "Starting firewall..."
purge
setup
;;
stop)
echo "Stopping firewall..."
purge
;;
restart)
$0 stop
$0 start
;;
status)
iptables -n -L
;;
*)
echo "Usage: $0 <start|stop|restart|status>"
;;
esac
</pre>

Note: This will only allow access to the HN from the hosts/networks defined in SEGMENT. If you'd like to open up the OKPORTS on the HN to everybody, you can remove the ''-s $SEGMENT'' parameters from the iptables commands under the "Firewall: Allowing access to HN" section. The modified lines would look like this:

<pre>
iptables -I INPUT -j ACCEPT -d $THISHOST --protocol tcp --destination-port $port
iptables -I INPUT -j ACCEPT -d $THISHOST --protocol udp --destination-port $port
</pre>

The above script can be called like this:
<pre>
service firewall start
service firewall stop
service firewall restart
service firewall status
</pre>

It will set up the firewall for the HN according to the parameters you specified for OKPORTS, DMZs, etc. and then it will call each file under /etc/firewall.d and process its configuration.

So create a file under /etc/firewall.d The exact filename isn't important, as long as it's meaningful to you, e.g. <code>ExampleCompany</code> or <code>ve12</code> and give it content like this:

<pre>
# This file is processed by /etc/init.d/firewall
CTID="1" # the container's ID#
CTNAME="Customer1" # A human-friendly label for the container
CTIP="192.168.1.34" # the IP address for this container
OPENPORTS="80 443" # ports that should be universally opened
# to the entire Internet
DMZS="1.2.3.0/24 5.6.7.8/32" # IPs and blocks that should have full access
# to the container's services
BANNED="" # IPs and blocks that should be entirely
# blocked from the container's services
</pre>

And there you go. Go ahead and start the firewall and check its status:
<pre>
service firewall restart
service firewall status
</pre>

As you can see, you can now add and edit the configurations for individual containers very easily. This method proves a lot easier to manage than Fedora's iptables-config mechamism!

To make the firewall service automatically start when the HN boots, run
<pre>
chkconfig --add firewall
</pre>

=== Debian Notes ===

The setup above works fine for Debian as well, however /etc/init.d/functions is missing. Here is a very simple version that you can use:

# /etc/init.d/functions

success() {
echo -n "...success"
}
failure() {
echo -n "...failure"
}

== Setting up a firewall that allows per-container configuration ==

This setup configures iptables on the HN to disallow access to all hosts, including the containers. However, it allows all traffic into the containers so they may define their own iptables rules and therefore manage their own firewall.

<pre>
iptables -P FORWARD ACCEPT
iptables -F FORWARD
</pre>

This will remove all rules for the FORWARD chain so all packets can pass back and forth between containers and the outside world.

If you want to use a firewall inside a container, please load these modules BEFORE starting the container:

<pre>
modprobe xt_tcpudp
modprobe ip_conntrack
</pre>

If you do not, you will get an error like this: "iptables: No chain/target/match by that name"

If you want to use stateful firewall rules (and you should!) you will also need to make sure that 'ipt_state' is in the 'IPTABLES' option in your vz.conf file:

<pre>
IPTABLES="ipt_REJECT ipt_tos ipt_limit ipt_multiport iptable_filter iptable_mangle ipt_TCPMSS ipt_tcpmss ipt_ttl ipt_length ipt_state"
</pre>

Also make sure the 'xt_state' module is loaded on the host:

<pre>
modprobe xt_state
</pre>

== See also ==

* [[Traffic accounting with iptables]]

[[Category: Networking]]

Setting up an iptables firewall

2010-11-11T23:00:21Z

DmitryKoterov:

This document consists of two parts. The first is setting up a firewall (using iptables) on the HN, which will restrict traffic to the containers. The effect would emulate, as far as the containers and their customers are concerned, an external hardware firewall controlled by the sysadmin. The second is setting up a firewall that protects the [[HN]] itself but still allows traffic to the containers, thus allowing individual containers to define their own iptables.

While the firewalls shown here can be accomplished using iptables manually (or using Fedora core's iptables service), the methods presented here are especially modular and easy to modify. This is important when you have 20+ containers and a lot of other things to be doing...

The scripts and pathnames given here are for Fedora Core 6, though they can probably be applied to most similar SysV-like systems with little modification.

== A little background ==

On our systems, we use the HN to provide privileged services which are not appropriate for access by the containers. For example, the HN acts as a backup server, runs Nagios for health monitoring, has a webserver for managing the 3ware RAID controller, etc. The containers are leased to customers, who can't entirely be trusted, especially if they get hacked. As such, our scenario is one in which the HN must be protected from all access (even from the containers) except for a few trusted hosts (e.g. my home-office).

The exception to this is the nameserver, which we want open to the world. We use it as a caching nameserver for our containers and also to host DNS for a few customer domain.

== Simple firewall configuration independent to IP addresses ==

''Vzfirewall'' tool allows you to open/close ports for incoming connections with no dependencies to foreign IP addresses. E.g. you may allow a hostname ''release.prod.example.com'' to connect to VE 1234 by modifying 1234.conf file adding multiline ''FIREWALL'' directive into it:

<pre>
...
PRIVVMPAGES="300000:300000"
HOSTNAME="example.com"
...
FIREWALL="
...
# Allow access to PostgreSQL port only from release.prod machine.
# Note that you may use domain names here.
[5432]
release.prod.example.com
release.test.example.com
...
"
</pre>

Note that you may use hostnames instead of IP addresses, so the configuration is persistent fore VE movements to different IP-address.

Vzfirewall and its documentation tool is available at [http://en.dklab.ru/lib/dklab_vzfirewall/ vzfirewall homepage.

== An alternative from the author of Shorewall ==

For those who might find the solution provided in this wiki article unsatisfactory (for whatever reason), the creator of Shorewall (Tom Eastep) has written a nice article explaining how to use Shorewall on an OpenVZ host node to manage the host node, containers, and more... with quite a complex setup as an example. The article IS NOT an introduction to Shorewall for beginners, so some pre-existing knowledge and understanding of Shorewall may be required.

Shorewall and OpenVZ by Tom Eastep - http://www.shorewall.net/OpenVZ.html

See also this OpenVZ Forum posting - http://forum.openvz.org/index.php?t=msg&goto=16406&

== Setting up a HN-based firewall ==

This setup emulates (to the containers anyway) an external hardware firewall. It protects the HN from any access and then defines what services and ports are allowed/banned for individual containers. This leaves the firewall controlled by the site administrator, not be individual containers and the hackers who've gotten into them. ;)

First off, let's disable Fedora's existing <code>iptables</code> service:
<pre>
service iptables stop
chkconfig iptables off
</pre>

Now create the new <code>firewall</code> service. This code should be <code>/etc/init.d/firewall</code> and then should be chmod'd 755.
<pre>
#!/bin/sh
# firewall Start iptables firewall
# chkconfig: 2345 08 92
# description: Starts, stops and saves iptables firewall
# This script sets up the firewall for the INPUT chain (which is for
# the HN itself) and then processes the config files under
# /etc/firewall.d to set up additional rules in the FORWARD chain
# to allow access to containers' services.

. /etc/init.d/functions

# the IP block allocated to this server
SEGMENT="192.168.0.0/24"
# the IP used by the hosting server itself
THISHOST="192.168.0.1"
# services that should be allowed to the HN;
# services for containers are configured in /etc/firewall.d/*
OKPORTS="53"
# hosts allowed full access through the firewall,
# to all containers and to this server
DMZS="12.34.56.78 90.123.45.67"

purge() {
echo -n "Firewall: Purging and allowing all traffic"
iptables -P OUTPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P INPUT ACCEPT
iptables -F
success ; echo
}

setup() {
echo -n "Firewall: Setting default policies to DROP"
iptables -P INPUT DROP
iptables -P FORWARD DROP
iptables -I INPUT -j ACCEPT -m state --state ESTABLISHED,RELATED
iptables -I FORWARD -j ACCEPT -m state --state ESTABLISHED,RELATED
iptables -I INPUT -j ACCEPT -i lo
iptables -I FORWARD -j ACCEPT --source $SEGMENT
success ; echo

echo "Firewall: Allowing access to HN"
for port in $OKPORTS ; do
echo -n " port $port"
iptables -I INPUT -j ACCEPT -s $SEGMENT -d $THISHOST --protocol tcp --destination-port $port
iptables -I INPUT -j ACCEPT -s $SEGMENT -d $THISHOST --protocol udp --destination-port $port
success ; echo
done
for ip in $DMZS ; do
echo -n " DMZ $ip"
iptables -I INPUT -i eth0 -j ACCEPT -s $ip
iptables -I FORWARD -i eth0 -j ACCEPT -s $ip
success ; echo
done

CTSETUPS=`echo /etc/firewall.d/*`
if [ "$CTSETUPS" != "/etc/firewall.d/*" ] ; then
echo "Firewall: Setting up container firewalls"
for i in $CTSETUPS ; do
. $i
echo -n " $CTNAME CT$CTID"
if [ -n "$BANNED" ]; then
for source in $BANNED ; do iptables -I FORWARD -j DROP --destination $CTIP --source $source ; done
fi
if [ -n "$OPENPORTS" ]; then
for port in $OPENPORTS ; do iptables -I FORWARD -j ACCEPT --protocol tcp --destination $CTIP --destination-port $port ; done
for port in $OPENPORTS ; do iptables -I FORWARD -j ACCEPT --protocol udp --destination $CTIP --destination-port $port ; done
fi
if [ -n "$DMZS" ]; then
for source in $DMZS ; do iptables -I FORWARD -j ACCEPT --protocol tcp --destination $CTIP --source $source ; done
for source in $DMZS ; do iptables -I FORWARD -j ACCEPT --protocol udp --destination $CTIP --source $source ; done
fi
[ $? -eq 0 ] && success || failure
echo
done
fi
}

case "$1" in
start)
echo "Starting firewall..."
purge
setup
;;
stop)
echo "Stopping firewall..."
purge
;;
restart)
$0 stop
$0 start
;;
status)
iptables -n -L
;;
*)
echo "Usage: $0 <start|stop|restart|status>"
;;
esac
</pre>

Note: This will only allow access to the HN from the hosts/networks defined in SEGMENT. If you'd like to open up the OKPORTS on the HN to everybody, you can remove the ''-s $SEGMENT'' parameters from the iptables commands under the "Firewall: Allowing access to HN" section. The modified lines would look like this:

<pre>
iptables -I INPUT -j ACCEPT -d $THISHOST --protocol tcp --destination-port $port
iptables -I INPUT -j ACCEPT -d $THISHOST --protocol udp --destination-port $port
</pre>

The above script can be called like this:
<pre>
service firewall start
service firewall stop
service firewall restart
service firewall status
</pre>

It will set up the firewall for the HN according to the parameters you specified for OKPORTS, DMZs, etc. and then it will call each file under /etc/firewall.d and process its configuration.

So create a file under /etc/firewall.d The exact filename isn't important, as long as it's meaningful to you, e.g. <code>ExampleCompany</code> or <code>ve12</code> and give it content like this:

<pre>
# This file is processed by /etc/init.d/firewall
CTID="1" # the container's ID#
CTNAME="Customer1" # A human-friendly label for the container
CTIP="192.168.1.34" # the IP address for this container
OPENPORTS="80 443" # ports that should be universally opened
# to the entire Internet
DMZS="1.2.3.0/24 5.6.7.8/32" # IPs and blocks that should have full access
# to the container's services
BANNED="" # IPs and blocks that should be entirely
# blocked from the container's services
</pre>

And there you go. Go ahead and start the firewall and check its status:
<pre>
service firewall restart
service firewall status
</pre>

As you can see, you can now add and edit the configurations for individual containers very easily. This method proves a lot easier to manage than Fedora's iptables-config mechamism!

To make the firewall service automatically start when the HN boots, run
<pre>
chkconfig --add firewall
</pre>

=== Debian Notes ===

The setup above works fine for Debian as well, however /etc/init.d/functions is missing. Here is a very simple version that you can use:

# /etc/init.d/functions

success() {
echo -n "...success"
}
failure() {
echo -n "...failure"
}

== Setting up a firewall that allows per-container configuration ==

This setup configures iptables on the HN to disallow access to all hosts, including the containers. However, it allows all traffic into the containers so they may define their own iptables rules and therefore manage their own firewall.

<pre>
iptables -P FORWARD ACCEPT
iptables -F FORWARD
</pre>

This will remove all rules for the FORWARD chain so all packets can pass back and forth between containers and the outside world.

If you want to use a firewall inside a container, please load these modules BEFORE starting the container:

<pre>
modprobe xt_tcpudp
modprobe ip_conntrack
</pre>

If you do not, you will get an error like this: "iptables: No chain/target/match by that name"

If you want to use stateful firewall rules (and you should!) you will also need to make sure that 'ipt_state' is in the 'IPTABLES' option in your vz.conf file:

<pre>
IPTABLES="ipt_REJECT ipt_tos ipt_limit ipt_multiport iptable_filter iptable_mangle ipt_TCPMSS ipt_tcpmss ipt_ttl ipt_length ipt_state"
</pre>

Also make sure the 'xt_state' module is loaded on the host:

<pre>
modprobe xt_state
</pre>

== See also ==

* [[Traffic accounting with iptables]]

[[Category: Networking]]