OpenVZ Virtuozzo Containers Wiki - User contributions [en]

ExecuteInAllVEs

2008-01-06T15:23:33Z

HostGIS:

I found myself often faced with a need to run the same command in all VEs, e.g. ''apachectl restart'' to restart all webservers or ''dmesg | tail'' to see the latest news from everybody.

You can loop through all VEs and execute this this easily:
<code>
for veid in `vzlist -Hoveid`; do vzctl exec $veid COMMAND; done
</code>

Knowing this, you can also save some typing later by making a simple utility out of it.
I like to call this ''/usr/sbin/vzexec'' so I can later just say ''vzexec apachectl restart''
<code>
#!/bin/bash
# vzexec -- execute a command on all VEs
if [ "$1" == "" ]; then
echo "$0 -- Execute a command on all VEs"
echo ""
echo "Example: vzexec service httpd restart"
echo ""
echo "Note that variables are expanded in the host's shell, and use ' ' to prevent this."
echo "Example: vzexec echo \$PWD and vzexec 'echo \$PWD' are different."
exit
fi
for veid in `vzlist -Hoveid`; do
echo "*** VE $veid"
vzctl exec $veid $@
done
</code>

ExecuteInAllVEs

2008-01-06T15:23:20Z

HostGIS:

I found myself often faced with a need to run the same command in all VEs, e.g. ''apachectl restart'' to restart all webservers or ''dmesg | tail'' to see the latest news from everybody.

You can loop through all VEs and execute this this easily:
<code>
for veid in `vzlist -Hoveid`; do vzctl exec $veid COMMAND; done
</code>

Knowing this, you can also save some typing later by making a simple utility out of it. I like to call this ''/usr/sbin/vzexec'' so I can later just say ''vzexec apachectl restart''
<code>
#!/bin/bash
# vzexec -- execute a command on all VEs
if [ "$1" == "" ]; then
echo "$0 -- Execute a command on all VEs"
echo ""
echo "Example: vzexec service httpd restart"
echo ""
echo "Note that variables are expanded in the host's shell, and use ' ' to prevent this."
echo "Example: vzexec echo \$PWD and vzexec 'echo \$PWD' are different."
exit
fi
for veid in `vzlist -Hoveid`; do
echo "*** VE $veid"
vzctl exec $veid $@
done
</code>

ExecuteInAllVEs

2008-01-06T15:22:42Z

HostGIS:

I found myself often faced with a need to run the same command in all VEs, e.g. ''apachectl restart'' to restart all webservers or ''dmesg | tail'' to see the latest news from everybody.

At its core is just this simple tidbit: a for loop around vzlist.
<code>
for veid in `vzlist -Hoveid`; do vzctl exec $veid COMMAND; done
</code>

Knowing this, you can also save some typing later by making a simple utility out of it. I like to call this ''/usr/sbin/vzexec'' so I can later just say ''vzexec apachectl restart''
<code>
#!/bin/bash
# vzexec -- execute a command on all VEs
if [ "$1" == "" ]; then
echo "$0 -- Execute a command on all VEs"
echo ""
echo "Example: vzexec service httpd restart"
echo ""
echo "Note that variables are expanded in the host's shell, and use ' ' to prevent this."
echo "Example: vzexec echo \$PWD and vzexec 'echo \$PWD' are different."
exit
fi
for veid in `vzlist -Hoveid`; do
echo "*** VE $veid"
vzctl exec $veid $@
done
</code>

ExecuteInAllVEs

2008-01-06T15:20:19Z

HostGIS:

I found myself often faced with a need to run the same command in all VEs, e.g. ''apachectl restart'' to restart all webservers or ''dmesg | tail'' to see the latest news from everybody. This simple utility is just a loop around ''vzctl exec'' but I find it very handy.

<code>
#!/bin/bash
# vzexec -- execute a command on all VEs
if [ "$1" == "" ]; then
echo "$0 -- Execute a command on all VEs"
echo ""
echo "Example: vzexec service httpd restart"
echo ""
echo "Note that variables are expanded in the host's shell, and use ' ' to prevent this."
echo "Example: vzexec echo \$PWD and vzexec 'echo \$PWD' are different."
exit
fi
for veid in `vzlist -Hoveid`; do
echo "*** VE $veid"
vzctl exec $veid $@
done
</code>

ExecuteInAllVEs

2008-01-03T18:47:32Z

HostGIS:

I found myself often faced with a need to run the same command in all VEs, e.g. ''apachectl restart'' to restart all webservers or ''dmesg | tail'' to see the latest news from everybody. This simple utility is just a loop around ''vzctl exec'' but I find it very handy.

<code>
#!/bin/bash
# vzexec -- execute a command on all VEs
CONFDIR="/etc/vz/conf"
if [ "$1" == "" ]; then
echo "$0 -- Execute a command on all VEs"
echo ""
echo "Example: vzexec service httpd restart"
echo ""
echo "Note that variables are expanded in the host's shell, and use ' ' to prevent this."
echo "Example: vzexec echo \$PWD and vzexec 'echo \$PWD' are different."
exit
fi
for veconf in $CONFDIR/*.conf ; do
veid=`basename $veconf .conf`
if [ "$veid" == "0" ]; then continue; fi
vename=`grep ^NAME $veconf | head -1 | sed -e 's@NAME=@@' | sed -e 's@"@@g'`
echo "*** VE $veid ($vename)"
vzctl exec $veid $@
done
</code>

ExecuteInAllVEs

2008-01-03T18:47:03Z

HostGIS:

I found myself often faced with a need to run the same command in all VEs, e.g. ''apachectl restart'' to restart all webservers or ''dmesg | tail'' to see the latest news from everybody. This simple utility is just a loop around ''vzctl exec'' but I find it very handy.

<code>
#!/bin/bash
CONFDIR="/etc/vz/conf"
if [ "$1" == "" ]; then
echo "$0 -- Execute a command on all VEs"
echo ""
echo "Example: vzexec service httpd restart"
echo ""
echo "Note that variables are expanded in the host's shell, and use ' ' to prevent this."
echo "Example: vzexec echo \$PWD and vzexec 'echo \$PWD' are different."
exit
fi
for veconf in $CONFDIR/*.conf ; do
veid=`basename $veconf .conf`
if [ "$veid" == "0" ]; then continue; fi
vename=`grep ^NAME $veconf | head -1 | sed -e 's@NAME=@@' | sed -e 's@"@@g'`
echo "*** VE $veid ($vename)"
vzctl exec $veid $@
done
</code>

ExecuteInAllVEs

2008-01-03T18:44:31Z

HostGIS:

<code>
#!/bin/bash
CONFDIR="/etc/vz/conf"
if [ "$1" == "" ]; then
echo "$0 -- Execute a command on all VEs"
echo ""
echo "Example: vzexec service httpd restart"
echo ""
echo "Note that variables are expanded in the host's shell, and use ' ' to prevent this."
echo "Example: vzexec echo \$PWD and vzexec 'echo \$PWD' are different."
exit
fi
for veconf in $CONFDIR/*.conf ; do
veid=`basename $veconf .conf`
if [ "$veid" == "0" ]; then continue; fi
vename=`grep ^NAME $veconf | head -1 | sed -e 's@NAME=@@' | sed -e 's@"@@g'`
echo "*** VE $veid ($vename)"
vzctl exec $veid $@
done
</code>

ExecuteInAllVEs

2008-01-03T18:44:09Z

HostGIS:

<code>
#!/bin/bash
CONFDIR="/etc/vz/conf"
if [ "$1" == "" ]; then
echo "$0 -- Execute a command on all VEs"
echo ""
echo "Example: vzexec service httpd restart"
echo ""
echo "Note that variables are expanded in the host's shell, and use '' to prevent this."
echo "Example: vzexec echo \$PWD and vzexec 'echo \$PWD' are different."
exit
fi
for veconf in $CONFDIR/*.conf ; do
veid=`basename $veconf .conf`
if [ "$veid" == "0" ]; then continue; fi
vename=`grep ^NAME $veconf | head -1 | sed -e 's@NAME=@@' | sed -e 's@"@@g'`
echo "*** VE $veid ($vename)"
vzctl exec $veid $@
done
</code>

ExecuteInAllVEs

2008-01-03T18:43:14Z

HostGIS:

<code>
#!/bin/bash
CONFDIR="/etc/vz/conf"
if [ "$1" == "" ]; then
echo "$0 -- Execute a command on all VEs"
echo ""
echo "Example: vzexec service httpd restart"
echo ""
echo "Note that variables are expanded in the host's shell, and use '' to prevent this."
echo "Example: vzexec echo \$PWD and vzexec 'echo \$PWD' are different."
exit
fi
for veconf in $CONFDIR/*.conf ; do
veid=`basename $veconf .conf`
if [ "$veid" == "0" ]; then continue; fi
vename=`grep ^NAME $veconf | head -1 | sed -e 's@NAME=@@' | sed -e 's@"@@g'`
echo "*** VE $veid ($vename)"
vzctl exec $veid $@
done
</code>

ExecuteInAllVEs

2008-01-03T18:42:48Z

HostGIS: program code, no wording

{{{
#!/bin/bash
CONFDIR="/etc/vz/conf"
if [ "$1" == "" ]; then
echo "$0 -- Execute a command on all VEs"
echo ""
echo "Example: vzexec service httpd restart"
echo ""
echo "Note that variables are expanded in the host's shell, and use '' to prevent this."
echo "Example: vzexec echo \$PWD and vzexec 'echo \$PWD' are different."
exit
fi
for veconf in $CONFDIR/*.conf ; do
veid=`basename $veconf .conf`
if [ "$veid" == "0" ]; then continue; fi
vename=`grep ^NAME $veconf | head -1 | sed -e 's@NAME=@@' | sed -e 's@"@@g'`
echo "*** VE $veid ($vename)"
vzctl exec $veid $@
done
}}}

Creating a template cache : Slackware or HostGIS Linux

2007-11-30T17:00:58Z

HostGIS: /* Zipping it up into a cache image */

== Creating a new Host Template Cache for HostGIS Linux 4.x or Slackware 11.x/12.x ==

This process uses VMWare to install the OS into a VM, then to trim down the VM's contents to only those items suitable for a VPS/VE environment, then to save a snapshot of the system as a host template cache for use in OpenVZ.

This document focuses on HostGIS Linux (a Slackware derivative) but aside from the specifics about installation settings, it should be 99% applicable to Slackware as well.

=== Create the VM in VMWare ===

Technically, you could probably do this on a hardware PC without VMWare, but VMWare does make it more convenient.

Start by creating a new VM in VMWare.
* The disk and RAM stats can be minimal, as the system will never see live use.
* There is no need to create the entire disk at once during the setup.
* Create the disk as SCSI.

Then install HGL.
* Create a small partition at the end of the disk for swap. Some swap is technically necessary, but since you'll never in fact be using it, a few MB should be fine.
* Set the passwords to 'password'
* Do set the timezone properly. The internal clock does not use UTC/GMT.
* Select the default mouse, but do NOT enable GPM at startup.
* Hostname: template Domain: internal.lan
* IP config: as appropriate for your LAN
* Nameserver: no

Reboot into your new HGL install, and log in.

== Delete unnecessary stuff ==

A lot of packages aren't relevant to a VPS setting, e.g. floppy disk utilities and kernel modules, even getty listening on the console.

<code>

# kernel, kernel modules, documentation, mount points
rm -rf /lib/modules /boot /dev/.udev /usr/doc /usr/info /media

# packages not applicable to a VPS setting
cd /var/log/packages
for pkg in \
hotplug-* hdparm-* devmapper-* udev-* usbutils-* pciutils-* module-init-tools-* \
mdadm-* floppy-* lvm2-* raidtools-* reiserfsprogs-* \
smartmontools-* sysfsutils-* syslinux-* wireless_tools.* quota-* iptables-*
do removepkg $pkg ; done

# prune init's getty
vi /etc/inittab # delete everything after entry l6 (runlevel 6)
init q

# clean out the fstab and mtab files
( cd /etc ; rm -f fstab mtab ; ln -s ../proc/mounts mtab )
echo "proc /proc proc defaults 0 0" >> /etc/fstab
echo "devpts /dev/pts devpts mode=0620 0 0" >> /etc/fstab

# the startup sequence and services
cd /etc/rc.d
rm -f rc.gpm-sample rc.gpm rc.hotplug rc.ip_forward rc.modules \
rc.scanluns rc.serial rc.udev rc.sysvinit
vi rc.syslog # delete all mentions of klogd
vi rc.M # delete the setterm entry
vi rc.S # delete the MOTD clobbering

</code>

== Fix permissions and ownerships ==

<code>

# clear out old/dummy SSL certificates
mv /etc/ssl/openssl.cnf /tmp ; rm -r /etc/ssl/* ; mv /tmp/openssl.cnf /etc/ssl

# set an ownership on any unowned files
find / -mount -nouser -exec chown root {} \; &
find / -mount -nogroup -exec chgrp root {} \; &

# remove the setuid bit from programs which nobody else should use
# you may want to review this list first, as some folks want their users
# able to edit cronjobs and to change their own passwords, etc.
for i in \
/bin/ping /bin/mount /bin/ping6 /bin/umount /usr/bin/chfn \
/usr/bin/chsh /usr/bin/crontab /usr/bin/chage /usr/bin/traceroute6 /usr/bin/traceroute \
/usr/bin/expiry /usr/bin/newgrp /usr/bin/passwd /usr/bin/gpasswd \
/usr/libexec/ssh-keysign /usr/libexec/pt_chown /usr/bin/wall /usr/bin/write
do chmod u-s $i ; done

</code>

== Changes to rc scripts ==

OpenVZ emulates rebooting with an external cronjob called vpsreboot and a dummy file called /reboot within the VPS, and emulates the /etc/mtab file by pointing it to /proc/mounts So, some small changes are necessary to the rc scripts.

<code>

# somewhere in rc.6 add this command: touch /reboot
vi /etc/rc.d/rc.6

# somewhere in rc.M, add this command: rm -f /etc/mtab ; ln -s /proc/mounts /etc/mtab
vi /etc/rc.d/rc.M

</code>

== Blanking settings ==

Lastly, you'll want to delete or blank out a bunch of files so they start fresh
when the VE is booted for its first time.

<code>

# stop all services
apachectl stop
killall syslogd klogd udevd crond
/etc/rc.d/rc.sendmail stop
/etc/rc.d/rc.inetd stop
/etc/webmin/stop
/etc/rc.d/rc.pgsql stop
/etc/rc.d/rc.mysqld stop
killall named proftpd

# blow away the network configuration with dummy strings for later replacement
# replace the IP address with __IPADDRESS_
# replace the netmask with __NETMASK__
# replace the GATEWAY with __GATEWAY__
vi /etc/rc.d/rc.inet1.conf

# disable the root and user accounts
# by changing the password for root and user to a ! character.
vi /etc/shadow

# refresh the 'locate' cache
/etc/cron.daily/slocate

# blank out the system logfiles
for logfile in \
/var/log/messages /var/log/syslog /var/log/debug /var/log/secure \
/var/log/maillog /var/log/spooler /var/log/proftpd.log /var/log/xinetd.log \
/var/log/dmesg /var/log/faillog /var/log/lastlog /var/log/wtmp \
/var/log/apache/access_log /var/log/apache/error_log \
/var/log/webmin/miniserv.error /var/log/webmin/miniserv.pid
do cp /dev/null $logfile ; done
rmdir /var/log/sa

# clear the SSH host key
rm -f /etc/ssh/ssh_host_*

# database server logfiles
rm -f /var/lib/mysql/*.err /var/lib/pgsql/logfile

# delete vi backup files, bash_history files, and other small application crud
unset HISTFILE
find / -name '*~' \
-o -name .bash_history \
-o -name .gnupg \
-o -name .lesshst \
-o -name .viminfo \
-o -name .rnd \
-delete

# anything under /tmp
rm -rf /tmp/*

</code>

== Zipping it up into a cache image ==

A VE cache is just a tar.gz file of the entire filesystem, excluding some very dynamic stuff which gets populated by the OS at runtime anyway:

<code>

tar zcvf /tmp/HostGIS_Linux_4.2_64bit.tar.gz --exclude='/sys/*' --exclude='/proc/*' --exclude='/tmp/*' /

</code>

Creating a template cache : Slackware or HostGIS Linux

2007-11-30T17:00:43Z

HostGIS: /* Changes to rc scripts */

Creating a template cache : Slackware or HostGIS Linux

2007-11-30T17:00:28Z

HostGIS: /* Fix permissions and ownerships */

Creating a template cache : Slackware or HostGIS Linux

2007-11-30T16:58:57Z

HostGIS: nearing the final version!

== Creating a new Host Template Cache for HostGIS Linux 4.x or Slackware 11.x/12.x ==

This process uses VMWare to install the OS into a VM, then to trim down the VM's contents to only those items suitable for a VPS/VE environment, then to save a snapshot of the system as a host template cache for use in OpenVZ.

This document focuses on HostGIS Linux (a Slackware derivative) but aside from the specifics about installation settings, it should be 99% applicable to Slackware as well.

=== Create the VM in VMWare ===

Technically, you could probably do this on a hardware PC without VMWare, but VMWare does make it more convenient.

Start by creating a new VM in VMWare.
* The disk and RAM stats can be minimal, as the system will never see live use.
* There is no need to create the entire disk at once during the setup.
* Create the disk as SCSI.

Then install HGL.
* Create a small partition at the end of the disk for swap. Some swap is technically necessary, but since you'll never in fact be using it, a few MB should be fine.
* Set the passwords to 'password'
* Do set the timezone properly. The internal clock does not use UTC/GMT.
* Select the default mouse, but do NOT enable GPM at startup.
* Hostname: template Domain: internal.lan
* IP config: as appropriate for your LAN
* Nameserver: no

Reboot into your new HGL install, and log in.

== Delete unnecessary stuff ==

A lot of packages aren't relevant to a VPS setting, e.g. floppy disk utilities and kernel modules, even getty listening on the console.

<code>

# kernel, kernel modules, documentation, mount points
rm -rf /lib/modules /boot /dev/.udev /usr/doc /usr/info /media

# packages not applicable to a VPS setting
cd /var/log/packages
for pkg in \
hotplug-* hdparm-* devmapper-* udev-* usbutils-* pciutils-* module-init-tools-* \
mdadm-* floppy-* lvm2-* raidtools-* reiserfsprogs-* \
smartmontools-* sysfsutils-* syslinux-* wireless_tools.* quota-* iptables-*
do removepkg $pkg ; done

# prune init's getty
vi /etc/inittab # delete everything after entry l6 (runlevel 6)
init q

# clean out the fstab and mtab files
( cd /etc ; rm -f fstab mtab ; ln -s ../proc/mounts mtab )
echo "proc /proc proc defaults 0 0" >> /etc/fstab
echo "devpts /dev/pts devpts mode=0620 0 0" >> /etc/fstab

# the startup sequence and services
cd /etc/rc.d
rm -f rc.gpm-sample rc.gpm rc.hotplug rc.ip_forward rc.modules \
rc.scanluns rc.serial rc.udev rc.sysvinit
vi rc.syslog # delete all mentions of klogd
vi rc.M # delete the setterm entry
vi rc.S # delete the MOTD clobbering

</code>

== Fix permissions and ownerships ==

<code>

# clear out old/dummy SSL certificates
mv /etc/ssl/openssl.cnf /tmp ; rm -r /etc/ssl/* ; mv /tmp/openssl.cnf /etc/ssl

# fix file permissions
find / -mount -nouser -exec chown root {} \; &
find / -mount -nogroup -exec chgrp root {} \; &
for i in \
/bin/ping /bin/mount /bin/ping6 /bin/umount /usr/bin/chfn \
/usr/bin/chsh /usr/bin/crontab /usr/bin/chage /usr/bin/traceroute6 /usr/bin/traceroute \
/usr/bin/expiry /usr/bin/newgrp /usr/bin/passwd /usr/bin/gpasswd \
/usr/libexec/ssh-keysign /usr/libexec/pt_chown /usr/bin/wall /usr/bin/write
do chmod u-s $i ; done

/code>

== Changes to rc scripts ==

OpenVZ emulates rebooting with an external cronjob called vpsreboot and a dummy file called /reboot within the VPS, and emulates the /etc/mtab file by pointing it to /proc/mounts So, some small changes are necessary to the rc scripts.

<code>

# somewhere in rc.6 add this command: touch /reboot
vi /etc/rc.d/rc.6

# somewhere in rc.M, add this command: rm -f /etc/mtab ; ln -s /proc/mounts /etc/mtab
vi /etc/rc.d/rc.M

</code>

== Blanking settings ==

Lastly, you'll want to delete or blank out a bunch of files so they start fresh
when the VE is booted for its first time.

<code>

# stop all services
apachectl stop
killall syslogd klogd udevd crond
/etc/rc.d/rc.sendmail stop
/etc/rc.d/rc.inetd stop
/etc/webmin/stop
/etc/rc.d/rc.pgsql stop
/etc/rc.d/rc.mysqld stop
killall named proftpd

# blow away the network configuration with dummy strings for later replacement
# replace the IP address with __IPADDRESS_
# replace the netmask with __NETMASK__
# replace the GATEWAY with __GATEWAY__
vi /etc/rc.d/rc.inet1.conf

# disable the root and user accounts
# by changing the password for root and user to a ! character.
vi /etc/shadow

# refresh the 'locate' cache
/etc/cron.daily/slocate

# blank out the system logfiles
for logfile in \
/var/log/messages /var/log/syslog /var/log/debug /var/log/secure \
/var/log/maillog /var/log/spooler /var/log/proftpd.log /var/log/xinetd.log \
/var/log/dmesg /var/log/faillog /var/log/lastlog /var/log/wtmp \
/var/log/apache/access_log /var/log/apache/error_log \
/var/log/webmin/miniserv.error /var/log/webmin/miniserv.pid
do cp /dev/null $logfile ; done
rmdir /var/log/sa

# clear the SSH host key
rm -f /etc/ssh/ssh_host_*

# database server logfiles
rm -f /var/lib/mysql/*.err /var/lib/pgsql/logfile

# delete vi backup files, bash_history files, and other small application crud
unset HISTFILE
find / -name '*~' \
-o -name .bash_history \
-o -name .gnupg \
-o -name .lesshst \
-o -name .viminfo \
-o -name .rnd \
-delete

# anything under /tmp
rm -rf /tmp/*

</code>

== Zipping it up into a cache image ==

A VE cache is just a tar.gz file of the entire filesystem, excluding some very dynamic stuff which gets populated by the OS at runtime anyway:

<code>

tar zcvf /tmp/HostGIS_Linux_4.2_64bit.tar.gz --exclude='/sys/*' --exclude='/proc/*' --exclude='/tmp/*' /

</code>

Creating a template cache : Slackware or HostGIS Linux

2007-11-30T16:48:19Z

HostGIS:

== Creating a new Host Template Cache for HostGIS Linux 4.x or Slackware 11.x/12.x ==

This process uses VMWare to install the OS into a VM, then to trim down the VM's contents to only those items suitable for a VPS/VE environment, then to save a snapshot of the system as a host template cache for use in OpenVZ.

This document focuses on HostGIS Linux (a Slackware derivative) but aside from the specifics about installation settings, it should be 99% applicable to Slackware as well.

=== Create the VM in VMWare ===

Technically, you could probably do this on a hardware PC without VMWare, but VMWare does make it more convenient.

Start by creating a new VM in VMWare.
* The disk and RAM stats can be minimal, as the system will never see live use.
* There is no need to create the entire disk at once during the setup.
* Create the disk as SCSI.

Then install HGL.
* Create a small partition at the end of the disk for swap. Some swap is technically necessary, but since you'll never in fact be using it, a few MB should be fine.
* Set the passwords to 'password'
* Do set the timezone properly. The internal clock does not use UTC/GMT.
* Select the default mouse, but do NOT enable GPM at startup.
* Hostname: template Domain: internal.lan
* IP config: as appropriate for your LAN
* Nameserver: no

Reboot into your new HGL install, and log in.

== Delete unnecessary stuff ==

<code>

# kernel, kernel modules, documentation, mount points
rm -rf /lib/modules /boot /dev/.udev /usr/doc /usr/info /media

# packages not applicable to a VPS setting, or which we don't use at HostGIS
# e.g. phpMyAdmin and phpPgAdmin are security holes
cd /var/log/packages
for pkg in \
hotplug-* hdparm-* devmapper-* udev-* usbutils-* pciutils-* module-init-tools-* \
mdadm-* floppy-* lvm2-* phpMyAdmin-* phppgAdmin-* raidtools-* reiserfsprogs-* \
smartmontools-* sysfsutils-* syslinux-* wireless_tools.* quota-* iptables-*
do removepkg $pkg ; done

# most folks don't use GeoServer, so disable it by default
chmod 644 /etc/rc.d/rc.geoserver

# prune init's getty
vi /etc/inittab # delete everything after entry l6 (runlevel 6)
init q

# clean out the fstab and mtab files
( cd /etc ; rm -f fstab mtab ; ln -s ../proc/mounts mtab )
echo "proc /proc proc defaults 0 0" >> /etc/fstab
echo "devpts /dev/pts devpts mode=0620 0 0" >> /etc/fstab

# the startup sequence and services, even the firewall
cd /etc/rc.d
rm -f rc.gpm-sample rc.gpm rc.hotplug rc.ip_forward rc.modules \
rc.scanluns rc.serial rc.udev rc.sysvinit rc.firewall
vi rc.syslog # delete all mentions of klogd
vi rc.local # delete smartd and inetd
vi rc.M # delete the setterm entry
vi rc.S # delete the MOTD clobbering
</code>

== Fix permissions and ownerships ==

<code>

# clear out old/dummy SSL certificates
mv /etc/ssl/openssl.cnf /tmp ; rm -r /etc/ssl/* ; mv /tmp/openssl.cnf /etc/ssl

# fix file permissions
find / -mount -nouser -exec chown root {} \; &
find / -mount -nogroup -exec chgrp root {} \; &
for i in \
/bin/ping /bin/mount /bin/ping6 /bin/umount /usr/bin/chfn \
/usr/bin/chsh /usr/bin/crontab /usr/bin/chage /usr/bin/traceroute6 /usr/bin/traceroute \
/usr/bin/expiry /usr/bin/newgrp /usr/bin/passwd /usr/bin/gpasswd \
/usr/libexec/ssh-keysign /usr/libexec/pt_chown /usr/bin/wall /usr/bin/write
do chmod u-s $i ; done

# fix Apache's configuration:
# add ServerTokens prod
# go to the htdocs Directory definition and change Indexes to -Indexes
# delete the entries for phpmyadmin and phppgadmin
vi /etc/apache/httpd.conf

# keep FTP users chrooted:
echo "" >> /etc/proftpd.conf
echo "# keep all users chrooted to their homedir" >> /etc/proftpd.conf
echo "DefaultRoot ~" >> /etc/proftpd.conf

# allow the mailq to be checked by anybody:
chgrp smmsp /var/spool/mqueue
chmod g+rx /var/spool/mqueue

</code>

== Changes to rc scripts ==

A VPS cannot actually reboot, since there's no power switch to power-cycle the machine
after the VE has been shut down. OpenVZ emulates this effect with an external cronjob
called vpsreboot (see /etc/cron.d/vz). In order to reboot a VPS that has been shut down
and which is expecting a reboot, the shutdown sequence must create a file named /reboot
in the VPS's filesystem.

Also, the /etc/mtab file should point to /proc/mounts so it can detect the / filesystem.

<code>

vi /etc/rc.d/rc.6
And add these two lines near the start:
# create the reboot flag so we get rebooted automatically
touch /reboot

vi /etc/rc.d/rc.M
And add these two lines near the start:
# replace the mtab file with a link to /proc/mounts so OpenVZ can find the / filesystem
rm -f /etc/mtab ; ln -s /proc/mounts /etc/mtab

</code>

== Blanking settings ==

Lastly, you'll want to delete or blank out a bunch of files so they start fresh
when the VE is booted for its first time.

<code>

# stop all services
apachectl stop
killall syslogd klogd udevd crond
/etc/rc.d/rc.sendmail stop
/etc/webmin/stop
/etc/rc.d/rc.pgsql stop
/etc/rc.d/rc.mysqld stop
killall named proftpd
killall xinetd

# blow away the network configuration with dummy strings for later replacement
# replace the IP address with __IPADDRESS_
# replace the netmask with __NETMASK__
# replace the GATEWAY with __GATEWAY__
vi /etc/rc.d/rc.inet1.conf

# disable the root and user accounts
# by changing the password for root and user to a ! character.
vi /etc/shadow

# refresh the 'locate' cache
/etc/cron.daily/slocate

# blank out the system logfiles
for logfile in \
/var/log/messages /var/log/syslog /var/log/debug /var/log/secure \
/var/log/maillog /var/log/spooler /var/log/proftpd.log /var/log/xinetd.log \
/var/log/dmesg /var/log/faillog /var/log/lastlog /var/log/wtmp \
/var/log/apache/access_log /var/log/apache/error_log \
/var/log/webmin/miniserv.error /var/log/webmin/miniserv.pid
do cp /dev/null $logfile ; done
rmdir /var/log/sa

# clear the SSH host key
rm -f /etc/ssh/ssh_host_*

# database server logfiles
rm -f /var/lib/mysql/*.err /var/lib/pgsql/logfile

# delete vi backup files, bash_history files, and other small application crud
unset HISTFILE
find / -name '*~' \
-o -name .bash_history \
-o -name .gnupg \
-o -name .lesshst \
-o -name .viminfo \
-o -name .rnd \
-delete

# the junk under /tmp
rm -rf /tmp/*

</code>

== Zipping it up into a cache image ==

A VE cache is just a tar.gz file of the entire filesystem, excluding some very dynamic stuff which gets populated by the OS at runtime anyway:

<code>

tar zcvf /tmp/HostGIS_Linux_4.2_64bit.tar.gz --exclude='/sys/*' --exclude='/proc/*' --exclude='/tmp/*' /

</code>

Creating a template cache : Slackware or HostGIS Linux

2007-11-30T16:29:41Z

HostGIS: Creating a template cache for SLackware or HostGIS Linux moved to Creating a template cache : Slackware or HostGIS Linux: Typo in title

Creating a new Host Template Cache
for HostGIS Linux 4.x or Slackware 11.x/12.x

This process uses VMWare to install the OS into a VM, then to trim down the VM's contents to only those items suitable for a VPS/VE environment, then to save a snapshot of the system as a host template cache for use in OpenVZ.

***** CREATING THE VM

Start by creating a new VM in VMWare. The stats can be minimal, and there
is no need to create the entire disk at once during the setup.
* Create the disk as SCSI.

Then install HGL.
* Create a small partition at the end of the disk for swap. Some swap
is technically necessary, but since you'll never in fact be using it,
a few MB should be fine.
* Set the passwords to 'password'
* Do set the timezone properly. The internal clock does not use UTC/GMT.
* Select the default mouse, but do NOT enable GPM at startup.
* Hostname: template Domain: internal.lan
* IP config: as appropriate for your LAN
* Nameserver: no
Reboot into your new HGL install.

Now we want to tweak it into a usable template.
Go ahead and login to the VM.

***** UPGRADES AND SECURITY PATCHES

The default HGL you used may require some software to be reinstalled, since new versions
and critical bugfixes may have been released since that version of HGL was released.
Follow these instructions, and also update them as necessary for the appropriate versions
and to remove paragraphs when a revision of HGL comes out that no longer requires them.

# HGL 4.2 - no necessary upgrades as of Nov 29 2007

***** REPLACE INETD WITH XINETD

Inetd is good but minimal. Xinetd offers security features, such as restricting service
to only certain IPs, and only listening on certain interfaces,.

removepkg inetd
rm -f /etc/inetd.conf* /etc/rc.d/rc.inetd

cd /tmp
wget --header="Host: xinetd.org" http://204.152.188.37/xinetd-2.3.14.tar.gz
tar zxvf xinetd*.gz
cd xinetd*
./configure --prefix=/usr --sysconfdir=/etc
make && make install
mkdir /etc/xinetd.d
cat >> /etc/rc.d/rc.local <<EOF

# xinetd
/usr/sbin/xinetd
EOF
cat > /etc/xinetd.conf <<EOF
defaults
{
log_type = SYSLOG daemon notice
log_on_success = HOST EXIT DURATION
log_on_failure = HOST ATTEMPT
instances = 30
cps = 50 10
}
includedir /etc/xinetd.d
EOF

***** NAGIOS: THE HEALTH-MONITORING SYSTEM

groupadd nagios
useradd -g nagios -d /usr/local/nagios -m nagios
echo "nrpe 5666/tcp # Nagios NRPE" >> /etc/services

cd /tmp
wget http://superb-east.dl.sourceforge.net/sourceforge/nagiosplug/nagios-plugins-1.4.10.tar.gz
tar zxvf nagios-plugins-*.tar.gz ; cd nagios-plugins-*
./configure && make all && make install
cd /tmp
wget http://umn.dl.sourceforge.net/sourceforge/nagios/nrpe-2.10.tar.gz
tar zxvf nrpe-2.10.tar.gz ; cd nrpe-2.10
./configure && make && cp src/nrpe /usr/local/nagios/nrpe

for plugin in \
check_wave check_users check_ups check_time check_tcp check_swap check_ssh check_ssmtp \
check_spop check_simap check_smtp check_sensors check_rpc check_real check_pop check_ping \
check_overcr check_oracle check_nwstat check_nt check_nntps check_nntp check_nagios \
check_mysql_query check_mrtgtraf check_mrtg check_log check_jabber check_ircd \
check_imap check_ifstatus check_ifoperstatus check_icmp check_http check_ftp check_flexlm \
check_file_age check_dummy check_disk_smb check_dig check_dhcp check_clamd check_by_ssh \
check_breeze check_apt check_udp
do rm -f /usr/local/nagios/libexec/$plugin ; done

cat > /usr/local/nagios/nrpe.cfg <<EOF
# NRPE Config File
pid_file=/var/run/nrpe.pid
debug=0
command_timeout=60
connection_timeout=300

# And now the list of allowed check-commands:
command[check_disk]=/usr/local/nagios/libexec/check_disk -w 20% -c 10% -m /
command[check_dns]=/usr/local/nagios/libexec/check_dns www.google.com
command[check_load]=/usr/local/nagios/libexec/check_load -w 5,5,5 -c 8,8,8
command[check_mailq]=/usr/local/nagios/libexec/check_mailq -w 10 -c 20
command[check_mysql]=/usr/local/nagios/libexec/check_mysql -d gisdata -H localhost -u gisdata -p password
command[check_pgsql]=/usr/local/nagios/libexec/check_pgsql -d gisdata -H localhost -l gisdata -p password
command[check_ntp]=/usr/local/nagios/libexec/check_ntp -H pool.ntp.org
command[check_crond]=/usr/local/nagios/libexec/check_procs -u root -c 1: --command=crond
command[check_syslogd]=/usr/local/nagios/libexec/check_procs -u root -c 1:1 --command=syslogd
command[check_xinetd]=/usr/local/nagios/libexec/check_procs -u root -c 1:1 --command=xinetd
EOF

cat > /etc/xinetd.d/nrpe <<EOF
# description: NRPE for Nagios
service nrpe
{
socket_type = stream
protocol = tcp
wait = no
user = nagios
server = /usr/local/nagios/nrpe
server_args = -c /usr/local/nagios/nrpe.cfg --inetd
only_from = __HOSTIP__
}
EOF

chown -R nagios:nagios /usr/local/nagios
chmod -R o-rwx /usr/local/nagios
chmod go-rwx /etc/xinetd.d

***** OTHER UNNECESSARY STUFF

# kernel, kernel modules, documentation, mount points
rm -rf /lib/modules /boot /dev/.udev /usr/doc /usr/info /media

# packages not applicable to a VPS setting, or which we don't use at HostGIS
# e.g. phpMyAdmin and phpPgAdmin are security holes
cd /var/log/packages
for pkg in \
hotplug-* hdparm-* devmapper-* udev-* usbutils-* pciutils-* module-init-tools-* \
mdadm-* floppy-* lvm2-* phpMyAdmin-* phppgAdmin-* raidtools-* reiserfsprogs-* \
smartmontools-* sysfsutils-* syslinux-* wireless_tools.* quota-* iptables-*
do removepkg $pkg ; done

# most folks don't use GeoServer, so disable it by default
chmod 644 /etc/rc.d/rc.geoserver

# prune init's getty
vi /etc/inittab # delete everything after entry l6 (runlevel 6)
init q

# clean out the fstab and mtab files
( cd /etc ; rm -f fstab mtab ; ln -s ../proc/mounts mtab )
echo "proc /proc proc defaults 0 0" >> /etc/fstab
echo "devpts /dev/pts devpts mode=0620 0 0" >> /etc/fstab

# the startup sequence and services, even the firewall
cd /etc/rc.d
rm -f rc.gpm-sample rc.hotplug rc.ip_forward rc.modules rc.scanluns rc.serial rc.udev rc.sysvinit rc.firewall
vi rc.syslog # delete all mentions of klogd
vi rc.local # delete smartd and inetd
vi rc.M # delete the setterm entry
vi rc.S # delete the MOTD clobbering

# blow away the network configuration with dummy strings for later replacement
# replace the IP address with __IPADDRESS_
# replace the netmask with __NETMASK__
# replace the GATEWAY with __GATEWAY__
vi /etc/rc.d/rc.inet1.conf

***** BASIC FILE SECURITY SETTINGS

# clear out old/dummy SSL certificates
mv /etc/ssl/openssl.cnf /tmp ; rm -r /etc/ssl/* ; mv /tmp/openssl.cnf /etc/ssl

# fix file permissions
find / -mount -nouser -exec chown root {} \; &
find / -mount -nogroup -exec chgrp root {} \; &
for i in \
/bin/ping /bin/mount /bin/ping6 /bin/umount /usr/bin/chfn \
/usr/bin/chsh /usr/bin/crontab /usr/bin/chage /usr/bin/traceroute6 /usr/bin/traceroute \
/usr/bin/expiry /usr/bin/newgrp /usr/bin/passwd /usr/bin/gpasswd \
/usr/libexec/ssh-keysign /usr/libexec/pt_chown /usr/bin/wall /usr/bin/write
do chmod u-s $i ; done

# fix Apache's configuration:
# add ServerTokens prod
# go to the htdocs Directory definition and change Indexes to -Indexes
# delete the entries for phpmyadmin and phppgadmin
vi /etc/apache/httpd.conf

# keep FTP users chrooted:
echo "" >> /etc/proftpd.conf
echo "# keep all users chrooted to their homedir" >> /etc/proftpd.conf
echo "DefaultRoot ~" >> /etc/proftpd.conf

# allow the mailq to be checked by anybody:
chgrp smmsp /var/spool/mqueue
chmod g+rx /var/spool/mqueue

# disable the root and user accounts
# by changing the password for root and user to a ! character.
vi /etc/shadow

***** REBOOTING

A VPS cannot actually reboot, since there's no power switch to power-cycle the machine
after the VE has been shut down. OpenVZ emulates this effect with an external cronjob
called vpsreboot (see /etc/cron.d/vz). In order to reboot a VPS that has been shut down
and which is expecting a reboot, the shutdown sequence must create a file named /reboot
in the VPS's filesystem.

Also, the /etc/mtab file should point to /proc/mounts so it can detect the / filesystem.

vi /etc/rc.d/rc.6
And add these two lines near the start:
# create the reboot flag so we get rebooted automatically
touch /reboot

vi /etc/rc.d/rc.M
And add these two lines near the start:
# replace the mtab file with a link to /proc/mounts so OpenVZ can find the / filesystem
rm -f /etc/mtab ; ln -s /proc/mounts /etc/mtab

***** DELETING AND BLANKING SETTINGS

Lastly, you'll want to delete or blank out a bunch of files so they start fresh
when the VE is booted for its first time.

# stop all services
apachectl stop
killall syslogd klogd udevd crond
/etc/rc.d/rc.sendmail stop
/etc/webmin/stop
/etc/rc.d/rc.pgsql stop
/etc/rc.d/rc.mysqld stop
killall named proftpd
killall xinetd

# refresh the 'locate' cache
/etc/cron.daily/slocate

# blank out the system logfiles
for logfile in \
/var/log/messages /var/log/syslog /var/log/debug /var/log/secure \
/var/log/maillog /var/log/spooler /var/log/proftpd.log /var/log/xinetd.log \
/var/log/dmesg /var/log/faillog /var/log/lastlog /var/log/wtmp \
/var/log/apache/access_log /var/log/apache/error_log \
/var/log/webmin/miniserv.error /var/log/webmin/miniserv.pid
do cp /dev/null $logfile ; done
rmdir /var/log/sa

# clear the SSH host key
rm -f /etc/ssh/ssh_host_*

# database server logfiles
rm -f /var/lib/mysql/*.err /var/lib/pgsql/logfile

# delete vi backup files, bash_history files, and other small application crud
unset HISTFILE
find / -name '*~' \
-o -name .bash_history \
-o -name .gnupg \
-o -name .lesshst \
-o -name .viminfo \
-o -name .rnd \
-delete

# the junk under /tmp
rm -rf /tmp/*

***** CREATING THE VE CACHE IMAGE

A VE cache is just a tar.gz file of the entire filesystem. So creating them is simple!

tar zcvf /tmp/HostGIS_Linux_4.2_64bit.tar.gz --exclude='/sys/*' --exclude='/proc/*' --exclude='/tmp/*' /

Ta-da! That's your new VE template cache. Just SFTP it to the VE server and you're all set!

Creating a template cache : Slackware or HostGIS Linux

2007-11-30T16:29:13Z

HostGIS:

Creating a template cache : Slackware or HostGIS Linux

2007-11-30T16:27:17Z

HostGIS: very raw initial version

Creating a new "Host Template Cache" for HostGIS Linux

This document describes how to use VMWare to create a new VM,
install HostGIS Linux (HGL) on it and tweak the system into shape,
and then create a Host Template Cache (a compressed VE image) for
use in OpenVZ.

***** CREATING THE VM

Start by creating a new VM in VMWare. The stats can be minimal, and there
is no need to create the entire disk at once during the setup.
* Create the disk as SCSI.

Then install HGL.
* Create a small partition at the end of the disk for swap. Some swap
is technically necessary, but since you'll never in fact be using it,
a few MB should be fine.
* Set the passwords to 'password'
* Do set the timezone properly. The internal clock does not use UTC/GMT.
* Select the default mouse, but do NOT enable GPM at startup.
* Hostname: template Domain: internal.lan
* IP config: as appropriate for your LAN
* Nameserver: no
Reboot into your new HGL install.

Now we want to tweak it into a usable template.
Go ahead and login to the VM.

***** UPGRADES AND SECURITY PATCHES

The default HGL you used may require some software to be reinstalled, since new versions
and critical bugfixes may have been released since that version of HGL was released.
Follow these instructions, and also update them as necessary for the appropriate versions
and to remove paragraphs when a revision of HGL comes out that no longer requires them.

# HGL 4.2 - no necessary upgrades as of Nov 29 2007

***** REPLACE INETD WITH XINETD

Inetd is good but minimal. Xinetd offers security features, such as restricting service
to only certain IPs, and only listening on certain interfaces,.

removepkg inetd
rm -f /etc/inetd.conf* /etc/rc.d/rc.inetd

cd /tmp
wget --header="Host: xinetd.org" http://204.152.188.37/xinetd-2.3.14.tar.gz
tar zxvf xinetd*.gz
cd xinetd*
./configure --prefix=/usr --sysconfdir=/etc
make && make install
mkdir /etc/xinetd.d
cat >> /etc/rc.d/rc.local <<EOF

# xinetd
/usr/sbin/xinetd
EOF
cat > /etc/xinetd.conf <<EOF
defaults
{
log_type = SYSLOG daemon notice
log_on_success = HOST EXIT DURATION
log_on_failure = HOST ATTEMPT
instances = 30
cps = 50 10
}
includedir /etc/xinetd.d
EOF

***** NAGIOS: THE HEALTH-MONITORING SYSTEM

groupadd nagios
useradd -g nagios -d /usr/local/nagios -m nagios
echo "nrpe 5666/tcp # Nagios NRPE" >> /etc/services

cd /tmp
wget http://superb-east.dl.sourceforge.net/sourceforge/nagiosplug/nagios-plugins-1.4.10.tar.gz
tar zxvf nagios-plugins-*.tar.gz ; cd nagios-plugins-*
./configure && make all && make install
cd /tmp
wget http://umn.dl.sourceforge.net/sourceforge/nagios/nrpe-2.10.tar.gz
tar zxvf nrpe-2.10.tar.gz ; cd nrpe-2.10
./configure && make && cp src/nrpe /usr/local/nagios/nrpe

for plugin in \
check_wave check_users check_ups check_time check_tcp check_swap check_ssh check_ssmtp \
check_spop check_simap check_smtp check_sensors check_rpc check_real check_pop check_ping \
check_overcr check_oracle check_nwstat check_nt check_nntps check_nntp check_nagios \
check_mysql_query check_mrtgtraf check_mrtg check_log check_jabber check_ircd \
check_imap check_ifstatus check_ifoperstatus check_icmp check_http check_ftp check_flexlm \
check_file_age check_dummy check_disk_smb check_dig check_dhcp check_clamd check_by_ssh \
check_breeze check_apt check_udp
do rm -f /usr/local/nagios/libexec/$plugin ; done

cat > /usr/local/nagios/nrpe.cfg <<EOF
# NRPE Config File
pid_file=/var/run/nrpe.pid
debug=0
command_timeout=60
connection_timeout=300

# And now the list of allowed check-commands:
command[check_disk]=/usr/local/nagios/libexec/check_disk -w 20% -c 10% -m /
command[check_dns]=/usr/local/nagios/libexec/check_dns www.google.com
command[check_load]=/usr/local/nagios/libexec/check_load -w 5,5,5 -c 8,8,8
command[check_mailq]=/usr/local/nagios/libexec/check_mailq -w 10 -c 20
command[check_mysql]=/usr/local/nagios/libexec/check_mysql -d gisdata -H localhost -u gisdata -p password
command[check_pgsql]=/usr/local/nagios/libexec/check_pgsql -d gisdata -H localhost -l gisdata -p password
command[check_ntp]=/usr/local/nagios/libexec/check_ntp -H pool.ntp.org
command[check_crond]=/usr/local/nagios/libexec/check_procs -u root -c 1: --command=crond
command[check_syslogd]=/usr/local/nagios/libexec/check_procs -u root -c 1:1 --command=syslogd
command[check_xinetd]=/usr/local/nagios/libexec/check_procs -u root -c 1:1 --command=xinetd
EOF

cat > /etc/xinetd.d/nrpe <<EOF
# description: NRPE for Nagios
service nrpe
{
socket_type = stream
protocol = tcp
wait = no
user = nagios
server = /usr/local/nagios/nrpe
server_args = -c /usr/local/nagios/nrpe.cfg --inetd
only_from = __HOSTIP__
}
EOF

chown -R nagios:nagios /usr/local/nagios
chmod -R o-rwx /usr/local/nagios
chmod go-rwx /etc/xinetd.d

***** OTHER UNNECESSARY STUFF

# kernel, kernel modules, documentation, mount points
rm -rf /lib/modules /boot /dev/.udev /usr/doc /usr/info /media

# packages not applicable to a VPS setting, or which we don't use at HostGIS
# e.g. phpMyAdmin and phpPgAdmin are security holes
cd /var/log/packages
for pkg in \
hotplug-* hdparm-* devmapper-* udev-* usbutils-* pciutils-* module-init-tools-* \
mdadm-* floppy-* lvm2-* phpMyAdmin-* phppgAdmin-* raidtools-* reiserfsprogs-* \
smartmontools-* sysfsutils-* syslinux-* wireless_tools.* quota-* iptables-*
do removepkg $pkg ; done

# most folks don't use GeoServer, so disable it by default
chmod 644 /etc/rc.d/rc.geoserver

# prune init's getty
vi /etc/inittab # delete everything after entry l6 (runlevel 6)
init q

# clean out the fstab and mtab files
( cd /etc ; rm -f fstab mtab ; ln -s ../proc/mounts mtab )
echo "proc /proc proc defaults 0 0" >> /etc/fstab
echo "devpts /dev/pts devpts mode=0620 0 0" >> /etc/fstab

# the startup sequence and services, even the firewall
cd /etc/rc.d
rm -f rc.gpm-sample rc.hotplug rc.ip_forward rc.modules rc.scanluns rc.serial rc.udev rc.sysvinit rc.firewall
vi rc.syslog # delete all mentions of klogd
vi rc.local # delete smartd and inetd
vi rc.M # delete the setterm entry
vi rc.S # delete the MOTD clobbering

# blow away the network configuration with dummy strings for later replacement
# replace the IP address with __IPADDRESS_
# replace the netmask with __NETMASK__
# replace the GATEWAY with __GATEWAY__
vi /etc/rc.d/rc.inet1.conf

***** BASIC FILE SECURITY SETTINGS

# clear out old/dummy SSL certificates
mv /etc/ssl/openssl.cnf /tmp ; rm -r /etc/ssl/* ; mv /tmp/openssl.cnf /etc/ssl

# fix file permissions
find / -mount -nouser -exec chown root {} \; &
find / -mount -nogroup -exec chgrp root {} \; &
for i in \
/bin/ping /bin/mount /bin/ping6 /bin/umount /usr/bin/chfn \
/usr/bin/chsh /usr/bin/crontab /usr/bin/chage /usr/bin/traceroute6 /usr/bin/traceroute \
/usr/bin/expiry /usr/bin/newgrp /usr/bin/passwd /usr/bin/gpasswd \
/usr/libexec/ssh-keysign /usr/libexec/pt_chown /usr/bin/wall /usr/bin/write
do chmod u-s $i ; done

# fix Apache's configuration:
# add ServerTokens prod
# go to the htdocs Directory definition and change Indexes to -Indexes
# delete the entries for phpmyadmin and phppgadmin
vi /etc/apache/httpd.conf

# keep FTP users chrooted:
echo "" >> /etc/proftpd.conf
echo "# keep all users chrooted to their homedir" >> /etc/proftpd.conf
echo "DefaultRoot ~" >> /etc/proftpd.conf

# allow the mailq to be checked by anybody:
chgrp smmsp /var/spool/mqueue
chmod g+rx /var/spool/mqueue

# disable the root and user accounts
# by changing the password for root and user to a ! character.
vi /etc/shadow

***** REBOOTING

A VPS cannot actually reboot, since there's no power switch to power-cycle the machine
after the VE has been shut down. OpenVZ emulates this effect with an external cronjob
called vpsreboot (see /etc/cron.d/vz). In order to reboot a VPS that has been shut down
and which is expecting a reboot, the shutdown sequence must create a file named /reboot
in the VPS's filesystem.

Also, the /etc/mtab file should point to /proc/mounts so it can detect the / filesystem.

vi /etc/rc.d/rc.6
And add these two lines near the start:
# create the reboot flag so we get rebooted automatically
touch /reboot

vi /etc/rc.d/rc.M
And add these two lines near the start:
# replace the mtab file with a link to /proc/mounts so OpenVZ can find the / filesystem
rm -f /etc/mtab ; ln -s /proc/mounts /etc/mtab

***** DELETING AND BLANKING SETTINGS

Lastly, you'll want to delete or blank out a bunch of files so they start fresh
when the VE is booted for its first time.

# stop all services
apachectl stop
killall syslogd klogd udevd crond
/etc/rc.d/rc.sendmail stop
/etc/webmin/stop
/etc/rc.d/rc.pgsql stop
/etc/rc.d/rc.mysqld stop
killall named proftpd
killall xinetd

# refresh the 'locate' cache
/etc/cron.daily/slocate

# blank out the system logfiles
for logfile in \
/var/log/messages /var/log/syslog /var/log/debug /var/log/secure \
/var/log/maillog /var/log/spooler /var/log/proftpd.log /var/log/xinetd.log \
/var/log/dmesg /var/log/faillog /var/log/lastlog /var/log/wtmp \
/var/log/apache/access_log /var/log/apache/error_log \
/var/log/webmin/miniserv.error /var/log/webmin/miniserv.pid
do cp /dev/null $logfile ; done
rmdir /var/log/sa

# clear the SSH host key
rm -f /etc/ssh/ssh_host_*

# database server logfiles
rm -f /var/lib/mysql/*.err /var/lib/pgsql/logfile

# delete vi backup files, bash_history files, and other small application crud
unset HISTFILE
find / -name '*~' \
-o -name .bash_history \
-o -name .gnupg \
-o -name .lesshst \
-o -name .viminfo \
-o -name .rnd \
-delete

# the junk under /tmp
rm -rf /tmp/*

***** CREATING THE VE CACHE IMAGE

A VE cache is just a tar.gz file of the entire filesystem. So creating them is simple!

tar zcvf /tmp/HostGIS_Linux_4.2_64bit.tar.gz --exclude='/sys/*' --exclude='/proc/*' --exclude='/tmp/*' /

Ta-da! That's your new VE template cache. Just SFTP it to the VE server and you're all set!

Setting up an iptables firewall

2007-05-16T17:12:39Z

HostGIS:

This document consists of two parts. The first is setting up a firewall (using iptables) on the HN, which will restrict traffic to the VEs. The effect would emulate, as far as the VEs and their customers are concerned, an external hardware firewall controlled by the sysadmin. The second is setting up a firewall that protects the HN itself but still allows traffic to the VEs, thus allowing individual VEs to define their own iptables.

While the firewalls shown here can be accomplished using iptables manually (or using Fedora core's iptables service), the methods presented here are especially modular and easy to modify. This is important when you have 20+ VEs and a lot of other things to be doing...

The scripts and pathnames given here are for Fedora Core 6, though they can probably be applied to most similar SysV-like systems with little modification.

== A little background ==

On our systems, we use the HN to provide privileged services which are not appropriate for access by the VEs. For example, the HN acts as a backup server, runs Nagios for health monitoring, has a webserver for managing the 3ware RAID controller, etc. The VEs are leased to customers, who can't entirely be trusted, especially if they get hacked. As such, our scenario is one in which the HN must be protected from all access (even from the VEs) except for a few trusted hosts (e.g. my home-office).

The exception to this is the nameserver, which we want open to the world. We use it as a caching nameserver for our VEs and also to host DNS for a few customer domain.

== Setting up a HN-based firewall ==

This setup emulates (to the VEs anyway) an external hardware firewall. It protects the HN from any access and then defines what services and ports are allowed/banned for individual VEs. This leaves the firewall controlled by the site administrator, not be individual VEs and the hackers who've gotten into them. ;)

First off, let's disable Fedora's existing <code>iptables</code> service:
<pre>
service iptables off
chkconfig iptables off
</pre>

Now create the new <code>firewall</code> service. This code should be <code>/etc/init.d/firewall</code> and then should be chmod'd 755.
<pre>
#!/bin/sh
# firewall Start iptables firewall
# chkconfig: 2345 08 92
# description: Starts, stops and saves iptables firewall
# This script sets up the firewall for the INPUT chain (which is for the HN itself)
# and then processes the config files under /etc/firewall.d to set up additional rules
# in the FORWARD chain to allow access to VEs' services.

. /etc/init.d/functions

# the IP block allocated to this server
SEGMENT="192.168.0.0/24"
# the IP used by the hosting server itself
THISHOST="192.168.0.1"
# services that should be allowed to the HN; services for VEs are configured in /etc/firewall.d/*
OKPORTS="53"
# hosts allowed full access through the firewall, to all VEs and to this server
DMZS="12.34.56.78 90.123.45.67"

purge() {
echo -n "Firewall: Purging and allowing all traffic"
iptables -P OUTPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P INPUT ACCEPT
iptables -F
success ; echo
}

setup() {
echo -n "Firewall: Setting default policies to DROP"
iptables -P INPUT DROP
iptables -P FORWARD DROP
iptables -I INPUT -j ACCEPT -m state --state ESTABLISHED,RELATED
iptables -I FORWARD -j ACCEPT -m state --state ESTABLISHED,RELATED
iptables -I INPUT -j ACCEPT -i lo
iptables -I FORWARD -j ACCEPT --source $SEGMENT
success ; echo

echo "Firewall: Allowing access to HN"
for port in $OKPORTS ; do
echo -n " port $port"
iptables -I INPUT -j ACCEPT -s $SEGMENT -d $THISHOST --protocol tcp --destination-port $port
iptables -I INPUT -j ACCEPT -s $SEGMENT -d $THISHOST --protocol udp --destination-port $port
success ; echo
done
for ip in $DMZS ; do
echo -n " DMZ $ip"
iptables -I INPUT -i eth0 -j ACCEPT -s $ip
iptables -I FORWARD -i eth0 -j ACCEPT -s $ip
success ; echo
done

VESETUPS=`echo /etc/firewall.d/*`
if [ "$VESETUPS" != "/etc/firewall.d/*" ] ; then
echo "Firewall: Setting up VE firewalls"
for i in $VESETUPS ; do
. $i
echo -n " $VENAME VE$VEID"
if [ -n "$BANNED" ]; then
for source in $BANNED ; do iptables -I FORWARD -j DROP --destination $VEIP --source $source ; done
fi
if [ -n "$OPENPORTS" ]; then
for port in $OPENPORTS ; do iptables -I FORWARD -j ACCEPT --protocol tcp --destination $VEIP --destination-port $port ; done
for port in $OPENPORTS ; do iptables -I FORWARD -j ACCEPT --protocol udp --destination $VEIP --destination-port $port ; done
fi
if [ -n "$DMZS" ]; then
for source in $DMZS ; do iptables -I FORWARD -j ACCEPT --protocol tcp --destination $VEIP --source $source ; done
for source in $DMZS ; do iptables -I FORWARD -j ACCEPT --protocol udp --destination $VEIP --source $source ; done
fi
[ $? -eq 0 ] && success || failure
echo
done
fi
}

case "$1" in
start)
echo "Starting firewall..."
purge
setup
;;
stop)
echo "Stopping firewall..."
purge
;;
restart)
$0 stop
$0 start
;;
status)
iptables -n -L
;;
*)
echo "Usage: $0 <start|stop|restart|status>"
;;
esac
</pre>

The above script can be called like this:
<pre>
service firewall start
service firewall stop
service firewall restart
service firewall status
</pre>

It will set up the firewall for the HN according to the parameters you specified for OKPORTS, DMZs, etc. and then it will call each file under /etc/firewall.d and process its configuration.

So create a file under /etc/firewall.d The exact filename isn't important, as long as it's meaningful to you, e.g. <code>ExampleCompany</code> or <code>ve12</code> and give it content like this:

<pre>
# This file is processed by /etc/init.d/firewall
VEID="1" # the VE's ID#
VENAME="Customer1" # A human-friendly label for the VE
VEIP="192.168.1.34" # the IP address for this VE
OPENPORTS="80 443" # ports that should be universally opened to the entire Internet
DMZS="1.2.3.0/24 5.6.7.8/32" # IPs and blocks that should have full access to the VE's services
BANNED="" # IPs and blocks that should be entirely blocked from the VE's services
</pre>

And there you go. Go ahead and start the firewall and check its status:
<pre>
service firewall restart
service firewall status
</pre>

As you can see, you can now add and edit the configurations for individual VEs very easily. This method proves a lot easier to manage than Fedora's iptables-config mechamism!

== Setting up a firewall that allows per-VE configuration ==

This setup configures iptables on the HN to disallow access to all hosts, including the VEs. However, it allows all traffic into the VEs so they may define their own iptables rules and therefore manage their own firewall.

<code>This content is missing. You are invited to fill it in, if you get to it before I do. :)</code>

== See Also ==

[[Traffic accounting with iptables]]

[[ Category: Networking ]]
[[ Category: Firewalls ]]

Setting up an iptables firewall

2007-05-16T17:11:46Z

HostGIS: /* A little background */

This document consists of two parts. The first is setting up a firewall (using iptables) on the HN, which will restrict traffic to the VEs. The effect would emulate, as far as the VEs and their customers are concerned, an external hardware firewall controlled by the sysadmin. The second is setting up a firewall that protects the HN itself but still allows traffic to the VEs, thus allowing individual VEs to define their own iptables.

While the firewalls shown here can be accomplished using iptables manually (or using Fedora core's iptables service), the methods presented here are especially modular and easy to modify. This is important when you have 20+ VEs and a lot of other things to be doing...

The scripts and pathnames given here are for Fedora Core 6, though they can probably be applied to most similar SysV-like systems with little modification.

== A little background ==

On our systems, we use the HN to provide privileged services which are not appropriate for access by the VEs. For example, the HN acts as a backup server, runs Nagios for health monitoring, has a webserver for managing the 3ware RAID controller, etc. The VEs are leased to customers, who can't entirely be trusted, especially if they get hacked. As such, our scenario is one in which the HN must be protected from all access (even from the VEs) except for a few trusted hosts (e.g. my home-office).

The exception to this is the nameserver, which we want open to the world. We use it as a caching nameserver for our VEs and also to host DNS for a few customer domain.

== Setting up a HN-based firewall ==

This setup emulates (to the VEs anyway) an external hardware firewall. It protects the HN from any access and then defines what services and ports are allowed/banned for individual VEs. This leaves the firewall controlled by the site administrator, not be individual VEs and the hackers who've gotten into them. ;)

First off, let's disable Fedora's existing <code>iptables</code> service:
<pre>
service iptables off
chkconfig iptables off
</pre>

Now create the new <code>firewall</code> service. This code should be <code>/etc/init.d/firewall</code> and then should be chmod'd 755.
<pre>
#!/bin/sh
# firewall Start iptables firewall
# chkconfig: 2345 08 92
# description: Starts, stops and saves iptables firewall
# This script sets up the firewall for the INPUT chain (which is for the HN itself)
# and then processes the config files under /etc/firewall.d to set up additional rules
# in the FORWARD chain to allow access to VEs' services.

. /etc/init.d/functions

# the IP block allocated to this server
SEGMENT="192.168.0.0/24"
# the IP used by the hosting server itself
THISHOST="192.168.0.1"
# services that should be allowed to the HN; services for VEs are configured in /etc/firewall.d/*
OKPORTS="53"
# hosts allowed full access through the firewall, to all VEs and to this server
DMZS="12.34.56.78 90.123.45.67"

purge() {
echo -n "Firewall: Purging and allowing all traffic"
iptables -P OUTPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P INPUT ACCEPT
iptables -F
success ; echo
}

setup() {
echo -n "Firewall: Setting default policies to DROP"
iptables -P INPUT DROP
iptables -P FORWARD DROP
iptables -I INPUT -j ACCEPT -m state --state ESTABLISHED,RELATED
iptables -I FORWARD -j ACCEPT -m state --state ESTABLISHED,RELATED
iptables -I INPUT -j ACCEPT -i lo
iptables -I FORWARD -j ACCEPT --source $SEGMENT
success ; echo

echo "Firewall: Allowing access to HN"
for port in $OKPORTS ; do
echo -n " port $port"
iptables -I INPUT -j ACCEPT -s $SEGMENT -d $THISHOST --protocol tcp --destination-port $port
iptables -I INPUT -j ACCEPT -s $SEGMENT -d $THISHOST --protocol udp --destination-port $port
success ; echo
done
for ip in $DMZS ; do
echo -n " DMZ $ip"
iptables -I INPUT -i eth0 -j ACCEPT -s $ip
iptables -I FORWARD -i eth0 -j ACCEPT -s $ip
success ; echo
done

VESETUPS=`echo /etc/firewall.d/*`
if [ "$VESETUPS" != "/etc/firewall.d/*" ] ; then
echo "Firewall: Setting up VE firewalls"
for i in $VESETUPS ; do
. $i
echo -n " $VENAME VE$VEID"
if [ -n "$BANNED" ]; then
for source in $BANNED ; do iptables -I FORWARD -j DROP --destination $VEIP --source $source ; done
fi
if [ -n "$OPENPORTS" ]; then
for port in $OPENPORTS ; do iptables -I FORWARD -j ACCEPT --protocol tcp --destination $VEIP --destination-port $port ; done
for port in $OPENPORTS ; do iptables -I FORWARD -j ACCEPT --protocol udp --destination $VEIP --destination-port $port ; done
fi
if [ -n "$DMZS" ]; then
for source in $DMZS ; do iptables -I FORWARD -j ACCEPT --protocol tcp --destination $VEIP --source $source ; done
for source in $DMZS ; do iptables -I FORWARD -j ACCEPT --protocol udp --destination $VEIP --source $source ; done
fi
[ $? -eq 0 ] && success || failure
echo
done
fi
}

case "$1" in
start)
echo "Starting firewall..."
purge
setup
;;
stop)
echo "Stopping firewall..."
purge
;;
restart)
$0 stop
$0 start
;;
status)
iptables -n -L
;;
*)
echo "Usage: $0 <start|stop|restart|status>"
;;
esac
</pre>

The above script can be called like this:
<pre>
service firewall start
service firewall stop
service firewall restart
service firewall status
</pre>

It will set up the firewall for the HN according to the parameters you specified for OKPORTS, DMZs, etc. and then it will call each file under /etc/firewall.d and process its configuration.

So create a file under /etc/firewall.d The exact filename isn't important, as long as it's meaningful to you, e.g. <pre>ExampleCompany</pre> or <pre>ve12</pre> and give it content like this:

<pre>
# This file is processed by /etc/init.d/firewall
VEID="1" # the VE's ID#
VENAME="Customer1" # A human-friendly label for the VE
VEIP="192.168.1.34" # the IP address for this VE
OPENPORTS="80 443" # ports that should be universally opened to the entire Internet
DMZS="1.2.3.0/24 5.6.7.8/32" # IPs and blocks that should have full access to the VE's services
BANNED="" # IPs and blocks that should be entirely blocked from the VE's services
</pre>

And there you go. Go ahead and start the firewall and check its status:
<pre>
service firewall restart
service firewall status
</pre>

As you can see, you can now add and edit the configurations for individual VEs very easily. This method proves a lot easier to manage than Fedora's iptables-config mechamism!

== Setting up a firewall that allows per-VE configuration ==

This setup configures iptables on the HN to disallow access to all hosts, including the VEs. However, it allows all traffic into the VEs so they may define their own iptables rules and therefore manage their own firewall.

<code>This content is missing. You are invited to fill it in, if you get to it before I do. :)</code>

== See Also ==

[[Traffic accounting with iptables]]

[[ Category: Networking ]]
[[ Category: Firewalls ]]

Setting up an iptables firewall

2007-05-16T17:07:19Z

HostGIS:

This document consists of two parts. The first is setting up a firewall (using iptables) on the HN, which will restrict traffic to the VEs. The effect would emulate, as far as the VEs and their customers are concerned, an external hardware firewall controlled by the sysadmin. The second is setting up a firewall that protects the HN itself but still allows traffic to the VEs, thus allowing individual VEs to define their own iptables.

While the firewalls shown here can be accomplished using iptables manually (or using Fedora core's iptables service), the methods presented here are especially modular and easy to modify. This is important when you have 20+ VEs and a lot of other things to be doing...

The scripts and pathnames given here are for Fedora Core 6, though they can probably be applied to most similar SysV-like systems with little modification.

== A little background ==

On our systems, we use the HN to provide privileged services which are not appropriate for access by the VEs. For example, the HN acts as a backup server, runs Nagios for health monitoring, has a webserver for managing the 3ware RAID controller, etc. The VEs are leased to customers, who can't entirely be trusted, especially if they get hacked. As such, our scenario is one in which the HN must be protected from all access (even from the VEs) except for a few trusted hosts (e.g. my home-office).

== Setting up a HN-based firewall ==

This setup emulates (to the VEs anyway) an external hardware firewall. It protects the HN from any access and then defines what services and ports are allowed/banned for individual VEs. This leaves the firewall controlled by the site administrator, not be individual VEs and the hackers who've gotten into them. ;)

First off, let's disable Fedora's existing <code>iptables</code> service:
<pre>
service iptables off
chkconfig iptables off
</pre>

Now create the new <code>firewall</code> service. This code should be <code>/etc/init.d/firewall</code> and then should be chmod'd 755.
<pre>
#!/bin/sh
# firewall Start iptables firewall
# chkconfig: 2345 08 92
# description: Starts, stops and saves iptables firewall
# This script sets up the firewall for the INPUT chain (which is for the HN itself)
# and then processes the config files under /etc/firewall.d to set up additional rules
# in the FORWARD chain to allow access to VEs' services.

. /etc/init.d/functions

# the IP block allocated to this server
SEGMENT="192.168.0.0/24"
# the IP used by the hosting server itself
THISHOST="192.168.0.1"
# services that should be allowed to the HN; services for VEs are configured in /etc/firewall.d/*
OKPORTS="53"
# hosts allowed full access through the firewall, to all VEs and to this server
DMZS="12.34.56.78 90.123.45.67"

purge() {
echo -n "Firewall: Purging and allowing all traffic"
iptables -P OUTPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P INPUT ACCEPT
iptables -F
success ; echo
}

setup() {
echo -n "Firewall: Setting default policies to DROP"
iptables -P INPUT DROP
iptables -P FORWARD DROP
iptables -I INPUT -j ACCEPT -m state --state ESTABLISHED,RELATED
iptables -I FORWARD -j ACCEPT -m state --state ESTABLISHED,RELATED
iptables -I INPUT -j ACCEPT -i lo
iptables -I FORWARD -j ACCEPT --source $SEGMENT
success ; echo

echo "Firewall: Allowing access to HN"
for port in $OKPORTS ; do
echo -n " port $port"
iptables -I INPUT -j ACCEPT -s $SEGMENT -d $THISHOST --protocol tcp --destination-port $port
iptables -I INPUT -j ACCEPT -s $SEGMENT -d $THISHOST --protocol udp --destination-port $port
success ; echo
done
for ip in $DMZS ; do
echo -n " DMZ $ip"
iptables -I INPUT -i eth0 -j ACCEPT -s $ip
iptables -I FORWARD -i eth0 -j ACCEPT -s $ip
success ; echo
done

VESETUPS=`echo /etc/firewall.d/*`
if [ "$VESETUPS" != "/etc/firewall.d/*" ] ; then
echo "Firewall: Setting up VE firewalls"
for i in $VESETUPS ; do
. $i
echo -n " $VENAME VE$VEID"
if [ -n "$BANNED" ]; then
for source in $BANNED ; do iptables -I FORWARD -j DROP --destination $VEIP --source $source ; done
fi
if [ -n "$OPENPORTS" ]; then
for port in $OPENPORTS ; do iptables -I FORWARD -j ACCEPT --protocol tcp --destination $VEIP --destination-port $port ; done
for port in $OPENPORTS ; do iptables -I FORWARD -j ACCEPT --protocol udp --destination $VEIP --destination-port $port ; done
fi
if [ -n "$DMZS" ]; then
for source in $DMZS ; do iptables -I FORWARD -j ACCEPT --protocol tcp --destination $VEIP --source $source ; done
for source in $DMZS ; do iptables -I FORWARD -j ACCEPT --protocol udp --destination $VEIP --source $source ; done
fi
[ $? -eq 0 ] && success || failure
echo
done
fi
}

case "$1" in
start)
echo "Starting firewall..."
purge
setup
;;
stop)
echo "Stopping firewall..."
purge
;;
restart)
$0 stop
$0 start
;;
status)
iptables -n -L
;;
*)
echo "Usage: $0 <start|stop|restart|status>"
;;
esac
</pre>

The above script can be called like this:
<pre>
service firewall start
service firewall stop
service firewall restart
service firewall status
</pre>

It will set up the firewall for the HN according to the parameters you specified for OKPORTS, DMZs, etc. and then it will call each file under /etc/firewall.d and process its configuration.

So create a file under /etc/firewall.d The exact filename isn't important, as long as it's meaningful to you, e.g. <pre>ExampleCompany</pre> or <pre>ve12</pre> and give it content like this:

<pre>
# This file is processed by /etc/init.d/firewall
VEID="1" # the VE's ID#
VENAME="Customer1" # A human-friendly label for the VE
VEIP="192.168.1.34" # the IP address for this VE
OPENPORTS="80 443" # ports that should be universally opened to the entire Internet
DMZS="1.2.3.0/24 5.6.7.8/32" # IPs and blocks that should have full access to the VE's services
BANNED="" # IPs and blocks that should be entirely blocked from the VE's services
</pre>

And there you go. Go ahead and start the firewall and check its status:
<pre>
service firewall restart
service firewall status
</pre>

As you can see, you can now add and edit the configurations for individual VEs very easily. This method proves a lot easier to manage than Fedora's iptables-config mechamism!

== Setting up a firewall that allows per-VE configuration ==

This setup configures iptables on the HN to disallow access to all hosts, including the VEs. However, it allows all traffic into the VEs so they may define their own iptables rules and therefore manage their own firewall.

<code>This content is missing. You are invited to fill it in, if you get to it before I do. :)</code>

== See Also ==

[[Traffic accounting with iptables]]

[[ Category: Networking ]]
[[ Category: Firewalls ]]

Setting up an iptables firewall

2007-05-16T15:52:31Z

HostGIS:

Setting up an iptables firewall

2007-05-16T15:48:33Z

HostGIS: New page: This document consists of two parts. The first is setting up a firewall (using iptables) on the HN, which will restrict traffic to the VEs. The effect would emulate, as far as the VEs and ...

Virtual Ethernet device

2007-05-09T06:55:23Z

HostGIS: /* Making a veth-device persistent */

'''Virtual ethernet device''' is an ethernet-like device which can be used inside a [[VE]]. Unlike
[[venet]] network device, veth device has a MAC address. Due to this, it can be used in configurations, when veth is bridged to ethX or other device and VE user fully sets up his networking himself,
including IPs, gateways etc.

Virtual ethernet device consist of two ethernet devices - one in [[VE0]] and another one
in VE. These devices are connected to each other, so if a packet goes to one
device it will come out from the other device.

== Virtual ethernet device usage ==

=== Kernel module ===
First of all, make sure the <code>vzethdev</code> module is loaded:
<pre>
# lsmod | grep vzeth
vzethdev 8224 0
vzmon 35164 5 vzethdev,vznetdev,vzrst,vzcpt
vzdev 3080 4 vzethdev,vznetdev,vzmon,vzdquota
</pre>

In case it is not loaded, load it:
<pre>
# modprobe vzethdev
</pre>

You might want to add the module to <code>/etc/init.d/vz script</code>, so it will be loaded during startup.

{{Note|since vzctl version 3.0.11, vzethdev is loaded by /etc/init.d/vz}}

=== Adding veth to a VE ===

{{Note|Use random MAC addresses. Do not use MAC addresses of real eth devices, because this can lead to collisions and MAC addresses must be entered in XX:XX:XX:XX:XX:XX format.}}

==== syntax vzctl version < 3.0.14 ====

<pre>
vzctl set <VEID> --veth_add <dev_name>,<dev_addr>,<ve_dev_name>,<ve_dev_addr>
</pre>

Here
* <tt>dev_name</tt> is the ethernet device name that you are creating on the [[VE0|host system]]
* <tt>dev_addr</tt> is its MAC address
* <tt>ve_dev_name</tt> is the corresponding ethernet device name you are creating on the VE
* <tt>ve_dev_addr</tt> is its MAC address

{{Note| that this option is incremental, so devices are added to already existing ones.}}

NB there are no spaces after the commas

Example:

<pre>
vzctl set 101 --veth_add veth101.0,00:12:34:56:78:9A,eth0,00:12:34:56:78:9B --save
</pre>
After executing this command <tt>veth</tt> device will be created for VE 101 and veth configuration will be saved to a VE configuration file.
Host-side ethernet device will have <tt>veth101.0</tt> name and <tt>00:12:34:56:78:9A</tt> MAC address.
VE-side ethernet device will have <tt>eth0</tt> name and <tt>00:12:34:56:78:9B</tt> MAC address.

==== syntax vzctl version >= 3.0.14 ====

Read Update infos about [http://openvz.org/news/updates/vzctl-3.0.14-1 vzctl 3.0.14]

<pre>
vzctl set <VEID> --netif_add <ifname>[,<mac>,<host_ifname>,<host_mac]
</pre>

Here
* <tt>ifname</tt> is the ethernet device name in the VE
* <tt>mac</tt> is its MAC address in the VE
* <tt>host_ifname</tt> is the ethernet device name on the host ([[VE0]])
* <tt>host_mac</tt> is its MAC address on the host ([[VE0]])

{{Note|All parameters except ifname are optional and are automatically generated if not specified.}}

Example:

<pre>
vzctl set 101 --netif_add eth0,00:12:34:56:78:9A,veth101.0,00:12:34:56:78:9B --save
</pre>

=== Removing veth from a VE ===

==== syntax vzctl version < 3.0.14 ====

<pre>
vzctl set <VEID> --veth_del <dev_name>
</pre>
Here <tt>dev_name</tt> is the ethernet device name in the [[VE0|host system]].

Example:
<pre>
vzctl set 101 --veth_del veth101.0 --save
</pre>
After executing this command veth device with host-side ethernet name veth101.0 will be removed from VE 101 and veth configuration will be updated in VE config file.

==== syntax vzctl version >= 3.0.14 ====
<pre>
vzctl set <VEID> --netif_del <dev_name>|all
</pre>

Here
* <code>dev_name</code> is the ethernet device name in the [[VE]].

{{Note|If you want to remove all ethernet devices in VE, use <code>all</code>.}}

Example:

<pre>
vzctl set 101 --netif_del eth0 --save
</pre>

== Common configurations with virtual ethernet devices ==
Module <tt>vzethdev</tt> must be loaded to operate with veth devices.

=== Simple configuration with virtual ethernet device ===

==== Start a VE ====
<pre>
[host-node]# vzctl start 101
</pre>

==== Add veth device to VE ====
<pre>
[host-node]# vzctl set 101 --veth_add veth101.0,00:12:34:56:78:9A,eth0,00:12:34:56:78:9B --save
</pre>

==== Configure devices in VE0 ====
<pre>
[host-node]# ifconfig veth101.0 0
[host-node]# echo 1 > /proc/sys/net/ipv4/conf/veth101.0/forwarding
[host-node]# echo 1 > /proc/sys/net/ipv4/conf/veth101.0/proxy_arp
[host-node]# echo 1 > /proc/sys/net/ipv4/conf/eth0/forwarding
[host-node]# echo 1 > /proc/sys/net/ipv4/conf/eth0/proxy_arp
</pre>

==== Configure device in VE ====
<pre>
[host-node]# vzctl enter 101
[ve-101]# /sbin/ifconfig eth0 0
[ve-101]# /sbin/ip addr add 192.168.0.101 dev eth0
[ve-101]# /sbin/ip route add default dev eth0
</pre>

==== Add route in [[VE0]] ====
<pre>
[host-node]# ip route add 192.168.0.101 dev veth101.0
</pre>

=== Virtual ethernet device with IPv6 ===

==== Start [[VE]] ====
<pre>
[host-node]# vzctl start 101
</pre>

==== Add veth device to [[VE]] ====
<pre>
[host-node]# vzctl set 101 --veth_add veth101.0,00:12:34:56:78:9A,eth0,00:12:34:56:78:9B --save
</pre>

==== Configure devices in [[VE0]] ====
<pre>
[host-node]# ifconfig veth101.0 0
[host-node]# echo 1 > /proc/sys/net/ipv6/conf/veth101.0/forwarding
[host-node]# echo 1 > /proc/sys/net/ipv6/conf/eth0/forwarding
[host-node]# echo 1 > /proc/sys/net/ipv6/conf/all/forwarding
</pre>

==== Configure device in [[VE]] ====
<pre>
[host-node]# vzctl enter 101
[ve-101]# /sbin/ifconfig eth0 0
</pre>

==== Start router advertisement daemon (radvd) for IPv6 in VE0 ====
First you need to edit radvd configuration file. Here is a simple example of <tt>/etc/radv.conf</tt>:
<pre>
interface veth101.0
{
AdvSendAdvert on;
MinRtrAdvInterval 3;
MaxRtrAdvInterval 10;
AdvHomeAgentFlag off;

prefix 3ffe:2400:0:0::/64
{
AdvOnLink on;
AdvAutonomous on;
AdvRouterAddr off;
};
};

interface eth0
{
AdvSendAdvert on;
MinRtrAdvInterval 3;
MaxRtrAdvInterval 10;
AdvHomeAgentFlag off;

prefix 3ffe:0302:0011:0002::/64
{
AdvOnLink on;
AdvAutonomous on;
AdvRouterAddr off;
};
};
</pre>

Then, start radvd:
<pre>
[host-node]# /etc/init.d/radvd start
</pre>

==== Add IPv6 addresses to devices in [[VE0]] ====
<pre>
[host-node]# ip addr add dev veth101.0 3ffe:2400::212:34ff:fe56:789a/64
[host-node]# ip addr add dev eth0 3ffe:0302:0011:0002:211:22ff:fe33:4455/64
</pre>

=== Virtual ethernet devices can be joined in one bridge ===
Perform steps 1 - 4 from Simple configuration chapter for several VEs and/or veth devices

==== Create bridge device ====
<pre>
[host-node]# brctl addbr vzbr0
</pre>

==== Add veth devices to bridge ====
<pre>
[host-node]# brctl addif vzbr0 veth101.0
...
[host-node]# brctl addif vzbr0 veth101.n
[host-node]# brctl addif vzbr0 veth102.0
...
...
[host-node]# brctl addif vzbr0 vethXXX.N
</pre>

==== Configure bridge device ====
<pre>
[host-node]# ifconfig vzbr0 0
[host-node]# echo 1 > /proc/sys/net/ipv4/conf/vzbr0/forwarding
[host-node]# echo 1 > /proc/sys/net/ipv4/conf/vzbr0/proxy_arp
</pre>

==== Add routes in [[VE0]] ====
<pre>
[host-node]# ip route add 192.168.101.1 dev vzbr0
...
[host-node]# ip route add 192.168.101.n dev vzbr0
[host-node]# ip route add 192.168.102.1 dev vzbr0
...
...
[host-node]# ip route add 192.168.XXX.N dev vzbr0
</pre>

Thus you'll have more convinient configuration, i.e. all routes to VEs will be through this bridge and VEs can communicate with each other even without these routes.

=== Making a veth-device persistent ===

At the moment, it is not possible to have the commands needed for a persistent veth being made automatically be vzctl. A bugreport ( http://bugzilla.openvz.org/show_bug.cgi?id=301 ) has already been made. Until then, here's a way to make the above steps persistent.

1. First, edit the VE's configuration to specify what the veth's IP address(es) should be, and to indicate that a custom script should be run when starting up a VE.
* Open up /etc/vz/conf/VEID.conf
* Comment out any IP_ADDRESS entries to prevent a VENET-device from being created in the VE
* Add or change the entry CONFIG_CUSTOMIZED="yes"
* Add an entry VETH_IP_ADDRESS="<VE IP>" The VE IP can have multiple IPs, separated by spaces

2. Now to create that "custom script". The following helper script will check the configuration file for IP addresses and for the veth interface, and configure the IP routing accordingly. Create the script /usr/sbin/vznetaddroute to have the following, and then <code>chmod 0500 /usr/sbin/vznetaddroute</code> to make it executable.

<pre>
#!/bin/bash
# /usr/sbin/vznetaddroute
# a script to bring up bridged network interfaces (veth's) in a VE

CONFIGFILE=/etc/vz/conf/$VEID.conf
. $CONFIGFILE
VZHOSTIF=`echo $NETIF |sed 's/^.*host_ifname=$.*$,.*$/\1/g'`

if [ ! -n "$VETH_IP_ADDRESS" ]; then
echo "According to $CONFIGFILE VE$VEIDI has no veth IPs configured."
exit 1
fi

if [ ! -n "$VZHOSTIF" ]; then
echo "According to $CONFIGFILE VE$VEIDI has no veth interface configured."
exit 1
fi

for IP in $VETH_IP_ADDRESS; do
echo "Adding interface $VZHOSTIF and route $IP for VE$VEID to VE0"
/sbin/ifconfig $VZHOSTIF 0
echo 1 > /proc/sys/net/ipv4/conf/$VZHOSTIF/proxy_arp
echo 1 > /proc/sys/net/ipv4/conf/$VZHOSTIF/forwarding
/sbin/ip route add $IP dev $VZHOSTIF
done

exit 0
</pre>

3. Now create /etc/vz/vznet.conf containing the following. This is what defines the "custom script" as being the vznetaddroute which you just created.

<pre>
#!/bin/bash
EXTERNAL_SCRIPT="/usr/sbin/vznetaddroute"
</pre>

4. Of course, the VE's operating system will need to be configured with those IP address(es) as well. Consult the manual for your VE's OS for details.

That's it! At this point, when you restart the VE you should see a new line in the output, indicating that the interface is being configured and a new route being added. And you should be able to ping the host, and to enter the VE and use the network.

=== Virtual ethernet devices + VLAN ===
This configuration can be done by adding vlan device to the previous configuration.

== See also ==
* [[Virtual network device]]
* [[Differences between venet and veth]]

== External links ==
* [http://www.tldp.org/HOWTO/Linux+IPv6-HOWTO/hints-daemons-radvd.html Linux IPv6 HOWTO, a chapter about radvd]

[[Category: Networking]]
[[Category: HOWTO]]

Postgresql and shared memory

2007-04-16T20:04:08Z

HostGIS: page created, prompted by Kirill's email

I tried running PostgreSQL as one of my first experiments, and hit upon some nuances of shared memory, which may be helpful to relate here.

One of the easiest ways to increase Postgres's performance is to turn up the shared_buffers parameter in the postgresql.conf file; this is basically the amount of shared memory which the postmaster will use for buffering everything: table data, indexes, etc. The default value is really small, and if you have RAM to spare you may want to crank it up to 128MB (that's 16384 shared buffers, for pgsql 8.1 and earlier) or even more. But if it then complains that it couldn't allocate the shared memory, this page explains why.

PostgreSQL uses shared memory. As such, you have to check two things about the VE.

1. UBC's shmpages setting for this VE. This dictates how many pages (8k apiece) are available to the VE, e.g. shmpages=16384 gives a limit of 128 MB of shared memory.

2. "/sbin/sysctl kernel.shmmax" This is the VE's self-imposed limit on how much shared memory may be allocated in a single request.

Get that? Not only does the HN impose a limit on the VE's total shared memory usage, but the VE itself has a setting for the maximum amount in a single chunk.

Some other notes:

* Note that shared memory is part of the VE's overall memory usage. It is not a second memory pool.

* Note that processes other than PostgreSQL may be using shared memory, and also that shmpages includes stuff other than IPC shared memory (tmpfs, shmem, etc) So don't set shmpages to exactly the amount you want to give Postgres.

* The sysctl kernel.shmmax value set in the HN/VE0 applies only to the HN, not to VEs.