Difference between revisions of "Shared webhosting"

From OpenVZ Virtuozzo Containers Wiki
Jump to: navigation, search
m (Robot: Automated text replacement (-VE ID +CT ID))
m (Protected "Shared webhosting": Excessive spamming ([edit=autoconfirmed] (indefinite) [move=autoconfirmed] (indefinite)))
 
(25 intermediate revisions by 13 users not shown)
Line 1: Line 1:
<pre>This document describes creating a "secure shared web hosting service" on « HN (Host Node) » It is « NOT » about per VE shared web hosting.</pre>
+
This document describes creating a "secure shared web hosting service" on [[HN]] (Hardware Node). It is '''NOT''' about per container shared web hosting.
  
 
{{roughstub}}
 
{{roughstub}}
 
  
 
== The problem ==
 
== The problem ==
  
One of the problems with shared webhosting (i.e. different people with each his/her own webpages) is that modern script languages such as PHP, Python, or Perl are too powerful. For example take the following PHP script:
+
One of the problems with shared web hosting (i.e. different people with each his/her own webpages) is that modern script languages such as PHP, Python, or Perl are too powerful. For example take the following PHP script:
  
 
<pre>
 
<pre>
Line 34: Line 33:
 
=== Minimal server ===
 
=== Minimal server ===
  
Create an VEx with your favorite distro. Give it an internal IP-address in one of the ranges 10.0.0.0/8, 172.16.0.0/12 or 192.168.0.0/16. Then strip away all unnecessary init.d scripts so only the bare minimum is started. That means as a minimum syslogd and ssh so the account holder can upload his/her files through SCP/SFTP in his/her own minimal server. For this to work you need to set up [[Using NAT for VE with private IPs|destination NAT on VE0]] from high numbered ports to port 22 on the given private IP address:
+
Create an CTx with your favorite distro. Give it an internal IP-address in one of the ranges 10.0.0.0/8, 172.16.0.0/12 or 192.168.0.0/16. Then strip away all unnecessary init.d scripts so only the bare minimum is started. That means as a minimum syslogd and ssh so the account holder can upload his/her files through SCP/SFTP in his/her own minimal server. For this to work you need to set up [[Using NAT for container with private IPs|destination NAT on CT0]] from high numbered ports to port 22 on the given private IP address:
  
 
<pre>
 
<pre>
Line 49: Line 48:
 
=== MySQL server ===
 
=== MySQL server ===
  
Most webhosting accounts use MySQL, but if you prefer another database server, go ahead. Create a new VEx with a lot more resources and again an internal IP-address. Now configure the accounts. As an extra security measure you can use the internal IP-address as well.
+
Most webhosting accounts use MySQL, but if you prefer another database server, go ahead. Create a new CTx with a lot more resources and again an internal IP-address. Now configure the accounts. As an extra security measure you can use the internal IP-address as well.
  
 
==== MySQL socket sharing ====
 
==== MySQL socket sharing ====
  
You can also share the socket of an VEx running MySQL, which is alot faster than TCP/IP.
+
You can also share the socket of an CTx running MySQL, which is alot faster than TCP/IP.
 
e.g.:
 
e.g.:
 
<pre>
 
<pre>
Line 68: Line 67:
 
==== Refreshing links to MySQL socket ====
 
==== Refreshing links to MySQL socket ====
  
Sharing the MySQL socket works really well until the MySQL database is restarted or the VE running MySQL is restarted.  When this happens, the socket file is removed and recreated.  In most cases, a different inode will be used, causing existing hard links to the mysql.sock file to no longer work.  The solution is to relink these sockets.  
+
Sharing the MySQL socket works really well until the MySQL database is restarted or the container running MySQL is restarted.  When this happens, the socket file is removed and recreated.  In most cases, a different inode will be used, causing existing hard links to the mysql.sock file to no longer work.  The solution is to relink these sockets.  
  
There are more elegant solutions to this problem, but the following script is a decent hack that can be run via a cron job every minute or two.  It will loop through all VEs between START_VEID and STOP_VEID and make sure that the links point to the correct socket.  If they do not, the link will be recreated.
+
There are more elegant solutions to this problem, but the following script is a decent hack that can be run via a cron job every minute or two.  It will loop through all containers between START_CTID and STOP_CTID and make sure that the links point to the correct socket.  If they do not, the link will be recreated.
  
 
<pre>
 
<pre>
Line 81: Line 80:
 
###################################
 
###################################
  
# Location of private VEs:
+
# Location of private containers:
 
PRIVATE=/vz/private
 
PRIVATE=/vz/private
  
# Starting CT ID.  VEIDs with this ID or greater will have mysql.sock link created
+
# Starting CT ID.  CTIDs with this ID or greater will have mysql.sock link created
START_VEID=1001
+
START_CTID=1001
  
# Stopping CT ID.  VEIDs with this ID or less will have mysql.sock link created
+
# Stopping CT ID.  CTIDs with this ID or less will have mysql.sock link created
STOP_VEID=2000
+
STOP_CTID=2000
  
 
# Shared Mysql CT ID:
 
# Shared Mysql CT ID:
MYSQL_VEID=201
+
MYSQL_CTID=201
  
 
# Location of mysql socket file
 
# Location of mysql socket file
Line 111: Line 110:
  
 
# Full path to socket
 
# Full path to socket
MYSQL_SOCK_FILE=${PRIVATE}/${MYSQL_VEID}${MYSQL_SOCK_DIR}/${MYSQL_SOCK}
+
MYSQL_SOCK_FILE=${PRIVATE}/${MYSQL_CTID}${MYSQL_SOCK_DIR}/${MYSQL_SOCK}
  
 
[ $QUIET -eq 0 ] && echo
 
[ $QUIET -eq 0 ] && echo
Line 119: Line 118:
 
oldDirectory=`pwd`
 
oldDirectory=`pwd`
  
# Check to see if MySQL VE socket exists
+
# Check to see if MySQL container socket exists
 
if [ -S "${MYSQL_SOCK_FILE}" ]; then
 
if [ -S "${MYSQL_SOCK_FILE}" ]; then
  
         # Get inode of MySQL VE socket
+
         # Get inode of MySQL container socket
 
         mysql_inode=`ls -i ${MYSQL_SOCK_FILE} | awk '{ print $1;}'`
 
         mysql_inode=`ls -i ${MYSQL_SOCK_FILE} | awk '{ print $1;}'`
  
         # Search through VEs
+
         # Search through containers
 
         cd $PRIVATE
 
         cd $PRIVATE
 
         for i in * ; do
 
         for i in * ; do
                 # The current VE to process
+
                 # The current container to process
 
                 veid=$i
 
                 veid=$i
  
                 # Check if VE should be processed
+
                 # Check if container should be processed
                 if [ $veid -ne $MYSQL_VEID -a $veid -ge $START_VEID -a $veid -le $STOP_VEID ]; then
+
                 if [ $veid -ne $MYSQL_CTID -a $veid -ge $START_CTID -a $veid -le $STOP_CTID ]; then
  
                         # Get this VE's socket
+
                         # Get this container's socket
 
                         vesock=${PRIVATE}/${veid}${MYSQL_SOCK_DIR}/${MYSQL_SOCK}
 
                         vesock=${PRIVATE}/${veid}${MYSQL_SOCK_DIR}/${MYSQL_SOCK}
  
Line 140: Line 139:
 
                         mkdir -p ${PRIVATE}/${veid}${MYSQL_SOCK_DIR}
 
                         mkdir -p ${PRIVATE}/${veid}${MYSQL_SOCK_DIR}
  
                         # Check to see if this VE has a socket already
+
                         # Check to see if this container has a socket already
 
                         if [ -S "${vesock}" ]; then
 
                         if [ -S "${vesock}" ]; then
                                 # Get inode of this VE socket
+
                                 # Get inode of this container socket
 
                                 ve_inode=`ls -i ${vesock} | awk '{ print $1;}'`
 
                                 ve_inode=`ls -i ${vesock} | awk '{ print $1;}'`
  
Line 193: Line 192:
 
=== Proxy webserver ===
 
=== Proxy webserver ===
  
Because we have only one public IP-address, we need an trick to access every minimal server based on the hostname in the HTTP request. For SSH we used different ports, but that is not an option for websites. Again we create an VEx with an internal IP-address. On this server we install Lighttpd as well, because the proxying is very simple. First we must forward port 80 to this server:
+
Because we have only one public IP-address, we need an trick to access every minimal server based on the hostname in the HTTP request. For SSH we used different ports, but that is not an option for websites. Again we create an CTx with an internal IP-address. On this server we install Lighttpd as well, because the proxying is very simple. First we must forward port 80 to this server:
  
 
<pre>
 
<pre>
Line 235: Line 234:
 
=== Other applications ===
 
=== Other applications ===
  
Create for other applications as mail, make sure that the minimal servers use this one for sending mail from webpages, DNS etc. VEx as needed. The resulting server is shown in the figure above.
+
Create for other applications as mail, make sure that the minimal servers use this one for sending mail from webpages, DNS etc. CTx as needed. The resulting server is shown in the figure above.  
 +
 
 +
OpenVZ can easily be installed on most Dedicated Server Hosting Services
  
 
[[Category:HOWTO]]
 
[[Category:HOWTO]]

Latest revision as of 06:31, 23 October 2011

This document describes creating a "secure shared web hosting service" on HN (Hardware Node). It is NOT about per container shared web hosting.

The problem

One of the problems with shared web hosting (i.e. different people with each his/her own webpages) is that modern script languages such as PHP, Python, or Perl are too powerful. For example take the following PHP script:

<?php

function get_content($filename) {
  $handle = fopen($filename, 'r');
  echo fread($handle, filesize($filename));
  fclose($handle);
}

get_content('/home/ppuk34/www/forum/config.inc.php');

?>

With PHP you could use open_basedir to prevent this, but there are more ways. For example PHP Shell, a script that is often mis-used by people with not-so-good intentions. Or think about the Santy-worm which mis-used phpBB. Again there is a solution in the form of safe_mode, but lots of PHP scripts break if you enable this. For Python, Perl, or CGI-scripts there are no easy ways and you have to use wrappers or other tricks to chroot these.

The solution

The OpenVZ way of shared webhosting

You can waste hours of time in securing all the possible things you don't want in your shared webhosting environment. And unless you are very familiar with all the things modern scripting languages can do, you probably miss dozens of alternative routes. In this process you frustrate your clients, because security always means that legitimate things break. As a side effect of your hard work, you can waste hours of extra time in educating your users. But in the end most users don't care about security, unless they are themselves victims of a compromised host. Learning the hard way is by far the most effective method. One possible solution is dedicated webhosting, but most users don't have the experience to maintain a server or it is way to expensive for them.

The main problem with shared webhosting is that by its very nature all files which are served through the web are public. Apache for example uses only one account to read all files. As said, you can use tricks with CGI wrappers to execute the scripting languages under its own credentials. However this kind of security depends on the wrappers ability to securely separate the users. We all know that if this is broken — and most often it will be broken — the result is a higher clearance on the underlying filesystem. For most systems you need more than one wrapper, so the number of possible security problems grow. The ultimate user separation is in the kernel and you can view the modifications OpenVZ has done in this light. Instead of CGI wrappers we go one step higher and give every user its own minimal server. In the rest of this article we describe how shared webhosting with OpenVZ could be implemented.

Minimal server

Create an CTx with your favorite distro. Give it an internal IP-address in one of the ranges 10.0.0.0/8, 172.16.0.0/12 or 192.168.0.0/16. Then strip away all unnecessary init.d scripts so only the bare minimum is started. That means as a minimum syslogd and ssh so the account holder can upload his/her files through SCP/SFTP in his/her own minimal server. For this to work you need to set up destination NAT on CT0 from high numbered ports to port 22 on the given private IP address:

dnat="-j DNAT --to-destination"

iptables -t nat -P PREROUTING ACCEPT
iptables -t nat -A PREROUTING -p TCP --dport 10122 $dnat 192.168.13.101:22
iptables -t nat -A PREROUTING -p TCP --dport 10222 $dnat 192.168.13.102:22
...

The other thing you want for webhosting is of course a webserver as well. To minimize the amount of needed memory, we choose Lighttpd instead of the common Apache. Then configure the scripting language of your choice to run under this webserver. It is possible to use different languages/setups for different accounts as well. Also problematic CGI-scripts are not problematic anymore...

MySQL server

Most webhosting accounts use MySQL, but if you prefer another database server, go ahead. Create a new CTx with a lot more resources and again an internal IP-address. Now configure the accounts. As an extra security measure you can use the internal IP-address as well.

MySQL socket sharing

You can also share the socket of an CTx running MySQL, which is alot faster than TCP/IP. e.g.:

ln /var/lib/vz/private/101/var/run/mysqld/mysqld.sock /var/lib/vz/private/102/var/run/mysqld/mysqld.sock

Inside of 102:

$ mysql -u root -p -S /var/run/mysqld/mysqld.sock
Enter password:
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 9 to server version: 5.0.32-Debian_7etch1-log

Refreshing links to MySQL socket

Sharing the MySQL socket works really well until the MySQL database is restarted or the container running MySQL is restarted. When this happens, the socket file is removed and recreated. In most cases, a different inode will be used, causing existing hard links to the mysql.sock file to no longer work. The solution is to relink these sockets.

There are more elegant solutions to this problem, but the following script is a decent hack that can be run via a cron job every minute or two. It will loop through all containers between START_CTID and STOP_CTID and make sure that the links point to the correct socket. If they do not, the link will be recreated.

#!/bin/sh

# Created by Tauren Mills (tauren at tauren dot com) 2007-11-15

###################################
# Start of Configuration settings #
###################################

# Location of private containers:
PRIVATE=/vz/private

# Starting CT ID.  CTIDs with this ID or greater will have mysql.sock link created
START_CTID=1001

# Stopping CT ID.  CTIDs with this ID or less will have mysql.sock link created
STOP_CTID=2000

# Shared Mysql CT ID:
MYSQL_CTID=201

# Location of mysql socket file
MYSQL_SOCK_DIR=/var/lib/mysql

# Mysql socket file name
MYSQL_SOCK=mysql.sock

#################################
# End of Configuration settings #
#################################

# Display output if quiet is 0
QUIET=0

if [ $# -eq 1 -a "$1" = "--quiet" ]; then
        QUIET=1
fi

# Full path to socket
MYSQL_SOCK_FILE=${PRIVATE}/${MYSQL_CTID}${MYSQL_SOCK_DIR}/${MYSQL_SOCK}

[ $QUIET -eq 0 ] && echo
[ $QUIET -eq 0 ] && echo "Relinking process starting..."

# Get current location so we can set it back later
oldDirectory=`pwd`

# Check to see if MySQL container socket exists
if [ -S "${MYSQL_SOCK_FILE}" ]; then

        # Get inode of MySQL container socket
        mysql_inode=`ls -i ${MYSQL_SOCK_FILE} | awk '{ print $1;}'`

        # Search through containers
        cd $PRIVATE
        for i in * ; do
                # The current container to process
                veid=$i

                # Check if container should be processed
                if [ $veid -ne $MYSQL_CTID -a $veid -ge $START_CTID -a $veid -le $STOP_CTID ]; then

                        # Get this container's socket
                        vesock=${PRIVATE}/${veid}${MYSQL_SOCK_DIR}/${MYSQL_SOCK}

                        # Make sure folder exists
                        mkdir -p ${PRIVATE}/${veid}${MYSQL_SOCK_DIR}

                        # Check to see if this container has a socket already
                        if [ -S "${vesock}" ]; then
                                # Get inode of this container socket
                                ve_inode=`ls -i ${vesock} | awk '{ print $1;}'`

                                # Test if sockets are the same
                                if [ $mysql_inode -eq $ve_inode ]; then
                                        # No action required
                                        [ $QUIET -eq 0 ] && echo "$veid VALID:  socket ${vesock}"
                                else
                                        # Remove existing file if any
                                        if [ -a "${vesock}" ]; then
                                                rm ${vesock}
                                        fi

                                        # Create hardlink to mysql socket file
                                        ln ${MYSQL_SOCK_FILE} ${vesock}

                                        [ $QUIET -eq 0 ] && echo "$veid FIXED:  socket ${vesock}"
                                fi
                        else
                                # Socket didn't exist or file wasn't a socket

                                # Remove existing file if any
                                if [ -a "${vesock}" ]; then
                                        rm ${vesock}
                                fi

                                # Create hardlink to mysql socket file
                                ln ${MYSQL_SOCK_FILE} ${vesock}

                                [ $QUIET -eq 0 ] && echo "$veid FIXED:  socket ${vesock}"
                        fi
                else
                        [ $QUIET -eq 0 ] && echo "$veid SKIPPED"
                fi
        done
else
        [ $QUIET -eq 0 ] && echo "${MYSQL_SOCK_FILE} does not exist. Is MySQL running?"
fi

cd $oldDirectory

[ $QUIET -eq 0 ] && echo "Relinking process complete."
[ $QUIET -eq 0 ] && echo

The following post on the OpenVZ forum has some other suggestions on how to deal with this issue: Shared webhosting - problem with mysql socket

Proxy webserver

Because we have only one public IP-address, we need an trick to access every minimal server based on the hostname in the HTTP request. For SSH we used different ports, but that is not an option for websites. Again we create an CTx with an internal IP-address. On this server we install Lighttpd as well, because the proxying is very simple. First we must forward port 80 to this server:

dnat="-j DNAT --to-destination"

iptables -t nat -P PREROUTING ACCEPT
iptables -t nat -A PREROUTING -p TCP -d <external IP-address> --dport 80 $dnat 192.168.13.11:80
iptables -t nat -A PREROUTING -p TCP --dport 10122 $dnat 192.168.13.101:22
iptables -t nat -A PREROUTING -p TCP --dport 10222 $dnat 192.168.13.102:22
...

Then we create for every website an section in /etc/lighttpd/lighttpd.conf as follows:

$HTTP["host"] == "ve101.armorica.tk" {
  proxy.server  = ( "" => ( ( "host" => "192.168.13.101" ) ) )
}

You can map more names to the same IP-address if needed. The last step is to add mod_proxy to the server.modules section.

For apache add a VirtualHost directive

<VirtualHost external-IP-address:80>
        ServerName mydomainnameishere.com
        RewriteEngine     On
        RewriteRule       ^(.*)$        http://192.168.2.101$1  [P]
        RewriteRule       ^(.*)$        http://mydomainnameishere.com$1   [P]
</VirtualHost>

<VirtualHost external-IP-address:80>
        ServerName mydomainnameishere.com
        RewriteEngine     On
        RewriteRule       ^(.*)$        http://192.168.2.101$1  [P]
        RewriteRule       ^(.*)$        http://www.mydomainnameishere.com$1   [P]
</VirtualHost>

Other applications

Create for other applications as mail, make sure that the minimal servers use this one for sending mail from webpages, DNS etc. CTx as needed. The resulting server is shown in the figure above.

OpenVZ can easily be installed on most Dedicated Server Hosting Services