Shared webhosting
This document describes creating a "secure shared web hosting service" on « HN (Host Node) » It is « NOT » about per VE shared web hosting.
Contents
The problem
One of the problems with shared webhosting (i.e. different people with each his/her own webpages) is that modern script languages such as PHP, Python, or Perl are too powerful. For example take the following PHP script:
<?php function get_content($filename) { $handle = fopen($filename, 'r'); echo fread($handle, filesize($filename)); fclose($handle); } get_content('/home/ppuk34/www/forum/config.inc.php'); ?>
With PHP you could use open_basedir to prevent this, but there are more ways. For example PHP Shell, a script that is often mis-used by people with not-so-good intentions. Or think about the Santy-worm which mis-used phpBB. Again there is a solution in the form of safe_mode, but lots of PHP scripts break if you enable this. For Python, Perl, or CGI-scripts there are no easy ways and you have to use wrappers or other tricks to chroot these.
The solution
You can waste hours of time in securing all the possible things you don't want in your shared webhosting environment. And unless you are very familiar with all the things modern scripting languages can do, you probably miss dozens of alternative routes. In this process you frustrate your clients, because security always means that legitimate things break. As a side effect of your hard work, you can waste hours of extra time in educating your users. But in the end most users don't care about security, unless they are themselves victims of a compromised host. Learning the hard way is by far the most effective method. One possible solution is dedicated webhosting, but most users don't have the experience to maintain a server or it is way to expensive for them.
The main problem with shared webhosting is that by its very nature all files which are served through the web are public. Apache for example uses only one account to read all files. As said, you can use tricks with CGI wrappers to execute the scripting languages under its own credentials. However this kind of security depends on the wrappers ability to securely separate the users. We all know that if this is broken — and most often it will be broken — the result is a higher clearance on the underlying filesystem. For most systems you need more than one wrapper, so the number of possible security problems grow. The ultimate user separation is in the kernel and you can view the modifications OpenVZ has done in this light. Instead of CGI wrappers we go one step higher and give every user its own minimal server. In the rest of this article we describe how shared webhosting with OpenVZ could be implemented.
Minimal server
Create an VEx with your favorite distro. Give it an internal IP-address in one of the ranges 10.0.0.0/8, 172.16.0.0/12 or 192.168.0.0/16. Then strip away all unnecessary init.d scripts so only the bare minimum is started. That means as a minimum syslogd and ssh so the account holder can upload his/her files through SCP/SFTP in his/her own minimal server. For this to work you need to set up destination NAT on VE0 from high numbered ports to port 22 on the given private IP address:
dnat="-j DNAT --to-destination" iptables -t nat -P PREROUTING ACCEPT iptables -t nat -A PREROUTING -p TCP --dport 10122 $dnat 192.168.13.101:22 iptables -t nat -A PREROUTING -p TCP --dport 10222 $dnat 192.168.13.102:22 ...
The other thing you want for webhosting is of course a webserver as well. To minimize the amount of needed memory, we choose Lighttpd instead of the common Apache. Then configure the scripting language of your choice to run under this webserver. It is possible to use different languages/setups for different accounts as well. Also problematic CGI-scripts are not problematic anymore...
MySQL server
Most webhosting accounts use MySQL, but if you prefer another database server, go ahead. Create a new VEx with a lot more resources and again an internal IP-address. Now configure the accounts. As an extra security measure you can use the internal IP-address as well.
MySQL socket sharing
You can also share the socket of an VEx running MySQL, which is alot faster than TCP/IP. e.g.:
ln /var/lib/vz/private/101/var/run/mysqld/mysqld.sock /var/lib/vz/private/102/var/run/mysqld/mysqld.sock
Inside of 102:
$ mysql -u root -p -S /var/run/mysqld/mysqld.sock Enter password: Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 9 to server version: 5.0.32-Debian_7etch1-log
Refreshing links to MySQL socket
Sharing the MySQL socket works really well until the MySQL database is restarted or the VE running MySQL is restarted. When this happens, the socket file is removed and recreated. In most cases, a different inode will be used, causing existing hard links to the mysql.sock file to no longer work. The solution is to relink these sockets.
There are more elegant solutions to this problem, but the following script is a decent hack that can be run via a cron job every minute or two. It will loop through all VEs between START_VEID and STOP_VEID and make sure that the links point to the correct socket. If they do not, the link will be recreated.
#!/bin/sh # Created by Tauren Mills (tauren at tauren dot com) 2007-11-15 ################################### # Start of Configuration settings # ################################### # Location of private VEs: PRIVATE=/vz/private # Starting VE ID. VEIDs with this ID or greater will have mysql.sock link created START_VEID=1001 # Stopping VE ID. VEIDs with this ID or less will have mysql.sock link created STOP_VEID=2000 # Shared Mysql VE ID: MYSQL_VEID=201 # Location of mysql socket file MYSQL_SOCK_DIR=/var/lib/mysql # Mysql socket file name MYSQL_SOCK=mysql.sock ################################# # End of Configuration settings # ################################# # Display output if quiet is 0 QUIET=0 if [ $# -eq 1 -a "$1" = "--quiet" ]; then QUIET=1 fi # Full path to socket MYSQL_SOCK_FILE=${PRIVATE}/${MYSQL_VEID}${MYSQL_SOCK_DIR}/${MYSQL_SOCK} [ $QUIET -eq 0 ] && echo [ $QUIET -eq 0 ] && echo "Relinking process starting..." # Get current location so we can set it back later oldDirectory=`pwd` # Check to see if MySQL VE socket exists if [ -S "${MYSQL_SOCK_FILE}" ]; then # Get inode of MySQL VE socket mysql_inode=`ls -i ${MYSQL_SOCK_FILE} | awk '{ print $1;}'` # Search through VEs cd $PRIVATE for i in * ; do # The current VE to process veid=$i # Check if VE should be processed if [ $veid -ne $MYSQL_VEID -a $veid -ge $START_VEID -a $veid -le $STOP_VEID ]; then # Get this VE's socket vesock=${PRIVATE}/${veid}${MYSQL_SOCK_DIR}/${MYSQL_SOCK} # Make sure folder exists mkdir -p ${PRIVATE}/${veid}${MYSQL_SOCK_DIR} # Check to see if this VE has a socket already if [ -S "${vesock}" ]; then # Get inode of this VE socket ve_inode=`ls -i ${vesock} | awk '{ print $1;}'` # Test if sockets are the same if [ $mysql_inode -eq $ve_inode ]; then # No action required [ $QUIET -eq 0 ] && echo "$veid VALID: socket ${vesock}" else # Remove existing file if any if [ -a "${vesock}" ]; then rm ${vesock} fi # Create hardlink to mysql socket file ln ${MYSQL_SOCK_FILE} ${vesock} [ $QUIET -eq 0 ] && echo "$veid FIXED: socket ${vesock}" fi else # Socket didn't exist or file wasn't a socket # Remove existing file if any if [ -a "${vesock}" ]; then rm ${vesock} fi # Create hardlink to mysql socket file ln ${MYSQL_SOCK_FILE} ${vesock} [ $QUIET -eq 0 ] && echo "$veid FIXED: socket ${vesock}" fi else [ $QUIET -eq 0 ] && echo "$veid SKIPPED" fi done else [ $QUIET -eq 0 ] && echo "${MYSQL_SOCK_FILE} does not exist. Is MySQL running?" fi cd $oldDirectory [ $QUIET -eq 0 ] && echo "Relinking process complete." [ $QUIET -eq 0 ] && echo
The following post on the OpenVZ forum has some other suggestions on how to deal with this issue: Shared webhosting - problem with mysql socket
Proxy webserver
Because we have only one public IP-address, we need an trick to access every minimal server based on the hostname in the HTTP request. For SSH we used different ports, but that is not an option for websites. Again we create an VEx with an internal IP-address. On this server we install Lighttpd as well, because the proxying is very simple. First we must forward port 80 to this server:
dnat="-j DNAT --to-destination" iptables -t nat -P PREROUTING ACCEPT iptables -t nat -A PREROUTING -p TCP -d <external IP-address> --dport 80 $dnat 192.168.13.11:80 iptables -t nat -A PREROUTING -p TCP --dport 10122 $dnat 192.168.13.101:22 iptables -t nat -A PREROUTING -p TCP --dport 10222 $dnat 192.168.13.102:22 ...
Then we create for every website an section in /etc/lighttpd/lighttpd.conf as follows:
$HTTP["host"] == "ve101.armorica.tk" { proxy.server = ( "" => ( ( "host" => "192.168.13.101" ) ) ) }
You can map more names to the same IP-address if needed. The last step is to add mod_proxy to the server.modules section.
For apache add a VirtualHost directive
<VirtualHost external-IP-address:80> ServerName mydomainnameishere.com RewriteEngine On RewriteRule ^(.*)$ http://192.168.2.101$1 [P] RewriteRule ^(.*)$ http://mydomainnameishere.com$1 [P] </VirtualHost> <VirtualHost external-IP-address:80> ServerName mydomainnameishere.com RewriteEngine On RewriteRule ^(.*)$ http://192.168.2.101$1 [P] RewriteRule ^(.*)$ http://www.mydomainnameishere.com$1 [P] </VirtualHost>
Other applications
Create for other applications as mail, make sure that the minimal servers use this one for sending mail from webpages, DNS etc. VEx as needed. The resulting server is shown in the figure above.