Tilera

Prerequisites

You have to have Tilera MDE installed in the following way:

gpg -d /path/to/tileramde-3.0.0.123096_tilepro.tar.bz2.gpg | tar xjf —
gpg -d /path/to/tileramde-3.0.0.123096_tilepro_src.tar.bz2.gpg | tar xjf —
eval $(TileraMDE-3.0.0.123096/tilepro/tile-env)
cd $TILERA_ROOT
gpg -d /path/to/tileramde-3.0.0.123096_tilepro_tile.tar.bz2.gpg | tar xjf —
cat << EOF > tile/.profile
HOME=/root
cd
EOF

In other words:

• tilepro and tilepro_src tarballs are unpacked to the same directory
• tilepro_tile tarball is unpacked under $TILERA_ROOT
• $TILERA_ROOT/tile/.profile is created as shown above

Creating OpenVZ template

A template is just a tarball of a Linux distribution installed into a chroot-like environment. Here, we can just use $TILERA_ROOT/tile contents, omitting just some less-needed contents:

cd $TILERA_ROOT/tile
tar cvzf ../tile.tar.gz . \
 --exclude ./boot \
 --exclude ./lib/modules \
 --exclude ./usr/share/doc \
 --exclude ./usr/share/locale \
 --exclude ./usr/include \
 --exclude ./usr/share/terminfo \
 --exclude ./usr/share/X11 \
 --exclude ./usr/share/fonts \
 --exclude ./usr/share/gtk-doc \
 --exclude ./usr/share/gnome \
 --exclude ./usr/share/mysql-test \
 --exclude ./usr/share/ghostscript \
 --exclude ./usr/share/alsa \
 --exclude ./lib/kbd

Note: The resulting tarball is still very big, but since Tilera MDE lacks any sort of package management / packages repository, I can't make it any better (ie using yum).

Compile/install vzctl

vzctl is a tool to control OpenVZ containers. To compile it:

cd $TILERA_ROOT
TAR=vzctl-tilera-20110429-3.tar.bz2
wget http://download.openvz.org/.kir/tilera/$TAR
tar xvjf $TAR
rm -rf vbuild
mkdir vbuild
cd vbuild
sh ../src/packages/vzctl/tile-extract
./tile-build --install

At this point you should have vzctl installed into $TILERA_ROOT/tile

In addition, put the OpenVZ template into an appropriate directory:

cd $TILERA_ROOT
cp tile.tar.gz tile/vz/template/cache/

Compile/install OpenVZ kernel

This kernel is based on RHEL6-2.6.32 plus OpenVZ patches 042test008.1, plus backported Tilera patches taken from Tilera MDE 3.0.0.123096 2.6.36 kernel source.

This is an initial port, with some known bugs and limitations.

FIXME: what options/features are not yet ported?

Here is how to compile OpenVZ kernel:

cd $TILERA_ROOT
TAR=kernel-rhel6-ovz-tilera-20110429.tar.bz2
wget http://download.openvz.org/.kir/tilera/$TAR
tar xjf $TAR
mkdir kbuild
cd kbuild
export ARCH=tile
sh $TILERA_ROOT/src/sys/rhel6-ovz-kernel/tile-prepare
make -j32
make modules_install INSTALL_MOD_PATH=$TILERA_ROOT/tile/

Running OpenVZ

Preparing a chroot script

cat << EOF > $TILERA_ROOT/tile/chroot.sh
cd /mnt
mount --bind /sys sys
mount --bind /proc proc
mount -t tmpfs tmpfs /mnt/dev
cp -a /dev/ /mnt/
chroot . /bin/bash
# Rollback
umount /mnt/dev
umount /mnt/sys
umount /mnt/proc
EOF

Running kernel

cd $TILERA_ROOT/kbuild
tile-monitor --vmlinux vmlinux --pci --mount $TILERA_ROOT/tile/ /mnt/

In another shell session, run screen in order to access console.

screen -S console /dev/ttyS1 115200

In the screen session, switch to chroot using the script prepared before:

/mnt/chroot.sh

You should now see bash prompt (bash-4.1#).

Next, start OpenVZ:

/etc/init.d/vzeventd restart
/etc/init.d/vz restart

Creating/starting container

Before creating a container, we should disable DISK_QUOTA in global OpenVZ configuration file (vzquota utility is not yet ported to Tilera):

sed -i s/^DISK_QUOTA=.*$/DISK_QUOTA=no/ /etc/vz/vz.conf

Now we can create our first container. The following command will take the tarball /vz/template/cache/tile.tar.gz and unpack it into /vz/private/101. It also creates /vz/root/101 and a container config file /etc/vz/conf/101.conf.

vzctl create 101 --ostemplate tile

NOTE: creation takes quite a long time because of sheperd-fuse mount being used. I do not know of any better way to access a host system's FS from inside the Tilera environment.

Next we can start the freshly created container:

vzctl start 101

Basic container operations

Show all containers:

vzlist -a

Show running containers:

vzlist

Enter the container (shell):

vzctl enter 101

Setting container physical memory usage limit (to 512 MB):

vzctl set 101 --physpages 0:512M --save

Setting container virtual swap limit (to 1GB):

vzctl set 101 --swappages 0:1G --save

Seeing current memory/swap usage:

vzctl exec 101 cat /proc/user_beancounters

Known issues

1. vzquota is not ported yet (no big need to, it's optional package)
2. Need to figure out how to access file system from tilera environment in a less slow way than shepherd-fuse
3. Need to add ip utility to $TILERA_ROOT/tile
4. Need to figure out networking setup in Tilera container
5. No RTC: hwclock: can't open '/dev/misc/rtc': No such file or directory
6. OS template cache (tile.tar.gz) is still very big -- to solve it Tilera needs proper package management