Traffic shaping with tc

From OpenVZ Virtuozzo Containers Wiki
Jump to: navigation, search

Sometimes it's necessary to limit traffic bandwidth from and to a container. You can do it using ordinary tc tool.

Packet routes

First of all, a few words about how packets travel from and to a container. Suppose we have Hardware Node (HN) with a container (CT) on it, and this container talks to some Remote Host (RH). HN has one "real" network interface eth0 and, thanks to OpenVZ, there is also "virtual" network interface venet0. Inside the container we have interface venet0:0. 

    venet0:0               venet0    eth0
CT >------------->-------------> HN >--------->--------> RH

    venet0:0               venet0    eth0
CT <-------------<-------------< HN <---------<--------< RH

Limiting outgoing bandwidth

We can limit container outgoing bandwidth by setting the tc filter on eth0. 

DEV=eth0
tc qdisc del dev $DEV root
tc qdisc add dev $DEV root handle 1: cbq avpkt 1000 bandwidth 100mbit
tc class add dev $DEV parent 1: classid 1:1 cbq rate 256kbit allot 1500 prio 5 bounded isolated
tc filter add dev $DEV parent 1: protocol ip prio 16 u32 match ip src X.X.X.X flowid 1:1
tc qdisc add dev $DEV parent 1:1 sfq perturb 10

X.X.X.X is an IP address of container.

Limiting incoming bandwidth

This can be done by setting the tc filter on venet0: 

DEV=venet0
tc qdisc del dev $DEV root
tc qdisc add dev $DEV root handle 1: cbq avpkt 1000 bandwidth 100mbit
tc class add dev $DEV parent 1: classid 1:1 cbq rate 256kbit allot 1500 prio 5 bounded isolated
tc filter add dev $DEV parent 1: protocol ip prio 16 u32 match ip dst X.X.X.X flowid 1:1
tc qdisc add dev $DEV parent 1:1 sfq perturb 10

Note that X.X.X.X is an IP address of container.

Limiting CT to HN talks

As you can see, two filters above don't limit container to HN talks. I mean a container can emit as much traffic as it wishes. To make such a limitation from the HN, it is necessary to use tc police on venet0: 

DEV=venet0
tc filter add dev $DEV parent 1: protocol ip prio 20 u32 match u32 1 0x0000 police rate 2kbit buffer 10k drop flowid :1

Limiting packets per second rate from container

To prevent dos atacks from the container you can limit packets per second rate using iptables. 

DEV=eth0
iptables -I FORWARD 1 -o $DEV -s X.X.X.X -m limit --limit 200/sec -j ACCEPT
iptables -I FORWARD 2 -o $DEV -s X.X.X.X -j DROP

Here X.X.X.X is an IP address of container.

An alternate approach using HTB

For details refer to the HTB Home Page 

#!/bin/sh
#
# Incoming traffic control
#
CT_IP1=$1
CT_IP2=$2
DEV=venet0
#
tc qdisc del dev $DEV root
#
tc qdisc add dev $DEV root handle 1: htb default 10
#
tc class add dev $DEV parent 1: classid 1:1 htb rate 100mbit burst 15k
tc class add dev $DEV parent 1:1 classid 1:10 htb rate 10mbit ceil 10mbit burst 15k
tc class add dev $DEV parent 1:1 classid 1:20 htb rate 20mbit ceil 20mbit burst 15k
tc class add dev $DEV parent 1:1 classid 1:30 htb rate 30mbit ceil 30mbit burst 15k
#
tc qdisc add dev $DEV parent 1:10 handle 10: sfq perturb 10
tc qdisc add dev $DEV parent 1:20 handle 20: sfq perturb 10
tc qdisc add dev $DEV parent 1:30 handle 30: sfq perturb 10
#
if [ ! -z $CT_IP1 ]; then
    tc filter add dev $DEV protocol ip parent 1:0 prio 1 u32 match ip dst "$CT_IP1" flowid 1:20 
fi
if [ ! -z $CT_IP2 ]; then
    tc filter add dev $DEV protocol ip parent 1:0 prio 1 u32 match ip dst "$CT_IP2" flowid 1:30 
fi
#
echo;echo "tc configuration for $DEV:"
tc qdisc show dev $DEV
tc class show dev $DEV
tc filter show dev $DEV
#
# Outgoing traffic control
#
DEV=eth0
#
tc qdisc del dev $DEV root
#
tc qdisc add dev $DEV root handle 1: htb default 10
#
tc class add dev $DEV parent 1: classid 1:1 htb rate 100mbit burst 15k
tc class add dev $DEV parent 1:1 classid 1:10 htb rate 10mbit ceil 10mbit burst 15k
tc class add dev $DEV parent 1:1 classid 1:20 htb rate 20mbit ceil 20mbit burst 15k
tc class add dev $DEV parent 1:1 classid 1:30 htb rate 30mbit ceil 30mbit burst 15k
#
tc qdisc add dev $DEV parent 1:10 handle 10: sfq perturb 10
tc qdisc add dev $DEV parent 1:20 handle 20: sfq perturb 10
tc qdisc add dev $DEV parent 1:30 handle 30: sfq perturb 10
#
if [ ! -z $CT_IP1 ]; then
    tc filter add dev $DEV protocol ip parent 1:0 prio 1 u32 match ip src "$CT_IP1" flowid 1:20
fi
if [ ! -z $CT_IP2 ]; then
    tc filter add dev $DEV protocol ip parent 1:0 prio 1 u32 match ip src "$CT_IP2" flowid 1:30
fi
#
echo;echo "tc configuration for $DEV:"
tc qdisc show dev $DEV
tc class show dev $DEV
tc filter show dev $DEV


Autogen rules for CT

#!/usr/bin/env python
# -*- coding: utf-8 -*-
######################################
## by VoidLess, lepus.su
######################################
import fnmatch
import os

######################################
## НАСТРОЙКИ СКРИПТА
confdir = "/etc/vz/conf/"
rulefile = "/etc/vz/shaper/rules.sh"

# шейпятся только эти тарифы
table_band = {
    #ORIGIN_SAMPLE : канал
    "SSD8": "100mbit",
    "SSD7": "100mbit",
    "SSD6": "100mbit",
    "SSD5": "100mbit",
    "SSD4": "100mbit",
    "SSD3": "100mbit",
    "SSD2": "100mbit",
    "SSD1": "100mbit"
}

######################################
## ШАБЛОН СКРИПТА НАСТРОЙКИ ШЕЙПЕРА

fileheader = """#!/bin/bash
# shaper configuration file

##### ЧИСТКА ОТ СТАРЫХ ПРАВИЛ #####
tc qdisc del dev eth0 root
tc -s qdisc ls dev eth0

tc qdisc del dev venet0 root
tc -s qdisc ls dev venet0
##### ЧИСТКА ОТ СТАРЫХ ПРАВИЛ #####
"""

# eth0

eth0header = """
DEV=eth0
tc qdisc del dev $DEV root 2>/dev/null
tc qdisc add dev $DEV root handle 1: htb default 1 r2q 3000
tc class add dev $DEV parent 1: classid 1:1 htb rate 1000mbit burst 10mb
"""

eth0template = """
# vzid {VPSID} tarif {BANDWIDTH} ip {IP}

tc class add dev $DEV parent 1:1 classid 1:{CLASS} htb rate {BANDWIDTH} ceil {BANDWIDTH} burst 1mb
tc qdisc add dev $DEV parent 1:{CLASS} handle {CLASS}: sfq perturb 5 #quantum 5000b
"""

eth0match = """
tc filter add dev $DEV protocol ip parent 1:0 prio 1 u32 match ip src {IP} flowid 1:{CLASS}
"""

# venet0

venet0header = """
DEV=venet0
tc qdisc del dev $DEV root 2>/dev/null
tc qdisc add dev $DEV root handle 1: htb default 1 r2q 3000
tc class add dev $DEV parent 1: classid 1:1 htb rate 1000mbit burst 10mb
"""

venet0template = """
# vzid {VPSID} tarif {BANDWIDTH} ip {IP}

tc class add dev $DEV parent 1:1 classid 1:{CLASS} htb rate {BANDWIDTH} ceil {BANDWIDTH} burst 1mb
tc qdisc add dev $DEV parent 1:{CLASS} handle {CLASS}: sfq perturb 5 #quantum 5000b
"""

venet0match = """
tc filter add dev $DEV protocol ip parent 1:0 prio 1 u32 match ip dst {IP} flowid 1:{CLASS}
"""

######################################
## КОД СКРИПТА
# получает список и настройки виртуалок из папки конфигов OpenVZ
def getvz():
    vzlist = []

    for file in os.listdir(confdir):
        if fnmatch.fnmatch(file, '*.conf') and file != "0.conf":
            #берем список конфигов
            vzid = file.split(".",1)[0]
            vztarif = None
            vzip = None

            #открываем файл, узнаем название тарифа
            with open(confdir + file) as f:
                for line in f:
                    if line[0]=="#":
                        continue

                    ar = line.split("=")

                    if ar[0] == "ORIGIN_SAMPLE":
                        vztarif = ar[1].replace('"', '').strip()
                        continue

                    if ar[0] == "IP_ADDRESS":
                        vzip = ar[1].replace('"', '').strip().split(" ")
                        continue

            #если в файле не написан тариф, или он левый, ничего не делаем
            if vztarif == None or vzip == None or vztarif not in table_band:
                continue
            
            # в список записываем инфу про все vps
            vzlist.append((vzid, table_band[vztarif], vzip))
    return vzlist

# записывает команды настройки шейпера в sh файл
def genconf(vzlist):
    with open(rulefile,"w+") as f:

        # file header
        f.write(fileheader)

        # eth0 rules
        f.write(eth0header)

        shaper_id = 100
        for vz in vzlist:
            f.write(eth0template
                        .replace("{VPSID}",vz[0])
                        .replace("{CLASS}",str(shaper_id))
                        .replace("{BANDWIDTH}",vz[1])
                        .replace("{IP}",str(vz[2])))
            for ip in vz[2]:
                f.write(eth0match
                            .replace("{VPSID}",vz[0])
                            .replace("{CLASS}",str(shaper_id))
                            .replace("{IP}",ip))
            shaper_id += 1

        # venet0 rules
        f.write(venet0header)

        shaper_id = 100
        for vz in vzlist:
            f.write(venet0template
                        .replace("{VPSID}",vz[0])
                        .replace("{CLASS}",str(shaper_id))
                        .replace("{BANDWIDTH}",vz[1])
                        .replace("{IP}",str(vz[2])))
            for ip in vz[2]:
                f.write(venet0match
                            .replace("{VPSID}",vz[0])
                            .replace("{CLASS}",str(shaper_id))
                            .replace("{IP}",ip))
            shaper_id += 1

    # make in runnable
    os.chmod(rulefile, 0755)

## основной код скрипта. выполнение функций
# получили список VPSок
vzlist = getvz()

# сгенерировали скрипт с правилами
genconf(vzlist)

# применили правила
os.system(rulefile)


External links

Retrieved from "https://wiki.openvz.org/index.php?title=Traffic_shaping_with_tc&oldid=14878"