Difference between revisions of "Multiple network interfaces and ARP flux"

From OpenVZ Virtuozzo Containers Wiki
Jump to: navigation, search
(New page: This page discusses how to setup a HN with multiple network interfaces on the same physical network and on the same IP network. For example: <pre> eth0 = 00:0c:29:b3:a2:54 with ip address...)
 
Line 1: Line 1:
−
This page discusses how to setup a HN with multiple network interfaces on the same physical network and on the same IP network.
+
== Overview ==
 +
This page discusses how to setup a HN with multiple network interfaces on the same physical network and on the same IP network.  Then how to setup multiple VE's to use only one of these interfaces.
  
−
For example:
+
For example, you want some of your VE's to always use eth3, and some to use eth4. But none of the VE traffic should use eth0, which is reserved for use by the HN only.  This makes sense if you have VE's that may generate or receive a lot of traffic and you don't want your remote administration of the server over eth0 to degrade or get blocked because of this.
−
<pre>
+
 
−
eth0 = 00:0c:29:b3:a2:54 with ip address 192.168.18.10
+
To make this clear we'll use the following HN configuration.
−
eth3 = 00:0c:29:b3:a2:68 with ip address 192.168.18.11
+
 
−
eth4 = 00:0c:29:b3:a2:fe with ip address 192.168.18.12
+
{| align="center" border="1" cellpadding=5
−
</pre>
+
! Interface !! MAC Address !! IP Address
−
The first issue is ARP flux.  Any client on the network who broadcasts an ARP "who has" message for any of these addresses will receive replies from all three interfaces.  This results in IP addresses that float between three MAC addresses, depending on which response a client accepts first.
+
|-
 +
| eth0 || 00:0c:29:b3:a2:54 || 192.168.18.10
 +
|-
 +
| eth3 || 00:0c:29:b3:a2:68 || 192.168.18.11
 +
|-
 +
| eth4 || 00:0c:29:b3:a2:fe || 192.168.18.12
 +
|}
 +
 
 +
=== HN ARP Flux ===
 +
The first issue is ARP flux.  Any client on the network broadcasting an ARP "who has" message for any of these addresses will receive replies from all three interfaces.  This results in IP addresses that float between three MAC addresses, depending on which response a client accepts first.
  
 
If all three network interfaces are on different IP networks (such as 10.x.x.x, 172.16.x.x, 192.168.x.x) then executing the following will work:
 
If all three network interfaces are on different IP networks (such as 10.x.x.x, 172.16.x.x, 192.168.x.x) then executing the following will work:
Line 27: Line 37:
 
</pre>
 
</pre>
 
Each of these should only generate responses from the network interface associated with that IP address on the HN.
 
Each of these should only generate responses from the network interface associated with that IP address on the HN.
−
----
+
 
−
Now, let's say you want some of your VE's to always use eth3, and some to use eth4. None of the VE traffic should use eth0, which is reserved for use by the HN only.  This makes sense if you have VE's that may generate or receive a lot of traffic and you don't want your remote administration of the server over eth0 to degrade or get blocked because of this.
+
=== Adding some VE's ===
  
 
Setup two VE's as follows:
 
Setup two VE's as follows:
Line 38: Line 48:
  
 
TODO: Discuss approach of <code>ip rule ...</code> and <code>ip route ...</code>.
 
TODO: Discuss approach of <code>ip rule ...</code> and <code>ip route ...</code>.
 +
 +
TODO: Add examples of ARP flux before fix is applied, and after.
 +
 +
[[Category:HOWTO]]
 +
[[Category:Networking]]

Revision as of 22:19, 16 February 2007

Overview

This page discusses how to setup a HN with multiple network interfaces on the same physical network and on the same IP network. Then how to setup multiple VE's to use only one of these interfaces.

For example, you want some of your VE's to always use eth3, and some to use eth4. But none of the VE traffic should use eth0, which is reserved for use by the HN only. This makes sense if you have VE's that may generate or receive a lot of traffic and you don't want your remote administration of the server over eth0 to degrade or get blocked because of this.

To make this clear we'll use the following HN configuration.

Interface MAC Address IP Address
eth0 00:0c:29:b3:a2:54 192.168.18.10
eth3 00:0c:29:b3:a2:68 192.168.18.11
eth4 00:0c:29:b3:a2:fe 192.168.18.12

HN ARP Flux

The first issue is ARP flux. Any client on the network broadcasting an ARP "who has" message for any of these addresses will receive replies from all three interfaces. This results in IP addresses that float between three MAC addresses, depending on which response a client accepts first.

If all three network interfaces are on different IP networks (such as 10.x.x.x, 172.16.x.x, 192.168.x.x) then executing the following will work:

sysctl -w net.ipv4.conf.all.arp_filter=1

However, if they are all on the same IP network, which is the case here, then the following solution will work. This can be added to your /etc/sysctl.conf file once you've tested it. 

sysctl -w net.ipv4.conf.all.arp_ignore=1
sysctl -w net.ipv4.conf.all.arp_announce=2

Using another system on the network you should be able to execute the following: 

arping -c3 192.168.18.10
arping -c3 192.168.18.11
apring -c3 192.168.18.12

Each of these should only generate responses from the network interface associated with that IP address on the HN.

Adding some VE's

Setup two VE's as follows: 

VEID = 101  IP = 192.168.18.101
VEID = 102  IP = 192.168.18.102

From another system on the network you should be able to ping both. However, looking at the ARP traffic with tcpdump or using arping you'll see that once again the physical address associated with each VE will be subject to ARP flux, drifting between all three IP addresses over time.

TODO: Discuss approach of ip rule ... and ip route ....

TODO: Add examples of ARP flux before fix is applied, and after.

Retrieved from "https://wiki.openvz.org/index.php?title=Multiple_network_interfaces_and_ARP_flux&oldid=2757"