Re: bonding

["John A. Sullivan III" <jsullivan@opensourcedevel.com>]

On Thu, 2011-04-07 at 16:26 +0200, Fabio DellaCorte wrote:
> Hello everybody ,
> i'm tryng to use two network device for improving performance as
> describe
> in http://wiki.debian.org/Bonding and http://www.debianadmin.com/linux-ethernet-bonding-configuration.html .
> I have two problem :
> 
> 
> 1)I don't see performance increase trying with ftp
> 2) i have used round roubin balance in in the first test  and now with
> balance-alb but i still see in /sys/class/net/bond0/bonding/mode
> "balance-rr 0"
> 
<snip>
There are a couple of issues in bonding which can bite the unsuspecting
(as they did me!). Round robin will load balance across multiple
interfaces but can produce serious issues with managing out of order TCP
packets.  Thus, the performance gain decreases dramatically with the
number of interfaces.  In other words, 2 NICs in RR mode will not give
2x the performance nor 3 NICs 3x performance.  I do not recall the exact
numbers off the top of my head but averages are something like:
2 NICs - 1.6x performance
3 NICs - 1.9x performance

The other modes (other than failover) eliminate the out of order TCP
problem but do so at a cost.  All traffic for a single traffic flow goes
across a single path.  The most common way to identify a single traffic
flow is matching source and destination MAC addresses.  Some bonding
algorithms allow matches on layer 3 or even layer 4 data but, if the
switch through which they flow only supports MAC to MAC flow
assignments, it will all devolve to matching MAC addresses anyway.

So what is the practical outcome using non-RR bonding? You have only one
combination of source and destination MAC address for each socket, e.g.,
if you are measuring a single FTP connection, there is only one
combination of source and destination MAC address.  Thus, no matter how
many NICs you have, all the traffic will flow across one combination of
NICs.  You will see no performance improvement.

In fact, depending on how the MAC addresses are advertised from the
systems with multiple NICs, all traffic between two systems may flow
across the same pair of NICs even if there are multiple, different
traffic streams.

On the other hand, if you are using bonding to provide a trunk carrying
traffic from many different source and destination MAC address
combinations, each separate stream will be limited to the maximum of the
individual NICs but the aggregate throughput should increase almost
linearly with the number of NICs.  Hope that helps - John