I have two servers running SLES11SP1 Xen kernel on S3420GP motherboard
with 2 integrated Intel LAN interfaces - 82574L and 82578DM configured
for LACP 802.3ad bonding. The switch ports are also configured for LACP,
and port statistics show that one port is mostly sending, another
mostly receiving packets on both severs. So I feel it as a kind of load
balancing.
The third SLES11SP1 server is running normal kernel and is configured
as iSCSI target for both Xen servers. It has SuperMicro X7SPA-H-O
motherboard with 2x Intel 82574L Gigabit LAN interfaces also configured
for LACP 802.3ad bonding. But switch port statistics show that one port
is nearly idle the whole time while another is loaded with both RX and
TX packets. No load balancing of any kind because only one port is
working.
If I disconnect the busy port, load goes to the idle one, so connection
is fault tolerant.
Is it possible to push anyhow the load balancing on this bond?
It may be that 802.3ad “Working As Designed, Design Is Bad” in that with
802.3ad, it ensure that > "All packets associated with a given ‘conversation’ are[color=blue]
transmitted on the same link to prevent mis-ordering." So could it be there the traffic is just divided unequally? In[/color]
other words with a small number of flows the division of labor could be
very unequal. Note also that this does not provide a solution for a
single flow needing both links worth of bandwidth.
But I went forward with experimenting. I made NFS shares on servers
with both types of motherboard (Intel and SuperMicro). Then I started
file copying (two files ~4GB each) on those shares from two Linux boxes
simultaneously in 3 modes:
Both reading
Both writing
One - reading, another - writing.
The results were strange again:
On both motherboards two simultaneous reading or writing operations
were served from the same port (no balancing between two different
connections to Linux boxes).
On Intel motherboard (2 different LAN chipsets) simultaneous reading
was through one LAN port, but simultaneous writing was through another.
On SuperMicro motherboard (2 same LAN chipsets) both simultaneous
reading and simultaneous writing were through one LAN port, another was
nearly idle (~300 B/s).
In one box reading, another - writing mode, Intel motherboard used 2
ports, but SuperMicro - again only one for both boxes.
I found good advice while googling the item:
ad_select=bandwidth xmit_hash_policy=layer2+3
This killed any difference in motherboard behavior - both Intel and
SuperMicro now look the same.
Nice balancing for parallel reading or writing - both interfaces are
working.
But when one client is reading and another - writing, the traffic from
both of them goes through one LAN interface.