Strange bonding on SLES11

I have two servers running SLES11SP1 Xen kernel on S3420GP motherboard
with 2 integrated Intel LAN interfaces - 82574L and 82578DM configured
for LACP 802.3ad bonding. The switch ports are also configured for LACP,
and port statistics show that one port is mostly sending, another
mostly receiving packets on both severs. So I feel it as a kind of load
The third SLES11SP1 server is running normal kernel and is configured
as iSCSI target for both Xen servers. It has SuperMicro X7SPA-H-O
motherboard with 2x Intel 82574L Gigabit LAN interfaces also configured
for LACP 802.3ad bonding. But switch port statistics show that one port
is nearly idle the whole time while another is loaded with both RX and
TX packets. No load balancing of any kind because only one port is
If I disconnect the busy port, load goes to the idle one, so connection
is fault tolerant.

Is it possible to push anyhow the load balancing on this bond?


andy_s’s Profile:
View this thread:

It may be that 802.3ad “Working As Designed, Design Is Bad” in that with
802.3ad, it ensure that > "All packets associated with a given ‘conversation’ are[color=blue]

transmitted on the same link to prevent mis-ordering." So could it be there the traffic is just divided unequally? In[/color]
other words with a small number of flows the division of labor could be
very unequal. Note also that this does not provide a solution for a
single flow needing both links worth of bandwidth.

– Bob

Bob Mahar – Novell Knowledge Partner
Do you do what you do at a .EDU?
“Programming is like teaching a jellyfish to build a house.”
More Bob: ‘Twitter’ ( ‘Blog’
( ‘Vimeo’ ( ←
Click And Be Amazed!

Bob-O-Rama’s Profile:
View this thread:

This idea is clear.

But I went forward with experimenting. I made NFS shares on servers
with both types of motherboard (Intel and SuperMicro). Then I started
file copying (two files ~4GB each) on those shares from two Linux boxes
simultaneously in 3 modes:

  1. Both reading
  2. Both writing
  3. One - reading, another - writing.

The results were strange again:

  1. On both motherboards two simultaneous reading or writing operations
    were served from the same port (no balancing between two different
    connections to Linux boxes).
  2. On Intel motherboard (2 different LAN chipsets) simultaneous reading
    was through one LAN port, but simultaneous writing was through another.
  3. On SuperMicro motherboard (2 same LAN chipsets) both simultaneous
    reading and simultaneous writing were through one LAN port, another was
    nearly idle (~300 B/s).
  4. In one box reading, another - writing mode, Intel motherboard used 2
    ports, but SuperMicro - again only one for both boxes.

Any ideas?


andy_s’s Profile:
View this thread:

I found good advice while googling the item:
ad_select=bandwidth xmit_hash_policy=layer2+3

This killed any difference in motherboard behavior - both Intel and
SuperMicro now look the same.
Nice balancing for parallel reading or writing - both interfaces are
But when one client is reading and another - writing, the traffic from
both of them goes through one LAN interface.


andy_s’s Profile:
View this thread: