Thank you Willem and Jens. Appreciate your interest.
Sorry about the delayed reply. I am just in charge of a small business and this is one of the things I do. I have attached my loadbalance script to this reply, in the hope that it helps. Please remember: I am just a advanced user, not a techy guy. Bear that in mind if my answers to your questions are way off.
My whole idea behind starting to use load balancing was to get over poor connections here AND to make maximum use of the charges the local telephone companies levies for usage. Usually it is very low upto a usage-point, then it shoots up. So having multiple providers and using their respective minuimum usage plans made economic sense. This setup only serves my companies web browsing needs. I know that the routes get cached and on days when I have large downloads, the entire traffic is thru just one interface. But 99% of the time, it is evenly distributed across all the interfaces and perfectly achieves my end.
The attached script is a mish-mash of everything I learned of the web and has been working for many years now. It continues to work without problems, except that I am now working on automatically detecting dead gateways and for that I need to be able to ping the real web through any/all my interfaces and detect which is down. Therein lies my problem. By default the script starts with 4-dsl routers load balanced assuming they are all having live connections. Later if/when one of them goes down, my ping script detects it and changes the load-balance in favour of the live connections. BUT: How do I know when it become lives again? The default route no longer uses that interface, as my script has changed it to use the other live ones. So once a interface is down, it is permanently down.
DO excuse, all the copyright info, it is undeserving and was purely to pamper my ego for achieving a working script in linux.
Jens, the nating happens on my linux box and on the dsl-routers. The firewall is also setup and uses nothing more than standard suse options. Your third question-- I do not understand.
[CODE]#! /bin/sh
######################################## FOR 4 CONNECTIONS ###########################################
Copyright (c) 2010-2012 T. N. Ramesh. All rights reserved.
T. N. Ramesh
script written on 17.2.2012
BEGIN INIT INFO
Provides: loadbal
Required-Start: $network $syslog
Required-Stop: $network $syslog
Default-Start: 3 5
Default-Stop: 0 1 2 6
Description: script for enabling loadbalancing through multiple internet routers
END INIT INFO
NOTE: During link failures must recall this function with live links
#######################################################################################################
In next version must consolidate hard-coded route entries to use variables & put them in a loop
Currently this script must be modified if default number of connections is different from 4 and
entries must be modified inside 2 functions “noparams” and “withparams”
Must also determine method to detect deadgateways and call script automatically with relevant
args and make it totally automatic.
#######################################################################################################
Read rc functions
Shell functions sourced from /etc/rc.status:
rc_check check and set local and overall rc status
rc_status check and set local and overall rc status
rc_status -v be verbose in local rc status and clear it afterwards
rc_status -v -r ditto and clear both the local and overall rc status
rc_status -s display “skipped” and exit with status 3
rc_status -u display “unused” and exit with status 3
rc_failed set local and overall rc status to failed
rc_failed set local and overall rc status to
rc_reset clear both the local and overall rc status
rc_exit exit appropriate to overall rc status
rc_active checks whether a service is activated by symlinks
. /etc/rc.status
Reset status of this service
rc_reset
#define aesthetics
statusCol=“echo -ne \\033[60G”
statusColorOK=“echo -ne \\033[1;32m”
statusColorFailed=“echo -ne \\033[1;31m”
statusColorNormal=“echo -ne \\033[0;39m”
#define script and config location and directory
LB_SCRIPT=loadbal.standalone.sh
LB_DIR=/usr/sbin/
#check it exist and has execute perms
if [ ! -x $LB_DIR$LB_SCRIPT ] ; then
echo -n “[”
${statusColorFailed}
echo -n " Load Balancing Script not installed or is not executable ! "
${statusColorNormal}
echo “]”
rc_status -s
exit 5
fi
#define sensible name for service to display in messages
SERVICE=“Load Balancing for multiple uplinks”
#define routes that load balancing should take by default
LB_ROUTERS=/etc/sysconfig/LB_routers
#read routes config file for passing default parameters.
[ -r $LB_ROUTERS ] && . $LB_ROUTERS
#define local interfaces
IF0=br0
#define gateway interfaces
#------###
IF1=vlan4
#------###
IF2=vlan5
#------###
IF3=vlan6
#------###
IF4=vlan7
#define LAN ip address of interfaces
IP0=192.168.2.4
IP1=192.168.4.4
IP2=192.168.5.4
IP3=192.168.6.4
IP4=192.168.7.4
#define WAN ip address of interfaces
P1=192.168.4.2
P2=192.168.5.2
P3=192.168.6.2
P4=192.168.7.2
#define network of interfaces
P0_NET=192.168.2.0/24
P1_NET=192.168.4.0/24
P2_NET=192.168.5.0/24
P3_NET=192.168.6.0/24
P4_NET=192.168.7.0/24
#move common route command to variable
RTCMD=“ip route add default scope global”
#retval=0
########################################################################
Route Tables check: found generally in /etc/iproute2/rt_tables
and presence of multiple tables
# define location of route tables
LB_RT_TABLES=rt_tables
LB_RT_TABLES_DIR=/etc/iproute2/
# ensure existence of rt_tables and it has read permission
if [ ! -r $LB_RT_TABLES_DIR$LB_RT_TABLES ] ; then
echo -n “[”
${statusColorFailed}
echo -n " $LB_RT_TABLES_DIR$LB_RT_TABLES does not exist or is not readable ! "
${statusColorNormal}
echo “]”
rc_status -s
exit 6
fi
# and check if additional table have been defined for load balancing
# NOTE: Tables must be name T1, T2, T3 and T4 for this function to work
# NOTE: grep searches for lines that do not begin with a # and contain t1 or
# t2 or t3 or t4
if [ “$(grep -Ec ‘^[^#].*t[1-4]’ $LB_RT_TABLES_DIR$LB_RT_TABLES)” -lt 2 ] ; then
echo -n “[”
${statusColorFailed}
echo -n “Must define atleast 2 tables in $LB_RT_TABLES for load balancing to function”
${statusColorNormal}
echo “]”
exit 6
fi
#########################################################################
Check number of routers specified in /etc/sysconfig/LB_routers
Display error if number is zero
N=1
for LBR in $LBR_AUTOSTART; do
if grep -q \; <<< “$LBR”; then
LBR_NAME[$N]=$(cut -d\; -f1 <<< “$LBR”)
else
LBR_NAME[$N]="$LBR"
fi
N=$(($N+1))
done
LBRs=${#LBR_NAME[*]}
if [ $LBRs -eq 0 ]; then
echo -n "["
${statusColorFailed}
echo -n "Starting $SERVICE: no routers configured"
echo -n "Check $LB_ROUTERS or pass args in cmd line"
${statusColorNormal}
echo "]"
rc_status -u
fi
First reset status of this service
rc_reset
############################################################################
SCRIPT CALLED WITHOUT PARAMETERS USES THIS FUNCTION: noparams
See last section for calls to this function
cleanuprt () {
########################################################################
### Start basic route clean up ####
ip route flush cache
ip route del default
echo -n "["
${statusColorOK}
echo -n "Setting up Load balancing... - Ramesh"
${statusColorNormal}
echo "]"
route del default
ip route flush cache
### end basic route clean up ###
########################################################################
}
########################################################################
### Final route command and display succesfully end of script ###
### This function is called last
finalrtcmd () {
# check if $RTCMD variable is appended with available routes
# (it should be different from initialised value)
if [ "$RTCMD" != "ip route add default scope global" ] ; then
#final route command to be executed. enclosed in backticks (ie.[shift]tilde)
`$RTCMD`
echo -n "["
${statusColorOK}
echo -n "Finished setting up load balancing --Ramesh"
${statusColorNormal}
echo "]"
else
echo -n "["
${statusColorFailes}
echo "FAILURE: routing entry remains unchanged. NOT load balancing."
${statusColorNormal}
echo "]"
fi
rc_status -v
exit 0
}
########################################################################
### Main routing setup starts here ###
### Must add more elif statements as more routers are added and ###
### also add respective routes to existing commands ###
noparams () {
cleanuprt
for LBR in $LBR_AUTOSTART; do
if [ $LBR -eq 4 ]; then
ip route add $P1_NET dev $IF1 src $IP1 table t1
ip route add default via $P1 table t1
ip route add $P1_NET dev $IF1 src $IP1
ip rule add from $IP1 table t1
ip route add $P0_NET dev $IF0 table t1
ip route add $P2_NET dev $IF2 table t1
ip route add $P3_NET dev $IF3 table t1
ip route add $P4_NET dev $IF4 table t1
ip route add 127.0.0.0/8 dev lo table t1
RTCMD="$RTCMD nexthop via ${P1} dev ${IF1} weight 1"
elif [[ $LBR -eq 5 ]]; then
ip route add $P2_NET dev $IF2 src $IP2 table t2
ip route add default via $P2 table t2
ip route add $P2_NET dev $IF2 src $IP2
ip rule add from $IP2 table t2
ip route add $P0_NET dev $IF0 table t2
ip route add $P1_NET dev $IF1 table t2
ip route add $P3_NET dev $IF3 table t2
ip route add $P4_NET dev $IF4 table t2
ip route add 127.0.0.0/8 dev lo table t2
RTCMD="$RTCMD nexthop via ${P2} dev ${IF2} weight 1"
elif [[ $LBR -eq 6 ]]; then
ip route add $P3_NET dev $IF3 src $IP3 table t3
ip route add default via $P3 table t3
ip route add $P3_NET dev $IF3 src $IP3
ip rule add from $IP3 table t3
ip route add $P0_NET dev $IF0 table t3
ip route add $P1_NET dev $IF1 table t3
ip route add $P2_NET dev $IF2 table t3
ip route add $P4_NET dev $IF4 table t3
ip route add 127.0.0.0/8 dev lo table t3
RTCMD="$RTCMD nexthop via ${P3} dev ${IF3} weight 1"
elif [[ $LBR -eq 7 ]]; then
ip route add $P4_NET dev $IF4 src $IP4 table t4
ip route add default via $P4 table t4
ip route add $P4_NET dev $IF4 src $IP4
ip rule add from $IP4 table t4
ip route add $P0_NET dev $IF0 table t4
ip route add $P1_NET dev $IF1 table t4
ip route add $P2_NET dev $IF2 table t4
ip route add $P3_NET dev $IF3 table t4
ip route add 127.0.0.0/8 dev lo table t4
RTCMD="$RTCMD nexthop via ${P4} dev ${IF4} weight 1"
else
echo -n "["
${statusColorFailed}
echo "$SERVICE: Failure"
${statusColorNormal}
echo "]"
fi
done
finalrtcmd
}
############################################################################
SCRIPT CALLED WITH PARAMETERS USES THIS FUNCTION: withparams
See the last section for calls to this function
withparams () {
cleanuprt
while [ $# -gt 0 ]; do
case “$1” in
4)
ip route add $P1_NET dev $IF1 src $IP1 table t1
ip route add default via $P1 table t1
ip route add $P1_NET dev $IF1 src $IP1
ip rule add from $IP1 table t1
ip route add $P0_NET dev $IF0 table t1
ip route add $P2_NET dev $IF2 table t1
ip route add $P3_NET dev $IF3 table t1
ip route add $P4_NET dev $IF4 table t1
ip route add 127.0.0.0/8 dev lo table t1
RTCMD="$RTCMD nexthop via ${P1} dev ${IF1} weight 1"
;;
5)
ip route add $P2_NET dev $IF2 src $IP2 table t2
ip route add default via $P2 table t2
ip route add $P2_NET dev $IF2 src $IP2
ip rule add from $IP2 table t2
ip route add $P0_NET dev $IF0 table t2
ip route add $P1_NET dev $IF1 table t2
ip route add $P3_NET dev $IF3 table t2
ip route add $P4_NET dev $IF4 table t2
ip route add 127.0.0.0/8 dev lo table t2
RTCMD="$RTCMD nexthop via ${P2} dev ${IF2} weight 1"
;;
6)
ip route add $P3_NET dev $IF3 src $IP3 table t3
ip route add default via $P3 table t3
ip route add $P3_NET dev $IF3 src $IP3
ip rule add from $IP3 table t3
ip route add $P0_NET dev $IF0 table t3
ip route add $P1_NET dev $IF1 table t3
ip route add $P2_NET dev $IF2 table t3
ip route add $P4_NET dev $IF4 table t3
ip route add 127.0.0.0/8 dev lo table t3
RTCMD="$RTCMD nexthop via ${P3} dev ${IF3} weight 1"
;;
7)
ip route add $P4_NET dev $IF4 src $IP4 table t4
ip route add default via $P4 table t4
ip route add $P4_NET dev $IF4 src $IP4
ip rule add from $IP4 table t4
ip route add $P0_NET dev $IF0 table t4
ip route add $P1_NET dev $IF1 table t4
ip route add $P2_NET dev $IF2 table t4
ip route add $P3_NET dev $IF3 table t4
ip route add 127.0.0.0/8 dev lo table t4
RTCMD="$RTCMD nexthop via ${P4} dev ${IF4} weight 1"
;;
esac
#check next parameter
shift
done
finalrtcmd
}
##############################################################################
test for parameters in command line and display usage instructions
Jump to function “noparams” or “withparams” based on call.
this has been coded last as bash needs to read functions to be
aware about them
Define function “usage” for common help message for error conditions
usage () {
echo “If ‘start’ or no parameters are specified on command line,”
echo “script will automatically pick config from $LB_ROUTERS”
echo “Command line Usage: $0 4 5 6 7”
echo " or $0 4 5 6"
echo “enable only connections that are active”
echo “CLI’s Airtel = 4 CLIPL’s Airtel = 5”
echo -n “CLI’s Bsnl = 6 TLO’s Bsnl = 7”
${statusColorNormal}
echo “]”
}
Test for sanity and proceed accordingly
if [ $# = 0 ]; then
echo -n "["
${statusColorOK}
usage
noparams
## If loaded on boot, the parameter "start" will automatically be passed
## hence, this must be handled.
elif [ $1 = "start" ]; then
echo -n "["
${statusColorOK}
usage
noparams
## If stopped during system shutdown, the parameter "stop" will automatically be passed
## hence, this must be handled.
elif [ $1 = "stop" ]; then
echo -n ""
${statusColorOK}
echo "Stop specified. Nothing to do. Exiting..."
exit
elif [ $# > 0 ]; then
if [ $# -lt 2 ]; then
echo -n "["
${statusColorFailed}
echo "Invalid parameters specified. Atleast 2 routes needed."
usage
exit 1
elif [ $# -gt 4 ]; then
echo -n "["
${statusColorFailed}
echo "Invalid parameters specified. Maximum 4 routes supported."
usage
exit 1
else
## move number of args to counter, and check each value for correct-range
## within loop
## do not use $# directly, as we need its values for next stage.
for args in "$@"
# above can also simply be -- for args -- "in "$@"" is default
do
if [ $args -lt 4 ]; then
echo -n "["
${statusColorFailed}
echo "Invalid parameters specified. Args must be between 4 and 7."
usage
exit 1
elif [ $args -gt 7 ]; then
echo -n "["
${statusColorFailed}
echo "Invalid parameters specified. Args must be between 4 and 7."
usage
exit 1
fi
done
## everything is fine, call withparams with original args intact
withparams "$@"
## the "$a" is to pass all args in the cmd line to the function
## normally cmd line args are not passed to user-functions
fi
fi
#############################################################################
exit
[/CODE]
[QUOTE=jmozdzen;3109]Hi Ramesh,
I’ve tried to catch what’s happening, but will probably need more facts to be helpful.
The first thing that catches my eyes is that it seems you’re trying to ping to the world with local addresses. Where does the NATing happen?
Secondly, on your Linux router I’d expect some firewalling to happen - how’s that set up?
Thirdly, you might want to do some tcpdumping of your traffic to get behind what’s actually happening on the wire: Maybe it’s not the ICMP echo request that’s being send to the wrong interface, but the echo reply? I don’t know how “ping” reacts to replies recieved on a different interface when running with “-I”.
Key should be to understand how the packets actually travel across the network, including where and how they’re NATed.
Regards,
Jens[/QUOTE]