firewall/router advise

[System1]

Hi all: We are still rocking with Border Manager on a NW6.5 sp8 box. We have a 16 block of static IPs hung off of the public interface (from ISP1). Downstream is our Cisco 2811 router. When traffic leaves the pulbic interface it shows it going out from one of our static IP addresses (ISP1).

OK, so far so good, everything is fine. But we are adding a Comcast business class cable circuit to act as a redundant internet access pipe and backup to some P2P T1s going to branch offices. The Comcast service gives us one static IP address (ISP2).

Now here is the problem, when traffic leaves Border Manager, it is coming from our static IP (ISP1) as that is our primary carrier. However, in the event of a failure, the traffic will shift over to Comcast (ISP2) at the router level, but the packets will still be coming from our BM server with an IP address associated with ISP1. Traffic will still go out OK, but it will ALWAYS try to comeback on ISP1 and of course will fail since that curcuit is down.

My question is this: What firewall appliance/software would allow us to make this system work? I know SuperLumin is the BM replacement, but I don’t know if it can handle this task. Meraki may work , but it is spendy.

Thanks for your time and input, Chris.

[system]

On 27/03/2013 14:00, Chris wrote:[color=blue]

My question is this: What firewall appliance/software would allow us
to make this system work? I know SuperLumin is the BM replacement,
but I don’t know if it can handle this task. Meraki may work , but
it is spendy.[/color]

There should be no special changes needed to your infrastructure -
depends on what you have now, of course.

Normally (and I am speaking with a cisco hat on here, but other
vendors would have similar functionality) you would be using policy
based routing on the upstream router to determine when ISP1 is down (if
you were fully multihomed then BGP would be the method, but that would
also imply that your external IPs were routable via both links) and
route the traffic to the backup.

By building a DMZ between that router and the internet, you can do the
final NAT on the router itself rather than the BM server, and then have
different nats between the inside (dmz) and two outside (ISP) interfaces.

[system]

I don’t see how this will preserve our static IPs for inbound traffic. We have a 16 block of IPs (x.y.z.1/28). Our main address is x,y,z,2. But I do remote access via ssh on x,y,z,4. If I add a NAT won’t I lose this ability?

[color=blue][color=green][color=darkred]

Dave HoweDaveHowe@invalid.dom 3/27/2013 10:21 AM >>>
[/color][/color][/color]
By building a DMZ between that router and the internet, you can do the
final NAT on the router itself rather than the BM server, and then have
different nats between the inside (dmz) and two outside (ISP) interfaces.

[system]

On 27/03/2013 15:06, Chris wrote:> I don’t see how this will preserve
our static IPs for inbound traffic.[color=blue]

We have a 16 block of IPs (x.y.z.1/28). Our main address is x,y,z,2.
But I do remote access via ssh on x,y,z,4. If I add a NAT won’t I lose
this ability?[/color]

no, you double-nat.

lets say you renumber your outside BM interface to 172.100.100.x

that means x.y.z.1 is now 172.100.100.1 though to x.y.z.30 and
172.100.100.30

now on the router, you map x.y.z.0/28 → 172.100.100.0/28 for ISP#1.

assuming you get a.b.c.128/28 from ISP#2, you map a.b.c.128/28 →
172.100.100.0/28

that means that traffic leaving the BM is tagged with its DMZ address,
but gets a different outside IP depending on which interface it leaves
via. This means of course that the IPs on #2 will not work until or
unless #1 is down, or replies will go out via #1, with the wrong IP.

but

if you want it, you can have a dmz range of /27 for the BM, and two sets
of forwards; by linking the inbound from #1 to the range 1-30 and the
range from #2 to 33-62, you can have both inbound paths valid (at least
as far as the border manager box) at the same time. It perhaps goes
without saying that you will then have to have different targets
internally for the port forwards to forward onto on the border manager,
but it is usually trivial to add a second IP to the limited number of
hosts you tend to want direct inbound mappings to, to facilitate that.

the first is an active/passive setup, with #2 only usable if #1 is down.
The second is active/active, and requires more setup, but only really
requires source routing on the router with the two ISP links on it, so
that it can route traffic from BM to the “right” interface.

[system]

Chris wrote:
[color=blue]

Now here is the problem, when traffic leaves Border Manager, it is
coming from our static IP (ISP1) as that is our primary carrier.
However, in the event of a failure, the traffic will shift over to
Comcast (ISP2) at the router level, but the packets will still be
coming from our BM server with an IP address associated with ISP1.[/color]

I assume your LAN uses private IP addresses so outgoing packets will
always be assigned one of the public IP addresses associated with the
interface from which it leaves. I also assume you have rules to ensure
that packets arriving via ISP1 will be assigned the appropriate IP
address for the reply.

When you have two public interfaces, you have to ensure the reply is
sent from the same interface from which the original packet arrived
otherwise the source IP address will be different on the outgoing
packets and when returned to the source machine, it will not be
recognised as a valid reply.

If you have a failure affecting your ISP1 interface or your ISP1, you
won’t have any packets arriving from that interface so the issue with
having them returned via the ISP2 interface is not a concern.

Having a second ISP helps with traffic originating from your LAN. You
may create rules to direct web traffic to the interface with the
greater bandwidth, for example, and can direct all traffic to one
interface if the other one fails but it won’t help for in-bound traffic
associated with one specific IP address from your block of sixteen
static addresses. If you have an email server associated with IP
address x.x.x.6 and you lose connectivity with your ISP1, you will not
receive any incoming email but with appropriate rules outgoing email
could still be sent via ISP2.

To make any of this work your router will have two public interfaces
and appropriate routing rules. I don’t know whether BorderManager alone
can do the job, even with two public interfaces. You may need SLES with
SuSEfirewall or a separate router.

–
Kevin Boyle - Knowledge Partner
If you find this post helpful and are logged into the web interface,
show your appreciation and click on the star below…

[system]

OK, I think I am getting it. However, ISP2 is only giving us 1 IP address, x.y.z.85/32.

[color=blue][color=green][color=darkred]

Dave HoweDaveHowe@invalid.dom 3/27/2013 12:36 PM >>>
[/color][/color][/color]
On 27/03/2013 15:06, Chris wrote:> I don’t see how this will preserve
our static IPs for inbound traffic.
[color=blue]
We have a 16 block of IPs (x.y.z.1/28). Our main address is x,y,z,2.
But I do remote access via ssh on x,y,z,4. If I add a NAT won’t I lose
this ability?
[/color]

no, you double-nat.

lets say you renumber your outside BM interface to 172.100.100.x

that means x.y.z.1 is now 172.100.100.1 though to x.y.z.30 and
172.100.100.30

now on the router, you map x.y.z.0/28 → 172.100.100.0/28 for ISP#1.

assuming you get a.b.c.128/28 from ISP#2, you map a.b.c.128/28 →
172.100.100.0/28

that means that traffic leaving the BM is tagged with its DMZ address,
but gets a different outside IP depending on which interface it leaves
via. This means of course that the IPs on #2 will not work until or
unless #1 is down, or replies will go out via #1, with the wrong IP.

but

if you want it, you can have a dmz range of /27 for the BM, and two sets
of forwards; by linking the inbound from #1 to the range 1-30 and the
range from #2 to 33-62, you can have both inbound paths valid (at least
as far as the border manager box) at the same time. It perhaps goes
without saying that you will then have to have different targets
internally for the port forwards to forward onto on the border manager,
but it is usually trivial to add a second IP to the limited number of
hosts you tend to want direct inbound mappings to, to facilitate that.

the first is an active/passive setup, with #2 only usable if #1 is down.
The second is active/active, and requires more setup, but only really
requires source routing on the router with the two ISP links on it, so
that it can route traffic from BM to the “right” interface.

[system]

Thanks for the reply. The 16 public IP are bound to the public interface of the BM server. However, the BM server routes traffic to our Cisco 2811 which has two public interfaces, one pointing to ISP1 and the other pointing to ISP2. Default routes have all traffic going to ISP1 like the following:

mail server is a secondary address on the BM public interface with ip of x.y.z.3. The default route point all traffic to x.y.z.1 which is the IP address of our serial0/0/0 going to ISP1. So the hops would be x.y.z.3 → x.y.z.1 → ISP1 or vise versa for inbound initiated traffic. This is NOT nat’ed.

Now, in the event of failure of serial0/0/0 to ISP1, the route would be x.y.z.3 → a.b.c.86 → ISP2 via cable. Since we only have the one static IP for the cable connection, this would have to be nat’ed. With a nat we can’t do the reverse which would be ISP2 → a.b.c.86 → a.b.c.85 (cable modem) → x.y.z.2 (or 3 or 4 etc) for inbound initiated traffic. This where I get really confused.

[color=blue][color=green][color=darkred]

KBOYLEKBOYLE@no-mx.forums.novell.com 3/27/2013 1:11 PM >>>
[/color][/color][/color]
Chris wrote:

[color=blue]

Now here is the problem, when traffic leaves Border Manager, it is
coming from our static IP (ISP1) as that is our primary carrier.
However, in the event of a failure, the traffic will shift over to
Comcast (ISP2) at the router level, but the packets will still be
coming from our BM server with an IP address associated with ISP1.
[/color]

I assume your LAN uses private IP addresses so outgoing packets will
always be assigned one of the public IP addresses associated with the
interface from which it leaves. I also assume you have rules to ensure
that packets arriving via ISP1 will be assigned the appropriate IP
address for the reply.

When you have two public interfaces, you have to ensure the reply is
sent from the same interface from which the original packet arrived
otherwise the source IP address will be different on the outgoing
packets and when returned to the source machine, it will not be
recognised as a valid reply.

If you have a failure affecting your ISP1 interface or your ISP1, you
won’t have any packets arriving from that interface so the issue with
having them returned via the ISP2 interface is not a concern.

Having a second ISP helps with traffic originating from your LAN. You
may create rules to direct web traffic to the interface with the
greater bandwidth, for example, and can direct all traffic to one
interface if the other one fails but it won’t help for in-bound traffic
associated with one specific IP address from your block of sixteen
static addresses. If you have an email server associated with IP
address x.x.x.6 and you lose connectivity with your ISP1, you will not
receive any incoming email but with appropriate rules outgoing email
could still be sent via ISP2.

To make any of this work your router will have two public interfaces
and appropriate routing rules. I don’t know whether BorderManager alone
can do the job, even with two public interfaces. You may need SLES with
SuSEfirewall or a separate router.

–
Kevin Boyle - Knowledge Partner
If you find this post helpful and are logged into the web interface,
show your appreciation and click on the star below…

[system]

Chris wrote:
[color=blue]

Now, in the event of failure of serial0/0/0 to ISP1, the route would
be x.y.z.3 → a.b.c.86 → ISP2 via cable. Since we only have the one
static IP for the cable connection, this would have to be nat’ed.
With a nat we can’t do the reverse which would be ISP2 → a.b.c.86 →
a.b.c.85 (cable modem) → x.y.z.2 (or 3 or 4 etc) for inbound
initiated traffic. This where I get really confused.[/color]

There should be no confusion. What you want can’t be done!

There are two things to consider. The first is how to get incoming
traffic from ISP2, all with the same public IP address, to a specific
host on your LAN. Since /all/ arriving packets have the same IP
address, you can’t use the IP address to determine the destination. You
could use port forwarding though. Say, for example, traffic with a
destination port number of 25 /could/ be routed to your mail server.
Similarly traffic with a destination port of 80 could be routed to your
web server.

The second thing to consider is how packets destined for your email
server or web server would ever arrive via ISP2. A DNS lookup to your
email server or web server will always return the same IP address, one
belonging to your block of sixteen from ISP1. It is that IP address
that is used to reach your server(s) therefore you will never receive
web and/or email traffic via ISP2.

A second ISP is a great redundant solution for outbound LAN traffic but
provides only limited benefits for incoming requests.

–
Kevin Boyle - Knowledge Partner
If you find this post helpful and are logged into the web interface,
show your appreciation and click on the star below…

[system]

Thanks.

[color=blue][color=green][color=darkred]

KBOYLEKBOYLE@no-mx.forums.novell.com 3/27/2013 3:29 PM >>>
[/color][/color][/color]
Chris wrote:

[color=blue]

Now, in the event of failure of serial0/0/0 to ISP1, the route would
be x.y.z.3 → a.b.c.86 → ISP2 via cable. Since we only have the one
static IP for the cable connection, this would have to be nat’ed.
With a nat we can’t do the reverse which would be ISP2 → a.b.c.86 →
a.b.c.85 (cable modem) → x.y.z.2 (or 3 or 4 etc) for inbound
initiated traffic. This where I get really confused.
[/color]

There should be no confusion. What you want can’t be done!

There are two things to consider. The first is how to get incoming
traffic from ISP2, all with the same public IP address, to a specific
host on your LAN. Since /all/ arriving packets have the same IP
address, you can’t use the IP address to determine the destination. You
could use port forwarding though. Say, for example, traffic with a
destination port number of 25 /could/ be routed to your mail server.
Similarly traffic with a destination port of 80 could be routed to your
web server.

The second thing to consider is how packets destined for your email
server or web server would ever arrive via ISP2. A DNS lookup to your
email server or web server will always return the same IP address, one
belonging to your block of sixteen from ISP1. It is that IP address
that is used to reach your server(s) therefore you will never receive
web and/or email traffic via ISP2.

A second ISP is a great redundant solution for outbound LAN traffic but
provides only limited benefits for incoming requests.

–
Kevin Boyle - Knowledge Partner
If you find this post helpful and are logged into the web interface,
show your appreciation and click on the star below…

[system]

Chris wrote:
[color=blue]

Traffic will still go out OK, but it will ALWAYS try to comeback on
ISP1 and of course will fail since that curcuit is down.[/color]

Not true!

Since outbound traffic via ISP2 is NATted, the outbound packets will
contain the ISP2 public IP address. This will ensure that replies will
return via ISP2.

Depending how your NAT is setup, it would substitute the ISP2 public IP
address on /all/ outgoing packets - even those containing one of your
static IP addresses from ISP1.

–
Kevin Boyle - Knowledge Partner
If you find this post helpful and are logged into the web interface,
show your appreciation and click on the star below…

[system]

On 27/03/2013 17:40, Chris wrote:[color=blue]

OK, I think I am getting it. However, ISP2 is only giving us 1 IP
address, x.y.z.85/32.[/color]

not as handy then. to make that work, you would need to overload
different services onto the single IP - you can do that, and it works
reasonably well, but obviously that means you can only have one of any
given service on its default port (so one smtp, one http, and so forth)

still works with either method (active/passive or active/active)