Is Your Bandwidth Getting Too Big for Your Pipes?

31 Mar

Posted by Joe as Commentary

Today many companies and IT departments are seeing their current DS3's being utilized to their maximum ability. Many are hitting their daily throughput of 40-45Mbs and don't know what the best migration path might be. With most companies still using serial HSSI interfaces that can't be upgraded where do you go from here?

Many Carriers are quickly moving to metro Ethernet as an easy migration path for their serial customers who need more bandwidth than that standard 45MBs. With Metro Ethernet you can have the freedom to start at 60Mbs with a burstable rate above this threshold. This gives you the breathing room for today and the abiliy to increase to a full 100MBs later as you bandwidth needs grow.

But What About the Router?

Many of you maybe thinking, "That's fine but what about the router?" Well leave it to Cisco to be thinking ahead and if you are one of the lucky folks who have a 2600, 2800, 3600, 3700, or 3800 series router, boy do they have a card for you! What can be easily migrated to from your standard serial card is a 16 or 32 port Ethernet Switch card like the NM-16ESW 16 port EtherSwitch Module.

These modules are designed to fit right in the existing network module slots in your router. Combining robust Layer 3 flexible WAN routing with low-density line-rate Layer 2 switching, the EtherSwitch modules provide straightforward configuration, easy deployment and integrated management in a single platform.

Configuring these modules are as easy as configuring any switch interface. But understand that these new interfaces are straight layer 2 interfaces and can not be assigned an ip address. To get around this we simply create a vlan interface, assign the needed ip address for your carrier uplink, then assigned the new vlan to the switch port you've plugged in your new metro ethernet carrier handoff. You can do this and keep your existing ip addresses so no routing or acl changes need to occur. Viola' your done!

One final thing to keep in mind if you want to move your current ip address from your serial interface to your new ethernet interface. You will need to setup a coordinated cutover with your service provider. This again is very simple and quick for most providers as its simply a matter of moving your ip address to a new ethernet interface. But be perpared. Do expect a breif outage during the cutover and as always have a backup of your config and be prepared to move back in the event of an emergency.

For more information on these EtherSwitch modules for your router check out the specs on Cisco's website.

2 Responses

JohnG
April 7th, 2009 at 1:20 pm
1
Just throwing in some real world experience around using older or lower/mid range routers on new high speed metro ethernet connections.
Our use case is using 100meg Metro ethernet for site to site back ups/ database replication, etc from a local hosted datacenter back to our office. This is occuring in our local metro area and we regularly fill the 100Meg pipe for sustained periods of time (1-6 hours).
Initially, we had some older 2600 series routers that were fine for serial WAN access. However, when used for supporting the 100meg metro ethernet WAN link. CPU capped at 100% with the ‘IP Input’ process using all the CPU at around 30Mbs/s of sustained throughput. (using CEF and no QOS, no ACLs or anything else that would cause packet to be process switched). Bottom line is that 30Mbs/s seems to be a cap for these old legacy 2600’s
Upgraded to a Cisco 2821 router which has CPU running at 20-25% at sustained 100Mbs/s, which is great. But if we upgraged the link to 1Gbs/s I’m speculating that the 2820 would hit 100% at 500-600Mbs/s. If we apply anything that causes the router to Process Switch packets (QOS, ACLs, etc), I’m guessing that we would get a fraction of that throughput.
The Metro Ethernet stuff is great. My point of all this is that just because a router has a 100meg or 1gig interface does not mean that particular router will be able to pump data at full line rate for a sustained period of time. (especially if you are dealing with routers at the lower end of the spectrum) You need to get an appropriately sized router for the interface speed, and if you are turning on features that will process switch traffic (ACLs, QOS, etc) you need to upsize the router even futher. Your constraint will end up being router CPU rather than the Router interface speed or the Metro Ethernet WAN speed (60Mbs/s, 100Mbs/s, 1Gbs/s, etc).
PS – Great articles and I love your blog.
Joe
April 7th, 2009 at 1:32 pm
2
John,
I completely agree and these are great points! I had installed an 1800 router some time ago that we had also included the crypto aim module to ensure better performing VPN. This router was going to be used for a remote office connection with back office connectivity over the internet via the ipsec vpn tunnel.
Of course the Cisco numbers said it could easily do what we wanted and for the price looked to be a great solution. However after putting it to a real world test, we could only get 14Mbs or so of traffic through the vpn tunnel due to the proc and memory.
Needless to say we ended up getting an ASA which was a little more but did the job perfectly!
Thanks again for the great comment!