High CPU utilisation on C2621XM

[CeriCob]

Hi,

I cant seem to utilize the 100Mbit bandwidth of my C2621 FastEthernet interfaces before it maxes the CPU utilization on my router (most CPU time seems to be taken at interrupt level). One interfaces serves as the LAN side and is acting as a nat inside interface and the Public interface is connected to a DMZ which is set as the nat outside interface. When downloading from a webserver on the DMZ to the LAN this is when the problem occurs

I did have quite a few ACL's set up on the inside interface, but disabled them temporarily to see if that was the issue, but at no avail. cef is enabled globally.

Does any one know what might be my issue here. Cisco specification says that the router supports 30kpps, but when checking during this activity it's showing only 500pps. Am i simply asking too much of this router to cope with 100Mbit/s traffic.

Thanks

 

[KiscoKid]

Do you know the version of IOS running on the router? I ask this because I noticed recently on the features supported by 12.4 that NAT traffic is now CEF-switched which would dramatically improve your pps.

For any IOS pre 12.4, I would anticipate a significant dropoff in pps as NAT traffic would need process switching which is an obvious burden on the CPU.

I can also tell you that PAT has a much bigger impact on CPU than static NAT for instance.

HTH

 

[CeriCob]

Hi KiscoKid,

Here is the 'show ver' output. It's a 12.2 as you can see. I think most of the packets are being fast switched though as the CPU seems to be spending most of it's time at interrupt level when the problem occurs.


Cisco Internetwork Operating System Software
IOS (tm) C2600 Software (C2600-IK9S-M), Version 12.2(27), RELEASE SOFTWARE (fc3)
Copyright (c) 1986-2004 by cisco Systems, Inc.
Compiled Tue 02-Nov-04 23:43 by kellmill
Image text-base: 0x8000808C, data-base: 0x8128E3A8

ROM: System Bootstrap, Version 12.2(7r) [cmong 7r], RELEASE SOFTWARE (fc1)

 

[KiscoKid]

Hi Cericob

I found the link below on the CCO which does mention that IP NAT translations can be a burden on CPU interrupts. In fact, I once worked with a company who was performing NAT overloading (PAT) on quite a high-end Cisco router. They suffered CPU problems a lot of the time. It was only when NAt was reconfigured not to use overloading did the CPU dramatically drop off.. something like from 99% utilisation down to around 20%.

Also I mentioned 12.4 previously and I had a look again to see what performance improvements have been made for NAT and I found the following excerpt which may be of interest to you:

2.8.25) NAT—Performance Related Enhancements
•Collection of enhancements aimed at improving overall performance of the Network Address Translation (NAT) feature within Cisco IOS Software

•Majority of the effort will be transparent to the end customer, however under certain circumstances they should see

–Optimized CPU utilization—taking longer to ramp to higher CPU percentages

–Will vary based on the IP type of traffic inspected by NAT, Specific platform in question, and other features active within the router

–Improved throughput when using NAT

•The specific enhancements are

•Support for Cisco Express Forwarding (CEF)

–TCP Flags—SYN, FIN and RST now handled in CEF

- Translation entry creation in the CEF path under

- Support for CEF

–Translation table optimization

- Improved creation and searching of translations

- Pool and Port List optimization

–Support of Fragmented Packets

Benefits

•Improved efficiency of CPU utilization when Network Address Translation is enabled in a router.

•Overall improved throughput, may vary slightly depending on the type and complexity of protocols NAT is inspecting.

http://www.cisco.com/en/US/products/ps6350/prod_bulletin09186a0080457b39.html#wp1053878

 

[CeriCob]

Hi KiscoKid

I do indeed have an overloaded NAT pool configured on the router, are there other ways I can allow the LAN clients access to the internet without using the overload option?

I will also persue upgrading my IOS to 12.4 for the NAT improvements.

Thanks

 

[KiscoKid]

Well this particular company that had this problem were only overloading for 300 people or so. So we took the decision to stop overloading and instead allocate dynamic NAT addresses from a pool.

However this approach may not work for you if you have a very large number of people who need NAT'ing or you simply don't have enough public address space.

So if you feel you have the resources to implement such a change, the following configuration would achieve this:

interface ethernet 1
ip address 10.10.20.1 255.255.255.0
ip nat inside

interface serial 0
ip address 100.100.100.1 255.255.255.0
ip nat outside

ip nat pool no-overload 100.100.100.2 100.100.100.63 prefix 24

ip nat inside source list 7 pool no-overload

access-list 1 permit 10.10.10.0 0.0.0.255

However if you don't have the resources to meet this (i.e. lack of public addresses to allocate to a NAT pool), you can improve things by being more aggressive with your NAT timers.

Various commands for modifying NAT timeouts include the following:

ip nat translation udp-timeout <seconds>
ip nat translation dns-timeout <seconds>
ip nat translation tcp-timeout <seconds>
ip nat translation finrst-timeout <seconds>

When port translation is configured, there is finer control over translation entry timeouts, because each entry contains more context about the traffic using it. Non-DNS UDP translations time out after 5 minutes; DNS times out in 1 minute. TCP translations time out after 24 hours, unless a RST or FIN is seen on the stream, in which case it times out in 1 minute.

It may be an idea to look at your active NAT translations (using show ip nat translations) to see what you tend to see a lot of. Are they UDP or TCP flows.

If you see a type of translation hanging out for a long time, try adjusting the appropriate timer above to make them time out quicker. This will hopefully give some relief to the CPU.

 

[CeriCob]

Hmmm, I dont think i'm going to have enough public IP's. There are only around 30 people accessing the internet but I only have around 15 public IP's available, the rest are taken up on the DMZ.

I guess I see IOS 12.4 coming up then. Thanks for your help.