Help!!! Output Queue Drops

[shan33]

I'm having some problems with my office internet connection lately. And there is nothing the ISP can do. I hope somebody can explain to me what is happening. Below are the info:

cisco1700#sh int ser0
Serial0 is up, line protocol is up
Hardware is PowerQUICC Serial
Internet address is x.x.x.6/29
MTU 1500 bytes, BW 1544 Kbit, DLY 20000 usec,
reliability 255/255, txload 13/255, rxload 1/255
Encapsulation HDLC, loopback not set
Keepalive set (10 sec)
Last input 00:00:00, output 00:00:00, output hang never
Last clearing of "show interface" counters 00:31:11
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 192
Queueing strategy: weighted fair
Output queue: 0/1000/64/192 (size/max total/threshold/drops)
Conversations 0/49/256 (active/max active/max total)
Reserved Conversations 0/0 (allocated/max allocated)
Available Bandwidth 1158 kilobits/sec
5 minute input rate 8000 bits/sec, 20 packets/sec
5 minute output rate 81000 bits/sec, 18 packets/sec
33824 packets input, 3383068 bytes, 0 no buffer
Received 321 broadcasts, 0 runts, 0 giants, 0 throttles
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort
36541 packets output, 12039205 bytes, 0 underruns
0 output errors, 0 collisions, 0 interface resets
0 output buffer failures, 0 output buffers swapped out
0 carrier transitions
DCD=up DSR=up DTR=up RTS=up CTS=up

Denied traffic log:
09-23-2003 19:14:36 Local7.Info 192.168.10.1 118105: 2w6d: %SEC-6-IPACCESSLOGP: list 101 denied tcp 61.145.199.112(37562) -> x.x.x.87(80), 1 packet
09-23-2003 19:14:21 Local7.Info 192.168.10.1 118104: 2w6d: %SEC-6-IPACCESSLOGRL: access-list logging rate-limited or missed 18 packets
09-23-2003 19:14:17 Local7.Info 192.168.10.1 118103: 2w6d: %SEC-6-IPACCESSLOGP: list 101 denied tcp 216.100.90.253(37643) -> 255.255.255.255(80), 1 packet
09-23-2003 19:13:40 Local7.Info 192.168.10.1 118102: 2w6d: %SEC-6-IPACCESSLOGP: list 101 denied tcp 202.196.68.113(4654) -> 255.255.255.255(80), 1 packet
09-23-2003 19:13:28 Local7.Info 192.168.10.1 118101: 2w6d: %SEC-6-IPACCESSLOGP: list 101 denied tcp 213.38.131.14(1787) -> x.x.x.6(445), 1 packet
09-23-2003 19:13:21 Local7.Info 192.168.10.1 118100: 2w6d: %SEC-6-IPACCESSLOGP: list 101 denied tcp 24.80.0.61(2829) -> 255.255.255.255(80), 1 packet
09-23-2003 19:12:21 Local7.Info 192.168.10.1 118099: 2w6d: %SEC-6-IPACCESSLOGRL: access-list logging rate-limited or missed 4 packets
09-23-2003 19:11:43 Local7.Info 192.168.10.1 118098: 2w6d: %SEC-6-IPACCESSLOGP: list 101 denied tcp 218.94.33.220(4428) -> 255.255.255.255(80), 1 packet
09-23-2003 19:11:28 Local7.Info 192.168.10.1 118097: 2w6d: %SEC-6-IPACCESSLOGP: list 101 denied tcp 218.1.188.152(3533) -> 255.255.255.255(80), 1 packet
09-23-2003 19:11:21 Local7.Info 192.168.10.1 118096: 2w6d: %SEC-6-IPACCESSLOGRL: access-list logging rate-limited or missed 21 packets
09-23-2003 19:11:16 Local7.Info 192.168.10.1 118095: 2w6d: %SEC-6-IPACCESSLOGP: list 101 denied tcp 218.199.143.67(2169) -> x.x.x.80(80), 1 packet
09-23-2003 19:11:01 Local7.Info 192.168.10.1 118094: 2w6d: %SEC-6-IPACCESSLOGP: list 101 denied tcp 202.149.215.77(2254) -> x.x.x.80(445), 1 packet
09-23-2003 19:10:59 Local7.Info 192.168.10.1 118093: 2w6d: %SEC-6-IPACCESSLOGP: list 101 denied tcp 24.112.228.4(4681) -> 255.255.255.255(80), 1 packet
09-23-2003 19:10:21 Local7.Info 192.168.10.1 118092: 2w6d: %SEC-6-IPACCESSLOGRL: access-list logging rate-limited or missed 12 packets
09-23-2003 19:10:08 Local7.Info 192.168.10.1 118091: 2w6d: %SEC-6-IPACCESSLOGP: list 101 denied tcp 218.75.22.210(62205) -> 255.255.255.255(80), 1 packet
09-23-2003 19:09:45 Local7.Info 192.168.10.1 118090: 2w6d: %SEC-6-IPACCESSLOGP: list 101 denied tcp 218.241.91.93(23765) -> x.x.x.81(80), 1 packet
09-23-2003 19:09:42 Local7.Info 192.168.10.1 118089: 2w6d: %SEC-6-IPACCESSLOGP: list 101 denied tcp 218.199.143.67(4319) -> 255.255.255.255(80), 1 packet
09-23-2003 19:09:38 Local7.Info 192.168.10.1 118088: 2w6d: %SEC-6-IPACCESSLOGP: list 101 denied tcp 213.244.185.68(1473) -> x.x.x.86(113), 1 packet
09-23-2003 19:09:36 Local7.Info 192.168.10.1 118087: 2w6d: %SEC-6-IPACCESSLOGP: list 101 denied tcp 218.241.91.93(23677) -> x.x.x.81(80), 1 packet
09-23-2003 19:09:21 Local7.Info 192.168.10.1 118086: 2w6d: %SEC-6-IPACCESSLOGRL: access-list logging rate-limited or missed 3 packets
09-23-2003 19:09:12 Local7.Info 192.168.10.1 118085: 2w6d: %SEC-6-IPACCESSLOGP: list 101 denied tcp 61.151.130.120(4415) -> x.x.x.84(80), 1 packet
09-23-2003 19:08:21 Local7.Info 192.168.10.1 118084: 2w6d: %SEC-6-IPACCESSLOGRL: access-list logging rate-limited or missed 2 packets
09-23-2003 19:07:27 Local7.Info 192.168.10.1 118083: 2w6d: %SEC-6-IPACCESSLOGP: list 101 denied tcp 213.244.185.68(1461) -> x.x.x.86(113), 1 packet
09-23-2003 19:07:21 Local7.Info 192.168.10.1 118082: 2w6d: %SEC-6-IPACCESSLOGRL: access-list logging rate-limited or missed 4 packets
09-23-2003 19:07:14 Local7.Info 192.168.10.1 118081: 2w6d: %SEC-6-IPACCESSLOGP: list 101 denied tcp 209.2.88.129(4257) -> x.x.x.84(80), 1 packet
09-23-2003 19:06:21 Local7.Info 192.168.10.1 118080: 2w6d: %SEC-6-IPACCESSLOGRL: access-list logging rate-limited or missed 14 packets
09-23-2003 19:06:02 Local7.Info 192.168.10.1 118079: 2w6d: %SEC-6-IPACCESSLOGP: list 101 denied tcp 195.25.227.2(2311) -> x.x.x.80(445), 1 packet
09-23-2003 19:05:24 Local7.Info 192.168.10.1 118078: 2w6d: %SEC-6-IPACCESSLOGP: list 101 denied tcp 202.96.179.244(4656) -> 255.255.255.255(445), 1 packet
09-23-2003 19:05:21 Local7.Info 192.168.10.1 118077: 2w6d: %SEC-6-IPACCESSLOGRL: access-list logging rate-limited or missed 5 packets
09-23-2003 19:05:04 Local7.Info 192.168.10.1 118076: 2w6d: %SEC-6-IPACCESSLOGP: list 101 denied tcp 202.110.229.97(2372) -> x.x.x.80(80), 1 packet
09-23-2003 19:04:21 Local7.Info 192.168.10.1 118075: 2w6d: %SEC-6-IPACCESSLOGRL: access-list logging rate-limited or missed 9 packets
09-23-2003 19:04:13 Local7.Info 192.168.10.1 118074: 2w6d: %SEC-6-IPACCESSLOGP: list 101 denied tcp 213.244.185.68(1434) -> x.x.x.86(113), 1 packet
09-23-2003 19:03:56 Local7.Info 192.168.10.1 118073: 2w6d: %SEC-6-IPACCESSLOGP: list 101 denied tcp 80.191.122.2(58142) -> x.x.x.85(80), 1 packet
09-23-2003 19:03:42 Local7.Info 192.168.10.1 118072: 2w6d: %SEC-6-IPACCESSLOGP: list 101 denied tcp 210.220.73.26(31547) -> 255.255.255.255(80), 1 packet
09-23-2003 19:03:40 Local7.Info 192.168.10.1 118071: 2w6d: %SEC-6-IPACCESSLOGP: list 101 denied tcp 68.123.28.137(3345) -> x.x.x.82(80), 1 packet
09-23-2003 19:03:26 Local7.Info 192.168.10.1 118070: 2w6d: %SEC-6-IPACCESSLOGP: list 101 denied tcp 202.110.229.97(60598) -> 255.255.255.255(80), 1 packet
09-23-2003 19:03:03 Local7.Info 192.168.10.1 118069: 2w6d: %SEC-6-IPACCESSLOGP: list 101 denied tcp 213.244.185.68(1426) -> x.x.x.86(113), 1 packet
09-23-2003 19:02:21 Local7.Info 192.168.10.1 118068: 2w6d: %SEC-6-IPACCESSLOGP: list 101 denied tcp 202.149.208.110(13091) -> 255.255.255.255(80), 2 packets
09-23-2003 19:02:02 Local7.Info 192.168.10.1 118067: 2w6d: %SEC-6-IPACCESSLOGP: list 101 denied tcp 195.25.227.2(4670) -> 255.255.255.255(445), 1 packet
09-23-2003 19:01:57 Local7.Info 192.168.10.1 118066: 2w6d: %SEC-6-IPACCESSLOGP: list 101 denied tcp 213.244.185.68(1415) -> x.x.x.86(113), 1 packet

There's a lot of traffic trying to come in but it has been blocked by the router's extended access-list. Imagine from the syslog server, I have about 228MPH for the denied log.
Can somebody pls help me????
Thanks

 

[tschouten]

Uhhhm, what dose your access list look like, and what are you rate limiting? Looks like you have port 80 limited so if you recieve excesive amounts of of that traffic (which you are bound to) the router drops the high volume packets.

I'm not certain on this, this is just a guess from what you have posted right now. By the way port 80 is website traffi if that is what you have rate-limited. If you could though, post your config or just the access lists and what you interface you have them applied to, that would help a lot.

 

[shan33]

My ACL as per below:
Inbound:
access-list 101 permit tcp any any established
access-list 101 permit icmp any any echo-reply
access-list 101 permit icmp any any time-exceeded
access-list 101 permit icmp any any packet-too-big
access-list 101 permit icmp any any traceroute
access-list 101 permit icmp any any unreachable
access-list 101 permit tcp any any eq 3389
access-list 101 permit tcp any any eq 5631
access-list 101 permit udp any any eq 5632
access-list 101 permit tcp any host x.x.x.86 eq 25
access-list 101 permit tcp any host x.x.x.86 eq 110
access-list 101 permit tcp host z.z.z.29 eq 25 any
access-list 101 permit tcp host z.z.z.29 eq 110 any
access-list 101 permit tcp any any eq 53
access-list 101 permit udp any any eq 53
access-list 101 permit tcp any eq 53 any
access-list 101 permit udp any eq 53 any
access-list 101 permit tcp any 192.168.10.0 0.0.0.255 eq 80
access-list 101 permit tcp any host x.x.x.83 eq 80
access-list 101 permit tcp any host x.x.x.86 eq 80
access-list 101 permit tcp any host x.x.x.6 eq 80
access-list 101 permit tcp any 192.168.10.0 0.0.0.255 eq 443
access-list 101 permit tcp any host x.x.x.86 eq 443
access-list 101 permit tcp any host x.x.x.6 eq 443
access-list 101 permit tcp any any eq 1863
access-list 101 permit udp any host x.x.x.85 eq 15330
access-list 101 permit udp any host x.x.x.85 eq 15331
access-list 101 permit udp any host x.x.x.85 eq 15332
access-list 101 permit udp any host x.x.x.85 eq 15333
access-list 101 permit tcp host y.y.y.179 any eq 137
access-list 101 permit udp host y.y.y.179 any eq 137
access-list 101 permit tcp host y.y.y.179 any eq 139
access-list 101 permit tcp host y.y.y.179 any eq 445
access-list 101 permit tcp host z.z.z.29 any eq 137
access-list 101 permit udp host z.z.z.29 any eq 137
access-list 101 permit tcp host z.z.z.29 any eq 139
access-list 101 permit tcp host z.z.z.29 any eq 445
access-list 101 permit tcp host 192.168.2.122 any eq 137
access-list 101 permit udp host 192.168.2.122 any eq 137
access-list 101 permit tcp host 192.168.2.122 any eq 139
access-list 101 permit tcp host 192.168.2.122 any eq 445
access-list 101 deny ip any any log

Outbound:
access-list 102 deny udp any any eq netbios-ns
access-list 102 deny udp any any eq netbios-dgm
access-list 102 deny udp any any eq netbios-ss
access-list 102 deny udp any any eq 8998
access-list 102 deny tcp any any eq 135
access-list 102 deny tcp any any eq 137
access-list 102 deny tcp any any eq 138
access-list 102 deny tcp any any eq 139
access-list 102 permit ip any any
access-list 102 deny ip any any log

Thanks

 

[tschouten]

Yipes that is huge, one thing you can cut down on the list by getting rid of all those tcp permit x.x.x.x The first tcp permit any any allows any tcp packet through. So you need to choose what you actually need there.
All of them or just what you want through.

Also you may want to place established after a few of those that you have blocked. Are you sure you don't have any rate-limiting lists as well?

 

[shan33]

Have go thru the list again, and I have remove some. Sorry I'm still quite new in Cisco router, I don't understand what u mean by rate-limiting list, pls explain. The router's config is rather simple, only have NAT and ACL.

Initially the connection was quite ok, until the MBLaster virus spread. I'd apply the fixes and patches on all workstations, but somehow the connection still not consistent. When I look at the denied log, it's really scarry!

I'd try to remove all connections from the switch and left only the router, I still can see a lot of denied traffic from the router. I'm not sure whether this is related to the internet connection problem. Pls explain why.