Slow data access from remote side of T1. 2

[TravisM]

I have a problem that I really am having a difficult time with. Essentially data moving from the orignating side of this T1 is crawling along, but data moving from the remote side to the originating side is just fine. Example;

I can log into a PC in this remote office, pick up a ~4mb file and dump it out to a machine in the home office in about 20 - 25 seconds. If I try to copy the file back to the machine on the remote side, it take ~120 second.

This is a Point-to-point full T1, the encapsulation was set to PPP on the link, I changed it to HDLC and that seemed to drop transfer times to ~60 - 70 seconds for the same file, but there is still a huge disperity. I have replaced the router (but not the CSU) on the remote side with another router with no luck, I've also replaced all cabling with known good cable - still no results. TELCO says the link tests clean, but that my equipment is spitting up some errors - the errors are very minimal though.

Does anyone have any ideas on this? The router at the main office is a 3640 servicing 6 sites total, none of the other sites are having a problem like this. The only things I can think of to replace now are the CSU/DSU's, but is it even possible they are the culprit?

Thanks for reading all of this, any ideas wouls be appreciated.

-Travis.

 

[tschouten]

Hey Travis,

First if you could post the errors, that would be a great help. Next try posting a show interface want to see that as well.

The only other thing I can think of is checking you CSU/DSU and make sure it is set correctly. To answer your question yes, the CSU/DSU can cause problems. Mostly due to incorrect configuration or faulty timing.

 

[TravisM]

Here is the show int on the remote side;

Serial0 is up, line protocol is up
Hardware is QUICC Serial
Description: connected to Cisco3620
Interface is unnumbered. Using address of Ethernet0 (192.168.3.1)
MTU 1500 bytes, BW 1544 Kbit, DLY 20000 usec,
reliability 255/255, txload 2/255, rxload 11/255
Encapsulation HDLC, loopback not set
Keepalive set (10 sec)
Last input 00:00:00, output 00:00:00, output hang never
Last clearing of "show interface" counters 14:38:15
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
Queueing strategy: weighted fair
Output queue: 0/1000/64/0 (size/max total/threshold/drops)
Conversations 0/7/256 (active/max active/max total)
Reserved Conversations 0/0 (allocated/max allocated)
Available Bandwidth 1158 kilobits/sec
5 minute input rate 67000 bits/sec, 9 packets/sec
5 minute output rate 13000 bits/sec, 9 packets/sec
136405 packets input, 76246023 bytes, 0 no buffer
Received 6121 broadcasts, 0 runts, 0 giants, 0 throttles
2 input errors, 0 CRC, 2 frame, 0 overrun, 0 ignored, 0 abort
137533 packets output, 30549927 bytes, 0 underruns
0 output errors, 0 collisions, 5 interface resets
0 output buffer failures, 0 output buffers swapped out
0 carrier transitions
DCD=up DSR=up DTR=up RTS=up CTS=up

Here is the show int from the host side;

Serial1/0 is up, line protocol is up
Hardware is QUICC with integrated T1 CSU/DSU
Description: connected to 4117
Interface is unnumbered. Using address of Ethernet0/0 (192.168.1.1)
MTU 1500 bytes, BW 1544 Kbit, DLY 20000 usec,
reliability 255/255, txload 10/255, rxload 1/255
Encapsulation HDLC, loopback not set
Keepalive set (10 sec)
Last input 00:00:05, output 00:00:00, output hang never
Last clearing of "show interface" counters 14:30:42
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
Queueing strategy: weighted fair
Output queue: 0/1000/64/0 (size/max total/threshold/drops)
Conversations 0/16/256 (active/max active/max total)
Reserved Conversations 0/0 (allocated/max allocated)
Available Bandwidth 1158 kilobits/sec
5 minute input rate 12000 bits/sec, 7 packets/sec
5 minute output rate 64000 bits/sec, 6 packets/sec
137898 packets input, 30628933 bytes, 0 no buffer
Received 6138 broadcasts, 0 runts, 0 giants, 0 throttles
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort
136791 packets output, 76431068 bytes, 0 underruns
0 output errors, 0 collisions, 1 interface resets
0 output buffer failures, 0 output buffers swapped out
0 carrier transitions
DCD=up DSR=up DTR=up RTS=up CTS=up

Here is the Show service-module from the host side;

Module type is T1/fractional
Hardware revision is 0.96, Software revision is 0.2,
Image checksum is 0x70F47262, Protocol revision is 0.1
Receiver has no alarms.
Framing is ESF, Line Code is B8ZS, Current clock source is line,
Fraction has 24 timeslots (64 Kbits/sec each), Net bandwidth is 1536 Kbits/sec.
Last module self-test (done at startup): Passed
Last clearing of alarm counters 17:23:36
loss of signal : 0,
loss of frame : 1, last occurred 15:56:00
AIS alarm : 0,
Remote alarm : 0,
Module access errors : 0,
Total Data (last 96 15 minute intervals):
510 Line Code Violations, 1786 Path Code Violations
19 Slip Secs, 332 Fr Loss Secs, 11 Line Err Secs, 113 Degraded Mins
7 Errored Secs, 1 Bursty Err Secs, 13 Severely Err Secs, 360 Unavail Secs
Data in current interval (864 seconds elapsed):
0 Line Code Violations, 0 Path Code Violations
0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins
0 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs

The CSU/DSU on the remote side is an adtran unit and it is not showing any alarms. I'm going to replace that unit right now with a Cisco T1 card, and hopefully that will make a change.

Thanks for the help.
-Travis.

 

[tschouten]

I can't say that I saw anything really all that wrong. Maybe Jimmyzz can see something, or fmonterio. Both are pretty good at spotting things. I'm still looking to see if I notice any errors. But, truthfully I'm not seeing it.

 

[TravisM]

That's why I'm just so stumped, I can't make heads or tails of this. I'll just continue to swap out this hardware until it's all verified, then call the TelCo again.

Thanks for looking at it for me, and getting back so quickly!

-Travis.

 

[tschouten]

Not a problem, I mean I see you have a few errors here and there. But everything is within exceptable limits nothing should be giving you a problem. At least nothing that I can see as of now. I can't think of anything else to try to look at (go figure).

Do you have any access-lists, rate-limiting, or are you running encryption of any kind? These are the only things I can think of that would slow a response down. Perhaps NAT, or something like that??

 

[tschouten]

looking at that wondering to myself where did I learn how to spell???

 

[TravisM]

None of the above. I just rewrote the config on the remote router last thing in the evening yesterday just to be sure, no luck there.

I'm starting to think more and more that this is a line issue, or maybe something between the CO and my Demarc.

-Travis.

 

[jimmyzz]

Travis,
Your T1 modules is showing some Line Code Violations:
510 Line Code Violations, 1786 Path Code Violations

Line code errors could indicate a framing mismatch. You are currently using line code B8ZS. You could check with your supplier to confirm the correct line code format, or try (out of business hours if possible) to change to the other line code alternative: AMI

Router(config-if)>linecode ami

If still a problem, please post your config for HQ and remote router.

JimmyZ

 

[TravisM]

I pulled that module show after the TelCo ran some intrusive tests on the lines, so I didn'tthink much of it at the time. I just pulled it again and it looks like there are still some Path viloations, but line code is showing 0. I am going to be hanging out tonight (switching the circuit to another WAN port on the HQ router, just to get that out of the way). Assuming that does nothing, I'll try the line type switch. The tag on the circuit it B8ZS, but you never really know with Qwest.

Thanks for the suggestions!

 

[jimmyzz]

Travis,
Most of the modern CSU/DSU will default to ESF/B8ZS framing/linecode format. Older T1 will use SF/AMI combo. If line code is now 0, then I'd probably leave the B8ZS setting (though its probably still worthwhile testing the AMI setting). It was just something I picked up in your earlier post.

You could try an extended ping test. Put your CSU/DSU into loopback mode. Do an extended ping and send 1500-byte packets and a data pattern of 0x0000 (all zeros). Repeat the process and send a data pattern of 0xFFFF (all ones). If your input errors do not increase, the all the local hardware (router, cables and CSU/DSU) should be okay. Do at both sites if possible. If the local hardware checks out ok at both ends, then the problem may be with the ISP.

JimmyZ

 

[TravisM]

Awesome info, I will give that a shot. I'm pretty certain at this point it's on the TelCo side. I just popped the circuit (was on s1/0) over to another wan port (s1/1) in the 3640 that a different circuit was running fine with. The problem stayed with the circuit despite the swap.

The only problem I have now is trying to get Qwest to fix it. The line is running clean from NIU to NIU, and they really don't want to do much about it.

Thanks for your time, gents.

-Travis.

 

[TravisM]

Oh wow... Could this be part of the problem?

Ethernet0/0 is up, line protocol is up
Hardware is AmdP2, address is 0009.b7a1.ad41 (bia 0009.b7a1.ad41)
Description: connected to EthernetLAN
Internet address is 192.168.1.1/24
MTU 1500 bytes, BW 10000 Kbit, DLY 1000 usec,
reliability 255/255, txload 7/255, rxload 5/255
Encapsulation ARPA, loopback not set
Keepalive not set
ARP type: ARPA, ARP Timeout 04:00:00
Last input 00:00:00, output 00:00:00, output hang never
Last clearing of "show interface" counters 01:37:26
Queueing strategy: fifo
Output queue 0/40, 0 drops; input queue 0/75, 0 drops
5 minute input rate 217000 bits/sec, 279 packets/sec
5 minute output rate 298000 bits/sec, 284 packets/sec
1651881 packets input, 225747033 bytes, 0 no buffer
Received 3433 broadcasts, 0 runts, 0 giants, 0 throttles
51018 input errors, 51018 CRC, 0 frame, 0 overrun, 0 ignored
0 input packets with dribble condition detected
1566344 packets output, 206982189 bytes, 0 underruns
270 output errors, 0 collisions, 0 interface resets
0 babbles, 0 late collision, 0 deferred
270 lost carrier, 0 no carrier
0 output buffer failures, 0 output buffers swapped out

Any suggestions or ideas on what could be causing that? Bad interface?

 

[Jynxx]

Not sure if this will help you out, but here is what I have configured for a PPP T1 between two 7200's for a backup link.

7200 #1

controller T1 1/1
framing esf
clock source internal
linecode b8zs
channel-group 0 timeslots 1-24
!
interface Serial1/1:0
bandwidth 1536000
ip address 172.22.1.1 255.255.255.252
no ip proxy-arp
no fair-queue


7200 #2


controller T1 1/1
framing esf
clock source line
linecode b8zs
channel-group 0 timeslots 1-24
!
interface Serial1/1:0
bandwidth 1536000
ip address 172.22.1.2 255.255.255.252
no ip proxy-arp
no fair-queue

As you can see, I do not configure ANY encapsulation for PPP, and set up one side for line clocking and the other for internal. Now...if you are using external CSU's, obviously this will be different, but I hope this helps.

In terms of your CRC and interface resets...it looks like a auto-negotiation/duplex mismatch. I would recommend forcing the Ethernet port to 100/full as well as the device that it connects to.