High Jitter Count, bad echo

[jtphoneman]

Hello, I am trying to troubleshoot a IP Office 500 voice network. The customer is complaining about bad echo and delay, sometimes even dropped calls. This voice network is voice only, no data. When I look at the QOS logs I see very high jitter counts. The IP Office is connected via LAN 1 to a Cisco 3560 PoE Switch running Auto Qos. Only 1 vlan in the switch (default vlan 1). All Cisco switchports are set to auto negotiate as are the phones ( Model 5621 IP Phone). According to the customer all network cables have been tested and certified. Ip Office was just changed out and is now running version 5.0 (18)

The server running IP Office Manager and VM Pro does have a second NIC to connect the the customers data network, I thought about asking the customer to change out that NIC just for process of elimination. The customer even said they sometime have echo when dialing extension to extension. Is there any settings in the IP Office that might be misconfigured or overlooked that could cause this issue?

Last Date Of Error Call ID Device IP Address Peer IP Address Jitter Round Trip Delay Packet Loss
-------------------------------------------------------------------------------------------------------------------------
4/21/2010 2:21:59 PM 9898 Extension 407 192.168.100.51 192.168.100.31 65.5ms

4/21/2010 2:47:02 PM 9962 Extension 410 192.168.100.80 192.168.100.31 119,480.9ms

4/21/2010 3:28:56 PM 10070 Extension 412 192.168.100.55 192.168.100.31 732.2ms

4/21/2010 3:31:35 PM 10052 Extension 414 192.168.100.66 192.168.100.31 73.9ms

4/21/2010 3:32:27 PM 10072 Extension 415 192.168.100.62 192.168.100.31 2,187.9ms

4/21/2010 3:50:04 PM 10101 Extension 407 192.168.100.51 192.168.100.31 69.4ms

4/21/2010 3:57:47 PM 10135 Extension 242 192.168.100.58 192.168.100.31 1,306.8ms

4/21/2010 4:32:05 PM 10193 Extension 251 192.168.100.61 192.168.100.31 1,003ms

 

[Ragnorak]

There's definitely something wrong. For starters get rid of the auto negotiate on the IP Office LAN port and force it to 100 half duplex.

 

[jtphoneman]

I will do that this afternoon when they close and then reset the QOS logs to see if that helps.

Thanks

 

[amriddle01]

Full not half, but the IP office is fixed at auto and having one fixed and one auto can cause the ports to "flap" making worse issues than this, see which works better Full or auto but I wouldn't pin it down to half

ACSS Small and Medium Enterprise (SME) Communications

"What the Crocodile Hat....was that?

 

[jtphoneman]

I am logged into the IP office now looking under Lan1, Lan settings and do not see anywhere to change the port speed, so I guess you guys are talking about setting the Cisco switchport not the actual port LAN1 on the IP Office?

 

[amriddle01]

Yes, as I say it's stuck on Auto so it can be best to match that or 100 Full on the Cisco try both

ACSS Small and Medium Enterprise (SME) Communications

"What the Crocodile Hat....was that?

 

[jtphoneman]

Okay, I have the Cisco switchport set to auto now, and has been like that for a week, I will set it to 100 full and give it a shot,

thanks

 

[jtphoneman]

I set the Cisco Switchport to 100 this morning, same results as before. So the the Cisco switchport connecting to the IP Office has been set to Auto and 100 full with the same results both times.

 

[NYSTECH]

Are the problems from IP phone to IP phone? Or IP phone to digital? Or Both?
Bad VCMs maybe.. what kind of cards do u have in IP 500

ACS- IPOffice Implement

 

[jtphoneman]

90% are incoming calls that are routed through a VoiceMail Pro server and then sent to a 10 phone call center or put in a waiting que until a agent is availible. I have ask the customer about extension to extension calls and he told me that they often have echo on those calls too. I will verify the card types Monday, if I am not mistaken their are only 4 or 5 phones that are not IP phones, they are analog.

 

[latitude12]

Re auto negotiation - if you set the data switch port to 100 full then the IP500 will set itself to 100 half (tip - this is bad). Ragnorak is correct, set the siwtch port to 100 half no avoid any auto negotiate issues.

Is direct media enabled for all IP phones?

Other suggestion from the Avaya IPT implement docs:
"Disable Cisco features – Cisco features that are not required by Avaya endpoints are auxiliaryvlan
(except for IP phones in a dual-VLAN setting...), channeling, cdp,
inlinepower, and any Cisco proprietary feature in general. Explicitly disable these features on ports
connected to Avaya devices, as they are non-standard mechanisms relevant only to Cisco devices and
can sometimes interfere with Avaya devices.

 

[Ragnorak]

I think it is more far more likely a network issue than an IP Office configuration issue.

Did this start when the IP Office was changed out? Has this been up and running on this network without issues before?

Do you have access to the Cisco 3560? If so do the logs on the Cisco show anything?

From your description I am inferring that only the IP Office, the VM Pro, and IP phones are attached to this switch. Is this the case or is there a default VLAN for untagged traffic on the same switch?

The round trip delay shows some huge spikes. It is either a defective piece of hardware or something is hammering your network. For example, you could have a device beyond the 100 meter cable limit that causes this whenever its sending broadcast packets. I know the customer said everything was tested but if if the Cisco logs show collisions that would be your answer. That's just an example, not saying that's the issue. The Cisco logs are probably your best troubleshooting friend at this point.

 

[soflaphoneguy]

Is there any documentation from Avaya regarding the full/half duplex setting?

 

[Ragnorak]

I feel bad now, like I derailed poor jtphoneman's jitter problem into a duplex debate.

http://marketingtools.avaya.com/knowledgebase/

has which switches support full/half. There have been a couple of Technical Tips over the years about changing the port from auto-negotiate to a fixed setting, the 412 being set to half and a 2.1 issue with VM Pro stand out in my mind. It isn't that the duplex setting on the ethernet port was the most likely cause it is just that the IP Office LAN port is common to all VoIP traffic and it is a quick and easy test. I suspect the vast majority of installations are left at auto-negotiate and never have an issue.

 

[fshaw]

Did you solve the issue? If so, what worked? I have a client with similar things.

 

[jtphoneman]

I had to take some unexpected time off last week but, it looks like we were able to solve the issue.

I have the all switchports set to auto. We actually made three changes that seems to have fixed the problem. 1st change was the firmware on the phones was 2.20 and we updated them to 2.9 the 2nd and 3rd change we made was in the IP Office, on each extension on the Voip tab the check box "allow direct media path" was unchecked and the compression mode was set to G.729.
We made sure we checked the box for allow direct media path and changed each compression mode to G.711 ULAW. So far the past 2 days we have only had 2 jitter errors in the QOS log. Before the 3 changes we would have at least 12 to 15 errors in a 2 hour time frame.

Hopefully the issue is resolved, thanks for all the help.