App Server VM issue

[chaunymony]

I'm running VM pro, Linux, 6.1.24 on an Intel based ESXi box that has a small Win7 partition for management. I have probably 6 installed the same way with no issues whatsoever except for this one.

What seems to be happening is it's not playing the audio files. I have DID's set to auto attendants and to voicemail (*17) and when the issue occurs, I can watch the call come in and hit voicemail/AA but I'm getting no audio. I restart the service through the webcontrol but that doesn't fix it. If I SSH into it and reboot the Cent OS box it will come back and work fine once restarted. No errors in SSA or in the webcontrol.

I was on a previous release of the App server and completely rebuilt the virtual machine and loaded 6.1.24 last night and imported the .mdb and audio files. It worked for 16 hours then just had the hiccup again.

Anyone seen this or have an idea where to start?

ACSS-SME
ACIS-SME

 

[chaunymony]

Here's a little more info that I was able to gather last night.

• Before restarting vmpro, the lsof output for it is like this: (skipping the boring stuff)
vmpro 2605 root 3u unix 0xf7ad8040 9092 socket
vmpro 2605 root 4u IPv4 9210 UDP *:time
vmpro 2605 root 5u IPv4 9184 UDP *:50791
vmpro 2605 root 6u IPv4 9185 TCP *:50791 (LISTEN)
vmpro 2605 root 7u IPv4 9186 TCP *:smtp (LISTEN)
vmpro 2605 root 8u IPv4 9187 TCP *:imap (LISTEN)
vmpro 2605 root 9u IPv4 9365 UDP *:60887
vmpro 2605 root 10u sock 0,5 60119 can't identify protocol
vmpro 2605 root 11r CHR 1,9 958 /dev/urandom
vmpro 2605 root 12u sock 0,5 60233 can't identify protocol
vmpro 2605 root 13u sock 0,5 60351 can't identify protocol
vmpro 2605 root 14u sock 0,5 60474 can't identify protocol
vmpro 2605 root 15u sock 0,5 60590 can't identify protocol
vmpro 2605 root 16u sock 0,5 77522 can't identify protocol
vmpro 2605 root 17u sock 0,5 77659 can't identify protocol
vmpro 2605 root 18u sock 0,5 78133 can't identify protocol
vmpro 2605 root 19u sock 0,5 78837 can't identify protocol


• After restarting:
vmpro 21922 root 3u unix 0xf7ad8040 104205 socket
vmpro 21922 root 4u IPv4 104206 UDP *:50791
vmpro 21922 root 5u IPv4 104207 TCP *:50791 (LISTEN)
vmpro 21922 root 6u IPv4 104209 TCP *:imap (LISTEN)
vmpro 21922 root 7u IPv4 104212 UDP *:time
vmpro 21922 root 8u IPv4 104437 TCP 10.0.0.65:50791->10.0.0.63:61564 (ESTABLISHED)
vmpro 21922 root 9u IPv4 104490 TCP *:smtp (LISTEN)

• Here are the errors spit out by vmpro when it is failing: (From /var/log/messages)
Jul 5 20:45:35 voicemailpro vmpro [2605]: ac8e0b90: IMAP4 - Connection received from 2.0.22.138
Jul 5 20:45:35 voicemailpro vmpro [2605]: ac8e0b90: IMAP4 - LinuxStartServerThread: Error creating thread for IMAP Server Session
Jul 5 20:45:38 voicemailpro vmpro [2605]: ad2e1b90: virtual void SMTPListener::CreateClient(SOCKET, long unsigned int, short unsigned int): Shutting down socket (0x00000014, 20), how=00000002
Jul 5 20:46:41 voicemailpro vmpro [2605]: ad2e1b90: virtual void SMTPListener::CreateClient(SOCKET, long unsigned int, short unsigned int): Shutting down socket (0x00000014, 20), how=00000002

• It takes a bloody eternity for the vmpro service to start after boot


ACSS-SME
ACIS-SME

 

[tlpeter]

Couldn't you just restart the service?
That takes two seconds.


BAZINGA!

I'm not insane, my mother had me tested!

 

[chaunymony]

Tried that. Restarting the service doesn't fix the issue. Still no audio after the restart, only reboot. Escalating to Avaya now too, just thought I'd poke around.

ACSS-SME
ACIS-SME

 

[tlpeter]

It shows IMAP.
Can you check if it selected or SMTP.

BAZINGA!

I'm not insane, my mother had me tested!

 

[chaunymony]

Sure, umm.. where would I check?

ACSS-SME
ACIS-SME

 

[tlpeter]

Open the vmpro client and press F9
Then go to the smtp setting.

BAZINGA!

I'm not insane, my mother had me tested!

 

[chaunymony]

It's the new voicemail settings. Email tab, MAPI is unchecked, I have 2 addresses along with the local host under SMTP Sender and the default local under recieve. Nothing under there for IMAP, I was thinking it was somewhere else other than preferences.

ACSS-SME
ACIS-SME

 

[tlpeter]

What happens if you remove those SMTP settings?
Also the VMPro service should start very quickly when you boot the server.


BAZINGA!

I'm not insane, my mother had me tested!

 

[IPGuru]

2.0.22.138 is a public ip address on the internet
why is it trying to connect to your VMpro?

could this be some kind of external attack (probably aimed at a windoze box that the VMPro dosen't like)?

I do not Have A.D.D. im just easily, Hey look a Squirrel!

 

[hairlessupportmonkey]

IPGuru,

I have a customer (we didnt set their network up) with an 82.xxx.xxx.xxx internal network

ive also had a customer set up a 169.xxx.xxx.xxx network based around a plotter they had.....

ACSS - SME
General Geek

 

[chaunymony]

So I installed Windows VM Pro last night on the windows machine we were using to manage the linux vmprom while I wait to hear from Avaya. Nothing has gone down since which is good but when I imported the accounts folder, none of the messages show up. The folder size does show up in vmpro client but no new, old, saved messages do. Am I missing something here for that or is it just not doable going from linux to windows?

ACSS-SME
ACIS-SME

 

[tlpeter]

That should work fine.
So i think you have found the problem a little bit.


BAZINGA!

I'm not insane, my mother had me tested!

 

[chaunymony]

Good, I thought so. I've moved vm servers plenty of times in the past without issue, just not from Linux to Windows. The grettings, names and AA .wav's all work fine. I'm hoping this will be a minor inconvenience to the fact that their voicemail is now working.

ACSS-SME
ACIS-SME

 

[chaunymony]

So, just did a latest 7.0 v2 install with linux vmpro this last week and this issue is happening again but I have new info also. I left vmpro client open and when voicemail would go silent I noticed that it said there were 40 active voicemail sessions going on. I went digging around with that info which led me to these two threads from 2008

http://www.tek-tips.com/viewthread.cfm?qid=1459558

http://www.tek-tips.com/viewthread.cfm?qid=1454926

Most seems to be pointing to an issue with time servers and another one points to issues with hunt groups keeping the channels open. After reading further, I'm thinking it may have something to do with the time sync's. I'm going to play around here this afternoon and see if I can't get a fix for this.



ACSS-SME
ACIS-SME

 

[tlpeter]

Do you see the port congestion in SSA too?


BAZINGA!

I'm not insane, my mother had me tested!

 

[chaunymony]

Not seeing the data channels. Just happened again after adjusting time settings. 52 active voicemail channels, no audio on AA, even though everything is up and running and no alarms. Restart the service and everything is running.

ACSS-SME
ACIS-SME

 

[chaunymony]

I switched them over to windows based vmpro and the same thing happened. 1 active channel stuck but I couldn't get it to play any audio after it was stuck, restarted the service, rebooted the vm server and IPO. Nothing. Ended up going back to the linux vm pro as at least I can restart the service and get that working. I'm just wondering if it has something to do with my config? I've been doing them the same for years though. . . .to Avaya I go, I guess.

ACSS-SME
ACIS-SME

 

[amriddle01]

You could quickly fire in a test config to see if the existing one is corrupt or not, do you have some particularly odd programming then? Could something else on the network be using the same ports as VM Pro or something like that?

http://www.ipoffice.tel

 

[chaunymony]

I was thinking configuration before but this one that's happening now is all new. So i have 2 new installs with the same issue. One at 6.1 and one at 7.0 (latest releases) I was thinking it had todo with auto attendants but nothing weird there. Sequential hunt group with an extension that is forwarded to a short code that goes to the auto attendant.

ACSS-SME
ACIS-SME