Switch or Voicemail problem ??

[frazierr]

We're experiencing a large number of rings before voicemail picks up. We have a System 11C using HMS-400 for voicemail. (The HMS-400 is a hospitality flavor version of CallPilot, near as I can determine).

When calling 399 (our voicemail number) or 2399 (alternate entry poing in HMS-400) we hear 4 rings, stutter, 3 rings, stutter and then voicemail picks up.
Watching on the HMS-400 during this process, in the Atlas State window, no activity is seen until about the instant the call is picked up. At this point, the Atlas State shows:
Ln3_>TDX_CST Ring recieved
Ln3_>Line Type 7
Ln3_> Raw COP=>399_313

ln3 is line 3 into the HMS-400, 399 is the voicemail number and 313 was the line I was calling from.

I don't see any activity indicated on the System Configuration Tab for HMS-400 while this is going on.

It would appear from this that the HMS-400 isn't getting the ring until after the second stutter...

I changed the NCFW for the 399 ACD to my extension and then called 399. Got an immediate forward to my extension, so changed it back to original number. Following is the Ld23 for 399:
REQ prt
TYPE acd
CUST 0
ACDN 399

TYPE ACD
CUST 0
ACDN 399
MWC YES
MAXP 1
SDNB NO
BSCW NO
ISAP NO
AACQ NO
RGAI NO
ACAA NO
FRRT
SRRT
NRRT
FROA NO
CALP POS
ICDD NO
NCFW 2399
FNCF NO
FORC NO
RTQT 0
SPCP NO
OBTN NO
RAO NO
CWTH 1
NCWL NO
BYTH 0
OVTH 2047
TOFT NONE
HPQ NO
OCN NO
OVDN
IFDN
OVBU LNK LNK LNK LNK
EMRT
MURT
RTPC NO
STIO
TSFT 20
HOML YES
RDNA NO
LABEL_KEY0 NO
DAL NO
RPRT YES
RAGT 4
DURT 30
RSND 4
FCTH 20
CRQS 100
IVR NO
OBSC NO
OBPT 5

MEM AVAIL: (U/P): 2402723 USED U P: 568326 76374 TOT: 3047423
DISK RECS AVAIL: 1152
ACD DNS
------------------------

Does this look like the problem is on the switch side, or the HMS-400 side? One part of me wants to blame HMS-400, another wants to believe the switch isn't sending the call to HMS-400 until the second stutter.... Does the immediate forward mean our switch is in night mode? (used to have a 2250 console but it was unplugged and is no longer on site).

Another question: It's getting pretty testy here with management, I may need to contract this out to an expert that can fix it for us. How do I go about finding a competent contractor to look into this? I know there's no recruiting or sales on this forum, but don't know where to start looking. I'm in Hawaii if that matters, but would probably need someone that can connect via pcanywhere or similar mechanism if it's a HMS problem.

Thanks
--Rick

 

[acewarlock]

How many phone ports do you have in the ACD Queue

 

[notapropergeordie]

Have you tried disabling switch TNs/ voicemail voice ports to see if you can isolate a bad one?

 

[acewarlock]

Sound to me like you only have 1 port working.

 

[frazierr]

Bingo! not one, but two bad lines (TN)...
Thankyou thankyou thankyou

I found the 10 ports going to HMS-400 by running a trace while calling 399, and found TN 4 3, DN 103 active during the call. Looking at DN 100 through 109, found that DN had FDN of 101, and so on. However, DN 106 had FDN of 107, but no DN 107 in system. Changed DN106 to use FDN 108 (the next DN connected to HMS-400), but this did not fix problem.
Corresponding range of TN is 4 0 through 4 5, then 8-12 through 8 15. (8-13 is housekeeping found at DN 668)

Did Disu 4 00 through 4 5, and now Voicemail picks up almost instantly (customer hears one stutter, then voicemail answers, outside caller hears 4 rings, stutter and VM pickup). Enlu starting at TN 4 5, coming up the card, and when got to TN 4 1, problem returned. Issue also returns with TN 4 0 enabled.

With both 4 0 and 4 1 dsbl, system now works normally, so for today I'm going to leave them disabled until I can figure out if it is a bad card. Don't have a spare digital card on site right now, so can't swap cards without potentially taking something else down, but if I can get a "quiet time" to do so, may swap to find out if card problem or not. We've only got three digital line cards in the system, so don't have many choices.

I'm supposed to go to "Nortel School" next fiscal year (begins in october 2007), and am hoping a week there will help fill in my lack of knowledge on this switch.

However, my next puzzle is how DN 399 knows to go to DN 100 in the first place.


Thanks
-_Rick

 

[jamie2]

You ACD 399 is a dummy ACD that is just call forwarding to 2399 'NCFW 2399'. Print out your DN 2399 in LD 20. You will most likely see that it is another ACD with agents built into it and DN 100 is probably one of the agents.

You should see all the TN's you found via your trace under the DNB printout for 2399.

 

[frazierr]

OK, printed 2399, but though the feature is PLDN, I don't know how it finds DN 100 to get to it...

REQ: PRT
TYPE: DNB
CUST 0
DN 2399
DATE
PAGE
DES
DN 2399
TYPE FFC
FEAT PLDN

NACT

Thanks
-_Rick

 

[jamie2]

Instead of 2399 being setup as an ACD, it is setup as a Hunt Group in LD 18 and 57.

REQ prt
TYPE ffc
CUST 0
CODE pldn

You will get a print out something like

CUST 00
FFCT YES
PLDN 5489
USE GPHT
LSNO 1
HTYP LIN
CFWI NO
MQUE 0

The LSNO 1 may be different for you, but this is the hunt group number.

Go to LD 20, and;
REQ: prt
TYPE: ght
LSNO 1 (What ever your LSNO was in Ld 57)

SIZE 4
RNGE

GHLN 0001
GHT

PLDN 2399
DNSZ 4
STOR 0 100
STOR 1 200
STOR 2 ???
STOR 3 ???

It should look something like the example above. To make changes to the hunt group that has to be done in LD 18.

 

[frazierr]

Very cool...

>ld 57
FFC000
MEM AVAIL: (U/P): 2402721 USED U P: 568326 76376 TOT: 3047423
DISK RECS AVAIL: 1152
REQ prt
TYPE ffc
CUST 0
CODE pldn

CUST 00
FFCT YES
PLDN 2399
USE GPHT
LSNO 100
HTYP LIN
CFWI NO
MQUE ALL

CODE
OVL000
>****
OVL000
>ld 20

PT0000
REQ: prt
TYPE: ght
LSNO 100
SIZE 10
RNGE

GHLN 0100
GHT

PLDN 2399
DNSZ 8
STOR 0 100
STOR 1 101
STOR 2 102
STOR 3 103
STOR 4 104
STOR 5 105
STOR 6 106
STOR 7
STOR 8 108
STOR 9 109

REQ:
AUD000

To drop 100 and 101, can I just make STOR 1 and 2 blanks in LD 18(temporarily, until I have a chance to test the card) or should I reorder them all beginning with STOR 1 as 102...
The blank at STOR 7 bothers me, but if it doesn't affect the working of the system, then I won't need to change it...

I'm getting the idea that this system is much like Unix in that there are multiple ways of obtaining the same effect, and none of them are necessarily wrong, just a bit harder to follow (especially if you're not very experienced).

Thanks
--Rick

 

[jamie2]

You should reorder them and take out the blanks. Keeps it cleaner.

How do you know that 100 and 101 are bad? Can you print the TNBs for those?

Build your phone and a Maintenance Set 'CLS MTC' and trying making a call out on those two.

 

[jamie2]

While looking at your GHT. See if you can printout DNB 107. It may have been bad at one time and removed from the list. You should get those trunks resolved.

 

[frazierr]

I'll reorder them and take out the blanks. I really do prefer having things clean...

Except for the expected obvious bits (different TN and DES), the printout from DN 100 and 101 don't look any different than 102 and up. Same for printing their TNB (4 0 does have an SSU of 0000, where the others are blank), but when I disu TN 4 0 and 4 1, the original symptom goes away and voicemail picks up right away. As I haven't had a chance to swap/test the card itself yet, I'm calling the two lines (tn 4 0 and 4 1) "bad" because I don't know anything else reasonable to do until I find out if I have a card problem.

During the earlier process, I couldn't find DN 107, (Error SCH0881, No Valid DN can be found starting with the specified Digits) so that's why I changed the FDN of DN 106 from 107 to 108 when trying to find what wasn't working.
I assume DN 107 was taken out when the Housekeeping interface was enabled on x668 (which corresponds to what was HMS8, and shows on HMS-400 as the housekeeping incoming line). Strangely, 668 had the same problem (multiple ring cycles) until I disu the two TNs... 668 is directly defined as TN 8 13, which is the slot for HMS8, and would have been DN 107 in the original list. The guys that initially set all this up about 18 months ago weren't very good about cleaning up when they made a change or finally enabled some of the features we bought. [sigh]

Prints follow:

OVL000
>ld 20

PT0000
REQ: prt
TYPE: dnb
CUST 0
DN 100
DATE
PAGE
DES

DN 100
CPND
NAME HMS Voicemail 1
XPLN 27
DISPLAY_FMT FIRST,LAST
TYPE SL1
TN 004 0 00 00 KEY 00 MARP DES S200 15 NOV 2005
(2616)


NACT
REQ: prt
TYPE: dnb
CUST 0
DN 101
DATE
PAGE
DES

DN 101
CPND
NAME HMS Voicemail 2
XPLN 27
DISPLAY_FMT FIRST,LAST
TYPE SL1
TN 004 0 00 01 KEY 00 MARP DES HMS2 15 NOV 2005
(2616)


NACT
REQ: prt
TYPE: dnb
CUST 0
DN 102
DATE
PAGE
DES

DN 102
CPND
NAME HMS Voicemail 3
XPLN 27
DISPLAY_FMT FIRST,LAST
TYPE SL1
TN 004 0 00 02 KEY 00 MARP DES HMS3 15 NOV 2005
(2616)


NACT
REQ: prt
TYPE: tnb
TN 4 0
DATE
PAGE
DES

DES S200
TN 004 0 00 00
TYPE 2616
CDEN 8D
CUST 0
AOM 0
FDN 101
TGAR 1
LDN NO
NCOS 0
SGRP 0
RNPG 0
SCI 0
SSU 0000
XLST
CLS CTD FBD WTA LPR MTD FNA HTD ADD HFD
MWA LMPN RMMD SMWD AAD IMD XHD IRD NID OLD VCE DRG1
POD DSX VMD CMSD SLKD CCSD SWD LND CNDA
CFTD SFD MRD DDV CNID CDCA MSID DAPA BFED RCBD
ICDD CDMD LLCN MCTD CLBD AUTU
GPUD DPUD DNDD CFXD ARHD FITD CLTD ASCD
CPFA CPTA ABDD CFHD FICD NAID BUZZ AGRD MOAD AHD
DDGA NAMA
DRDD EXR0
USRD ULAD RTDD RBDD RBHD PGND OCBD FLXD FTTC MCBN
CDMR
CPND_LANG ENG
RCO 0
HUNT
PLEV 02
CSDN
AST
IAPG 0
AACS NO
ITNA NO
DGRP
MLWU_LANG 0
DNDR 0
KEY 00 SCR 100 0 MARP
CPND
NAME HMS Voicemail 1
XPLN 27
DISPLAY_FMT FIRST,LAST
01
02
03 TRN
04 MCK
05 MIK
06
07
08
09 AO6
10
11
12
13
14
15
DATE 15 NOV 2005


NACT
REQ: prt
TYPE: tnb
TN 4 1
DATE
PAGE
DES

DES HMS2
TN 004 0 00 01
TYPE 2616
CDEN 8D
CUST 0
AOM 0
FDN 102
TGAR 1
LDN NO
NCOS 0
SGRP 0
RNPG 0
SCI 0
SSU
XLST
CLS CTD FBD WTA LPR MTD FNA HTD ADD HFD
MWA LMPN RMMD SMWD AAD IMD XHD IRD NID OLD VCE DRG1
POD DSX VMD CMSD SLKD CCSD SWD LND CNDA
CFTD SFD MRD DDV CNID CDCA MSID DAPA BFED RCBD
ICDD CDMD LLCN MCTD CLBD AUTU
GPUD DPUD DNDD CFXD ARHD FITD CLTD ASCD
CPFA CPTA ABDD CFHD FICD NAID BUZZ AGRD MOAD AHD
DDGA NAMA
DRDD EXR0
USRD ULAD RTDD RBDD RBHD PGND OCBD FLXD FTTC MCBN
CDMR
CPND_LANG ENG
RCO 0
HUNT
PLEV 02
CSDN
AST
IAPG 0
AACS NO
ITNA NO
DGRP
MLWU_LANG 0
DNDR 0
KEY 00 SCR 101 0 MARP
CPND
NAME HMS Voicemail 2
XPLN 27
DISPLAY_FMT FIRST,LAST
01
02
03 TRN
04 MCK
05 MIK
06
07
08
09 AO6
10
11
12
13
14
15
DATE 15 NOV 2005


NACT
REQ: prt
TYPE: tnb
TN 4 2
DATE
PAGE
DES

DES HMS3
TN 004 0 00 02
TYPE 2616
CDEN 8D
CUST 0
AOM 0
FDN 103
TGAR 1
LDN NO
NCOS 0
SGRP 0
RNPG 0
SCI 0
SSU
XLST
CLS CTD FBD WTA LPR MTD FNA HTD ADD HFD
MWA LMPN RMMD SMWD AAD IMD XHD IRD NID OLD VCE DRG1
POD DSX VMD CMSD SLKD CCSD SWD LND CNDA
CFTD SFD MRD DDV CNID CDCA MSID DAPA BFED RCBD
ICDD CDMD LLCN MCTD CLBD AUTU
GPUD DPUD DNDD CFXD ARHD FITD CLTD ASCD
CPFA CPTA ABDD CFHD FICD NAID BUZZ AGRD MOAD AHD
DDGA NAMA
DRDD EXR0
USRD ULAD RTDD RBDD RBHD PGND OCBD FLXD FTTC MCBN
CDMR
CPND_LANG ENG
RCO 0
HUNT
PLEV 02
CSDN
AST
IAPG 0
AACS NO
ITNA NO
DGRP
MLWU_LANG 0
DNDR 0
KEY 00 SCR 102 0 MARP
CPND
NAME HMS Voicemail 3
XPLN 27
DISPLAY_FMT FIRST,LAST
01
02
03 TRN
04 MCK
05 MIK
06
07
08
09 AO6
10
11
12
13
14
15
DATE 15 NOV 2005


NACT
REQ:

Thanks
--Rick

 

[jamie2]

Unfortunately, I don't know anything about the HMS-400, but you are using M2616 sets for intergration, like the Octel systems do, using the MCK and MIK keys on the sets to turn on/off message waiting lights. You may want to try and IDU the TNs in LD 20, to see if you have physical sets attached to the TN's. May be a wiring issue, bad set, or port.

 

[frazierr]

There are no physical instruments connected to any of the lines, because they connect instead to the HMS-400. It was worked fine until about a month ago, when we suddenly got the multple ring cycle problem. No changes had been made to the switch, just a symptom popped up. Right now, my guesses are either bad wiring or bad port. We've had a few bad wiring issues crop up over the 18 months or so since the system has been installed, so I wouldn't be totally surprised if it turned out to be that. On the other hand, we've taken more than a couple of lightning hits within the past year, (mostly took out trunk cards and an analog card if I remember correctly) so this may even be bad ports as a delayed reaction to that....

Thanks for all of the help!
Aloha

--Rick