IP Trunks - not connecting 1

[AnthonyHardy]

I have two CS1000 switches connected via ip trunks. We changed our core switch (layer 3, also routing) out yesterday and everything was fine. This morning, you cannot dial between the two CS1000's.

In ld 96, the dch's both said EST, yet no calls would pass. I'm not sure how to debug anything here (I am a network guy, been admin'ing the nortel stuff for nearly 10 years, but I'm not a setup guy, just basic MAC stuff).

If I disables the dch on one side and enl it and it "looks" EST on both sides, but no calls go through. On occassion the connection actually establishes itself after a dis/enl of the dch, but only for a short period of time (1-2 minutes).

I restarted the signalling server on the remote side. Once back up, the dch on that side still said EST. The main switch's dch wouldn't go to EST, so I'm restarting the signalling server at the main switch.

Any ideas? Where can I look for a debug of some sort? I have access to the Element Manager on both sides.

This is on 4.0 btw.

THanks for any help!

 

[Reusser]

The virtual IP Dchannel will "est"ablish as long as there is a good e-lan link to the SigServer - has nothing to do with wether or not it can see the far end.

If DCHannel is down at main - that would be a good place to start -reboot Sigserver to see if somes up.

Once up turn on Dchannel messages to see if you can tell what is happening with call. If you get a no circ available - this is an indication local side can not talk to far side.

From here there are a couple thing to do - from the Signaling Server OAM prompt you can turn on H323 or SIP signaling
Or
do a Sniffer trace on network.

 

[allenmac]

I would seriously suspect something is not configured correctly on the layer 3 switch that you just replaced.

Everything was working fine, and then after 1 piece of equipment was replaced, something broke.

Coincidence?

 

[pcat200]

If you have any VGMC cards in the system you may need to reset them via Element Manager. It may be that the IP trunks are connecting but the digital/analog sets are not being transcoded properly. Are you in Brewton?

 

[AnthonyHardy]

ThAnks for the responses guys. I am in brewton. Right now, I'm doing a cold start on the whole system. Normally I'd say that it was the new switch too, except that it is only the routing that changed as the nortel eq is still plugged into the same switch as before and routing is working and this nortel is VERY buggy.

I will post back in a bit with results. Thanks guys!

 

[AnthonyHardy]

OK, full restart at main location (three actually, as the system wouldn't come up properly for the first two). We're back up, but in the same situation.

I'm sticking my nose into the telnet session of mmm . not sure what it's called (oam prompt). Before, I've only used it to do a cslogin to get to the switch for MACs.

electShow at the main end shows that the MC card isn't registered (TN 3 0). Host type is SMC. There is another registered (Host type ISP). I'm guessing that is the signalling server since the ips are signalling server ips.

The remote end shows both the SMC and ISP as registered. I'm guessing this is similar to what pcat200 mentioned?

 

[AnthonyHardy]

yup, something tells me it's the element manager. Does it use any sort of broadcast info to connect to the remote end? Here's the syslog from element manager ..fun fun fun:


JUN 29 14:27:02 CSV: Error Unsupported CS S/W version; upgrade CS S/W
JUN 29 14:27:03 CSV: Error Unsupported CS S/W version; upgrade CS S/W
JUN 29 14:27:03 CSV: Error Unsupported CS S/W version; upgrade CS S/W
JUN 29 14:27:03 CSV: Error Unsupported CS S/W version; upgrade CS S/W
JUN 29 14:27:03 CSV: Error Unsupported CS S/W version; upgrade CS S/W
JUN 29 14:27:03 CSV: Error Unsupported CS S/W version; upgrade CS S/W
JUN 29 14:27:03 CSV: Error Unsupported CS S/W version; upgrade CS S/W
JUN 29 14:27:03 CSV: Error Unsupported CS S/W version; upgrade CS S/W
JUN 29 14:27:03 CSV: Error Unsupported CS S/W version; upgrade CS S/W
JUN 29 14:27:03 CSV: Error Unsupported CS S/W version; upgrade CS S/W
JUN 29 14:27:03 CSV: Error Unsupported CS S/W version; upgrade CS S/W
JUN 29 14:27:03 CSV: Error Unsupported CS S/W version; upgrade CS S/W
JUN 29 14:27:03 CSV: Error Unsupported CS S/W version; upgrade CS S/W
JUN 29 14:27:03 CSV: Error Unsupported CS S/W version; upgrade CS S/W
JUN 29 14:27:03 CSV: Error Unsupported CS S/W version; upgrade CS S/W
JUN 29 14:27:03 CSV: Error Unsupported CS S/W version; upgrade CS S/W
JUN 29 14:27:03 CSV: Error Unsupported CS S/W version; upgrade CS S/W
JUN 29 14:27:03 CSV: Error Unsupported CS S/W version; upgrade CS S/W
JUN 29 14:27:03 CSV: Error Unsupported CS S/W version; upgrade CS S/W
JUN 29 14:27:04 ELC: Warning Recall timer timeout, recall election to discard mastership
JUN 29 14:27:04 tNetTask: Info arptnew failed on f6aff
JUN 29 14:27:06 ELC: Info VGW card event 0x4 ignored
JUN 29 14:27:06 ES: Info esSocketClose: ES2 removing socket 68
JUN 29 14:27:08 CSV: Info Dropping connection to 10.101.0.84
JUN 29 14:27:08 CSV: Info Dropping connection to 10.101.0.55
JUN 29 14:27:08 CSV: Info Dropping connection to 10.101.0.203
JUN 29 14:27:08 CSV: Info Dropping connection to 10.101.0.89
JUN 29 14:27:08 CSV: Info Dropping connection to 10.101.0.20
JUN 29 14:27:08 CSV: Info Dropping connection to 10.10.1.65
JUN 29 14:27:08 CSV: Info Dropping connection to 10.101.0.150
JUN 29 14:27:08 CSV: Info Dropping connection to 10.101.0.249
JUN 29 14:27:08 CSV: Info Dropping connection to 10.101.0.52
JUN 29 14:27:08 CSV: Info Dropping connection to 10.101.0.210
JUN 29 14:27:08 CSV: Info Dropping connection to 10.101.0.218
JUN 29 14:27:08 CSV: Info Dropping connection to 10.101.0.99
JUN 29 14:27:08 CSV: Info Dropping connection to 10.101.0.136
JUN 29 14:27:08 CSV: Info Dropping connection to 10.101.0.201
JUN 29 14:27:08 CSV: Info Dropping connection to 10.101.0.98
JUN 29 14:27:08 CSV: Info Dropping connection to 10.101.0.204
JUN 29 14:27:08 CSV: Info Dropping connection to 10.201.0.252
JUN 29 14:27:08 CSV: Info Dropping connection to 10.10.2.251
JUN 29 14:27:08 CSV: Info CSV disabled
JUN 29 14:35:02 SNTPC: Warning SNTP listen timeout
JUN 29 14:43:34 SNTPC: Warning SNTP listen timeout
JUN 29 14:52:06 SNTPC: Warning SNTP listen timeout
JUN 29 15:00:38 SNTPC: Warning SNTP listen timeout
JUN 29 15:08:07 SYS: Info IPL 4.00.55 - Sunday January 30 13:22:52 EST 2005
JUN 29 15:08:07 SYS: Info Completed addition of ITG MIB tree 1.3.6.1.4.1.562
JUN 29 15:08:07 SYS: Info ITG5000 Card initialized, all alarms cleared. (202)
JUN 29 15:08:07 SYS: Info ELAN IP = 10.110.0.13
JUN 29 15:08:07 tLogTask: Info MMsend called by appl 9 before CS 1000 Manager Init!
JUN 29 15:08:07 tLogTask: Info MMsend called by appl 9 before CS 1000 Manager Init!
JUN 29 15:08:07 tLogTask: Info MMsend called by appl 9 before CS 1000 Manager Init!
JUN 29 15:08:14 SYS: Alert No bootp servers responding: S_bootpLib_TIME_OUT
JUN 29 15:08:14 SYS: Info Card TN = 0 0 3
JUN 29 15:08:14 SYS: Info XA : - Create MsgQ (Pointer)
JUN 29 15:08:14 SYS: Info XA : - Register with CS 1000 Manager
JUN 29 15:08:14 SYS: Info XA : - Spawn xaTask()
JUN 29 15:08:14 tXA: Info Task Spawned OK
JUN 29 15:08:15 SYS: Info XA : - Query 8051XA for Dongle ID
JUN 29 15:08:15 tXA: Info Got Dongle Response
JUN 29 15:08:15 tXA: Info Invalid Family Code 0x0
JUN 29 15:08:15 SYS: Info XA to IXP1200 polling enabled
JUN 29 15:08:15 tXA: Info Got Version Response
JUN 29 15:08:15 tXA: Info Version : MC Firmware Rls 8.2
JUN 29 15:08:15 SYS: Info WARNING: mac21440MCastAddrAdd called. addr=0x2e201f0
JUN 29 15:08:15 SYS: Info Configuring Voice LAN on ixpMac0, IP = 10.101.0.14
JUN 29 15:08:15 SYS: Info itgStartAppBegin
JUN 29 15:08:15 SYS: Info Starting applications
JUN 29 15:08:15 SYS: Info rudpMgrRun
JUN 29 15:08:15 RDP: Info RUDP waiting for messages.
JUN 29 15:08:15 SYS: Info ssdLibInit
JUN 29 15:08:15 SYS: Info mamInit
JUN 29 15:08:15 MAM: Info MAM started
JUN 29 15:08:15 MAM: Info DspEchoOpcode is not configured (default 28)
JUN 29 15:08:15 MAM: Info DspEchoNlpFix is not configured (default 0)
JUN 29 15:08:15 MAM: Info DspEchoConfig is not configured (default 64)
JUN 29 15:08:15 MAM: Info DspEchoNlpAggress is not configured (default 0)
JUN 29 15:08:15 MAM: Info DspEchoCnConfig is not configured (default 0)
JUN 29 15:08:15 SYS: Info dimInit
JUN 29 15:08:15 SYS: Info Detected 8 DSP devices, 32 channels
JUN 29 15:08:15 SYS: Info MXP environment is created
JUN 29 15:08:15 SYS: Info ommInit
JUN 29 15:08:15 SYS: Info ommFileSystemInit: OmmdayOfWeek = 4
JUN 29 15:08:15 SYS: Info pbxLibInit
JUN 29 15:08:15 SYS: Info pbxLinkStart
JUN 29 15:08:15 SYS: Warning Cannot open the file /C:/data/reduip.txt to read.
JUN 29 15:08:15 SYS: Alert ITS1009 Call server communication link: 10.110.0.11 down (20)
JUN 29 15:08:15 SYS: Info Waiting for PBX RUDP link to be established...
JUN 29 15:08:15 SYS: Info electInit
JUN 29 15:08:15 ELC: Warning ELAN link is still down, putting election into sleeping mode
JUN 29 15:08:15 ELC: Info ELC Started
JUN 29 15:08:15 SYS: Info lineStart
JUN 29 15:08:16 VTM: Info tos bytes is 0xb8
JUN 29 15:08:16 VTM: Info TOS: voice tos 0xb8, precedence 0x0
JUN 29 15:08:16 VTM: Info DSP: jitter 0x2, HighWaterMark 0x4
JUN 29 15:08:16 VTM: Info TOS: 802.1p flag 0x0
JUN 29 15:08:16 VTM: Info TOS: 802.1p tag bit 0x6
JUN 29 15:08:16 VTM: Info VTM Started
JUN 29 15:08:16 VTM: Info VTM initialized
JUN 29 15:08:16 vtiLib: Info vtiLib initializing
JUN 29 15:08:16 SET: Info Started
JUN 29 15:08:16 SET: Info tone table replaced
JUN 29 15:08:16 SET: Info cadence table replaced
JUN 29 15:08:16 VTM: Info Client logged on 0x2B30F30
JUN 29 15:08:16 SET: Error Unsupported CS S/W Version, Upgrade CS
JUN 29 15:08:16 SET: Info dsetTimerInit: setPollingPeriod data is NOT ready yet, force setPollingPeriod to default 30
JUN 29 15:08:16 SET: Info server <Hostname> node <0> online announce
JUN 29 15:08:16 SET: Warning tcp & rudp not up... msg not sent
JUN 29 15:08:16 VTM: Info Client logged on 0x2BCABBC
JUN 29 15:08:16 WAP: Error Unsupported CS S/W Version, Upgrade CS
JUN 29 15:08:16 WAP: Info Web Application Proxy initialized
JUN 29 15:08:16 CSV: Info CSV task started
JUN 29 15:08:16 TPSAR: Warning tpsARReadConfigFile: can not read SW_GateKeeper from config.ini
JUN 29 15:08:16 TPS: Info Security Check is Disabled
JUN 29 15:08:16 TPS: Info Overriding password to <>
JUN 29 15:08:16 TPS: Info TPS Started
JUN 29 15:08:16 TPS: Info DBSync Request sent to CS
JUN 29 15:08:16 TPS: Warning tcp & rudp not up... msg not sent
JUN 29 15:08:16 TPS: Info Error in retrieving ES configuration. Default values will be used
JUN 29 15:08:16 TPS: Info esConfigSet: ES[1].assignedIP = 10.101.0.14(0xe00650a), port = 10000, Port err=0
JUN 29 15:08:16 TPS: Info esConfigSet: ES[2].assignedIP = 10.101.0.13(0xd00650a), port = 10000, Port err=0
JUN 29 15:08:16 TPS: Info Loss Plan timer started for 30 seconds !
JUN 29 15:08:16 ES: Info tES task started
JUN 29 15:08:16 ES: Info esSocketCreate: UDP Socket: ip = 10.101.0.14(0xe00650a), port = 10000 created
JUN 29 15:08:16 FWBK: Info F/W backup task started as tfwBk
JUN 29 15:08:16 UMS: Info server stub task started as tUMS
JUN 29 15:08:16 UMS: Info UMS set F/W backup task priority to 255
JUN 29 15:08:16 SYS: Info UMS init: Fw download Sequence incomplete due to CS not up.
JUN 29 15:08:16 SYS: Info umsConfigGetRamFwfileFromMaster -- System Password is not available, download firmware file for i2004 later
JUN 29 15:08:16 SYS: Info umsConfigGetRamFwfileFromMaster -- System Password is not available, download firmware file for i2002 later
JUN 29 15:08:16 SYS: Info umsConfigGetRamFwfileFromMaster -- System Password is not available, download firmware file for i2001 later
JUN 29 15:08:16 VTM: Info Client logged on 0x22ED9D8
JUN 29 15:08:16 VTM: Info VTM received notification that UMC is initialized.
JUN 29 15:08:16 UMC: Info client stub registered with UMS server as 0x22edc30
JUN 29 15:08:16 UFTP: Info UFTP task started
JUN 29 15:08:16 SYS: Info All line tps modules started!
JUN 29 15:08:16 SYS: Info vgwStart
JUN 29 15:08:16 RTP: Info vgw rtp task started
JUN 29 15:08:16 RTCP: Info vgw rtcp task started
JUN 29 15:08:16 VGW: Info vgw task started
JUN 29 15:08:16 VGW: Info channel setup, trunkType 0
JUN 29 15:08:16 VGW: Info vgwTimerInit: setPollingPeriod data is NOT ready yet, force setPollingPeriod to default 30
JUN 29 15:08:16 VGW: Info VGW: Audio stream TOS is 0xb8
JUN 29 15:08:16 VGW: Info VGW: Control stream TOS is 0xa0
JUN 29 15:08:16 SYS: Info All vgw modules started!
JUN 29 15:08:16 SYS: Info itgXmsgStart
JUN 29 15:08:16 SYS: Info spawn XmsgSocketServerStart OK!
JUN 29 15:08:16 SYS: Info itgStartAppEnd
JUN 29 15:08:16 SYS: Info All applications started
JUN 29 15:08:16 MAM: Info SNMP: Own card IP = 10.110.0.13
JUN 29 15:08:16 MAM: Info SNMP: System Host Name = MRSMITH
JUN 29 15:08:16 MAM: Info SNMP: System Location = BREWTON
JUN 29 15:08:16 MAM: Info SNMP: System Contact =
JUN 29 15:08:16 MAM: Info TOS: Voice ToS = 0xb8
JUN 29 15:08:16 MAM: Info TOS: Control ToS = 0xa0
JUN 29 15:08:16 MAM: Info routes are added for tos values, controlPrio = 160, voicePro = 184
JUN 29 15:08:16 MAM: Info Route: Empty configRouteTable!
JUN 29 15:08:16 DLOG: Info DLOG task started
JUN 29 15:08:16 ICMPMON: Info task started
JUN 29 15:08:18 SYS: Info patchInit
JUN 29 15:08:18 SYS: Info 3145728 byte memory partition created for patches
JUN 29 15:08:27 PBXResetLink: Info Attempting to establish PBX link with link 0
JUN 29 15:08:29 SYS: Info patchReIns
JUN 29 15:08:29 SYS: Info patchInitSanity
JUN 29 15:08:29 SYS: Warning ITG4040 Last Reset Reason: (202)
JUN 29 15:08:44 TPS: Info Received tps loss plan message
JUN 29 15:16:46 SNTPC: Warning SNTP listen timeout
JUN 29 15:23:16 UMS: Info 15 Mins passed: Not all F/W downloaded successfully.
JUN 29 15:25:18 SNTPC: Warning SNTP listen timeout
JUN 29 15:32:56 SYS: Info IPL 4.00.55 - Sunday January 30 13:22:52 EST 2005
JUN 29 15:32:56 SYS: Info Completed addition of ITG MIB tree 1.3.6.1.4.1.562
JUN 29 15:32:56 SYS: Info ITG5000 Card initialized, all alarms cleared. (202)
JUN 29 15:32:56 SYS: Info ELAN IP = 10.110.0.13
JUN 29 15:32:56 tLogTask: Info MMsend called by appl 9 before CS 1000 Manager Init!
JUN 29 15:32:56 tLogTask: Info MMsend called by appl 9 before CS 1000 Manager Init!
JUN 29 15:32:56 tLogTask: Info MMsend called by appl 9 before CS 1000 Manager Init!
JUN 29 15:33:03 SYS: Alert No bootp servers responding: S_bootpLib_TIME_OUT
JUN 29 15:33:03 SYS: Info Card TN = 0 0 3
JUN 29 15:33:03 SYS: Info XA : - Create MsgQ (Pointer)
JUN 29 15:33:03 SYS: Info XA : - Register with CS 1000 Manager
JUN 29 15:33:03 SYS: Info XA : - Spawn xaTask()
JUN 29 15:33:03 tXA: Info Task Spawned OK
JUN 29 15:33:04 SYS: Info XA : - Query 8051XA for Dongle ID
JUN 29 15:33:04 tXA: Info Got Dongle Response
JUN 29 15:33:04 tXA: Info Invalid Family Code 0x0
JUN 29 15:33:04 SYS: Info XA to IXP1200 polling enabled
JUN 29 15:33:04 tXA: Info Got Version Response
JUN 29 15:33:04 tXA: Info Version : MC Firmware Rls 8.2
JUN 29 15:33:04 SYS: Info WARNING: mac21440MCastAddrAdd called. addr=0x2e201f0
JUN 29 15:33:04 SYS: Info Configuring Voice LAN on ixpMac0, IP = 10.101.0.14
JUN 29 15:33:04 SYS: Info itgStartAppBegin
JUN 29 15:33:04 SYS: Info Starting applications
JUN 29 15:33:04 SYS: Info rudpMgrRun
JUN 29 15:33:04 RDP: Info RUDP waiting for messages.
JUN 29 15:33:04 SYS: Info ssdLibInit
JUN 29 15:33:04 SYS: Info mamInit
JUN 29 15:33:04 MAM: Info MAM started
JUN 29 15:33:04 MAM: Info DspEchoOpcode is not configured (default 28)
JUN 29 15:33:04 MAM: Info DspEchoNlpFix is not configured (default 0)
JUN 29 15:33:04 MAM: Info DspEchoConfig is not configured (default 64)
JUN 29 15:33:04 MAM: Info DspEchoNlpAggress is not configured (default 0)
JUN 29 15:33:04 MAM: Info DspEchoCnConfig is not configured (default 0)
JUN 29 15:33:04 SYS: Info dimInit
JUN 29 15:33:04 SYS: Info Detected 8 DSP devices, 32 channels
JUN 29 15:33:04 SYS: Info MXP environment is created
JUN 29 15:33:04 SYS: Info ommInit
JUN 29 15:33:04 SYS: Info ommFileSystemInit: OmmdayOfWeek = 4
JUN 29 15:33:04 SYS: Info pbxLibInit
JUN 29 15:33:04 SYS: Info pbxLinkStart
JUN 29 15:33:04 SYS: Warning Cannot open the file /C:/data/reduip.txt to read.
JUN 29 15:33:04 SYS: Alert ITS1009 Call server communication link: 10.110.0.11 down (20)
JUN 29 15:33:04 SYS: Info Waiting for PBX RUDP link to be established...
JUN 29 15:33:04 SYS: Info electInit
JUN 29 15:33:04 ELC: Warning ELAN link is still down, putting election into sleeping mode
JUN 29 15:33:04 ELC: Info ELC Started
JUN 29 15:33:04 SYS: Info lineStart
JUN 29 15:33:05 VTM: Info tos bytes is 0xb8
JUN 29 15:33:05 VTM: Info TOS: voice tos 0xb8, precedence 0x0
JUN 29 15:33:05 VTM: Info DSP: jitter 0x2, HighWaterMark 0x4
JUN 29 15:33:05 VTM: Info TOS: 802.1p flag 0x0
JUN 29 15:33:05 VTM: Info TOS: 802.1p tag bit 0x6
JUN 29 15:33:05 VTM: Info VTM Started
JUN 29 15:33:05 VTM: Info VTM initialized
JUN 29 15:33:05 vtiLib: Info vtiLib initializing
JUN 29 15:33:05 SET: Info Started
JUN 29 15:33:05 SET: Info tone table replaced
JUN 29 15:33:05 SET: Info cadence table replaced
JUN 29 15:33:05 VTM: Info Client logged on 0x2B30F30
JUN 29 15:33:05 SET: Error Unsupported CS S/W Version, Upgrade CS
JUN 29 15:33:05 SET: Info dsetTimerInit: setPollingPeriod data is NOT ready yet, force setPollingPeriod to default 30
JUN 29 15:33:05 SET: Info server <Hostname> node <0> online announce
JUN 29 15:33:05 SET: Warning tcp & rudp not up... msg not sent
JUN 29 15:33:05 VTM: Info Client logged on 0x2BCABBC
JUN 29 15:33:05 WAP: Error Unsupported CS S/W Version, Upgrade CS
JUN 29 15:33:05 WAP: Info Web Application Proxy initialized
JUN 29 15:33:05 CSV: Info CSV task started
JUN 29 15:33:05 TPSAR: Warning tpsARReadConfigFile: can not read SW_GateKeeper from config.ini
JUN 29 15:33:05 TPS: Info Security Check is Disabled
JUN 29 15:33:05 TPS: Info Overriding password to <>
JUN 29 15:33:05 TPS: Info TPS Started
JUN 29 15:33:05 TPS: Info DBSync Request sent to CS
JUN 29 15:33:05 TPS: Warning tcp & rudp not up... msg not sent
JUN 29 15:33:05 TPS: Info Error in retrieving ES configuration. Default values will be used
JUN 29 15:33:05 TPS: Info esConfigSet: ES[1].assignedIP = 10.101.0.14(0xe00650a), port = 10000, Port err=0
JUN 29 15:33:05 TPS: Info esConfigSet: ES[2].assignedIP = 10.101.0.13(0xd00650a), port = 10000, Port err=0
JUN 29 15:33:05 TPS: Info Loss Plan timer started for 30 seconds !
JUN 29 15:33:05 ES: Info tES task started
JUN 29 15:33:05 ES: Info esSocketCreate: UDP Socket: ip = 10.101.0.14(0xe00650a), port = 10000 created
JUN 29 15:33:05 FWBK: Info F/W backup task started as tfwBk
JUN 29 15:33:05 UMS: Info server stub task started as tUMS
JUN 29 15:33:05 UMS: Info UMS set F/W backup task priority to 255
JUN 29 15:33:05 SYS: Info UMS init: Fw download Sequence incomplete due to CS not up.
JUN 29 15:33:05 SYS: Info umsConfigGetRamFwfileFromMaster -- System Password is not available, download firmware file for i2004 later
JUN 29 15:33:05 SYS: Info umsConfigGetRamFwfileFromMaster -- System Password is not available, download firmware file for i2002 later
JUN 29 15:33:05 SYS: Info umsConfigGetRamFwfileFromMaster -- System Password is not available, download firmware file for i2001 later
JUN 29 15:33:05 VTM: Info Client logged on 0x22ED9D8
JUN 29 15:33:05 VTM: Info VTM received notification that UMC is initialized.
JUN 29 15:33:05 UMC: Info client stub registered with UMS server as 0x22edc30
JUN 29 15:33:05 UFTP: Info UFTP task started
JUN 29 15:33:05 SYS: Info All line tps modules started!
JUN 29 15:33:05 SYS: Info vgwStart
JUN 29 15:33:05 RTP: Info vgw rtp task started

 

[AnthonyHardy]

oh . no edit. Well, this is the latest:

JUN 29 15:50:07 UMS: Info Retry firmware download: from IP 10.101.0.14 for terminal i2001
JUN 29 15:52:07 UMS: Info Retry firmware download: from IP 10.101.0.14 for terminal i2004
JUN 29 15:52:07 UMS: Info Retry firmware download: from IP 10.101.0.14 for terminal i2002
JUN 29 15:52:07 UMS: Info Retry firmware download: from IP 10.101.0.14 for terminal i2001
JUN 29 15:54:07 UMS: Info Retry firmware download: from IP 10.101.0.14 for terminal i2004
JUN 29 15:54:07 UMS: Info Retry firmware download: from IP 10.101.0.14 for terminal i2002
JUN 29 15:54:07 UMS: Info Retry firmware download: from IP 10.101.0.14 for terminal i2001
JUN 29 15:54:54 tNetTask: Info duplicate IP address 0a65000d sent from ethernet address 00:02:b3:f7:7f:33
JUN 29 15:55:01 tNetTask: Info duplicate IP address 0a65000d sent from ethernet address 00:02:b3:f7:7f:33
JUN 29 15:55:03 tNetTask: Info duplicate IP address 0a65000d sent from ethernet address 00:02:b3:f7:7f:33
JUN 29 15:55:06 UMS: Info 15 Mins passed: Not all F/W downloaded successfully.
JUN 29 15:55:06 UMS: Warning ITG4041 F/W file(s) not received but IP Phone have registered. May have mixed F/W versions across phones. When F/W file(s) received, IP Phone will automatically be updated. (4)
JUN 29 15:55:16 tNetTask: Info duplicate IP address 0a65000d sent from ethernet address 00:02:b3:f7:7f:33
JUN 29 15:55:31 tMVX_XSPY: Info 0000150537 - DIM: 2:3 - MGB_DM_ERROR_INDICATION (Channel-Dependent): Error Code: 2 TCID 11
JUN 29 15:56:21 tNetTask: Info duplicate IP address 0a65000d sent from ethernet address 00:02:b3:f7:7f:33
JUN 29 15:56:28 tNetTask: Info duplicate IP address 0a65000d sent from ethernet address 00:02:b3:f7:7f:33
JUN 29 15:57:25 tMVX_XSPY: Info 0000161992 - DIM: 3:3 - MGB_DM_ERROR_INDICATION (Channel-Dependent): Error Code: 2 TCID 15
JUN 29 15:57:26 RDP: Error Delinquent connection 0x22818e8 lost between [10.101.0.13:4100] and [10.101.0.215:5000], previous status was -1
JUN 29 15:57:47 tNetTask: Info duplicate IP address 0a65000d sent from ethernet address 00:02:b3:f7:7f:33
JUN 29 15:57:58 tNetTask: Info duplicate IP address 0a65000d sent from ethernet address 00:02:b3:f7:7f:33
JUN 29 15:58:16 CSV: Warning Terminal 10.10.1.65 failed node 1 lookup
JUN 29 15:58:23 CSV: Warning ITS4011 10.10.1.65 connecting to node 0, TN: 65-00, 0x6404 [Power up, code: 0] (201)
JUN 29 15:58:24 UMS: Info 10.10.1.65 decision-never by DEFAULT_IPPH2
JUN 29 15:58:24 VTM: Info ITS5008 Terminal connection status: 10.10.1.65 ok (20)
JUN 29 15:58:25 SET: Info 10.10.1.65 TN 65-00 Registered with CS
JUN 29 15:58:39 SNTPC: Warning SNTP listen timeout
JUN 29 15:59:15 tNetTask: Info duplicate IP address 0a65000d sent from ethernet address 00:02:b3:f7:7f:33
JUN 29 15:59:17 RDP: Error Delinquent connection 0x22817e4 lost between [10.101.0.13:4100] and [10.101.0.215:5000], previous status was -1
JUN 29 15:59:19 ICMPMON: Warning Channel 16 is transmitting to a closed port. Reregistering the channel.
JUN 29 15:59:19 ICMPMON: Info VGW offline announce channel 16
JUN 29 15:59:19 VGW: Error channel 16 NOT registered. Audio Command: 0x12
JUN 29 15:59:19 VGW: Error channel 16 NOT registered. Audio Command: 0x22
JUN 29 15:59:20 tNetTask: Info duplicate IP address 0a65000d sent from ethernet address 00:02:b3:f7:7f:33
JUN 29 15:59:21 VGW: Info Channel 16, registered with CS, tn 0x010c
JUN 29 16:00:36 tNetTask: Info duplicate IP address 0a65000d sent from ethernet address 00:02:b3:f7:7f:33
JUN 29 16:01:48 tNetTask: Info duplicate IP address 0a65000d sent from ethernet address 00:02:b3:f7:7f:33
JUN 29 16:01:55 tNetTask: Info duplicate IP address 0a65000d sent from ethernet address 00:02:b3:f7:7f:33
JUN 29 16:01:55 tNetTask: Info duplicate IP address 0a65000d sent from ethernet address 00:02:b3:f7:7f:33
JUN 29 16:03:19 tNetTask: Info duplicate IP address 0a65000d sent from ethernet address 00:02:b3:f7:7f:33
JUN 29 16:07:06 XMSG: Info Got request of syslog.0 file from client.
JUN 29 16:07:11 SNTPC: Warning SNTP listen timeout

 

[AnthonyHardy]

haha . well, i reset the MC card and now our systems are talking, however, the analog and digital is disconnected from the IP at the main site. Maybe a reset of the ss?

 

[allenmac]

I would check to see what is causing the duplicate IP address message. Do you know which piece of equipment MAC address 00:02:b3:f7:7f:33 corresponds to?

 

[AnthonyHardy]

yup. i noticed that. that's the signalling server.

The SMC is still not registered in electShow. Maybe a reset of the ss and then the MC card?

 

[AnthonyHardy]

yup. definitely the signalling server per hex to ip:

http://www.johngaughan.net/toys/ipconv.php

and the mac address exists only on one of the two interfaces the ss is plugged into.

 

[allenmac]

Is it possible that the SMC thinks it should be the Leader, and is trying to use the Node IP address?

Can you connect a terminal to the SMC card? If so, what is the response to electShow?

I would unseat the SMC card for a couple minutes to let everything settle down, then plug the SMC card back in, and monitor it booting up if you can.

 

[AnthonyHardy]

I think you're right. That's probably my whole issue. I used element manager to get to the console of the SMC card and it thinks the ss isn't registered (the reverse of the ss).

How to I properly disable the MC card so I can unseat it? just ld 20 and disc 3?

 

[allenmac]

Can you display the ARP tables in the network switch? It should be able to tell you which port that IP address is connected to.

You could also try flushing the ARP tables.

 

[allenmac]

Yes. LD 20 disc <card TN>.

 

[AnthonyHardy]

Just as a note for anyone who finds this. It appears as though having the command "ip proxy-arp" in the Foundry switch is what was causing the problems. That appears to be what was causing the duplicate ip address reports in the signalling server logs.

I just needed a good night of sleep to wrap my head around the issue so the solution could bubble up to the top.

Thanks for ALL of your help!

 

[AnthonyHardy]

Aarrrrgh!

OK .. that wasn't the only problem apparently. I can now get the ip trunks to function for almost exactly 10 minutes and then they stop, giving a fast busy. Watching the Virtual Terminal for the MC card, I can see the message "disconnect from SS" on the screen when it obviously loses connection and the trunks stop working.

I can restart the SS server and get another ten minutes of functionality. The SS console never gave any message at all. The SS and MC are on the same vlan with nothing else and can ping each other without problem. I'm completely stumped. Guess it's time to get a PO and call tech support.

 

[AnthonyHardy]

More information. I actually know what the problem is (or the primary symptom of the problem), but not how to fix it.

The MC32 card isn't properly registering with the signaling server. When the TPS election is called, they BOTH believe they are master TPS's and take the node IP address (which explains the duplicate ip messages).

What this means to me is that something in our switch change has affected the TPS election process so the two systems are not talking to each other.

Does anyone know how the election process works? I'm assuming some sort of broadcast, but goodness, the ss and mc32 are both on the same subnet!

Can I set the MC32 default master? Right now, it's 0.0.0.0. I don't see a location for that in the element manager, but I'm going to look again.

 

[allenmac]

You can disable LTPS on the MC32, and that should prevent it from trying to become the master. I would use the forcedisTPS command, as there should not be any phones registered to the MC32 in the first place.

Also, if Spanning-Tree is enabled on your T-LAN ports, then disabling it for those ports may help.

electShow will display the Election status of the element you are logged into. I believe an Election is held every 15 minutes, with the winner being either the Leader SS, Follower SS, or SMC with the longest UP Time.

I had this problem once with an SMC32 card that did not upgrade properly and was on an older software release than the Signaling Server and PBX. swVersionShow will show the software release of the SMC32, and it should match the version running on the Signaling server.