Server Edition 11.1.2.3.0 Work Place Stopped Working

[PrimeABTech]

Hi Folks, it seems like yesterday we had a Primary Server failover to the secondary server. Since then, even with rebooting both Primary and secondary in 7071, Work Place is broken.

Only connecting via 5060. via the DNS as the server address, Domain listed and port 5060.

I get this alarm.

40301739mS ERR: TPKTUser Receive from x.x.x.x: Invalid header tpkt->data[0]==13 tpkt->data[1]==10 tpkt->data[2]==13 tpkt->data[3]==10 tpkt->data[4]==0
40302901mS ERR: TPKTUser Receive from x.x.x.x: Invalid header tpkt->data[0]==13 tpkt->data[1]==10 tpkt->data[2]==13 tpkt->data[3]==10 tpkt->data[4]==0

Anyone have any ideas on the Invalid header?

If I change it to 5061 I can reach the IPO and it sees it, but will blacklist it. Tried to SIP UI whitelist it.

Thoughts?

 

[PrimeABTech]

Here is what we find in the logs in 7071

one-X Portal 2023-02-15 15:46:50,012 ERROR Log4JRollingBackupImpl - Error while archiving log file : 1XIPODirServiceRollingFile-4.log Message: duplicate entry: 1XIPODirServiceRollingFile-4.log
one-X Portal 2023-02-15 15:46:50,012 ERROR Log4JRollingBackupImpl - Error while archiving log file : 1XIZWI-9.log Message: duplicate entry: 1XIZWI-9.log
one-X Portal 2023-02-15 15:46:50,012 ERROR Log4JRollingBackupImpl - Error while archiving log file : 1XStatusReporterRollingFile-7.log Message: duplicate entry: 1XStatusReporterRollingFile-7.log
one-X Portal 2023-02-15 15:46:50,012 ERROR Log4JRollingBackupImpl - Error while archiving log file : 1XMidLayerRollingFile-8.log Message: duplicate entry: 1XMidLayerRollingFile-8.log
one-X Portal 2023-02-15 15:46:50,012 ERROR Log4JRollingBackupImpl - Error while archiving log file : 1XSCSWebConferenceRollingFile-1.log Message: duplicate entry: 1XSCSWebConferenceRollingFile-1.log
one-X Portal 2023-02-15 15:46:50,012 ERROR Log4JRollingBackupImpl - Error while archiving log file : 1XOverallRollingFile-6.log Message: duplicate entry: 1XOverallRollingFile-6.log
one-X Portal 2023-02-15 15:46:50,012 ERROR Log4JRollingBackupImpl - Error while archiving log file : 1XSCSServicesRollingFile-2.log Message: duplicate entry: 1XSCSServicesRollingFile-2.log
one-X Portal 2023-02-15 15:46:58,252 ERROR TspiMessage - Exception while unpacking header .getTspiMessage unknown packet type 41
one-X Portal 2023-02-15 15:47:00,253 ERROR TspiMessage - Exception while unpacking header .getTspiMessage unknown packet type 41
one-X Portal 2023-02-15 15:47:06,036 ERROR SystemLoadManager - Rate:5.6666665E-4, count: 34
one-X Portal 2023-02-15 15:47:28,301 ERROR HealthThread - HEAP Used: 167Mb HEAP Committed: 506Mb NON-HEAP Used: 207Mb NON-HEAP Committed: 207Mb Threads: 404 ratio1: 0.33083667790426435 ratio2: 0.33083667790426435
one-X Portal 2023-02-15 15:47:28,719 ERROR TspiMessage - Exception while unpacking header .getTspiMessage unknown packet type 41
one-X Portal 2023-02-15 15:47:28,719 ERROR CallInfoMsg - CallInfoMsg: iCallStateMode=0
one-X Portal 2023-02-15 15:47:29,281 ERROR CallInfoMsg - CallInfoMsg: iCallStateMode=0
one-X Portal 2023-02-15 15:47:29,321 ERROR TspiMessage - Exception while unpacking header .getTspiMessage unknown packet type 41
one-X Portal 2023-02-15 15:47:29,621 ERROR TspiMessage - Exception while unpacking header .getTspiMessage unknown packet type 41
one-X Portal 2023-02-15 15:47:29,621 ERROR CallInfoMsg - CallInfoMsg: iCallStateMode=0
one-X Portal 2023-02-15 15:47:29,921 ERROR TspiMessage - Exception while unpacking header .getTspiMessage unknown packet type 41
one-X Portal 2023-02-15 15:47:29,921 ERROR CallInfoMsg - CallInfoMsg: iCallStateMode=0
one-X Portal 2023-02-15 15:47:30,241 ERROR TspiMessage - Exception while unpacking header .getTspiMessage unknown packet type 41
one-X Portal 2023-02-15 15:47:30,241 ERROR CallInfoMsg - CallInfoMsg: iCallStateMode=0
one-X Portal 2023-02-15 15:47:30,251 ERROR CallInfoMsg - CallInfoMsg: iCallStateMode=0
one-X Portal 2023-02-15 15:47:38,624 ERROR TspiMessage - Exception while unpacking header .getTspiMessage unknown packet type 41
one-X Portal 2023-02-15 15:47:38,624 ERROR CallInfoMsg - CallInfoMsg: iCallStateMode=1
one-X Portal 2023-02-15 15:47:38,635 ERROR CallInfoMsg - CallInfoMsg: iCallStateMode=1
one-X Portal 2023-02-15 15:47:49,602 ERROR TspiMessage - Exception while unpacking header .getTspiMessage unknown packet type 41
one-X Portal 2023-02-15 15:47:49,602 ERROR CallInfoMsg - CallInfoMsg: iCallStateMode=0
one-X Portal 2023-02-15 15:47:49,764 ERROR CallInfoMsg - CallInfoMsg: iCallStateMode=0
one-X Portal 2023-02-15 15:47:49,803 ERROR TspiMessage - Exception while unpacking header .getTspiMessage unknown packet type 41
one-X Portal 2023-02-15 15:47:49,987 ERROR TspiMessage - Exception while unpacking header .getTspiMessage unknown packet type 41
one-X Portal 2023-02-15 15:47:49,987 ERROR CallInfoMsg - CallInfoMsg: iCallStateMode=0
one-X Portal 2023-02-15 15:47:50,214 ERROR TspiMessage - Exception while unpacking header .getTspiMessage unknown packet type 41
one-X Portal 2023-02-15 15:47:50,214 ERROR CallInfoMsg - CallInfoMsg: iCallStateMode=0
one-X Portal 2023-02-15 15:47:50,214 ERROR CallInfoMsg - CallInfoMsg: iCallStateMode=0
one-X Portal 2023-02-15 15:47:51,033 ERROR OpenApiRestServlet - Time required in millis to process rest request :status and request resource : startupstatusforwebconsole is : 0
one-X Portal 2023-02-15 15:48:05,217 ERROR TspiMessage - Exception while unpacking header .getTspiMessage unknown packet type 41
one-X Portal 2023-02-15 15:48:05,218 ERROR CallInfoMsg - CallInfoMsg: iCallStateMode=1
one-X Portal 2023-02-15 15:48:06,037 ERROR SystemLoadManager - Rate:9.999833E-4, count: 60
one-X Portal 2023-02-15 15:48:06,501 ERROR TspiMessage - Exception while unpacking header .getTspiMessage unknown packet type 41
one-X Portal 2023-02-15 15:48:06,501 ERROR CallInfoMsg - CallInfoMsg: iCallStateMode=0
one-X Portal 2023-02-15 15:48:06,780 ERROR CallInfoMsg - CallInfoMsg: iCallStateMode=0
one-X Portal 2023-02-15 15:48:06,820 ERROR TspiMessage - Exception while unpacking header .getTspiMessage unknown packet type 41
one-X Portal 2023-02-15 15:48:11,877 ERROR TspiMessage - Exception while unpacking header .getTspiMessage unknown packet type 41
one-X Portal 2023-02-15 15:48:11,877 ERROR CallInfoMsg - CallInfoMsg: iCallStateMode=0
one-X Portal 2023-02-15 15:48:11,928 ERROR TspiMessage - Exception while unpacking header .getTspiMessage unknown packet type 41
one-X Portal 2023-02-15 15:48:13,879 ERROR TspiMessage - Exception while unpacking header .getTspiMessage unknown packet type 41
one-X Portal 2023-02-15 15:48:21,501 ERROR TspiMessage - Exception while unpacking header .getTspiMessage unknown packet type 41
one-X Portal 2023-02-15 15:48:23,504 ERROR TspiMessage - Exception while unpacking header .getTspiMessage unknown packet type 41
one-X Portal 2023-02-15 15:48:28,301 ERROR HealthThread - HEAP Used: 121Mb HEAP Committed: 506Mb NON-HEAP Used: 207Mb NON-HEAP Committed: 207Mb Threads: 406 ratio1: 0.23989965684762493 ratio2: 0.23990363298118444
one-X Portal 2023-02-15 15:48:53,202 ERROR TspiMessage - Exception while unpacking header .getTspiMessage unknown packet type 41
one-X Portal 2023-02-15 15:48:53,202 ERROR CallInfoMsg - CallInfoMsg: iCallStateMode=0
one-X Portal 2023-02-15 15:48:53,806 ERROR OpenApiRestServlet - Time required in millis to process rest request :status and request resource : startupstatusforwebconsole is : 1
one-X Portal 2023-02-15 15:49:06,037 ERROR SystemLoadManager - Rate:5.6666665E-4, count: 34
one-X Portal 2023-02-15 15:49:08,202 ERROR TspiMessage - Exception while unpacking header .getTspiMessage unknown packet type 41
one-X Portal 2023-02-15 15:49:10,203 ERROR TspiMessage - Exception while unpacking header .getTspiMessage unknown packet type 41
one-X Portal 2023-02-15 15:49:28,301 ERROR HealthThread - HEAP Used: 125Mb HEAP Committed: 506Mb NON-HEAP Used: 207Mb NON-HEAP Committed: 207Mb Threads: 403 ratio1: 0.24826480929017583 ratio2: 0.24826480929017583
one-X Portal 2023-02-15 15:49:35,465 ERROR TspiMessage - Exception while unpacking header .getTspiMessage unknown packet type 41
one-X Portal 2023-02-15 15:49:35,465 ERROR CallInfoMsg - CallInfoMsg: iCallStateMode=0
one-X Portal 2023-02-15 15:49:35,465 ERROR CallInfoMsg - CallInfoMsg: iCallStateMode=0
one-X Portal 2023-02-15 15:49:35,516 ERROR TspiMessage - Exception while unpacking header .getTspiMessage unknown packet type 41
one-X Portal 2023-02-15 15:49:37,064 ERROR CallInfoMsg - CallInfoMsg: iCallStateMode=0
one-X Portal 2023-02-15 15:49:37,064 ERROR TspiMessage - Exception while unpacking header .getTspiMessage unknown packet type 41
one-X Portal 2023-02-15 15:49:37,064 ERROR CallInfoMsg - CallInfoMsg: iCallStateMode=0
one-X Portal 2023-02-15 15:49:37,065 ERROR CallInfoMsg - CallInfoMsg: iCallStateMode=0
one-X Portal 2023-02-15 15:49:37,155 ERROR TspiMessage - Exception while unpacking header .getTspiMessage unknown packet type 41
one-X Portal 2023-02-15 15:49:37,466 ERROR TspiMessage - Exception while unpacking header .getTspiMessage unknown packet type 41
one-X Portal 2023-02-15 15:49:37,769 ERROR TspiMessage - Exception while unpacking header .getTspiMessage unknown packet type 41
one-X Portal 2023-02-15 15:49:37,769 ERROR CallInfoMsg - CallInfoMsg: iCallStateMode=0
one-X Portal 2023-02-15 15:49:38,749 ERROR TspiMessage - Exception while unpacking header .getTspiMessage unknown packet type 41
one-X Portal 2023-02-15 15:49:38,749 ERROR CallInfoMsg - CallInfoMsg: iCallStateMode=0
one-X Portal 2023-02-15 15:49:39,065 ERROR TspiMessage - Exception while unpacking header .getTspiMessage unknown packet type 41
one-X Portal 2023-02-15 15:49:39,065 ERROR CallInfoMsg - CallInfoMsg: iCallStateMode=0
one-X Portal 2023-02-15 15:49:39,406 ERROR TspiMessage - Exception while unpacking header .getTspiMessage unknown packet type 41
one-X Portal 2023-02-15 15:49:39,406 ERROR CallInfoMsg - CallInfoMsg: iCallStateMode=0
one-X Portal 2023-02-15 15:49:39,837 ERROR TspiMessage - Exception while unpacking header .getTspiMessage unknown packet type 41
one-X Portal 2023-02-15 15:49:39,837 ERROR CallInfoMsg - CallInfoMsg: iCallStateMode=0
one-X Portal 2023-02-15 15:49:40,736 ERROR TspiMessage - Exception while unpacking header .getTspiMessage unknown packet type 41
one-X Portal 2023-02-15 15:49:40,736 ERROR CallInfoMsg - CallInfoMsg: iCallStateMode=0
one-X Portal 2023-02-15 15:49:41,107 ERROR TspiMessage - Exception while unpacking header .getTspiMessage unknown packet type 41
one-X Portal 2023-02-15 15:49:41,107 ERROR CallInfoMsg - CallInfoMsg: iCallStateMode=0
one-X Portal 2023-02-15 15:49:41,528 ERROR TspiMessage - Exception while unpacking header .getTspiMessage unknown packet type 41
one-X Portal 2023-02-15 15:49:41,528 ERROR CallInfoMsg - CallInfoMsg: iCallStateMode=0
one-X Portal 2023-02-15 15:49:41,816 ERROR TspiMessage - Exception while unpacking header .getTspiMessage unknown packet type 41
one-X Portal 2023-02-15 15:49:41,816 ERROR CallInfoMsg - CallInfoMsg: iCallStateMode=0
one-X Portal 2023-02-15 15:49:42,245 ERROR TspiMessage - Exception while unpacking header .getTspiMessage unknown packet type 41
one-X Portal 2023-02-15 15:49:42,245 ERROR CallInfoMsg - CallInfoMsg: iCallStateMode=0
one-X Portal 2023-02-15 15:49:42,606 ERROR TspiMessage - Exception while unpacking header .getTspiMessage unknown packet type 41
one-X Portal 2023-02-15 15:49:42,606 ERROR CallInfoMsg - CallInfoMsg: iCallStateMode=0
one-X Portal 2023-02-15 15:49:45,931 ERROR TspiMessage - Exception while unpacking header .getTspiMessage unknown packet type 41
one-X Portal 2023-02-15 15:49:46,632 ERROR TspiMessage - Exception while unpacking header .getTspiMessage unknown packet type 41
one-X Portal 2023-02-15 15:49:46,632 ERROR CallInfoMsg - CallInfoMsg: iCallStateMode=0
one-X Portal 2023-02-15 15:49:47,735 ERROR TspiMessage - Exception while unpacking header .getTspiMessage unknown packet type 41
one-X Portal 2023-02-15 15:49:47,735 ERROR CallInfoMsg - CallInfoMsg: iCallStateMode=0
one-X Portal 2023-02-15 15:49:55,179 ERROR TspiMessage - Exception while unpacking header .getTspiMessage unknown packet type 41
one-X Portal 2023-02-15 15:49:55,179 ERROR CallInfoMsg - CallInfoMsg: iCallStateMode=1
one-X Portal 2023-02-15 15:49:56,599 ERROR OpenApiRestServlet - Time required in millis to process rest request :status and request resource : startupstatusforwebconsole is : 0
one-X Portal 2023-02-15 15:50:06,037 ERROR SystemLoadManager - Rate:0.0015, count: 90
one-X Portal 2023-02-15 15:50:26,309 ERROR OpenApiRestServlet - Time required in millis to process rest request :status and request resource : startupstatusforwebconsole is : 0
one-X Portal 2023-02-15 15:50:28,302 ERROR HealthThread - HEAP Used: 133Mb HEAP Committed: 506Mb NON-HEAP Used: 207Mb NON-HEAP Committed: 207Mb Threads: 401 ratio1: 0.26435793877236274 ratio2: 0.26435793877236274
one-X Portal 2023-02-15 15:50:31,153 ERROR OpenApiRestServlet - Time required in millis to process rest request :status and request resource : startupstatusforwebconsole is : 0
IP Office Failed to close SERVICE pipe stream.
IP Office Failed to close SERVICE pipe stream.

 

[Travis Harper]

The first set of errors are related to archiving log files, and they indicate that there are duplicate entries for several log files. This could be caused by a problem with the archiving process, or it could indicate that the system is generating more log files than expected.

The second set of errors is related to the "TspiMessage" class, and they indicate that there is an exception occurring when attempting to unpack a header. The error message states that the packet type is unknown, which suggests that there may be a problem with the network communication between the client and the server.

The third set of errors are related to the "CallInfoMsg" class, and they indicate that the system is reporting changes in the call state mode. It's unclear what might be causing these changes or whether they are related to the other errors that are occurring.

Finally, there is an error related to the "SystemLoadManager" class, which indicates that the system is experiencing a high rate of CPU usage. This could be due to a problem with the one-X Portal application itself, or it could be caused by other processes running on the same machine.

Without more information, it's difficult to determine the root cause of these errors, but they suggest that there may be some issues with the one-X Portal system that should be investigated further.

 

[Travis Harper]

If I had to guess, I would say there is something not quite right with he One-X Portal resilience configuration.

Maybe spend some time validating your resiliency configuration.

https://ipofficekb.avaya.com/busine...vaya Workplace Client for IP Office_en-us.pdf

 

[Travis Harper]

Regarding your first post on invalid header.

The error message suggests that the header of the TPKT message received from the IP address x.x.x.x is invalid. Specifically, the first five bytes of the header have the values 13, 10, 13, 10, and 0, which do not match the expected values for a valid TPKT header.

Possible causes of this error include:

Network issues: There may be a problem with the network that is causing data corruption or loss, which could result in an invalid TPKT header being received.
Firewall or security software: It is possible that a firewall or other security software is blocking or modifying TPKT traffic, which could result in an invalid header being received.

The TPKT header is a 4-byte header used to wrap data being transmitted over the TCP/IP protocol. The expected values of the TPKT header are:

Byte 1 (tpkt->data[0]): It should be equal to 3, which is the TPKT version number.

Byte 2 (tpkt->data[1]): It should be equal to 0, which is the reserved byte and should always be set to 0.

Bytes 3 and 4 (tpkt->data[2] and tpkt->data[3]): They should contain the length of the data in the TPKT packet, including the TPKT header itself. The length is represented in two bytes as an unsigned 16-bit integer in network byte order.

So in the error message you provided, the TPKT header is invalid because the values of the first four bytes (tpkt->data[0] through tpkt->data[3]) do not match the expected values of the TPKT header. Specifically, the bytes have the values 13, 10, 13, and 10, respectively, which do not match the expected values of 3, 0, and the length of the data in network byte order.

 

[PrimeABTech]

Thanks Travis its looking like the system LAN has UDP, TCP and TLS all checked off. Which forever has been the case. Its looking like when we turn off TLS all issues go away. We just need to adjust the 46xx file settings I think to remove TLS from Work Place as it runs internal only and I think we are in the chips. Not sure how a failover would create the problem communicating but its what has fixed it... Weird