I'm running AES6.3 and both our web and aeservices certs expired. We failed to catch this unfortunately. This is used for remote call control with MS OCS/Lync, using DMCC. We provide certs from our own PKI and went through the process to provide new ones. The services were all restarted; However, the DMCC sessions seem to keep dropping. Looks like maybe 20 or so remain steady, the count then increases a few and then drops back down. The count of sessions created since service boot keeps incrementing, but established sessions are not. Is there anything to look out for when cutting certs for this server?
Tek-Tips is the largest IT community on the Internet today!
Members share and learn making Tek-Tips Forums the best source of peer-reviewed technical information on the Internet!
-
Congratulations MikeeOK on being selected by the Tek-Tips community for having the most helpful posts in the forums last week. Way to Go!
AES aeservice certificate
- Thread starter trilogy8
- Start date
- Thread starter
- #2
Doesn't sound like certs. You'd get nothin' if it wasn't trusted. Some secure.log in /var/log/avaya/aes or tshark/tcpdump would show TLS fatal blablabla
How many MS servers talk to AES? Is it all thru 1 MS box to 1 AES? Turn up your DMCC and TSAPI logging in the web page and follow the breadcrumbs!
How many MS servers talk to AES? Is it all thru 1 MS box to 1 AES? Turn up your DMCC and TSAPI logging in the web page and follow the breadcrumbs!
- Thread starter
- #4
Yea, was on the same thought about it no longer being the certs. There's a pool of Lync FE servers that point at this AES. There was consistency with about every other call working, which leads me to believe there's a quirky Lync server somewhere. That team is looking into that. Avaya didn't indicate any root cause after collecting logs and their recommendation is for me are to rebuild the dial plan and re-import certs. I found that to be a bit preposterous. How does that suddenly go bad after it working for years.
- Thread starter
- #5
Still having this issue with inconsistency. Avaya has said that all the TR/87 tests pass and are indicating this is not an AES issue. And are also indicating there is nothing indicative of a s/w bug. Are those 3 tests enough to go on for this?
Crank up the AES logging and check the Lync side.
If you got many Lync servers talking to aes, each should have a log of its IP when using those services.
Gonna have to get your hands dirty.
AES's self contained tests are 'good enough' to show AES can run a little client for DMCC or tsapi on localhost and connect to its own service - which is really just step 1 in proving you don't have a global aes failure.
Maybe you'll see something like 2 FE servers doing the same thing for the same user at the same time at AES that confuses it and leads you down a path of your FE servers aren't sync'd up or something. I don't know anything about Lync, but I do know about 'two app servers using 1 AES in tandem doing the same thing at the same time is bad news'
Just a thought trying to point you in the right direction...
If you can reproduce it - even one out of a dozen times, just log it to death and look for the difference. If you're really nice to your AES guy and provide those logs with a halfway decent effort of debugging them, you might just fish the answer out of him.
If you got many Lync servers talking to aes, each should have a log of its IP when using those services.
Gonna have to get your hands dirty.
AES's self contained tests are 'good enough' to show AES can run a little client for DMCC or tsapi on localhost and connect to its own service - which is really just step 1 in proving you don't have a global aes failure.
Maybe you'll see something like 2 FE servers doing the same thing for the same user at the same time at AES that confuses it and leads you down a path of your FE servers aren't sync'd up or something. I don't know anything about Lync, but I do know about 'two app servers using 1 AES in tandem doing the same thing at the same time is bad news'
Just a thought trying to point you in the right direction...
If you can reproduce it - even one out of a dozen times, just log it to death and look for the difference. If you're really nice to your AES guy and provide those logs with a halfway decent effort of debugging them, you might just fish the answer out of him.
- Thread starter
- #7
When accessing the AES log section there are a bazillion different files. Are there specific ones to try and sniff through? Avaya actually had turned up the logging and had me duplicate the issue and then pulled them off. That's when they mentioned the dial plan rebuild. A follow up from another shift SME said he didn't think that was the issue and wouldn't go down that path. I'd be open to rebuilding the dial plan, but I'm not clear how that should be done.
Finally, they said they've spent quite a bit of time looking into this issue and aren't seeing anything indicative of a problem on AES and further investigation is billable.
I have a duplicated AES server in another DC and am going to have the MS team point the Lync pool at to eliminate this specific AES server.
Finally, they said they've spent quite a bit of time looking into this issue and aren't seeing anything indicative of a problem on AES and further investigation is billable.
I have a duplicated AES server in another DC and am going to have the MS team point the Lync pool at to eliminate this specific AES server.
Aes logs aren't that complicated...theres a logging level for each CTI service and you're only using DMCC probably with tr87 and tsapi. Then there's the g3 which I believe is the one for msgs to/from CM. Maybe turn the logging levels down a bit. Fine/finest/debug will get you a gargantuan amount of data for a phone call.
What are you able to configure on the Lync side? Like, does each FE server have a CTI service that can be configured independently? If so, maintenance window and try each one in isolation one at a time?
Any MS logs server side or client side that might give a hint what's going on in there when a failure occurs?
What are you able to configure on the Lync side? Like, does each FE server have a CTI service that can be configured independently? If so, maintenance window and try each one in isolation one at a time?
Any MS logs server side or client side that might give a hint what's going on in there when a failure occurs?
- Status
Similar threads
- Locked
- Question