Exchange server is still unresponsive at times... Anyone?

[reynolwi]

Ok, i am totally lost and dont know where to turn. I finally have our exchange aware backup solution reporting in exchange so exchange is now happy that the logs are purged but exchange is still becoming unresponsive at times. It keeps showing these errors:


Event Type: Error
Event Source: MSExchangeSA
Event Category: General
Event ID: 9153
Date: 1/13/2008
Time: 7:08:52 AM
User: N/A
Computer: COMPNAME
Description:
Microsoft Exchange System Attendant reported an error '0x80004005' when setting DS notification.

-- -- -- --

Event Type: Warning
Event Source: MSExchangeAL
Event Category: LDAP Operations
Event ID: 8031
Date: 1/13/2008
Time: 7:08:22 AM
User: N/A
Computer: COMPNAME
Description:
Unable to open LDAP session on directory 'SRV1' using port number 389. Directory returned the LDAP error:[0x1] Operations Error.

-- -- -- --

Event Type: Error
Event Source: MSExchangeSA
Event Category: General
Event ID: 1031
Date: 1/13/2008
Time: 7:11:48 AM
User: N/A
Computer: COMPNAME
Description:
One of the System Attendant's task is blocked.
Function: COffLineABScanTask::Work

-- -- -- --

Event Type: Error
Event Source: MSExchangeSA
Event Category: General
Event ID: 1031
Date: 1/13/2008
Time: 7:14:48 AM
User: N/A
Computer: COMPNAME
Description:
One of the System Attendant's task is blocked.
Function: CMonitoringTask::Work

-- -- -- --

Event Type: Error
Event Source: MSExchangeSA
Event Category: General
Event ID: 1031
Date: 1/13/2008
Time: 7:23:48 AM
User: N/A
Computer: COMPNAME
Description:
One of the System Attendant's task is blocked.
Function: CPollDomainExchangeServersGroup::Work

-- -- -- --

Event Type: Warning
Event Source: MSExchangeMU
Event Category: General
Event ID: 1040
Date: 1/13/2008
Time: 8:31:15 AM
User: N/A
Computer: COMPNAME
Description:
Metabase Update failed replication 5 times with error 80004005 (Unspecified error). Please change the diagnostic logging level of MSExchangeMU to 'minimum' or greater to find the source of the problem.

-- -- -- --

Event Type: Error
Event Source: MSExchangeSA
Event Category: Monitoring
Event ID: 9097
Date: 1/13/2008
Time: 5:32:46 AM
User: N/A
Computer: TXEXCH
Description:
The MAD Monitoring thread was unable to connect to WMI, error '0x800705aa'.



Event ID 8031 repeats the most. Most of the app event log is all 8031 saying it cant open a connection. I have searched high and low and still cant figure out why its doing this. And once it starts about 24 hours later the server becomes unresponsive and you have to reboot it. Its like exchange just completely shuts down and doesnt respond.

Someone please tell me what is happening...

Wm. Reynolds
RRWDS | TxPSS

http://www.rrwds.com

http://www.txpubsafety.com

- - - - - - - - - - - - -
Network Error:
Hit any user to continue

 

[Zelandakh]

What happened when you ran policytest when you last asked this question?

 

[xmsre]

For the 8031s check your enterprise RUS and the the RUS for each domain. Check each endpoint carefully; are they valid?

 

[reynolwi]

Everything showed it was fine lasttime. I was trying to re-run one but stupid me forgot how to do it. How would the Reciepient Update Service affect anything. Theres only 1 enterprise domain and then an additional domain in exchange and its only assigned to very few ppl.

I dont know whats going on but the exchange server is freaking out. I have new error messages today and it completely stopped responding. I couldnt even get it to restart the system attendant service. I had to completely reboot the server again. I do not know whats going on but its all going downhill fast.

Here are the new errors:

Event Type: Warning
Event Source: MSSQL$COMMVAULTQINETIX
Event Category: (8)
Event ID: 19011
Date: 1/14/2008
Time: 6:17:56 PM
User: N/A
Computer: COMPNAME
Description:
The description for Event ID ( 19011 ) in Source ( MSSQL$COMMVAULTQINETIX ) cannot be found. The local computer may not have the necessary registry information or message DLL files to display messages from a remote computer. You may be able to use the /AUXSOURCE= flag to retrieve this description; see Help and Support for details. The following information is part of the event: (SpnRegister) : Error 8206.

-- -- -- -- --

Event Type: Error
Event Source: Server ActiveSync
Event Category: None
Event ID: 3005
Date: 1/14/2008
Time: 6:15:46 PM
User: DOMAIN\user
Computer: COMPNAME
Description:
Unexpected Exchange mailbox Server error: Server: [server.domain.net] User: [user@domain.net] HTTP status code: [503]. Verify that the Exchange mailbox Server is working correctly.

For more information, see Help and Support Center at

http://go.microsoft.com/fwlink/events.asp.

-- -- -- -- --

Event Type: Error
Event Source: MSExchangeSA
Event Category: MAPI Session
Event ID: 9175
Date: 1/14/2008
Time: 6:15:44 PM
User: N/A
Computer: COMPNAME
Description:
The MAPI call 'OpenMsgStore' failed with the following error:
The Microsoft Exchange Server computer is not available. Either there are network problems or the Microsoft Exchange Server computer is down for maintenance.
The MAPI provider failed.
Microsoft Exchange Server Information Store
ID no: 8004011d-0526-00000000

-- -- -- -- --

Event Type: Error
Event Source: MSExchangeSA
Event Category: Monitoring
Event ID: 1005
Date: 1/14/2008
Time: 6:13:52 PM
User: N/A
Computer: COMPNAME
Description:
Unexpected error <<0xc1050000 - The Microsoft Exchange Server computer is not available. Either there are network problems or the Microsoft Exchange Server computer is down for maintenance. The MAPI provider failed. Microsoft Exchange Server Information Store ID no: 8004011d-0526-00000000>> occurred.

-- -- -- -- --

Event Type: Error
Event Source: MSExchangeFBPublish
Event Category: General
Event ID: 8206
Date: 1/14/2008
Time: 6:13:20 PM
User: N/A
Computer: COMPNAME
Description:
Unable to prepare message table for polling thread processing on virtual machine COMPNAME. The error number is 0x80040115. Make sure Microsoft Exchange Store is running.

-- -- -- -- --

Event Type: Error
Event Source: MSExchangeSA
Event Category: OAL Generator
Event ID: 9126
Date: 1/14/2008
Time: 6:12:46 PM
User: N/A
Computer: COMPNAME
Description:
OALGen encountered error 80040115 while calculating the offline address list for address list '\Global Address List'. This offline address list will not be available for client download.
- Default Offline Address List

-- -- -- -- --

Event Type: Error
Event Source: MSExchangeSA
Event Category: OAL Generator
Event ID: 9331
Date: 1/14/2008
Time: 6:12:46 PM
User: N/A
Computer: COMPNAME
Description:
OALGen encountered error 80040115 (internal ID 50101df) accessing the public folder store while generating the offline address list for address list '\Global Address List'.
- Default Offline Address List

-- -- -- -- --


Event Type: Error
Event Source: MSExchangeSA
Event Category: OAL Generator
Event ID: 9360
Date: 1/14/2008
Time: 6:12:46 PM
User: N/A
Computer: COMPNAME
Description:
OALGen encountered an error while generating the changes.oab file for version 2 and 3 differential downloads of address list '\Global Address List'. The offline address list has not been updated so clients will not be able to download the current set of changes. Check other logged events to find the cause of this error.
If the cause of the problem was intentional or cannot be resolved, OALGen can be forced to post a full offline address list by creating the DWORD registry key 'HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\MSExchangeSA\Parameters\OAL post full if diff fails' and setting it to 1 on this server. When OALGen next generates the offline address list, clients will perform a full OAB download. After that time, the registry key should be removed to prevent further full downloads.
- Default Offline Address List

-- -- -- -- --

Event Type: Error
Event Source: MSExchangeSA
Event Category: General
Event ID: 1031
Date: 1/14/2008
Time: 6:10:17 PM
User: N/A
Computer: COMPNAME
Description:
One of the System Attendant's task is blocked.
Function: COffLineABGenTask::Work

-- -- -- -- --

Event Type: Warning
Event Source: Server ActiveSync
Event Category: None
Event ID: 3007
Date: 1/14/2008
Time: 6:00:37 PM
User: Domain\user
Computer: COMPNAME
Description:
Exchange mailbox Server response timeout: Server: [server.domain.net] User: [user@domain.net]. Exchange ActiveSync Server failed to communicate with the Exchange mailbox server in a timely manner. Verify that the Exchange mailbox Server is working correctly and is not overloaded.

For more information, see Help and Support Center at

http://go.microsoft.com/fwlink/events.asp.

-- -- -- -- --

Event Type: Warning
Event Source: MSExchangeSA
Event Category: OAL Generator
Event ID: 9116
Date: 1/14/2008
Time: 4:39:42 PM
User: N/A
Computer: COMPNAME
Description:
OALGen encountered an error while generating the binpatch.oab file for differential downloads of address list '\Global Address List'. Clients will not be able to incrementally update to the new version of the offline address list, they will perform a full download instead. This is normal if this is the first time this offline address list has been generated. Check other logged events to see if this is a serious error.
- Default Offline Address List

-- -- -- -- --

Event Type: Error
Event Source: MSExchangeSA
Event Category: General
Event ID: 1031
Date: 1/14/2008
Time: 4:18:17 PM
User: N/A
Computer: COMPNAME
Description:
One of the System Attendant's task is blocked.
Function: CMonitoringTask::Work




Wm. Reynolds
RRWDS | TxPSS

http://www.rrwds.com

http://www.txpubsafety.com

- - - - - - - - - - - - -
Network Error:
Hit any user to continue

 

[Zelandakh]

An additional domain in Exchange? Can you clarify how things are arranged on your network?

 

[xmsre]

Certainly there are some permissions problems and some OAB generation problems. Try

http://support.microsoft.com/kb/328229/en-us

There are a whole series of problems that have an error code of 80040115, which is basicly a connectivity issue. The 9126s in particular would lead me to believe that your exchange server is having issues contacting a GC. Is the Exchange server also a GC? Was it ever a DC/GC? Has the AD role changed since Exchange was installed?

 

[reynolwi]

Ok in the enterprise we have 2 ad servers, exchange server, and a file server. 1 ad, exchange, and the file servers are here at corporate and the other ad server is at the remote site. Both ad servers are DCs and are the GCs as well. (each ad server is DC/GC and there are 2)

exchange is set to only pull from the dc/gc here at corporate because it was acting weird when both DCs were listed. The enterprise is actually 2 companies merged on 1 network so the main domain is covered as primary under exchange which we will call domain-main.net
Since some of the employees belong to the other company i listed a second domain which i will call domain-sec.net within exchange as a new reciepient policy or whatever its called and is then assigned to those employees as their primary email instead of the main exchange. If havent had any problems with this setup and it has worked since it was setup like that probably 8 months ago.

There have been no major AD changes in the enterprise and i havent added any servers or changed up anything. The exchange server has never and will never be a DC because that causes serious issues. I read that article....


Wm. Reynolds
RRWDS | TxPSS

http://www.rrwds.com

http://www.txpubsafety.com

- - - - - - - - - - - - -
Network Error:
Hit any user to continue

 

[reynolwi]

Does anyone have any ideas? Im at a loss here and the more it happens the more downtime i have. I dont know if its a microsoft update that didnt take well and its corrupt something or what. Ive searched microsoft, called them (that was a mistake), i just cant seem to find an explanation. I know its gotta be starring right at me and i just cant see it.

Wm. Reynolds
RRWDS | TxPSS

http://www.rrwds.com

http://www.txpubsafety.com

- - - - - - - - - - - - -
Network Error:
Hit any user to continue

 

[xmsre]

You have two GCs, and exchange is hard coded to access one of them. The 80040115 errors indicated a connectivity issue between exchange and the GC. I'd start troubleshooting there.

 

[reynolwi]

ok answer me this. What could have changed within AD to make it lose connectivity? Could it possibly be a server problem with that one AD server? I did a dcdiag test on the AD server and everything passed. I looked thru event logs on that AD server and found KCC errors for sites and services and i do believe i fixed it but they dont happen on the days that exchange goes down. No errors occur on the AD server when exchange goes down. Im wondering if maybe the onboard gbit nic is having problems.

Wm. Reynolds
RRWDS | TxPSS

http://www.rrwds.com

http://www.txpubsafety.com

- - - - - - - - - - - - -
Network Error:
Hit any user to continue

 

[reynolwi]

Anybody have any other ideas?

Wm. Reynolds
RRWDS | TxPSS

http://www.rrwds.com

http://www.txpubsafety.com

- - - - - - - - - - - - -
Network Error:
Hit any user to continue

 

[Zelandakh]

Go down the path that it is not AD that is the problem but an Exchange comms problem. You need to look at the GC connectivity. Ensure that both GCs are listed and report back any changes you have made and exactly where you have made them.

 

[reynolwi]

ok i have added the other server as a gc. i can not get it to list as a domain controller though. im working on that problem know because the remote dc is having problems replicating gpo stuff with the dc here at corporate. I dont know if thats part of the problem but every time i try and list the remote dc as a domain controller in exchange it tells me it can not find the server and to check the name and try again.

Wm. Reynolds
RRWDS | TxPSS

http://www.rrwds.com

http://www.txpubsafety.com

- - - - - - - - - - - - -
Network Error:
Hit any user to continue