Applying Windows 2003 SP1 1

[silentsam33]

I have a Windows 2003 Ent Server running Exchange 2003 SP1, GFI's ME v.9, McAfee Groupshield v.5.2 and Veritas BackupExec 9.1.
I applied Windows 2003 SP1 on the weekend and all went well. The only error I got was that a service or driver failed to load. This was a result of the load balance service could not start as we had set it to disabled.
So far, 4 days later, all applications icluding Exchange are running without a problem.
Hopefully all other Exchange Admin will have a similar experience.

 

[silentsam33]

Well, last night we did encounter a problem. Users in our remote offices (connected via multiple VPN sessions) could not access Exchange via the Outlook client. But they could via Web. Users in the Main office where the mail server is were fine.
On the server I only noticed one problem, I could not use explorer (that is double-click My Computer), it would hang and freeze. Once it was killed via TS the screen would go blue for about 10 seconds and then come back. It did this via TS at my home and logged in localy to the server.
The error message listed in the logs are,

ID:10005
ID:1002
ID:8230/8206

Has anyone seen anything similar? This has never occurred before and I am left wondering if a security change from Windows 2003 SP1 is the root cause.

 

[silentsam33]

Well it did it to me again on the weekend. I have now turned up logging for RPC as I believe this is the cause of my problems. I am thinking about rolling back Windows 2003 SP1 as this problem only occurred after it was applied over a week ago.

Again when the problem starts,
-All Web Mail is fine.
-All locally connected computers can connect fine.
-All remote VPN connection (User from at home and our remote offcies)cannot access e-mail via Outlook Client. But can via webmail.

Anyone esle seen this?

 

[Tightpants]

I had a similar problem when I installed SP1 on our two servers last week. The problem was due to the Windows Firewall now installed on the servers as a part of SP1. Check your Event Viewer Security log to see if you are getting any failures due to the Windows Firewall.

After installing SP1 I was getting warnings about DNS and error saying that the RPC server was unavailable. Disabling the firewall in Control Panel, Administrative Tools, Services solved the problem.

 

[PuzzledUNIX]

The issue is only in firewall setting on SP1, so many companies are burned out on this, just read the deployment paper on SP1 and you will be fine.

 

[silentsam33]

Thanks for the replies. In our case the Windows 2003 servers are set not to use the Windows Firewall. This is one of our Group Policy settings for all servers and disables that service.
I have not yet rolled back the service pack, but may do it tonight if I can't find another solutions.

 

[silentsam33]

Well it happened again this morning at 5:30am EST. I have removed the SP1 update. I am going to bite the bullet and call Microsoft support for help on this one.

 

[JustinRobbins]

I am having the same issue

We have 4 vpn sites - each are connected using Cisco Pix firewalls with VPN-IPSEC tunnels to each site.

After 2 days of heavy usage, the ability to access OUTLOOK, PRINTERS or anything IP related appears to deterroirate over time. For the first few days from a reboot - the server operates fine. Then people cannot print, they cannot use OUTLOOK, or anything.

It's wierd because once I UNINSTALLED the service pack, the problem STILL existed.

It wasn't there prior and now I know it is a bug since I am not the only one experiencing the exact same issue.

I have contacted Microsoft and they claim that it's something on our network.

Strangely though the problem goes away from a reboot until the port numbers gets high in the sequence (which takes up to 2 days).

Modifying the tcp stack to reflect additional tcp connections and changes in the timeout values and mss does not resolve the problem.

One more thing - this started in Windows XP SP2.
We had issues where about 15 of our machines would suddenly lose their network connections - they could PING - but you cannot use anything Winsock related - including Internet Explorer or FTP. Even WS_FTP fails to function until a reboot of the system.

It's almost as if buffers or something are becoming saturated and then they are not expunging the garbage the correct way.

PLEASE if you have a fix for this, please post info about it. This is killing us. We reboot about 3 times a week because of this and an uninstall of the service pack did not resolve the problem.

P.S. we have this on our fileserver, and our SQL server as well as a result of sp1.

 

[sab4you]

I found this tech info useful. We havent come across the problem, but maybe this is what ails you?

http://support.microsoft.com/default.aspx/kb/898060?

 

[JustinRobbins]

I am researching this now

SAB4YOU thank you so much. I have applied the hotfix
and will see what happens now.

Thanks,
Justin

 

[silentsam33]

JustinRobbins,
That was the exact same articule that I came across yesterday. I showed it to my boss (network Admin) and he believes that is our problem as well. He has seen similiar behaviour on some of our other Windows 2003 servers.
Our setup is very similar to your, multiple VPN connected remote offcies using IPSEC tunnels and a head office hosting the majority of services. We also have about 50-100 users connecting in via high speed VPN sessions.

Did you have to call MS to get the hotfix and did they charge you for it?

 

[JustinRobbins]

I called MS and it was free for the hotfix

I have not been able to VERIFY that it does indeed correct
the problem but I applied it to 9 of my servers last night and rebooted them all. We will see shortly what happens.

 

[silentsam33]

Thanks Justin for the update. I assume that you have not removed SP1 off these servers? Let us know if the hotfix does fix the issue. Thanks, Chris.

 

[TChachra]

SAB4YOU - THANK YOU FOR THE HOTFIX.
I was getting killed with Active Directory problems in an Exchange 2003 environment. I applied this hotfix to both of my ADS GC servers and my problems have dissapeared.

One NOTE of caution...this hotfix made our exchange 2003 sp1 on windows 2003 standard sp1 HANG!!! However as I said above...the hotfix solved our GC problems.

Thanks SAB4YOU.

 

[JustinRobbins]

SAB4YOU thank you also for the hotfix

This appears to have fixed the networking issues but our exchange server has "hung" once since the application of the hotfix.

I later did a eseutil /defrag on the database, which cleaned up 10 gigs out of it
and we haven't had a problem since.

thanks for your help,

Justin

 

[silentsam33]

Justin/SAB4YOU,
What exactly do you mean by 'hung' in reference to the application of the Hotfix? FYI....since I removed Windows 2003 SP1, my server has not experienced the same issue.

 

[PeterRD]

I have had the same problems which I have been working with Microsoft to resolve.

We looked at two issues. The first was DCOM calls which other people have refered to in this thread but the patch did not work for me.

The real cause was a problem with RPC calls over the WAN via VPN links. This was traced to SP1 code being written in an attempt to improve security. Basically it refuses to work properly with MTU sizes below about 576 bytes. The result is packets being lost which causes RD and mapping and Outlook problems connecting to exchange server.

If your VPN uses a small MTU size of say 512 bytes then this is the cause of your problems.

You can check this by using the ping <remote server> -l 512 -f command

If the result is a fragmented packet with DF set then this is almost certainly the cause of the problem

To fix this you need to contact Microsoft to get a fix for the dll file rpcrt4.dll and make two registry changes. This will reduce the size of your MTU which will slow your network down. I had to reduce mine to 484 bytes. I use Watchguard Firebox II firewalls which has a maximum VPN MTU of 512.

The only real fix it seems is to replace your firewalls with a VPN MTU size that is large enough to work properly with SP1.

Quite an expensive upgrade.

You are lucky though. This has cost me well over 130 man hours to fix and I lost my exchange server when I tried to roll SP1 back in an attempt to recover.

In the end I had to rebuild from scratch and do a disater recovery which worked OK.

I did not want to restore from backup because I need to be secure and SP1 seems the only way to achieve this.

Our network is now all at SP1 level and working OK but slowly until I get new firewalls in place so I can increase the MTU again.

Perhaps I should buy some stock in some firewall companies...

Good luck