Impersonation very slow on initial connection

[CrashDome]

Our company receives very large files from our customers and we decided to toss out FTP because it is a full-time job keeping track of whose file is whose or what are logins/passwords.

I had the bright idea to use a file upload control (from MediaChase) and attach files to a ticket system similar to a FedEx or UPS tracking number system.

It works great but in order for our webserver to store these very large files it needs to access our storage server on another domain. Since I do not want to allow full trust to the ASP.NET account I created a second account specifically for impersonation.

My code does all the work needed transfering to the webserver (temp file) and then impersonates just before finally copying all the files from the webserver to the storage server. It does it once (per ticket) and does not impersonate for each file.

My experience is that after being idle for awhile, a user who does an upload will get a 20-30 second delay while authenticating (which works great btw). After this, everything else is immediate... even subsequent tickets.
After another idle period, the first user that comes along gets the delay again.

Is it something I am missing? or is it normal?

ASP Page:
if (impersonateValidUser(UploadUser,UploadDomain,UploadPass))
{
for(int Index = 0; Index<Request.Files.Count; Index++)
{
McHttpPostedFile PostedFile = new McHttpPostedFile(Request.Files[Index]);

try
{
if(PostedFile.FileName!="")
{
//Write File
}
}
catch(Exception ex)
{
//Report Error
}

//Free Temp Files
try
{
if(PostedFile.InputStream is FileStream)
{

FileStream fs = PostedFile.InputStream as FileStream;
fs.Close();
File.Delete(fs.Name);
}
}
catch(Exception ex)
{
//Report Error
}
}

bool cEmailSent = sendCustomerEmail(tbEmail.Text,ticketNum,ticketID);

undoImpersonation();
}

Impersonation Code:
private bool impersonateValidUser(String userName, String domain, String password)
{
WindowsIdentity tempWindowsIdentity;
IntPtr token = IntPtr.Zero;
IntPtr tokenDuplicate = IntPtr.Zero;

if(RevertToSelf())
{
if(LogonUserA(userName, domain, password, LOGON32_LOGON_INTERACTIVE,
LOGON32_PROVIDER_DEFAULT, ref token) != 0)
{
if(DuplicateToken(token, 2, ref tokenDuplicate) != 0)
{
tempWindowsIdentity = new WindowsIdentity(tokenDuplicate);
impersonationContext = tempWindowsIdentity.Impersonate();
if (impersonationContext != null)
{
CloseHandle(token);
CloseHandle(tokenDuplicate);
return true;
}
}
}
}
if(token!= IntPtr.Zero)
CloseHandle(token);
if(tokenDuplicate!=IntPtr.Zero)
CloseHandle(tokenDuplicate);
return false;
}

private void undoImpersonation()
{
if (impersonationContext != null)
impersonationContext.Undo();
}