5000 computers all connected to 1 server share using vb app

[rmoyes02]

Hi, I have created a vb application, and I have 5,000 users using this application all at the same time... Right now I have this application using:

if fso.fileexists(app.path & "\test.txt") then[/color blue] 'etc[/color green]

The application has a timer which runs every 30 seconds, and checks for this file using fso.. Well as you can imagine, it bogs down the server. Normally the files test.txt is not there.. Its only there about 3 times a day.

Is there a better way to check for the existance of this file? Can I have 5,000 computers monitoring the share for a newly created file called test.txt, or will this kill the server?

Thanks

 

[ousoonerjoe]

By chance does your application connect to a database?

"If I were to wake up with my head sewn to the carpet, I wouldn't be more surprised than I am right now.

 

[rmoyes02]

No, no database connections, just 1 call to fso.fileexists in a timer that runs every 30 seconds.

Thanks

 

[ousoonerjoe]

why does the timer need to hit every 30 seconds? can you not space that timer out to about 15 minutes? That alone will relieve a lot of stress from the server.

"If I were to wake up with my head sewn to the carpet, I wouldn't be more surprised than I am right now.

 

[dilettante]

Improving local performance is as simple as caching the constant filename and calling GetAttr() instead of using the FSO.

However it's clear you're doing something completely funky here. If App.Path points to a location in a remote file share you're using VB in an inappropriate manner. VB programs are meant to be deployed directly onto the client machines, for a number of reasons.

But let's set that aside.

Your real problem is the high cost of file polling across a LAN.

There are two obvious approaches to reducing this overhead. Both require some service that will do this polling for file presence and pass a lightweight notification to your 5000 clients.

One way is to have the service broadcast an alert to each client. If we were not limiting ourselves to VB6 this could also be done via multicasting. However in VB6 there are two simple ways to approach broadcasting: UDP datagrams and second-class Mailslots.

The other way is to have the clients connect to the notification service via TCP or Named Pipes. Named Pipes are not cleanly supported in VB6, and unlike Mailslots require bit more API programming.

The downside of broadcasting is that the notifications will not be forwarded by most routers. Thus if you LAN is internally routed this will probably not be an option. There is also no delivery guarantee, so it can make sense to send each alert broadcast several times with a short delay in between. The broadcast message should contain a timestamp of the event time, and clients can track the last alart timestamp to determine whether any received alert is new or a duplicate.

The downside of 5000 TCP connections is... 5000 TCP connections. This is a fairly heavy load for a VB6-based service but may well be viable with such a light traffic load. However the server hosting such a service may require some TCP parameter tweaking if the clients tend to come up and all connect within a short interval. This is a sort of "manycasting" option - you'll have to send the alert 5000 times, once to each client.

Another alternative might be a DCOM server to monitor for the file and send callback or event alerts.


The required service is fairly trivial, as is the required client logic. Any of these options is quite a bit more code than your current file test however.

There are probably many more parameters to your problem that would determine whether any of the choices above is practical or even possible. For one thing you'd need a server where the alerter service can run. Another question is "Who removes this file?" If a client is expected to do so, a simple alerter service may not be your answer.


If the only purpose of the file is to permit one client to notify all others you might eliminate the file altogether. In such a case one client might broadcast to all others, or the client might forward an alert to an alerter service that simply relays the alert to all connected clients.

 

[rmoyes02]

Thanks for your replies, I am a novice. Would something like this work any better than what I'm currently doing?

http://www.devguru.com/features/tutorials/watchfolderact/watchfolder.asp

Thanks

 

[strongm]

That's for VB.NET. Is that what you are using? If so, you probably want to visit forum796.

If not, and you are actually using VB6 (or even 5) then I was considering suggesting a folder watching solution (as described in that article as 'not very simple and required lots of coding'; just do a keyword search in this forum for FileWatch and you should find at least on of my examples of this) - however I hate to think what would happen if NTFS tried to send an IOCompletion notification to 5000 event monitors the same time, so I initially decided against it posting.

 

[rmoyes02]

I'm using vb6, but would consider trying .net there was an easier solution using .net. So using a FileWatch, do you think it would be worse than using fso every 30 seconds for 5000 clients?

 

[strongm]

The 'advantage' of the 30 second solution is that the queries would likely be spread out so the system might only have to be handling 167 requests a second which, if you use a more lightweight request than FSO might not be too bad. But with the file watching solution all 5000 clients would be serviced at the same time. I don't know what that would do to system performance.

 

[rmoyes02]

Ok, so as far as alternatives to fso, dilettante mentioned GetAttr()... Is that an alternative? Are there other lightweight alternatives?

Thanks

 

[three57m]

Not sure how viable this solution would be because i have no way to test in a situation similar to yours but if this helps so be it.

'This tests for file every 1 millisecond so I would think every
'30 seconds would be just fine

'TO TEST Add a command button and a timer control
Private Declare Function GetInputState Lib "USER32" () As Long


Public Function DoesFileExist(strFile As String) As Boolean
    Dim lsize As Long
    On Error Resume Next
    If GetInputState = 0 Then
        lsize = -1
        lsize = FileLen(strFile)
        If lsize >= 0 Then: DoesFileExist = True
    DoEvents
    Else
    End If
End Function

Private Sub Command1_Click()
    If DoesFileExist("C:\test.txt") = True Then: Exit Sub
    'create file
    Open "C:\test.txt" For Output As #1
    Close #1
End Sub

Private Sub Form_Load()
    Timer1.Interval = 1
End Sub

Private Sub Timer1_Timer()
    If DoesFileExist("C:\test.txt") = True Then
        MsgBox "FileExists"
        Kill "C:\test.txt"
    End If
End Sub

 

[ousoonerjoe]

Is there not a way to push the file locally to the users and check the local directory? Or if you could use a database, set the flag there as whether or not the file is present and then the clients could go get the file as needed. However, this could cause a "traffic jam" with 5000 users hitting the same file at the same time.

If you could upload the file into a database, most of these issues would clear up. Database servers are built for this kind of traffic. Also, you would be updating one location as opposed to 5000.

How big and what type of a file are we talking about here, typically?

"If I were to wake up with my head sewn to the carpet, I wouldn't be more surprised than I am right now.

 

[jadams0173]

I'm sure there is a reason it hasn't been mentioned yet, but what about Dir function? Wouldn't it be lighter than FSO?

 

[dilettante]

GetAttr() won't help much here since your main problem isn't FSO methods, it's network traffic and server load. This would be a pointless micro-optimization for a check each 30 seconds, but it might be a better way to get used to as a way of verifying file presence in general. In this case it's hardly worth the savings gained to spend the effort changing from FSO.FileExists() in already-written code though.

FSO, Dir$(), etc. all have the disadvantage of high overhead.

I suggested some form of alerter service because only a single process needs to poll for the file (or subscribe to a notification) and only light traffic on the wire need occur, and even then only when the file "appears." In theory the alerter could run on the file server itself to minimize network traffic.