Faster File Management on Networks?

[ajharn]

My users deal with thousands of documents and move them around frequently as part of a workflow process. They are often dealing with large directories with as many as 30,000 files in them.

Windows Explorer/My Computer are terrible at dealing with directories this big - especially over the network!

I am trying to find professional, commercial grade file management software that can handle sorting, viewing and working with files in large directories, over the network w/o choking. Speed in loading directories and the ability to search/filter files based on filenames are the most important features to me.

I would really appreciate everyone's recommendations. I am not having much luck finding this product on my own.

Thanks,
AJ Harn

 

[bcastner]

Explain what you mean by dirctories. If you have a Folder with 30,000 files in it, you need to rethink how you manage files. If you have 30,000 files on a volume this is not an issue.

Explain what server OS, the server OS level, and service pack level you are using. In general, there should not be an issue with Windows 2003 at Service Pack 1 talking to clients that are Windows XP at Service Pack 2. There are substantial enhancements in the way in which opportunistic locking, SMB signing issues, and the client redirector speed with this combination.

There are so many possible issues involved: from your physical network layer, your cable plant, to the adequacy of your path support and driver upgrade support involved; your server hardware and client hardware; your server settings and your client settings; that I doubt an adequate question can be obtained from a newsgroup posting.

Without knowing, in addition, any more details on the workflow, it becomes impossible to give you a reasonable answer.

I can assure you that the Windows file server system can hold its own with any OS alternative you can think of. The "...can handle sorting, viewing and working with files in large directories, over the network w/o choking. Speed in loading directories and the ability to search/filter files based on filenames are the most important features to me." are going to be improved substantially with WinFS in about a year. But for the present: get every aspect of the LAN or WAN right with existing OS products.

 

[ajharn]

Thanks for your reply. Sorry for slow response. Been a busy work week.

Alright...I left the network underpinnings and directory structures out of the story because that is all in transition. I am in the middle of a thorough network upgrade. So lets focus on functionality, which is really what I need most. Lets assume performance will sort itself out.

Windows Explorer has very limited functionality and I'm looking for something that has some richer file management capabilites. The users move a lot of files from one location to another after analyzing them (ananlyzing may be as simple as looking at the file name or may involve viewing or modifying the file contents). Primarily, they need to be able to move files from a small number of possible source directories (one primary source) to one of several possible destination directories(lets say 6 or 7). I'm looking for something configurable where I can tweak the interface to be intuitive for the users. Also, something with strong filtering capability, a particularly weak point in explorer.

Bottom Line: Configurable interface. Filtering tools (e.g. input wildcard criteria). Intuitive and easy to train for end users.

Something like the old Norton Commander, etc.

Any thoughts or experiences appreciated.

Thanks,
AJ Harn

F.Y.I. - The network this will end up on will be Windows Server 2003 with XP Pro clients working on files stored on a dedicated file server. Small workgroup w/ < 10 users.

And yes...the directories will have < 30K files in them.

 

[bcastner]

http://www.altap.cz/salam_en/norton_commander.html

http://winnc.net/en/?page=25

 

[ajharn]

Thanks.

I'm familiar w/ both of these.
Any experience with them yourself?

I'm primarily looking here for someone to recommend something they have used and liked.

I'm interested to see if there is a product in this marketplace targeted for business use that might include some more advanced features like full text search (w/ indexing).

 

[ajharn]

A Note about the performance issue...

Here is an example of a performance test on the same PC searching the same files on a network drive.

Directory used for this test contains ~3600 files. Test is to load the directory and apply a wildcard filter (e.g. xxy*.doc) so that only files matching the filter are returned in the file list.

Using the DOS based Word Perfect 6.1 file manager this operation takes approximately 1 second.

The latest Servant Salamander build takes 13 seconds to accomplish this same task.

Same network, same PC. Why is the DOS prog so much faster? Isn't the VM just translating calls from the DOS prog out to the Windows shell?

Please educate me. I don't get this stuff.

 

[bcastner]

Icons, thumbnails and extended attributes.

 

[ajharn]

So there's now way around it then? It's just slow and will be until WinFS? (Assuming of course, that WinFS will be faster in this sense).

Last I heard, WinFS was pulled from Longhorn. You said you think we may see it next year, though?