Working with large files causes system to become unstable 1

[lab99]

I am running Win 2000 Pro using an Asus motherboard with an Intel P4 2 GHZ processor, 640 Mb RAM. The problem is this, when I am working with large files or a large number of files the system becomes unstable. Example, I copied 17 GB of files across the network to a 2000 Server for backup purposes. The copy process took over an hour (87 minutes on a 100 Mb network?). Once done when I tried to open applications that normally load in a few seconds it took 30 to 60 seconds for them to load and they were very slugish or didn't work at all.

I do a lot of video work, a project I completed this week, once rendered created an 11.8 GB AVI file in 1 hour and 20 minutes. When completed the above problems occured. I clipped, edited and converted a 149 Mb MPG file to DVD format last evening and this morning and the system is still not acting right.

When these problems occur I restart the machine. Half the time the system goes back to normal, the other half it doesn't. Sometimes it takes several hours or several reboots to get the system working properly.

On an average I have 350 to 400 Mb of free ram available. When working with a large number of, or large files, once what ever I am doing with them is coplete I have 550+ Mb available. Having that much more when done than when I started tells me that something is forcing needed programs out of RAM. Rebooting, as I said sometimes solves the problem sometimes not.

I am running MemTurbo to monitor memory usage. When I scrub the RAM with MemTurbo available memory goes up to 500+ Mb and the system becomes unstable as stated above. This is what has lead me to the assumption that needed items are being forced out of RAM. After a RAM scrub a reboot solves the problem every time though.

I don't have a clue as to what to do to solve this problem or if it is just a bug in Windows as I had simular problems under 98, any ideas?

 

[InfrArch]

hmmm... did you tryed to monitor sawp file usage?
Try to enlarge it twicely + put it on another faster local disk (RAID5 preferred).
Monitor HDD read/write performance also. It can be a bottleneck in your case.




Victor K
psas@canada.com
MCSE+I;MCSA;MCSE(w2k);CNE(5.1);CNE(6);CIWSP;CIWSA;Net+;CCNA

 

[lab99]

The swap file is set to 1.28 Gb, 2X my actural RAM . I only have one hard drive in the system, an 80 GB 7200 RPM ATA WD drive so I can't relocate the swap file. I am going to add another HD as the 50 GB free space I have now isn't anywhere close to enough when dealing with uncompressed AVI files.

The swap file usage when rendering usually runs around 50 to 60%, never seen it go much above that.

The TSR's I am running are... CPUIdle, Motherboard Monitor, MemTurbo, Zone Alarm, NAV & System Doctor and the ATI and Adaptec stuff. I tried setting up another user with only the ATI & Adaptec drivers loading, which pretty much have to load. The results were the same.

 

[InfrArch]

I would recommend you to monitor the 3 critical counters directly at the time you 'r working with the large files.
Monitor :
1. RAM read/write
2. hdd read/write/queue lenght
3. CPU usage.
4. swap file usage

By getting this statistics you can poit out where the problem could arise by analyzing this data in complex.
Post the results.





Victor K
psas@canada.com
MCSE+I;MCSA;MCSE(w2k);CNE(5.1);CNE(6);CIWSP;CIWSA;Net+;CCNA

 

[leo8888]

How do you monitor these items in 2000? I'm used to 9x system monitor.
Thanks.
Dan

 

[InfrArch]

Hi,

Just click Start/Run and type perfmon
That's all.
No need to use 9x.




Victor K
psas@canada.com
MCSE+I;MCSA;MCSE(w2k);CNE(5.1);CNE(6);CIWSP;CIWSA;Net+;CCNA

 

[NickFerrar]

Are you running an anti-virus scanner? IME they don't seem to cope very well during large file transfers and can often crash or get stuck using a lot of CPU time. I disable mine before working with large files that I know have already been scanned.

 

[leo8888]

Victor,
Thanks, thats exactly what I was looking for!
Dan

 

[lab99]

Took me awhile to get back to this thread but I did find NAV to be the problem. I disabled Auto Protect and copied large files and large amounts of files across the network and to CD without incident. I also found rendering video projects faster with NAV off. The rendering process creates several temp files sometimes larger than the final output file which NAV appearently was looking at as they were being created. Since these temp files are changing almost as quickly as they are being written NAV was going into a tail spin using a lot of RAM and processer resources to try and scan them. This is, at least I think it is, what caused the bottleneck and the system to become unstable, NAV is still trying to do something to files that in some cases don't even exist anymore. When the system became unstable I would have to reboot several times. After the second or third reboot I would receive errors saying, could not complete task, unable to open file xxxx or file inaccessable. Once I got this error the system stabilized.