How could I create / or fake merge directories

[jamesdodd]

I have a series of files spanned over several HDD's in the same server.

I really need to consolidate these HDD's into one.

These drives have been purchased over the last few years as needs have expanded.

We're presently using junction tool to create symbolic links
but having this like dated folders is a bit of a hassle.

If it was possible I beleive the best way would be some sort of fake directory through MYSQL which would appear as a folder somewhere on the drive.
Each folder would be an entry in the mysql "folders" table and each file would be an entry in the mysqk "files" table in the corresponding category.

I can't find anything that can do this nor anything such as merging the drives in some sort of fake/faux partition, just so it would seem as if the contents of all the drives would be as one folder and subfolders or even a new drive.


If anyone has any ideas of how this could be done I would be extremely grateful.

I'm not afraid of a bit of coding, and I'm fine with mysql php, java and the occasional C++

 

[drew1701d]

Have you considered using DFS? You could change links behind the scenes while the DFS path would remain the same to your end users or MySQL app

"I'm certifiable, not certified. It just means my answers are from experience...not a book

 

[LWComputingMVP]

Have you looked into mounting the drives as folders (junctions)?

http://support.microsoft.com/kb/205524

http://en.wikipedia.org/wiki/NTFS_junction_point

 

[jamesdodd]

lwcomputing:
as i mentioned in my post I'm presently using juntions aka symbolic links

the problem with these is that although I can mount drives as folders there is no way that I can merge it so that 2 drives could be mounted into the same folder...

drew1701d:
I think this applies to DFS to, but I will look into it...
although at the moment we don't have R2 of 2k3, so this might mess things up a little... but heck if it works who cares!


The reason I mentioned MySQL was just to see if anyone knew a way to derive a folder from mysql information???
It doesn't sound likely and googles getting harder to use day by day with all that bloat

thanks

 

[LWComputingMVP]

I skimmed your question and seemed to read everything except that part.

Question... this is a server, right? And you're not using RAID? This seems horribly unwise. I would recommend at this point purchasing a RAID controller and probably EXTERNAL storage box and some hard drives and create a nice RAID volume to protect against single disk failure (using RAID 5). This would provide enhanced reliability and give you the opportunity to merge all disks into one logical one on the RAID 5.

This doesn't have to be expensive. You can get a 4-disk SATA enclosure that runs an eSATA link to the server and an eSATA capable RAID card that supports RAID 5. Total Cost, before Hard Drives, $250-$350 (includes enclosure, cable, RAID card). Then get 4x500 GB SATA drives and you've got 1.5 TB of usable, RAID 5 space.

 

[jamesdodd]

lwcomputing:

Were covered for backups etc, and the data isn't in a situation where accessability is required on a daily basis, were talking huge archives that might be needed.

It just doesn't seem to justifiable to spend £1000 on new HDD's and raid especially considering that we have used over 1TB of storage at the moment and this is all collected within the last year.

I need a software solution to this...

I think linux would be more likely to be able to offer me a solution but since I'm in windows at the moment I though it would be worth a shot.

thanks again

 

[yencoj]

If I were you, I'd get a SAN --anything approaching a TeraByte should be on a robust disk subsystem that you can expand volume size as you add disk so your data could be on a single logical drive. (I'd say the same for Linux systems too)

Based on your orginal question, DFS will not consolidate the data to a single drive; it will give the impression to the users that it on a single volume.

What you say about lack of Raid 5 and relying on backups is frightening. Disks will fail and you WILL loose your 1.o TeraByte of data. Restoring that much data to a Windows server would take hours from tape provided your tapes are within easy grasp and fairly "recent" that is close to the point of data loss. Plus the time it would take to replace the failed drive. You are taking a big risk and putting your staff in a tough situation. Whereas with Raid 5 you can loose a disk and your users would never be inconvenienced. If you can't justify £1000 for this... I'd quit.

 

[jamesdodd]

like I said it isn't a risk

the data isn't that important... we have other systems with integral data on it and using a raid 5 system for this isn't justifiable

I'm talking archives... stuff that exists on disks and tapes and stuff and is rarely used at all... its an extra to have it on disk to be honest

 

[markdmac]

I agree with the advice given above. Get yourself an inexpensive SATA RAID system and move the date to that to consolidate it all in one location. You could also stripe the disks if you wanted to avoid loosing space for the parity checks, but I'd stick with the RAID 5 just to make your life easier. RAID disks are super cheap right now so this need not cost you a lot.

I hope you find this post helpful.

Regards,

Mark

Check out my scripting solutions at

http://www.thespidersparlor.com/vbscript

 

[jamesdodd]

really can't be spending money on more disks and controllers... Its not the sort of project we can just chuck money at as I keep saying

a cheap raid controller card would end up being software and be pretty pointless...

I think I'm going to look towards linux anyway (we've moved most of our servers across in the last year leaving only a couple (this being one of them)).


Thanks for your help anyway

 

[markdmac]

I recently purchased a PCI SATA raid controller for around $65.00.

I get a lot of mixed information here. it is not important, yet it is bugging the heck out of you.

I hope you find this post helpful.

Regards,

Mark

Check out my scripting solutions at

http://www.thespidersparlor.com/vbscript

 

[jamesdodd]

I never said it was "bugging the heck out of me"

Its a slight annoyance that if I can resolve I will, but I'm not willing to spend loads of money...

The PCI SATA raid controller will more than likely be a software raid solution for the price its at.
Similar to something like NvRaid which isn't hardware raid...

I'll look into the scripts, I think the VBScript might be the way to go

I'll let you know how I get on


Cheers