Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[benn600]

After converting our 300 DVDs, I couldn't be happier or sadder.  It is so amazing to be able to watch any DVD we own through Media Center and on my Xbox Media Center in our kitchen.  However, on my 2.5TB array, all these movies take 1.6TB and I obviously have a few hundred GB's of music, pictures, documents, profiles, podcasts, etc.

5 drives go to the active array and 5 go for my backup but I have an 11th which holds my most important data.

Yesterday, I got pretty sick with the flu and had a rough night (I should be sleeping now) but just now it occurred to me--what if I used all 10 drives?  That would be 5TB!  That would hold another 400 movies!  I definitely need more space for other projects like when I start scanning in our 30,000 analog pictures & slides.

The problem is this.  Since I have two arrays but want to combine them, I don't know if it's possible.  In order to rebuild the active array, I would have to delete it--erasing all the data--just as I would have to do for the backup array.  Therefore, the only way to create this 10 drive RAID0 array (which sounds scary--one drive failure!) would be to literally delete all my data except the data on that 11th drive.  The 11th drive pretty much has everything except the video and podcasts--those of which could be backed up to another computer--but I don't happen to have an extra 1.6TB laying around to temporarily store the data until the new array is created.

How would a big business handle a situation like this?  I know they would never sacrifice a backup and I doubt I would ever pursue this but it just occurred to me how horrible the situation is.  I wonder if there is a way in Windows to tell it to add 5 drives to the array and then give it about a week to rearrange all the bits--that sounds very risky.  10 drives sounds just as risky.

Then I thought maybe 10 in RAID5.  Again I wasn't a huge fan of RAID5 software-wise.  It slows everything down and if I'm not extra careful when I unhook the array (which I probably would never have to do--but just restarting) it can prompt a regenerate which takes 24 hours for 5 drives.  I always feel like the filesystem or files got messed up when it prompts a regenerate so when that has happened, I usually just deleted the active array and restored from my backup array--that's around 3 solid days of copying--and 3 solid days of having all the drives inside the house and not offsite.

Getting sick is no fun.  I do not recommend hunting down sick people to catch their sickness.  I hope my RAID0 array's don't catch my flu or whatever I have.

[johnnyboy]

In real life terms, a big business would just buy additional storage.
They'd buy a big storage server with however many TB on it and then just add that to the available network storage and leave the old one or if they wanted to decommission the old one, they would simply copy the files across and then dump it.

A storage farm would never be added to the network unless it was properly setup and configured.

If for some random reason (just to give you an answer) they ever had some storage that was being used that they wanted to re-use in a different way, they'd simply use a tape library to backup all the data onto a few tapes, then re-configure it and then copy the data back from the tapes.

What you have to realise is that for a big business it's all about long term bottom dollar and when they're paying their staff $200 a day, 1.6Tb of data is going to be several staff's work over several weeks. Even if it was just 5 staff members over 5 days (as a complete low example) thats still 25 man days at $200 a day is $5000.

Home users are never in the same boat as big businesses for this very reason.

If you're going to try put 10 drives into a software RAID 1 you are, without fail, long term going to pay the price for it.

If you want to have all 10 drives then just use them as 10 seperate drives or do RAID 1 over two drives so you end up with '5 double sized drives' - at least that way you can only loose 1/5th of your data.


Going Raid 5 is definitely the way to go with this huge a collection of drives with this much data. Software RAID 5 with this much data isn't worth it.
To fill these drives is going to take you days / weeks worth of your own personal free time and energy. Compare this to a cheaper low end card and its not worth it.


I'd personally say just use all 10 drives as stand alone drives. I know for a fact that XBMC can easily map to 10 seperate drive shares with no hassles so it wouldn't cause any problems with that.
In windows XP you can mount a drive as a folder so could map all your drives as folders under one drive if you want them all to appear as one drive.

If you do them as seperate drives as well it gives you the advantage that you can easily just break down one array, copy the files over to each of the single drives then break down the other array - no worries about where to put the files inbetween.

[benn600]

I would like using the drive independently because it would be a lot simpler to move data around but I always have issues with my folder structure.  Right now, it is as perfect as can be.  Everything just works so nicely.  I fear that if they were all different drives, everything would go crazy.  What I really want is a program that creates a virtual hard drive which would allow a user to then add folders from other drives.  So, if you request a folder from the fake drive, it would check all 10 drives for the contents of that folder and ADD them together.  If you request DVD Movies from the fake drive, it would go to all 10 drives looking for the DVD Movies folder.  If it finds that folder, it would take the contents -- its folders -- and combine them with the contents from all the other drives.

I don't know what to do.  I still have about 100 GB free but that really isn't anything (unless you're on a 120 GB drive).  100 GB out of 2,500 GB is quite a drop.  It will be finished mirroring (copying) to my 2nd backup array in 10 hours from now.  At that point, I'll have to just think about what to do next.  If I have all 10 drives in use, then I can have no offsite backup.  It really limits my complete backup ability--where as now I can actually keep all my movies backed up and offsite.

[johnnyboy]

well like I said - if you just put them all as stand alone drives you get the best in terms of stability and ease of use from a stability point of view.

Your only issue then is that you want them all showing as one drive so you can easily see all files together type thing.

with XBMC this isn't an issue as it would instantly show the contents of as many folders as you set it to show you when you add a new location.

From your desktop however its not quite as simple.
Best way to manage them then is just to be using some software which shows them all together regardless of where they are physically saved (ie MC).

Between the two you'd never be dealing with the real physical location of the items, more just with which item it is you want.


If one hard drive goes, you loose 500Gb - not quite the same as loosing 2500Gb!

[benn600]

Well, right now, I have 2 identical array so if one goes, I lose nothing...but, I could use 5 independent drives and in that case, there is less chance for losing data.

The issue now is that I have everything backed up.  If I remove the backup for my less important data, I could use some of those extra 5 drives for more data & movies.  It's complicated, though, because then I obviously lose my backup!  I can't really afford to buy another 4 drives (2 for each array) to gain an extra TB.  I've already spent too much.

[johnnyboy]

You dont need a backup of your ripped DVD's.
You already have a backup - the DVD's themselves. Chances of a drive going aren't that high and you usually get indicators before it finally goes so you could back up everything onto a new drive.
Worst case scenario though and you loose the drive then you just have to rip them again and by then there'll probably be newer and better codec's out anyway so you can use one of them to get higher quality and smaller file sizes.

With music and stuff that involves a lot more tagging etc once it's ripped then just back that up.
It's the DVD's that are gobbling up the huge majority of the space.

[benn600]

Good point