Topic: Switching to Raid 0? (Read 7246 times)

benn600 · « **on:** October 03, 2006, 07:20:30 pm »

I don't get it! I had no problems with my RAID 5 array for about 2 weeks...then it kept giving me regenerating errors and saying drives were failing. I then tried remaking the array, etc...but regenerating a RAID5 array takes around 36 hours. I did it twice and both times didn't work. Therefore, I am thinking of switching to a plain old RAID 0. Since I keep it all backed up on another RAID 0 array, that seems somewhat safe. I have never had a drive truly fail (hardware wise) other than one time a drive was making noise and I switched it out--didn't really matter because it was the boot drive so it had no important data on it. And I never actually saw it fail, it worked fine but made strange noises (lol--as long as I kept using it).

Does that sound like an OKAY plan? It also slows everything way down when I have to use RAID 5. If I was willing to put $300 more into the project, sure, I could do RAID 5 right...but I'd rather invest that money in more drives since I desperately need at least one more drive.

Basically, I can't handle a drive failure in both arrays at the same time.

jgreen · « **Reply #1 on:** October 03, 2006, 08:13:16 pm »

www.3ware.com

If that's too pricey for you (cheaper than Adaptec), go pick up a Promise card, a no-name company with superb products. I've seen Promise RAID cards for around $50.

What you're describing is a RAID 0+1. It should be easier on your overhead, but you still have a gaping hole between the "save" command and the destination disk. Even a minor screwup in the PC will cause a data loss, AFAICT. A RAID card will be bulletproof the moment you hit "save".

benn600 · « **Reply #2 on:** October 03, 2006, 09:13:34 pm »

I don't know if you can use Windows XP for a RAID 0+1. I am currently using only RAID 0 for my main array and my backup array...I'm actually in the process of copying 1,000 GB from my backup array to the newly created array. By the way, I've modified XP to allow RAID 5 and mirrored volumes..but I don't know how I can enable a stripped plus mirrored volume. Besides, I had 2.0TB of RAID5 space but now I have 2.5TB of stripped space. If I switched to Raid 0+1 I would only have 1TB or 1.5 if I bought another drive. Neither of those are options because I am about 30 GB over the 1TB limit and still have a lot more data that I am creating from ripping new CDs (I am currently buying CDs at a rate of about 10-20 per week), scanning our 20,000 old photos and slides, ripping more DVDs, etc. I basically don't mind having 2.5TB!

But I can't use that all because I only have 1TB of backup space so I'm desperately waiting for the WD My Book's to go on sale for under $200 again so I can upgrade. I realized my backup array doesn't need much more than what I have but the main array can have a lot more because it means I don't have to re-create the entire array when I upgrade the space.

johnnyboy · « **Reply #3 on:** October 03, 2006, 09:24:13 pm »

benn - I'm aching to say it, but I cant

lol.
That would just be mean!

I said it at the beginning and I'm just gonna say it again - I think you're trying to way over think and overdo the whole thing.

1. Why does the backup need to be raided? What's wrong with just using them as seperate drives? Dont say that being limited to 300Gb transfers per time is too big a hassle - even on a blazing machine you're still talking about a large transfer time just for that.

If you have it all backed up to a seperate drive just either do Raid properly or scrap it all together and just do them as seperate drives - you dont loose anything.

1 300Gb drive in your system goes to 1 300Gb backup drive - easy peasy!

You're saying you cant afford to do it properly, yet you can afford to fork out all this cash to buy the drives to store who knows what?

What format are you ripping all your stuff to? Have you considered just ripping it to something smaller, especially at the rate you're increasing your collection - you're going to be expanding ridiculasly fast.

Also, for something that big a project, I'd say you're just shooting yourself in the foot. Spending all the time and money and hassle to set all this up with hacks and work arounds.

HACKING xp to allow SOFTWARE RAID of USB Drives is just too many hacks together.
You're already seeing problems and you've barely got going.

For a project that big you really want a dedicated file server running Raid 5 using backup tapes and a backup job to safely back it all up.

You say it'll cost you more but it wont because all of a sudden you get twice the storage capacity as you aren't wasting half your drives on backup and you dont need to buy two 300Gb hard drives every time you want 300Gb more space.

newsposter · « **Reply #4 on:** October 03, 2006, 10:01:56 pm »

ah, told ya so, not just me, but about 6 other people..........

Promise cards are 'fakeraid', no better than anything else Benn has tried so far. Such cards don't do much more than add another layer of complexity ready to fail at first opportunity. If benn is going to insist on doing 'fakeraid' he should just hook up the drives to the mobo IDE ports and stop using USB with all of its performance problems as well as no error correction, arbitration, or anything else needed to support reliable large-scale disk.

The serverelements.com NAS distro is free for ghods sakes. The way Benn is trying to set things up he will still have risks but at least the serverelements (and naslite.com for $30- too) distros use a decent high-performance journaled filesystem that is easier to recover than NTFS.

Forget NTFS unless you have top of the line hardware and Server 2003.

Before putting any more physical work into the project Benn really needs to do a load of reading on raid levels as well as the capabilities of various NAS implementations, filesystems (XFS, JFS, etc) and realraid (hardware) vs fakeraid (cheap cards with simple x-or chips plus software drivers). Don't forget to read up on real disk interfaces like PATA, SATA, SAS, and SCSI. Firewire (IEEE 1394) as an intermediate interface is viable but all implementations of USB are NOT.

Only then will the grasshopper be able to grab the pebble from our hands.........

benn600 · « **Reply #5 on:** October 03, 2006, 10:08:16 pm »

I decided to switch back to RAID5.

I can't imagine taking the risk of RAID 0 but unfortunately, this brings me back to my complaints.

1. Why does Windows take 36 hours to regenerate a new RAID 5 array? I think this means it goes through and makes sure the parity is all correct--or maybe it sets up the drives, configuring where the parity will be stored for various areas of the drive?
2. When I'm writing to the RAID 5 array, it shows strange write speeds. It starts out at a healthy 4MB/second (over network) but then drops way down to about 1 MB/second. If I write to it while regenerating, it is 4MB/second in the beginning but then drops even lower than 1MB/second.

I'm going to dedicate a computer to being the RAID server here pretty soon.

Again, I'll remind everyone that NOTHING BAD HAS HAPPENED YET. I have lost NO data. I have wasted NO money. My current cost is around $1600 for 7 drives or about 3.5TB total space. What I was hoping for was really an answer to the questions I've listed above. Other than that, it will just take time. 36 hours until the array is regenerated and then I can copy the data over to it.

InflatableMouse · « **Reply #6 on:** October 05, 2006, 05:25:42 pm »

YOU got me cracking up. Your hassling with this crap is all too familiar. Been there, done that

So you got what? 3,5Tb in how many volumes? I sure hope more than one.

Tell you what. I work in a datacenter with hundreds of servers including clustered file servers with dual and tripple fibrechannel connections to enterprise level SAN solutions. These servers are mostly HP's, dual or quad opterons etc with 2 or 4gb of memory. We use 500gb LUN's MAX (easy explanation of a LUN = a physical disk in Windows). Depending on the usage (sessions, transactions etc), the 500Gb volumes are giving us huge performance issues. This has to do with file size, cluster size, volume defragmentation and a few other things. We are in the process of migrating some of these volumes to 250gb LUNs. Point is, there is SOO MUCH MORE to RAID than there is to an ordinary disk.... you have no idea what you are dealing with, trust me.

All the objections you hear here are valid. SATA or PATA is far from ideal for RAID solutions and I can only give one advice: STEER CLEAR FROM IT!

You want RAID-5 in a reliable and trouble free solution? Get yourself a Compaq ML-530 (or alike) from Ebay and motherload of hotpluggable 72gb disks (12 will fit in there). That "thing" will make more noise than your mother in law, but I understand its in the basement so no worries I would guess. Fit a gigabit ethernet adapter in there and get a good gigabit switch. You will pump 40-50mb per second to your server. The RAID adapter in there has its own accu so no worries in case of a power failure. You should use the ATA disks for backup of your server.

Apart from the occasional failed disk you won't have any worries mate. But as long as you keep on doing things you are doing now, you're gonna keep having issues. Believe me, I've been there too.

benn600 · « **Reply #7 on:** October 05, 2006, 05:57:40 pm »

Lol. If only I had another $3K to spend! Those 72GB 10K RPM drives can get expensive and 12 only gets to around 800GB. HA! I currently have 5 of the 7 disks on the RAID5 array, giving me 2TB. My backup array is a RAID0 (kept offline most of the time) with 2 drives = 1TB. I'm in the market for another drive because I'm just now surpassing the limit of my backup at around 1200GB, so one more drive should hold me for a while on the backup end.

The RAID5 volume finished renerating this morning at around 12:00pm and I started the copy process from the backup to the new array. It has been going smoothly for 6 hours and has copied close to 60GB, so it's copying at around 10GB/hour or 2.8MB/sec. That is about 2/3 of what I read a user with a perfect RAID5 setup was getting--dedicated case, SATA RAID cards, etc. It may have also been Windows RAID, but he did use SATA instead of my solution of using USB drives. As long as I don't run into the problems I had before, everything should go very well. The biggest disappointment is that this entire process won't even be done until Tuesday and then I have another 200GB I've compiled from ripping CDs/DVDs/downloading podcasts, etc that doesn't fit on the backup that I'll have to move to the array.

My problem last time was, I think, that my usb hub was failing. I was using an ultra portable 4 port hub. I have switched to a more robust hub and have a 7 port hub on the way so I can move all the drives to the single hub, leaving the server USB ports open.

I hope it works flawlessly--it did once before but I made the mistake of shutting the computer off HARD while it was writing, screwing it all up...soon I'll get it all on a big UPS and shouldn't have to worry about that!

johnnyboy · « **Reply #8 on:** October 05, 2006, 11:02:55 pm »

You do of course realise that USB has a maximum bandwidth so by putting all your drives onto one USB channel to your computer you are limiting the performance of them all even further?

I hope you please keep updating us as this goes along, especially when (if) it all goes awful! lol.
I can see you having a load of problems but thanks to how extreme you're being with it all, you shouldn't loose any data from any of it - just time and hassles.

The best thing about all this however is that it's reminded me to do a backup (I'm the far extreme of you with hassle - in my years of computing I've only ever had one hardware failure and I knew that one was coming (not including my USB drive packing up on me which lost a bunch of data for me

)).
I'm gonna backup my 16,000 photo's collected over the last 9yrs that I have on there - definitely something worth backing up

As for the rest, I'm thinking about investing in a tape drive if I can find one cheap enough second hand on ebay. Nice easy slow proven way of backing up all my data.
It'll do nice incremental backups for me and it's got commercial enterprise level software thats proven for reliability to do the jobs with.

Sounds solid to me

InflatableMouse · « **Reply #9 on:** October 06, 2006, 07:20:44 am »

Quote from: johnnyboy on October 05, 2006, 11:02:55 pm

...As for the rest, I'm thinking about investing in a tape drive if I can find one cheap enough second hand on ebay...

Tapes are unreliable, unless you get into the upper class SCSI drives like (s)DLT. Even then I would not solely rely on a tape backup. DLT Tapes are rediculously expensive (30-40 $$ for a 40/80 DLT) if you buy them new and 2nd hand, I wouldnt trust them. larger, professional bussinesses (read smarter

) use tapes for 1 year and then they are replaced with new ones. We had a customer for which I was the System Administrator for a few years. They had more than 60 departments with every department a server running local backups to DLT (40/80). When the tapes expired after a year we gave them a proposal to replace these tapes (21 tapes times 64 departments = 40,000 $$). They really didn't like the idea and signed a waiver to take the risk if restores would fail. They really believed these tapes would still be reliable after 1 year of daily and weekly use. So wrong. The first year wasnt too bad, but the 2nd and 3rd years were horrible. In the 3rd and 4th year they finally started replacing tapes, realizing the loss of data and the inability to restore was costing them so much more.

You will also run into other issues like which software you will use for tape backups and their pros and cons.

Here is what I do. I invested in a bunch of SATA drives and a piece of software from Acronis called True Image. I use version 9. Check out their website www.acronis.com if you are interested, it creates images from hard drives or logical volumes. I make incremental backups of my media disk to a dedicated backup disk. Next to this, I use quickpar to create 15% parity volumes for my lossless collection per album and backup my lossless media folder including these parity files to DVD5. DVDs are cheap but isnt very reliable either, but the 15% parity files will allow me to recover from bad media. This is still a hassle as bad dvd media can lock up your dvd drive or even your pc. In that case I use tools like IMGBURN to create an image to disk of the dvd and mount it with deamon tools. And thats the difference with tape. Bad tapes are very hard to recover from while this (in my opinion) is much more accessible than tapes. The format on the tape depends on the software you use and you are therefore limited to the software's recovery capabilities. These are often limited. DVD format is standardized and the internet is flooded with free, open source tools that will read bad media and recover from it. The quickpar files will help recover your files if they are damaged.

I realize these methods are a too savvy for most regular users but worth a thought IMO. It works good for me and I went through all the hassles Benn is seeing. A long time ago I tried tape backups as well quickly realized it was far from ideal for personal use.

newsposter · « **Reply #10 on:** October 06, 2006, 07:09:49 pm »

Benn, a USB hub does not do what you think it does........

benn600 · « **Reply #11 on:** October 07, 2006, 05:00:26 pm »

You guys are just a huge laugh! I cannot believe how deep we are getting into this discussion. It's partly my fault for creating new threads all the time but I have long since addressed the issues presents. First, I have no speed requirements at all. Second, my USB card has 6 ports on it. Are all 6 independent with their own bandwidth? This is hard to believe. I always thought at least two at a time were shared so it had 3 ports *2 = 6 available ports. I want them all on one hub because it will make it easier to keep track of. I can keep all the cables plugged into the hub and remove the hub if I need to.

It has been absolutely rock solid for the past 4 days (lol)! I'm up to about 500GB copied of a total of 1300GB. I desperately need another drive for my backup array which only holds 1TB (2 drives) because then I'll be able to backup everything.

Nothing bad has happened yet. I have lost data twice in my lifetime and it was always due to formatting errors--not physical drive problems. One time I was moving files and suddenly the computer restarted, ran a scan disk, and all my files were gone. Long story short, It happened to start copying the files I HAD backed up, leaving the unbacked up ones for later and I basically lost nothing that I know of today. I had my documents backed up on another drive but not the video (or other way around...) and it started moving the documents first...restarted, all those are gone. Video was next in line but the restart stopped the moving process. Recover tools I have didn't help me either.

newsposter · « **Reply #12 on:** October 08, 2006, 02:49:45 am »

Brushing the issues away with a wave of the hand is not quite the same as resolving them........

On the other hand, I'm on my third 1.5 Tb array. They take about 60 minutes to build from parts, load the OS, format the volumes, hang in the basement closet, and make the storage available on the network. The storage arrays back each other up on a minute by minute basis without intervention from me.

Never a data loss in 4 years which is roughly 400x your 'experience'. This is being done with zero 'drama' and wasted time hacking about with 'good ideas read off the net' and instead uses good practices and design.

But by all means, carry on. This thread is loads more entertaining than the endless arguments about flac vs mp3 vs mp4 vs ape..........

InflatableMouse · « **Reply #13 on:** October 08, 2006, 04:10:37 am »

Quote from: benn600 on October 07, 2006, 05:00:26 pm

You guys are just a huge laugh!

I'm glad we entertain you. I've been cracking up over your stories too

I'm glad its working for you and I hope it will stay that way. I gave my 2 cents about the whole deal and I've learned to leave it at that. It's just not worth trying to convince others to do otherwise. Respect for other people's choices makes better friends

benn600 · « **Reply #14 on:** October 08, 2006, 09:41:00 am »

I agree completely. Every day or more I pull up JRmediacenter.com, click on Interact, and then middle click on MC & Other Hardware. MC I quickly scan through and close. Then, whenever I see my RAID threads with new posts I just can't wait to read them. They really are so much better because there is a lot of drama and yelling across the keyboards. Maybe I am crazy? Perhaps I'm just plain nuts? But also perhaps I've found a solution that works for me? Maybe there are better ways to setup my array--believe me, I spent hours on eBay looking for good prices on drive cases, power supplies, sata cards/cables, etc. Then I found a way around spending hundreds more than I've already spent! My new array should be data complete tomorrow morning around 9am. At that point, I can sit back and relax, examining the good work I've done.

Unfortunately, I have a "new" "server" on the way (P4 2GHz) that will replace my P4 2.4GHz I was using temporarily. I must say that going from 600MHz to 2.4GHz REALLY made a difference. When I was writing to the array before, I could hardly even read a bit of it. Now, I can read it with only a small decrease in speed. But when I get the new computer, it will be interesting to see if I can get it moved to the new computer without running into problems of it not recognizing it--oh WAIT, I'm using software raid in Windows! There's no problem there! Hardware raid is the one where if you accidentally boot to a raid drive you could lose everything! Oh yea, and there are 100 million computers in the world that could read my array right now! I forgot how versatile my setup is! Oh yea, and every computer in the world (mostly) has usb! So with my 7 port hub and 5 drives attached, I could get my data from almost any computer in the world! My mistake!

By the way, flac is better than mp3.

jgreen · « **Reply #15 on:** October 08, 2006, 12:14:45 pm »

Quoting Benn600:

"I could get my data from almost any computer in the world! My mistake!"

I seem to recall from a previous post that you said you weren't running any kind of firewall on your internet-facing systems. So I guess that's a true statement, then.

benn600 · « **Reply #16 on:** October 08, 2006, 02:32:55 pm »

I mean I can plug my hub w/drives into any Windows 2000 server or modified Windows XP computer and get my data--not necessarily over the internet.

newsposter · « **Reply #17 on:** October 09, 2006, 05:51:51 pm »

You are grossly over estimating the 'portability' of NTFS volumes, in particular any kind of raid volume, and especially any raid volume created and used with your favorite XP raid hack.

Microsoft themselves say that not only is NTFS raid portability unsupported and that all data on any such volumes moved between systems is at risk for total loss.

johnnyboy · « **Reply #18 on:** October 09, 2006, 07:21:06 pm »

For the backup tapes - they might be expensive but I can get 10-20 brand new ones from work for nothing as they bought a bunch of new ones and we now have a new backup drive (200/400 instead of 40/80) and so all the 40/80 ones I can just take.

All it'll cost me is getting a drive and I'll have more than enough backup tapes

As for shouting over a keyboard - none of that going on here mate, just casually reading, learning and counting down the days till the inevitible happens again lol.

I dont trust windows to be reliable with a single hard drive running stand alone, what to speak of trusting it to maintain data accross multiple drives but thats just me

benn600 · « **Reply #19 on:** October 09, 2006, 09:42:07 pm »

Forget the error that occured before. It was equivilent to a sudden power outage tnat that corrupted part of the array. Nothing could have avoided that--except a UPS which I'll be adding shortly.

Aside from that, I feel that my setup is absolutely, positively rock solid and the ultimate solution combining both low cost, amazing reliability and compatibility, and great uptime in the event an error does occur. I am continually amazed to see how rock-solid my solution is! In theory it is a bad idea. In practice, it is a dream come true.

johnnyboy · « **Reply #20 on:** October 09, 2006, 11:04:13 pm »

lol - that is then what we call in the business 'ReSuLt!!'

benn600 · « **Reply #21 on:** October 10, 2006, 11:31:14 am »

rock solid and flawless

INTERACT FORUM

Author Topic: Switching to Raid 0? (Read 7246 times)