Topic: Terrabyte Raid Server for about $5000 (Read 4928 times)

JimH · « **on:** October 03, 2002, 10:25:40 am »

Just found an interesting article on a big RAID server:

http://staff.sdsc.edu/its/terafile/

Quisp · « **Reply #1 on:** October 03, 2002, 11:34:07 am »

Does it do real-time crossfades?

KingSparta · « **Reply #2 on:** October 03, 2002, 11:50:16 am »

I Think I Will Pick Up 10 Or So Today

Robert Taylor · « **Reply #3 on:** October 03, 2002, 01:40:04 pm »

Speaking of BIG RAID, how about a mention of SMALL RAID?

I'm sure a lot of people have a lot of MP3s on their PCs.

Do many of them stop to consider what would (will) happen when the disk with their precious cargo dies?

There are an increasing number of motherboards out there with a simple RAID controller built in.

I have an MSI KT3-Ultra2. This has a Promise RAID controller on board, and lets me striped or mirrored RAID on TWO hard drives (apart from the standard four drive IDE controllers on board).

I am running this with 2 X 80GB drives mirrored, so if one drive dies, my machine keeps humming until I replace the failed drive, which when replaced is automatically re-mirrored. This means, obviously, that I lose half of my purchased capacity, but with drives so cheap, who cares?

The RAID controller only supports 2 drives in a RAID configuration, but a further 2 can be added as "normal" IDE drives.

It's well worth investigating if you fear for the safety of your precious MP3s (like me).

FYI

sekim · « **Reply #4 on:** October 03, 2002, 03:36:11 pm »

Hook four of these up to RAID card and get 1.28 terabytes.

http://www.maxtor.com/en/products/ata/enterprise_applications/maxline_ii/index.htm

JimH · « **Reply #5 on:** October 03, 2002, 03:54:27 pm »

So, let's see if we can get the hardware gurus to step in and make corrections.

How RAID works. First you find someone who knows what it is, and then.... oh, uh....

Here's what I know. There are several levels of RAID, starting at 0 and going to at least 5. Each does something different. The pieces I know about are these:

Mirroring -- Everything is written to two drives at the same time. If a drive fails, the other one can be used to make a new copy on a new drive.

Striping -- Alternate tracks are written on alternate drives because it is twice as fast. Track 0 goes on drive A, track 1 goes on drive B, track 2 goes on drive A, and so on.

Mirroring and striping -- I know what it means but I can't explain how it works.

But the important aspect (to me) of RAID is that it uses a "parity drive" to be able to magically increase the storage. For example, three 100 GB drives can be used to get 200 GB of storage that can be rebuilt if any one of the three drives fails. I think that the way this must work is by comparing every byte on the first two drives and writing a 0 or a 1 on the third in order to remember the state so either drive can be rebuilt.

The original post I made is a little different from solutions that provide RAID for a single PC. It provides a RAID server that any machine on the network can use.

OK, Hardware People, it's your chance to have fun with me.

skidoo · « **Reply #6 on:** October 03, 2002, 04:27:36 pm »

Quote

I'm sure a lot of people have a lot of MP3s on their PCs.

Here's what I do. I have a directory structure on my file server that looks like this:

\media
\media\stage
\media\clean
\media\library
\media\library\music
\media\library\video

All new stuff (downloaded or ripped) goes into the stage directory and gets imported into MJ. I have a SmartList defined called Stage:

folder=[\\botetourt\media\stage"

I go to this SmartList and edit the tags of the files in the stage directory. Once a file is cleaned up (tagged correctly, renamed) it gets moved (from within MJ, via the SmartMove plugin) into the \media\clean directory. Of course, I also have a SmartList called Clean:

folder=[\\botetourt\media\clean"

As soon as I get 650 megs or so of songs in the clean directory, I drag them onto the tree node for my burner and write them to CD (as DATA). I copy this list of songs from the grid (CTRL+C) and paste it into Excel, recording the artist, track name, and CD number. I use a Sharpie to write the number on the CD.

Once files have been written to CD I move them (again, from within MJ, using the SmartMove plugin) to the \media\library\music folder.

My files all get backed up to CD. So if I lose a hard drive or two, no big deal.

JaredH · « **Reply #7 on:** October 03, 2002, 07:57:25 pm »

I would run a RAID setup, but im not necissarily worried about backup, i keep all of my music/pictures/documents/important stuff on a USB 2.0 external, havent had any complaints thus far although murphys law will step in now that ive said that and something will piss me off.

Moonshine · « **Reply #8 on:** October 03, 2002, 07:59:50 pm »

Jimh,

You're roughly on track, so there isn't much to have at.

Basically there are a variety of RAID levels to choose from, each of which have different performance characteristics, pros/cons and drive requirements.

For large storage servers (like terrabyte+) people usually look to RAID5 as it requires only one extra drive and offers the best cost / performance ratio of any RAID levels.

RAID 5 requires 3 or more drives in the disk array.
Data and parity information are striped at block level across all the drives in an array though. Because parity data is distributed across the whole disk array, there is no single point of failure. (As in RAID4, which nobody uses(?))

Using an arithmetic function (exclusive OR), data on any drive can be computed from parity data stored on the other drives in the array.

Anyway there is a lot of info out there on RAID as more people use it every day. Heck, even Windows now has the ability to do software RAID1 in the operating system. (Started in Windows 2K Server I think)

I've built a number of RAID systems using Redhat Linux though and they run great. Currently I have a 320GB RAID5 system that acts as a file server for all of my MP3's and digital photos. (With a network of Turtle Beach Audiotrons to play music around the house)

Recently I've been looking to re-rip my whole collection to APE and thus need to replace the server with something much larger. Once Maxtor's 320GB models are available in mass (hopefully a month or two

) I should be able to build a nice 1.2 terrabyte RAID5 setup for ~$3500 in rackmount form with redundant power supplies. Install Samba and you have a monster drive to share.

ghappe · « **Reply #9 on:** October 03, 2002, 11:49:11 pm »

He Folks, Raid is fine, off site backup's are better!!!

Moonshine · « **Reply #10 on:** October 04, 2002, 08:04:10 am »

But when trouble hits with RAID you can still stream your music from the server while you are restoring things

Chris Shaw · « **Reply #11 on:** October 04, 2002, 08:44:32 am »

Very few integrated RAID controllers offer RAID5. Most offer:

RAID0 : Striping for full capacity, higher speed but lower reliability (1 drive fails and you lose all your data).
RAID1 : Mirroring for reduced capacity, normal speed but higher reliability.
RAID0+1: Reduced capacity but higher speed and higher reliability.

e.g. 4x80Gb disks at RAID0+1 mirrors disk 1 onto disk 2 and disk 3 onto disk 4, but stripes between the pairs. Thus you get 160Gb of storage at nearly twice the speed of one drive, and any one drive can fail without losing data. Two drives can fail so long as they're not a mirrored pair.
RAID5 would give you 240Gb at the same speed as one drive, but any one drive could still fail and you'd be OK.
It's getting cheaper though. Promise Fasttrak SX4000 does RAID5 for about $200

marketability · « **Reply #12 on:** October 04, 2002, 09:23:52 am »

I use a standard PC (large case) with 2x 3Ware Escalade IDE RAID controllers (an 8 port and a 4 port)

This allows IDE RAID5 and with
8x80Gb = 560Gb
4x120Gb = 360Gb

gives about 800Gb of useable disk space

The costs are pretty good (about £500 for the cards and about £1000 for the disks)

Works a treat and has saved my bacon a few times when disks have failed. Seemed expensive until I lost my first standalone 80Gb 's worth of sounds!

How DO you backup 500Gb sensibly?

KingSparta · « **Reply #13 on:** October 04, 2002, 09:29:13 am »

> How DO you backup 500Gb sensibly?
On IBM Punch Cards

marketability · « **Reply #14 on:** October 04, 2002, 09:57:43 am »

KingSparta - I have a few 3.5" Floppies - will that work?

Moonshine · « **Reply #15 on:** October 04, 2002, 12:17:15 pm »

Quote

Very few integrated RAID controllers offer RAID5.

Well, as you mentioned, Promise and others have hardware options for a couple hundred dollars now though. And there is always software RAID. Linux and Windows both have support. Not as galmorous performance-wise, but it really works quite well now. We have about 5 systems running software RAID1 and RAID5 under Redhat 7.2 and it works like a champ.

PhatPhreddy · « **Reply #16 on:** October 05, 2002, 10:36:39 am »

Time to re enter the fray (48 hours without sleep and counting).. RAID is well worth it and with drives getting cheaper and cheaper the best and cheapest way to back up your massive RAID array is to build another RAID array and co locate

!!!

I run 4 Maxtor 160GB's in RAID 0.. This is risky but up to now all data is my own.. No P2P here so there is no data on this array I dont own... From this I get the full 640GB...

I will probably get another 160 GB and go RAID 5 now I am back in Europe and can use corporate buying accounts to fund my HW addictions... This wil give me the redundancy I need and my promise controller will do RAID 5...

Basically the only types of RAID that are commonly interesting for us non clustered folks are RAID 0 1 and 5...

0 is fast and the full storage with no fault tolerance... 1 drive dies and the whole array is trashed...

1 is slightly slower and is mirrored... You get 1/2 the storage but full redundancy... Data is pretty much safe unless the machine is in a fire etc...

5 is with 1 redundant drive per array.. a 4 drive array will have 3 live drives and that much storage.. 1 drive can fail at a a time (I have no idea how it manages this neat trick) and the rest of the array's data is safe... RAID 5 make one hell of a lot of sense to the copying crowd.. Large storage but safe data..

BTW one of the AVS crowd went into the terrabyte storage biz... Above 2 terrabyte last time I looked but with drive capacities and prices this is a moving target... They had >2.2 or 2.6 terrabyte last time I looked and were pretty cost effective only adding a smal margin on actual drive and chassis costs IMHO... Of course they were going for the full hot swap high end market but if a RAID solution is interesting they are easy to talk to... http://www.pandora.com/pdms/

Lots of the AVS finge crowd are ripping entire DVD collections to HD in full DVD (no DIVX or WMP compression) format, and then using GUI front end launchers to have DVD by menu choice in a whole house setup... Too rich for my blood but its merely and extension of the ripping CD's and it wont be many years before terrabytes sound like kilobytes....

marketability · « **Reply #17 on:** October 06, 2002, 12:48:17 am »

"ripping entire DVD collections to HD in full DVD"

Scary!

I thought I was being extravagent in going for APE!

Suddenly 750Gb seems v small!

konicky · « **Reply #18 on:** October 06, 2002, 05:36:03 am »

750 seems a lot to me, I only have 2

JorgeGVB · « **Reply #19 on:** October 07, 2002, 03:11:15 pm »

I thought RAID was the way the go until a friend of my mine had his power supply burned up. It fried everything attached, including his hard drives. I have 3,500+ mp3s, not sure what the best way is to go now.

PhatPhreddy · « **Reply #20 on:** October 08, 2002, 02:18:54 am »

If a PSU did go bad and smoke the whole system (and this would be the first time I have ever heard of such a serious calamaty) then what alternative system (toehr than offsite tape backup) would solve this for you....

Working with DVD-R's is a pain as the volumes are not mounted all the time and even then they dont give you that much room....

JorgeGVB · « **Reply #21 on:** October 08, 2002, 05:03:35 am »

Yes, that is what occurred. My friend said the there was actually fire coming out the back of the PSU. I agree with you, I never heard of this before either. However, when he went to a local computer store to pick up a new PSU, the owner said he had seen this before.

I don't really have an alternate solution for backups. The thought of burning everything to CD/DVD is not very appealing. Like I said before, I thought RAID was the best answer.

Frankly, if you are using a high quality PSU, you are probably pretty safe. However, I am a little nervous now than I was a week ago.

zurn · « **Reply #22 on:** October 08, 2002, 08:26:06 am »

Regarding exploding power supplies, this just happened to me, and while my drives appear safe, the MB seems non-functional and i'm still assessing the reach of the damage.

Luckily this was not my music PC!

But i too have been considering the backup options...i guess all i've been asking myself revolves around time vs. money. It would be cheap but time consuming to use CD-R's, but i could just shell out for an additional HD and occasionally update...

Anyone have some advice for this decision?

PhatPhreddy · « **Reply #23 on:** October 08, 2002, 08:40:29 am »

Well the only solution to large RAID array backups (that makes any fiscal sense) is more large RAID arrays

!!! That may sound nuts but the cheapest terrabyte backup solution is a RAID array so if it is mission critical (and by that I dont mean our CD collection) then a RAID array in another location with a suitable backup structure is your only real solution....

Actually was talking a biz idea with an Auzzie in Thialand last week who provdes off site backup services for his clients under a similar but more basic structure... Hotel reservation systems need good backups... This servce his making him a nice living and is near 0 maintainance or effort.... He has a monthly charge system and almost 0 to do... I was impressed...

JorgeGVB · « **Reply #24 on:** October 08, 2002, 10:54:22 am »

I found out my friend's power supply was made by Deer. They are often found in eMachines and generic cases (for those who build their own computers). Apparently, these PSUs are known to be cheap and problematic. It may be worth it to open up your case to see what you have inside. It may be time for a PSU upgrade! Better safe than sorry....

marketability · « **Reply #25 on:** October 08, 2002, 12:00:38 pm »

definition of musical paranoia...

going on holiday and removing the drive cages from your server and hiding them in the loft so that if anyone nicks the server the music remains!

...are there drugs for this? maybe just more beer would work

zurn · « **Reply #26 on:** October 08, 2002, 01:01:12 pm »

Quote

I found out my friend's power supply was made by Deer.....

Same here! Yep==check those power supplies!

steve

INTERACT FORUM

Author Topic: Terrabyte Raid Server for about $5000 (Read 4928 times)

sekim