INTERACT FORUM

Please login or register.

Login with username, password and session length
Advanced search  
Pages: [1]   Go Down

Author Topic: RAID for movie storage?  (Read 23998 times)

Audioseduction

  • World Citizen
  • ***
  • Posts: 125
  • World Class Referance Playback!
RAID for movie storage?
« on: May 23, 2012, 06:57:51 am »

I like to store my DVDs & Blu-Rays onto a 10-12TB RAID NAS. What do you recommand? What do you use to store your videos?
Logged

MrHaugen

  • Regular Member
  • Citizen of the Universe
  • *****
  • Posts: 3774
Re: RAID for movie storage?
« Reply #1 on: May 23, 2012, 09:54:33 am »

I use a Norco DS-24D. 24 bay unit with SAS expander. Just add a server with a decent raid card to run it, and you have a hell of a good storage solution for a reasonable price (http://www.norcotek.com). I have not seen this affordable and "professional" units ever before. It's at least professional for a home users perspective.

I run mine in a 24 x 2TB disk setup in rad 6. 24 TB x 2 (minus 4 x 2 TB disks  for parity data). 40 TB total disk space, but I'm using a sync app to have a daily sync with a delayed deletion on the backup array :) So, it's just 20 TB in storage today. When I run out of space I can just replace the disks with larger disks. Or purchase another norco unit :D
Logged
- I may not always believe what I'm saying

Audioseduction

  • World Citizen
  • ***
  • Posts: 125
  • World Class Referance Playback!
Re: RAID for movie storage?
« Reply #2 on: May 23, 2012, 04:14:54 pm »

After some research I decided to purchase a Drobo S Beyond RAID 5-Bay  and 5 3TB 7200RPM drives. I plan to set it up as RAID 5 and attach it directory to my HTPC via firewire or USB 3.0.  I decided I don't need it on the network.

Logged

MrHaugen

  • Regular Member
  • Citizen of the Universe
  • *****
  • Posts: 3774
Re: RAID for movie storage?
« Reply #3 on: May 24, 2012, 02:13:21 am »

That will work, and it's certainly cheaper. Do have in mind that this is not a backup solution though. At least not if you store everything here. Remember a secondary backup of the most important things. Raid can fail, even with raid 0, 5, 6 and so on.
Logged
- I may not always believe what I'm saying

glynor

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 19608
Re: RAID for movie storage?
« Reply #4 on: May 24, 2012, 06:47:23 pm »

I use a Norco DS-24D. 24 bay unit with SAS expander. Just add a server with a decent raid card to run it, and you have a hell of a good storage solution for a reasonable price (http://www.norcotek.com). I have not seen this affordable and "professional" units ever before. It's at least professional for a home users perspective.

I've read your posts on that bad-boy before.  I'm almost certainly going to pick one of those up for the office next year, when my newly-upgraded-to-6TB RAID runs out of space again.

Two questions though... My home RAID box is running on miniSAS 8088 cables too (only two of them, though), but they're short runs as the RAID box is an Adaptec desktop job sitting right next to the server.  It is loud though... Substantially louder than the PC itself (stupid little fans - I really wish they made rackmount cases with bigger, slower fans more often).

Anyhow... The 8088 cables were absurdly expensive, even for short ones.  Makes the Apple Thunderbolt cable look like a good deal.

1. Do you know of any secret place to buy them where they don't require you to qualify for a loan first?
2. What kind of cable length restrictions am I looking at, if I want to put the Norco in another room or a closet or something?  Can I run it way down the hall to our telecom closet, for example?
Logged
"Some cultures are defined by their relationship to cheese."

Visit me on the Interweb Thingie: http://glynor.com/

glynor

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 19608
Re: RAID for movie storage?
« Reply #5 on: May 24, 2012, 06:48:37 pm »

PS. Hopefully by the time I do this again at home, they'll have something that runs over Thunderbolt.  PCIe is so much easier to deal with, and I'll already have it on my motherboard.  Put the RAID hardware in the drive case where it belongs.
Logged
"Some cultures are defined by their relationship to cheese."

Visit me on the Interweb Thingie: http://glynor.com/

Hendrik

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 10786
Re: RAID for movie storage?
« Reply #6 on: May 25, 2012, 12:43:20 am »

2. What kind of cable length restrictions am I looking at, if I want to put the Norco in another room or a closet or something?  Can I run it way down the hall to our telecom closet, for example?

For SAS cables, its around 10 meters (33 ft)

I was pondering on an external SAS attached drive bay as well, but sadly those are either hard to get (at least in Europe those Norco cases are not that good on availability), or *very* expensive. Plus i really would want thats not noisy.
For now, i still have room for a little bit of expansion inside the server, so its a project for next year, i guess.
Logged
~ nevcairiel
~ Author of LAV Filters

MrHaugen

  • Regular Member
  • Citizen of the Universe
  • *****
  • Posts: 3774
Re: RAID for movie storage?
« Reply #7 on: May 25, 2012, 02:17:58 am »

First of, noise:
The Norco cases also comes shipped with jet engines for fans. I ordered some almost silent but effective 80mm fans at the same time as I bought the case, and now I mostly hear the drives them self. Even quieter disks makes more or the same amount as those 4-7? fans in that case. The drives get hot, but I'm not really worried.

Cables:
I also think the max length I've seen is about 10m/33ft. The case I mentioned uses ONE single SFF-8088 for output to the raid card. It also have another SFF-8088 input for daicy chaining up to 128 of those boxes :D In my case I have 2 mini SAS ports on m\y raid controller, so by the time I run out of disk space, I'll probably just purchase another DS-24D and attach that to the raid card. It will give better performance.

Price:
The cables them self are not that expensive. At least when you consider you only have to purchase one. They are priced at around 25 bucks and up I think. More expensive as the length goes up I think.
The DS24-D unit is quite expensive. But if you look at similar alternatives, most of them are much much more expensive! I think you get the unit for around $1400. I purchased mine for around 1100 bucks on ebay, and with the dollar value being very low at that time, it was a steal for me.

Availability:
For European guys you have Ri-vier in the Netherlands (?). They do stock much Norco products. http://cybershop.ri-vier.nl/? Ri-vier also have many cheaper alternatives for storage and servers. I purchased 2 servers from them. Just the case though. One 2U case and one 4U case. They are both decent cases for rack mounting, and very cheap.
I think US users have much more options getting a hold of this exact unit.
Logged
- I may not always believe what I'm saying

Hendrik

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 10786
Re: RAID for movie storage?
« Reply #8 on: May 25, 2012, 04:36:52 am »

Nice, i didnt know there was a shop dedicated to sell the Norco cases in Europe.
The pure storage cases are indeed extremely expensive, but i would probably be fine with one of their 4U server cases with 24 HDD bays (no SAS expanders, though), and those are relatively cheap.

For now, i can still fit 4 more drives into my case if i get a 5.25 -> 3.5 cage (and have 4 ports on my SAS controller), so thats still plenty of space to be had.
As a bonus, i also still have a PCIe slot free, where i could put a SAS card with external connectors if that time comes.

Currently on 9x2TB in Raid6, so 14TB of usable space, with around 8.5 used.
Logged
~ nevcairiel
~ Author of LAV Filters

glynor

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 19608
Re: RAID for movie storage?
« Reply #9 on: May 25, 2012, 09:37:53 am »

I think US users have much more options getting a hold of this exact unit.

Yeah, they sell them on Newegg, so I'm golden there.

The case I mentioned uses ONE single SFF-8088 for output to the raid card. It also have another SFF-8088 input for daicy chaining up to 128 of those boxes :D In my case I have 2 mini SAS ports on m\y raid controller, so by the time I run out of disk space, I'll probably just purchase another DS-24D and attach that to the raid card. It will give better performance.

Yeah, I'm likely not going to go with the one with the built-in port expanders.  I'm looking at a smaller system (probably the D-12D, or something similar), and it'll be going to a Mac Pro.  I have a couple issues with port expanders generally, but mostly, the RAID card I'd end up getting will come with 4 8088 connectors.  Choices are somewhat limited on Macs, and I'm not going with a cheap Highpoint card for this system.  They support port expanders, of course, but I've seen flakiness in the past and it isn't worth the trouble (and expense).  Plus, I need a pile of SPEED off of this RAID.

I'm using it to edit video off of a RED Epic (and assorted DVCProHD cameras).  The RED 4-5K footage is big.

I mean... Really big.  The footage I have from our production company runs between 3-5GB (that's gigabytes, not bits) per minute of footage.  If you enable HDRx mode, it is double that size.  Layer two of those on a timeline and try to play it off of your average WD Black drive and see how that works.  ;)

I don't typically actually use the RED footage directly in my edits.  I transcode them to 1080p Apple ProRes HQ instead.  (But those data rates are nothing to sneeze at either.)  But, if I'm going to build a new storage system, I'm going to spec it out for the future, not my current needs.

I might end up going with a 24-bay job and save half of them for future expansion.  I don't know.  We'll see.

The cables them self are not that expensive. At least when you consider you only have to purchase one. They are priced at around 25 bucks and up I think. More expensive as the length goes up I think.

Man, that wasn't my experience.  For a little 2 meter cable I was looking at around $85-90 on Amazon and other places.  The 8-10 meter cables were substantially more expensive.

Anyone know if there is an easy way to go from miniSAS to fiber and back again?  It would be really nice to put it in the telecom room, or at least out of my office.
Logged
"Some cultures are defined by their relationship to cheese."

Visit me on the Interweb Thingie: http://glynor.com/

glynor

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 19608
Re: RAID for movie storage?
« Reply #10 on: May 25, 2012, 10:07:32 am »

For SAS cables, its around 10 meters (33 ft)

Thanks, by the way.

I'd seen cables stocked in the 10-12 meter range, but nothing bigger.  That usually means either the length limits or special order, and I wasn't sure which.
Logged
"Some cultures are defined by their relationship to cheese."

Visit me on the Interweb Thingie: http://glynor.com/

MrHaugen

  • Regular Member
  • Citizen of the Universe
  • *****
  • Posts: 3774
Re: RAID for movie storage?
« Reply #11 on: May 25, 2012, 02:28:21 pm »

I might just remember the small price for those tiny cables I needed :) I only ordered 0.5 meter cables I think.
Logged
- I may not always believe what I'm saying

glynor

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 19608
Re: RAID for movie storage?
« Reply #12 on: May 25, 2012, 02:41:35 pm »

I just realized that monoprice sells 8088 cables.  Man, I feel silly.  Should have checked there before.

They don't sell long ones, but they have 0.5-2 meter cables for reasonable prices.
Logged
"Some cultures are defined by their relationship to cheese."

Visit me on the Interweb Thingie: http://glynor.com/

FastKayak

  • World Citizen
  • ***
  • Posts: 196
Re: RAID for movie storage?
« Reply #13 on: May 30, 2012, 11:11:22 am »

When Windows 8 ships - see  http://blogs.msdn.com/b/b8/archive/2012/01/05/virtualizing-storage-for-scale-resiliency-and-efficiency.aspx I'm assuming the various drive bay vendors will co-announce new hardware and if we are lucky there will be lower prices and/or some black Friday deals.

At least I have my fingers crossed that is what happens. 

FastKayak / Larry
Logged

Sparks67

  • World Citizen
  • ***
  • Posts: 162
Re: RAID for movie storage?
« Reply #14 on: July 15, 2012, 11:36:56 pm »

Anyone know if there is an easy way to go from miniSAS to fiber and back again?  It would be really nice to put it in the telecom room, or at least out of my office.

If your motherboad has a thunderbolt port, then that is an option with these new storage system from Areca.   http://www.areca.com.tw/products/thunderbolt.htm
There is another company that is making a fiber thunderbolt cable.  http://www.storagenewsletter.com/news/connection/sumitomo-electric-thunderbolt-cable
I haven't seen those cables hit the USA market yet.   Probably when the new Gigabyte motherboard ships, the cables are shown in this video.  http://www.youtube.com/watch?v=c7gY8iE3Ltg
If you have Areca raid card, they are making a driver to give you raid capability with their new storage setup.  http://www.areca.com.tw/news/2012/n0605_0112.htm
The Areca cards work in a Mac Pro too.  

This would be the best method, because I have fiber to the desktop at work.  Optical fiber cable connections are rather easy to break.  
Logged

Audioseduction

  • World Citizen
  • ***
  • Posts: 125
  • World Class Referance Playback!
Re: RAID for movie storage?
« Reply #15 on: July 17, 2012, 11:52:17 am »

That will work, and it's certainly cheaper. Do have in mind that this is not a backup solution though. At least not if you store everything here. Remember a secondary backup of the most important things. Raid can fail, even with raid 0, 5, 6 and so on.

I have 2 Drobo S, one backs up the other. They both are connected via USB 3.0 and it works great! Transfer rate is VERY fast! Almost 50MB per second from one drobo to the other drobo.  ;D
Logged

Daydream

  • Citizen of the Universe
  • *****
  • Posts: 770
Re: RAID for movie storage?
« Reply #16 on: July 17, 2012, 05:20:56 pm »

First of, noise:
The Norco cases also comes shipped with jet engines for fans.

Just for people into this kind of thing, for the Norco RPC-4224, RPC-4220, RPC-4216, RPC-4116, RPC-4020, RPC-4164 (AFAIK) the fan bracket can be replaced, doing a 4 x 80mm screamers to 3 x 120mm [your choice] fan conversion.
Logged
Pages: [1]   Go Up