Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[JustinChase]

I've got something like 17TB of video stored on my server, and only have a few TB of empty space left (again).  I can upgrade a few of my drives from 1 or 2TB to 4TB drives, but I have no more room in my server to put any additional drives.  I'm not out of space yet, but can see that at some point in the next year or 2, I'll run out of space.

So, I've been considering converting my current files to h.265 format, which is supposed to keep the same quality at around 1/2 the size.  Before I spend too much more time looking into this, I wanted to see if this is something that will give MC problems in playback (I suspect not).

I also wonder if anyone else has started (or finished) converting their videos, and if they have any opinions on how it went.

Any problems or 'gotchas' that I should be aware of?

I probably won't start to do anything for a little while yet, but I wanted to start doing a bit of research and gathering opinions in the meantime.

[Hendrik]

A couple of thoughts:

- There is no encoder yet that can actually reach that promised goal (50% size at same quality). Best result right now is marginal space savings at the same quality. Expect years before that goal is reached, it took years for H.264 encoders to become "good".
- The only encoder being able to deliver that (x265) is much slower than a H.264 encoder would be, right now you can plan for like 12 hours for one movie, to get to a decent quality/space level.
- Decoding of H.265 is much slower than H.264, and there is no hardware acceleration, forget about playing those on a low-end system for the coming years (or god beware, mobile devices).

Personally, I think its at least 1-2 years too early to invest in H.265/HEVC.

[JustinChase]

Thanks for the insight.  I guess I'll keep upgrading drives for now, and if space gets too tight, I'll delete stuff I don't really "need".

[capfan]

Assuming all of your video files are already compressed (and you do not have access to the uncompressed video), recompressing them will almost always introduce loss of quality if your goal is to reduce space by a meaningful amount.

[JustinChase]

Well, most/all video is compressed, even BluRay is already compressed; but I've not compressed any more than that.  i.e. I have stored all my blurays in their original format, so they are all 20GB to 30GB in size as they sit now.  I'd like to compress the files, but keep the same quality, if/when this becomes possible.

[Hendrik]

You could easily compress them down to around half of their original Blu-ray size using H.264 with no perceptual quality loss (but mathematical loss, which your H.265 endavours would also include). Blu-rays use extremely high bitrates because they can, but its not always required for their level of quality - plus, many "commercial" encoders aren't as efficient as x264, which means they can also waste a bit of bitrate.

x264 could do wonders here, after tuning its option to your taste of quality/size vs. speed.
As a baseline, crf 18 results in around 10-11 mbit on some Blu-ray material I tested recently, and looks pretty darn fine.

[6233638]

You could easily compress them down to around half of their original Blu-ray size using H.264 with no perceptual quality loss (but mathematical loss, which your H.265 endavours would also include). Blu-rays use extremely high bitrates because they can, but its not always required for their level of quality - plus, many "commercial" encoders aren't as efficient as x264, which means they can also waste a bit of bitrate.

x264 could do wonders here, after tuning its option to your taste of quality/size vs. speed.
As a baseline, crf 18 results in around 10-11 mbit on some Blu-ray material I tested recently, and looks pretty darn fine.

I would argue that this is highly dependent on your setup, and what you consider to be acceptable quality.
Recompressing Blu-ray in order to save space has always resulted in noticeable artifacts in my opinion, and things just get worse as you move to larger displays and start upscaling to higher resolutions.
 
Similarly, while there are claims of H.265 being able to compressed to the same quality at 50% of the size, that assumes you start with the original source file before compression.
Re-compressing already compressed video will require higher bitrates.
 
You're right about x264 though, it can be quite a bit better than the commercial encoders.
I believe there are now some Blu-ray discs out there which used it for the encoding.
 
 
The best thing for saving space, as long as you only care about the main feature, would be to start converting your rips to MKV with a tool like MakeMKV.
 
Storage is a real pain though. I'm in a similar position myself now - I'm almost out of space again, don't have any spare drive bays, and the largest drives which are actually reasonably priced are still only 4TB. (the prices for 5/6TB disks are insane right now)
Sometimes I wonder if the people buying disc changers had the right idea after all.

[Hendrik]

I would argue that this is highly dependent on your setup, and what you consider to be acceptable quality.
Recompressing Blu-ray in order to save space has always resulted in noticeable artifacts in my opinion, and things just get worse as you move to larger displays and start upscaling to higher resolutions.

Watching strictly on a 1080p screen, even if somewhat larger, re-compressing Blu-rays to 10-15 gb always seemed visually lossless to me.
I still don't do it, since I have plenty space left and re-encoding at proper quality settings takes easily 3-5x the runtime of the movie, but I never managed to see a quality difference on that screen - and just judging from the bitrates allowed for that file size, its plenty to encode really high-quality streams. Its far above the bitrates the so called "scene" uses for its releases, or any online streaming services would ever use.
 

Sometimes I wonder if the people buying disc changers had the right idea after all.

Unless you can squeeze a 1000 disc changer into comfortable size and make it easy to use, I doubt it.
My movie collection is at around 300 Blu-rays to date, at only 7 TB of space - thats an average of nearly 25gb per movie, which means roughly 40-45 movies per Terabyte - or with 4TB disks, 160-180 movies per disk. Much easier to run 4 of those disks than a huge clunky changer with spotty support.

And my new file server has 40 TB usable space right now, not counting parity discs, and still space for 8 more discs (assuming I buy an extra controller). I think I'm setup for a while now!

[jmone]

Justin, 6TB drives are now out <$300.  I predict you will have a port free soon (One of your HDD is bound to die) = 5TB upgrade!

[JustinChase]

Justin, 6TB drives are now out <$300.  I predict you will have a port free soon (One of your HDD is bound to die) = 5TB upgrade!

I did not know that (haven't looked at drives for a little while).  Sadly, for me to use a higher than 4TB drive, I actually need to buy 2.  With unRAID, none of the storage drives can be larger than the parity drive, so I'd have to replace the parity with a 6TB before I could add another 6TB do the array.  I would use the current 4TB drive to replace a smaller 1TB drive, so I'd still benefit, but not as much as you would like.

As it is now, I've got a few TB free, and can replace several smaller drives before I'm really out of upgrade ability.

[glynor]

I'd just like to add +1 to everything Hendrik said.

And note that:

1. None of you have access to the real source.
2. BluRay uses those high bitrates for the reasons discussed above and also because the studios often do a shockingly bad job of encoding.

The latter is mostly cultural. Hollywood has not been good at valuing tech workers. In fact, they're often adversarial and treat all but the most connected individuals poorly.

So, they don't always have the best engineers, and the good ones they do have aren't the "encoding lackies".

[6233638]

Watching strictly on a 1080p screen, even if somewhat larger, re-compressing Blu-rays to 10-15 gb always seemed visually lossless to me.
I still don't do it, since I have plenty space left and re-encoding at proper quality settings takes easily 3-5x the runtime of the movie, but I never managed to see a quality difference on that screen - and just judging from the bitrates allowed for that file size, its plenty to encode really high-quality streams. Its far above the bitrates the so called "scene" uses for its releases, or any online streaming services would ever use.

Well it has always stood out to me. I was surprised when I bought the Surface Pro 2, just how much better watching real Blu-ray looked on that display instead of the encodes I had made for my iPad. (which were about 15GB in size)
It's funny, because at normal "television" sizes these things are often not that apparent, but when you shrink it down to tablet size, due to the sheer sharpness of a display like that, it stands out more than you would think.
And of course, if you have, or plan to use a projector, the differences can be quite obvious there.
 
I've always found re-encoded formats that appear to be "visually lossless" on a television can start showing their flaws when you move to a larger display or a higher resolution one and are having to upscale.
 

Unless you can squeeze a 1000 disc changer into comfortable size and make it easy to use, I doubt it.
My movie collection is at around 300 Blu-rays to date, at only 7 TB of space - thats an average of nearly 25gb per movie, which means roughly 40-45 movies per Terabyte - or with 4TB disks, 160-180 movies per disk. Much easier to run 4 of those disks than a huge clunky changer with spotty support.

Sony's 400 disc changer actually seemed quite reasonable.
Of course the units were very expensive, and at the end of the day you have to deal with a Blu-ray player interface, and we all know how awful that is with the forced ads/menus etc.
 
I've got about 400 Blu-rays and 200 DVDs on my server right now.
The averages seem to be:

Blu-ray ISO: 25.4 GB
Blu-ray MKV: 22.3 GB
DVD ISO: 5.0 GB
DVD MKV: 4.3 GB

I have a mixture of ISO and MKV because ripping to MKV requires some interaction, but I can deal with ISO later.

And my new file server has 40 TB usable space right now, not counting parity discs, and still space for 8 more discs (assuming I buy an extra controller). I think I'm setup for a while now!

The issue is that I originally built a gaming PC that is now predominantly used for media, so there are only five hard drive bays. And my motherboard only has eight SATA connectors. (1x SSD, 2x Blu-ray, 5x HDD)
I really need to do a complete rebuild, but have been putting it off for some time now, choosing to upgrade drive capacity before anything else. I should start another topic for this in the hardware section.
 

Justin, 6TB drives are now out <$300.  I predict you will have a port free soon (One of your HDD is bound to die) = 5TB upgrade!

Less than $300!? Here, the cheapest drives are the new Seagates, and those are $700 - more than twice what I paid for my 4TB WD Se drives.

[Hendrik]

Less than $300!? Here, the cheapest drives are the new Seagates, and those are $700 - more than twice what I paid for my 4TB WD Se drives.

Same here, 5TB is actually available for rather cheap, the Seagate Desktop HDD 5TB is only 180 € ($250), but 6TB is only available over 450 € ($600)

[jmone]

http://www.bhphotovideo.com/c/product/1051991-REG/seagate_stbd6000100_6tb_7200rpm_3_5_internal.html Also seen a post claiming $275 but I don't. Know from where.  These are the new desktop Seagate drives which are not on their website.  Mate got 2 shipped to Oz last week and as usual arrived in only a couple of days so stock does exist.

[jmone]

Justin. Like you I'm down to a few free TB on my pools but and my data ports are full.  After pondering the same growth issue I will wait till I get close and start replacing with these higher capacity drives and/or when they fail (and given I have over a dozen it will not be long)

[Sparks67]

Storage is a real pain though. I'm in a similar position myself now - I'm almost out of space again, don't have any spare drive bays, and the largest drives which are actually reasonably priced are still only 4TB. (the prices for 5/6TB disks are insane right now)
Sometimes I wonder if the people buying disc changers had the right idea after all.

I have 3 Sony BDP-CX7000ES, but JRiver never wanted to do the RS-232 port.  I was using home automation software to control them.  Changers allow you to have the Bluray menu's, but I do have my collection now on 2 large arrays.  A changer allows for 20Tb, with 400 Discs.  When Sony quit making them in 2012, there was people buying the scratch and dent models for $200 USD.  I bought 2 new ones for $500 USD. Compare the cost and you see that a changer is most economical.    I never compress the moves, which is huge mistake. 

The other option is Sas expander, but it is costly as well. http://www.sasexpanders.com/ 

[Hendrik]

3 of those changers are huge and take up a whole load of space, and for $500 you can buy 3x4TB drives these days, which hold 400-500 movies easy, even more if you compress them - with the added benefit that ripped movies are much easier to use.

Maybe they were more economical when they were new and big hard drives were still more expensive, but not anymore. Obviously Sony realized this, since they don't sell them anymore.
If you have space to put up another of these changers, you could also use the space to place a few external disks.

[glynor]

For those maxed out on their motherboard's sata ports (and bays) consider:
http://yabb.jriver.com/interact/index.php?topic=86285.msg604159#msg604159

[Sparks67]

3 of those changers are huge and take up a whole load of space, and for $500 you can buy 3x4TB drives these days, which hold 400-500 movies easy, even more if you compress them - with the added benefit that ripped movies are much easier to use.

Maybe they were more economical when they were new and big hard drives were still more expensive, but not anymore. Obviously Sony realized this, since they don't sell them anymore.
If you have space to put up another of these changers, you could also use the space to place a few external disks.

I have 32TB online space as well, but if you compare the cost.  The problem with going with a server is that you have to match the drive with Areca hard drive controller.  I have tried the cheap drives, and one lost 3tb of music. (I never buy a cheap drive again, and that was a seagate that lasted 6 months). I have a mix of 3Tb Seagate enterprise drives, and 4tb WD SE drives.   The Sony changers are cheaper.  The changers use less electricity, than a sever.  I had in the past a rackmount server with a zippy server power supply, but it would cost me about $60 a month to run the server 24/7. I never notice the cost to run the changers, because it is based on demand. I have them stored in my Salmander home theater cabinet, but I can make them rack mount.  Space will be about the same as rackmount server.   The reason that Sony quit making them, is not because of the size or expense.  The demand is still there for Bluray changers, and they currently sell at least $1500 on ebay.  Sony quit making them, because Bluray is about to die.  Bluray is known as third generation format.  (Bluray will be like HD DVD, and die a slow death) The reason is UHDTV1 (4K) and UHDTV2 (8k).

Here is the research from NHK.  http://www.nhk.or.jp/strl/english/aboutstrl1/r5-2-2.htm

I am sure some of you might remember the bankrupt of InPhase http://www.timescall.com/ci_19148132.   Companies never die because of their patents, but one of the PhD's started a new company.  http://akoniaholographics.com/  Just follow the management team, but the owner of the company has published a new paper to develop a new holographic storage system that is compatible a bluray structure.  Well, Then you  have Hitachi that has developed a faster holographic format.  http://www.infostor.com/blogs_new/henry_newman/stranger-than-fiction-hitachi-discusses-holographic-storage.html  The disc would initially be 2TB, but the future would be up to 12TB.  Hitachi is designed around storage, but NHK is only showing 2 different formats.  Hitachi's design would be the end of Bluray.

[Sparks67]

A couple of thoughts:

- There is no encoder yet that can actually reach that promised goal (50% size at same quality). Best result right now is marginal space savings at the same quality. Expect years before that goal is reached, it took years for H.264 encoders to become "good".
- The only encoder being able to deliver that (x265) is much slower than a H.264 encoder would be, right now you can plan for like 12 hours for one movie, to get to a decent quality/space level.
- Decoding of H.265 is much slower than H.264, and there is no hardware acceleration, forget about playing those on a low-end system for the coming years (or god beware, mobile devices).

Personally, I think its at least 1-2 years too early to invest in H.265/HEVC.

No, they are coming out.  Here is the broadcast quality one.  http://www.design-reuse.com/news/33893/hardware-hevc-h-265-encoder-ip.html, and here is the companies that Allegro has license the decoder technology.  http://www.allegrodvt.com/semiconductor/products2/al-hevc-es-25.html  Don't see Apple?  Broadcom is developing their equipment. 

Directv 14 is Ka band, and it is set to launch in October 2014.  DIRECTV-14 is a powerful, high-capacity spacecraft that will use Ka-band and the new “Reverse” DBS band to expand high definition and other new consumer services.  UHDTV has been tested in Ku band as well.  This company is saying 2015.  http://www.intelsat.com/blog/media-blog/ultra-hd-programming-beginning-to-materialize/  Intelsat is better quality. 

Technically, you are right it was announced that it be 2016.   

[fitbrit]

Anyone looking to buy storage related items might want to take a look at the unraid classifieds board:

http://lime-technology.com/forum/index.php?board=40.0

[Hendrik]

No, they are coming out.  Here is the broadcast quality one.  http://www.design-reuse.com/news/33893/hardware-hevc-h-265-encoder-ip.html, and here is the companies that Allegro has license the decoder technology.  http://www.allegrodvt.com/semiconductor/products2/al-hevc-es-25.html  Don't see Apple?  Broadcom is developing their equipment. 

Directv 14 is Ka band, and it is set to launch in October 2014.  DIRECTV-14 is a powerful, high-capacity spacecraft that will use Ka-band and the new “Reverse” DBS band to expand high definition and other new consumer services.  UHDTV has been tested in Ku band as well.  This company is saying 2015.  http://www.intelsat.com/blog/media-blog/ultra-hd-programming-beginning-to-materialize/  Intelsat is better quality. 

Technically, you are right it was announced that it be 2016.   

Encoders for broadcast are usually not "very good", they have a different goal, many of them need to work in real-time (or close to) to encode the outgoing broadcasts.
Also for the goal in this thread, our local media storage, they don't matter at all.

And yes, it'll be at least 2016 before that becomes viable. Today, noone has the decoding capability at home, which means long before they can start sending it to you, they need to make sure everyone got a hardware upgrade.

[Sparks67]

And yes, it'll be at least 2016 before that becomes viable. Today, noone has the decoding capability at home, which means long before they can start sending it to you, they need to make sure everyone got a hardware upgrade.

Allegro DVT is the French company that developed the HEVC h.265 specification.  Allegro DVT has licensed HEVC h.265 to 35 IP vendors in May 2013.  Actually, Allegro had HEVC encoder in 2012, and it was going to be firmware upgrade.  Although, since the specification has been released, then there was 25 IC Silicon licensed in October 2013. Intel, Broacom, Google, LG, Nec, Sigma, etc.  http://www.allegrodvt.com/semiconductor/products2/al-hevc-es-25.html   The HEVC specification has a 2 tier level contained in it. HEVC specification defines them differently. Actually, all products have consumer and Pro level.  This fall is the consumer release, but in 2015 to 2016 it will be Pro release. 

Nvidia Maxwell has been announced that it will have HEVC support. (remember the first implentation of HDMI in a video card?  It was called hdmi support)   I suspect that you are waiting for Nivida Pascal, which has most potential.  As far as HEVC is concerned it will be supported in set top boxes this fall.  I am following it by the actual chip.  For example, here is Broadcom's press release http://www.broadcom.com/products/Cable/Cable-Set-Top-Box-Solutions/BCM7445
Broadcom is one of Apple's hardware manufacturers, so expect an Apple TV set top box this fall. 
Accordiing to this article, if you want an Intel.  Broadwell will have it.  http://www.cpu-world.com/news_2014/2014050101_Intel_Broadwell_graphics_enhancements.html Companies are in business to make money, so delaying HEVC has caused some companies to change the production schedule. 

Denon is going to bring out a receiver with Dolby Atmos.   Well, it is implied in the article, but starts on model X4100, X5200, X7200.   http://www.cinenow.fr/articles/28834-denon-avr-x3100w-avr-x4100w-avr-x5200w-et-avr-x7200w-demandez-le-programme  You have to translate it.  Yes, there will be lot of people that will want the latest technology, which these electronics companies will sell to them. 

Content is available streaming in HEVC now.  Although, it is over the internet. I found several companies.