Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[Z0001]

Hi

As the title suggests, I need to expand my storage from 5 internal 4tb drives to an expandable external solution that can also be accessed by other computers on my network.

However, I want to ensure that it will handle 4k video bandwidth requirements so that I have future proofed and don't need to reinvest again (however if 4k movies are going to use 100gb of disk space I guess that begs the question of when will hdd $/gb be low enough).

So I guess my question is, assuming the DRM on UHD Blu Ray content gets cracked and I can back up my media, what throughput do I need to design to?

I am thinking of using an Addonics hardware solution that allows me to choose a case, install their port multiplier and connect to a NAS adapter via a single USB 3.0, allowing jbod mode over the network. I can then use a USB hub to expand when I need to. And I don't need to build a separate server, get to grips with a NAS software and maintain it.

Would appreciate thoughts on how others are planning for 4k.

Thanks
Z

[jmone]

I use this setup on my Main PC (no external HDD enclosure) that also runs MC Server, and I'm serving UHD BD now without issue:
- Tower Case with 9 HDD Bays
- Added a M1015 Adapter flashed to IT Mode to give me 8 more SATA ports
- Run DrivePool over the physical HDDs to present it as one Drive (currently a 60TB Pool)
- Share the DrivePool to the Network

All works well.  I do have a second server that I use to backup everything but that is a different story.

I've always found the limiting factor cost wise is getting more Sata Ports and Bays.  I have also tried external NAS setups and expansion chassis and apart from being relatively expensive (on a per drive basis), they were also low powered, or cheaply made, or unreliable with their connection.  In the end I just got a decent case (Lian Li) that with 9HDD Bays & the M1015 (for the extra SATA ports), and it has served me well for years.  I then just buy the large size value for $ HDD on the market at the time.  When I need more room I swap out the older smaller hdd for newer bigger ones.  The smaller hdd go into my 24-Bay rack mounted backup server.

[wildcatknh]

So I guess my question is, assuming the DRM on UHD Blu Ray content gets cracked and I can back up my media, what throughput do I need to design to?

DRM on UHD BluRays has been cracked for quite a while now and is not an issue for ripping UHD BluRays.

[jmone]

Not all discs - more info here - https://yabb.jriver.com/interact/index.php/topic,112653.0.html

[Z0001]

I use this setup on my Main PC (no external HDD enclosure) that also runs MC Server, and I'm serving UHD BD now without issue:
- Tower Case with 9 HDD Bays
- Added a M1015 Adapter flashed to IT Mode to give me 8 more SATA ports
- Run DrivePool over the physical HDDs to present it as one Drive (currently a 60TB Pool)
- Share the DrivePool to the Network

All works well.  I do have a second server that I use to backup everything but that is a different story.

I've always found the limiting factor cost wise is getting more Sata Ports and Bays.  I have also tried external NAS setups and expansion chassis and apart from being relatively expensive (on a per drive basis), they were also low powered, or cheaply made, or unreliable with their connection.  In the end I just got a decent case (Lian Li) that with 9HDD Bays & the M1015 (for the extra SATA ports), and it has served me well for years.  I then just buy the large size value for $ HDD on the market at the time.  When I need more room I swap out the older smaller hdd for newer bigger ones.  The smaller hdd go into my 24-Bay rack mounted backup server.

I see. So what's the benefit of having a dedicated windows server PC rather than just storage on the network? Feels like an extra layer to manage.

Is the benefit of running a central MC server that you keep one library database?

What's the benefit of drive pooling in the context of MC accessing the files?

In my situation the storage solution will be out of the way in the basement with no monitor and not terribly convenient for managing.

Thanks
Z

[jmone]

I see. So what's the benefit of having a dedicated windows server PC rather than just storage on the network? Feels like an extra layer to manage.

These drives are just in my "main" PC that I use everyday.  It doubles as the MC Server, so I'm actually combine them into a PC I already have (plus it is a powerful PC with plenty of grunt for transcoding duties should a mobile or DLNA device request a video that needs to be transcoded).  I do have a seperate Windows Server 2012R2 Essentials PC for purely backup reasons (the one running 24 bays).

Is the benefit of running a central MC server that you keep one library database?

Correct.  All my MC clients then pull the Database from this one.

What's the benefit of drive pooling in the context of MC accessing the files?

It is the easy of accessing and managing the files though a single share / drive map.  I have a single "M:" drive = 60TB and I've then used Windows sharing (Homegroup) so all items in the Database appear as "\\MAIN\Media\Video\somemovie.m2ts" and all the HTPC's can then access this directly.  With direct access to the share, there is no need to transcode for the HTPC but it can still transcode for devices like Phones / Tablets / DLNA devices as needed.

In my situation the storage solution will be out of the way in the basement with no monitor and not terribly convenient for managing.

That is what I do with my Backup server (well it does have a monitor and keyboard but I mainly RDP to it).

[jmone]

The other benefit of using an existing PC is that it keeps the cost and complexity down especially if you already have a pretty good PC.  Potentially you are just up for a Case that can hold enough physical discs for your needs and a SATA card if you need more ports.  You also don't need any Pooling SW as you could share each physical drive and manage the content on each drive manually.... but it is just easier to have one "big" drive and the SW is cheap and works.

[Z0001]

The other benefit of using an existing PC is that it keeps the cost and complexity down especially if you already have a pretty good PC.  Potentially you are just up for a Case that can hold enough physical discs for your needs and a SATA card if you need more ports.  You also don't need any Pooling SW as you could share each physical drive and manage the content on each drive manually.... but it is just easier to have one "big" drive and the SW is cheap and works.

So in terms of that server PC, what sort of power is needed if all it's really doing is sitting in the basement making a library available? Eg does the server do the upscaling or transcoding if it's needed or does that happen on the client at playback?

I'm not really sure I understand the full performance requirements of the server.

Really appreciate your help
Z

[JimH]

Conversion is usually done on the server. 

[jmone]

Lets break it into several parts (and these can be separate or all on one PC pending what you want to do).
- File Store : Hosts the actual media and shares it to the Network.  Needs to have plenty of SATA Ports, Bays, with good throughput.  Does not need a high end CPU or GPU.  Could be a PC, NAS, or Direct Attached Storage (eg box with a Sata or USB connection).  From my experience many of the cheaper NAS or Attached Storage boxs have poor throughput or unreliable connections.  Does not

- MC Server : Runs MC and shares the Database to MC Clients.  It will also be the box that will do any of the "Transcoding" if needed by the clients.  If transcoding you will need a strong CPU.  I have a i7-8700K Overclocked and it can transcode all my UHD with the excpetion of Billy Lynn's Long Halftime Walk (which is 4k/60p - eg much higher frame rate than 46/24p on most discs).  Hendrik is currently also looking at adding GPU Transcoding and if this happens in MC then this PC will need a strong GPU.

- MC Client / HTPC : Runs MC as a Client, and will Load the Library from the MC Server, and then play the media from the File Store.  Will need a strong GPU if using ROHQ (madVR) and a reasonable CPU.

As I said you can have all or some of these on One PC, (or two, or three etc). 

[Z0001]

This is great.

As well as wanting to set up a storage solution I was also considering a rebuild of my flakey HTPC, and beefing it up for DAW application.

I'm now thinking of a higher powered machine in the study which would absorb all HTPC ripping and management duties as well as my DAW plus I can keep the back up duies I here away from the lounge room, which will be a selling point of the investment(!)

Thanks!
Z

[SkyBox]

- MC Server : Runs MC and shares the Database to MC Clients.  It will also be the box that will do any of the "Transcoding" if needed by the clients.  If transcoding you will need a strong CPU.  I have a i7-8700K Overclocked and it can transcode all my UHD with the excpetion of Billy Lynn's Long Halftime Walk (which is 4k/60p - eg much higher frame rate than 46/24p on most discs).  Hendrik is currently also looking at adding GPU Transcoding and if this happens in MC then this PC will need a strong GPU.

As I said you can have all or some of these on One PC, (or two, or three etc).

I'm building a new media server with 20 bays, and I will push off my backups to this box as well. Regarding your comments about CPU and GPU for UHD playback, I am planning to get an i3-8100 or i5-8400 and let the GPU do the heavy lifting.

Are you saying that JR MC is not currently built to play back this way or just wont transcode to clients with a weak CPU?  I won't be streaming the movies on other clients, just directly from server to projector.

[jmone]

If you are using a HTPC for playback using ROHQ to your PJ then you want a decent GPU, regardless of where the media is stored.  If you also need your Server to transcode video (say for devices not directly accessing the file store on your server,  like tablets, phones, TV etc) then you want a strong CPU in your server to do this job. 

[SkyBox]

I have not checked out ROHQ yet - I'm trialing JRiver.  My current HTPC (11 years old-Athlon 4200) has a Radeon HD 5670 1GB GPU and my new ASRock Z370 board has onboard GPU with the coffee lake intel UHD Graphics 630.  So I doubt either of these 2 GPU options will be sufficient for ROHQ, although the requirements stated Nvidia GTX 550 at a minimum.

Once I get the new machine built, I'll tinker around with it, but holding off on purchasing an additional GPU at this time.  My current HTPC has been playing back blu-ray files for years, but it's dying and needs replaced.

[Manfred]

I have now the following:

A central Media Server:
- Case:Lian Li PC-Q25B; PSU:Corsair SF450 Gold; 2 x Kingston DDR2400 ECC RAM; Asus P10S-I+KVM Management Module; i3-7100 3MB 3.9 GHz (ECC Support); 64 GB Transcend MTS400 M.2 SSD; Windows 10 Pro; Cooling: NH-9Li; NH-S12A PWM; NH-A14 PWM

- 2 x 8 TB WD Red (Windows Storage Space configured): I wanted some high availability to all my documents, photo's,  purchased High Res Audio files etc.

- 2 x 10 TB WD Red filled by ripped movies and TV Shows backup-ed once (static content) with EaseUS to old disk and put to another place
-DLNA server(s)

In the living room I have a MC based HTPC client with an GTX 960 (no media stored) for playing video from the server to a 4k TV and use my Devialet (in 2 weeks back from Paris) with Core Infinity Streaming module as UpnP client to the server for Audio.

Comments & Experience:

-The i3 CPU on the server is more than enough. Also Windows is very stable. No Problems so far.

-I have also tried to use my LG OLED TV as DLNA renderer and push the video through JRemote directly from the server to the LG TV - from a picture point of view, the up-scaled quality using the LG TV for up-scaling even for 720p or SD content is more or less the same as using the HTPC with GTX 960 and madVR. Disadvantage is that all my video files with 24 bit audio stream are not supported by LG TV. That is the reason why I will stay with HTPC as a client to the server.

[SkyBox]

I have now the following:

A central Media Server:
- Case:Lian Li PC-Q25B; PSU:Corsair SF450 Gold; 2 x Kingston DDR2400 ECC RAM; Asus P10S-I+KVM Management Module; i3-7100 3MB 3.9 GHz (ECC Support); 64 GB Transcend MTS400 M.2 SSD; Windows 10 Pro; Cooling: NH-9Li; NH-S12A PWM; NH-A14 PWM

- 2 x 8 TB WD Red (Windows Storage Space configured): I wanted some high availability to all my documents, photo's,  purchased High Res Audio files etc.

- 2 x 10 TB WD Red filled by ripped movies and TV Shows backup-ed once (static content) with EaseUS to old disk and put to another place
-DLNA server(s)

In the living room I have a MC based HTPC client with an GTX 960 (no media stored) for playing video from the server to a 4k TV and use my Devialet (in 2 weeks back from Paris) with Core Infinity Streaming module as UpnP client to the server for Audio.

Comments & Experience:

-The i3 CPU on the server is more than enough. Also Windows is very stable. No Problems so far.

Nice setup - Always been a fan of the Lian Li cases, One day I'll replace my old Antec tank with one. 

Are you replicating your movie/tv files from one 10tb to the other?  Or are these drives serving as the backups for your original discs?  I'm trying to get an idea of what works for others.  I haven't decided if I'm going to setup RAID or use Macrium Reflect.

Glad to hear the I3 7100 is working out for you - that gives me some confidence in my decision to save $100 for a better GPU.  Also have my 1st  M.2 SSD coming for this build and going with Win10 Pro as well.

[Manfred]

I don't replicate my movie files and I also don't use RAID for this. I fill the first 10 TB disks with mkv files, if the disk is full I fill the next 10 TB disk.
- 1 x copy on the original source DVD/BD,
- 1 x copy on disk (not replicated, no RAID)
- 1 x copy on external disk which I transport to a different physical location. I use my old disk for this.

playing FHD, 4K videos with GTX 960 results in < 20% CPU utilization. GPU utilization depends on madVR algorithm and source (SD, HD).

[wildcatknh]

Not all discs - more info here - https://yabb.jriver.com/interact/index.php/topic,112653.0.html

You may be talking about having the software and hardware to rip your personal UHD BluRays on your own and I will admit that I am not very knowledgeable in that specific area.  I was referring to the ripped UHD BluRays that I've seen on various sites around the internet available to download.  The number of ripped UHD BluRays that are out there is fairly large and is growing pretty quickly every day.  So someone out there is able to rip a significant number of them. 

*** If this specific part of this conversation (downloading ripped content) I've introduced is forbidden on these forums, then please remove this post and I won't bring it up again.

[Z0001]

Lets break it into several parts (and these can be separate or all on one PC pending what you want to do).
- File Store : Hosts the actual media and shares it to the Network.  Needs to have plenty of SATA Ports, Bays, with good throughput.  Does not need a high end CPU or GPU.  Could be a PC, NAS, or Direct Attached Storage (eg box with a Sata or USB connection).  From my experience many of the cheaper NAS or Attached Storage boxs have poor throughput or unreliable connections.  Does not

- MC Server : Runs MC and shares the Database to MC Clients.  It will also be the box that will do any of the "Transcoding" if needed by the clients.  If transcoding you will need a strong CPU.  I have a i7-8700K Overclocked and it can transcode all my UHD with the excpetion of Billy Lynn's Long Halftime Walk (which is 4k/60p - eg much higher frame rate than 46/24p on most discs).  Hendrik is currently also looking at adding GPU Transcoding and if this happens in MC then this PC will need a strong GPU.

- MC Client / HTPC : Runs MC as a Client, and will Load the Library from the MC Server, and then play the media from the File Store.  Will need a strong GPU if using ROHQ (madVR) and a reasonable CPU.

As I said you can have all or some of these on One PC, (or two, or three etc).

Just a follow-up on ripping and then serving 4k UHD content (once there is a ripping solution for our personal discs).

What minimum hardware would I need for this? Clearly a UHD disc drive, but would my mobo and CPU need to of a particular spec? I currently have gigabyte P55A-UD4P mobo and an Intel i7 870 2.93 GHz.

I'd like to be able to put a UHD drive in this machine and use it to rip (once there is a reliable solution) and as my server. I don't plan to do any transcoding (eg playing to an iPad) I just play the media content as is and watch on the lounge room TV.

Many thanks
Z

[jmone]

All you need is a the "right" drive and SW to rip the UHD.  Once ripped it is just another HEVC Video and there is no constraint on what Mobo/CPU/OS/SW needed.  Have a read of the FAQ on this - https://yabb.jriver.com/interact/index.php/topic,112653.0.html and follow the links for more info.

[lynes88]

So jmone, back to one thing you mentioned in an earlier post.  So, when you get to the point of wanting to upgrade a HDD to a larger size as they become available and $ is right... how do you actually go about that?  So, lets say your old HDD that is being replaced is 2TB with around 80 BD rips.  And the new HDD that will replace it is 6TB. 

What do you do with the roughly 2TB of BD Rips so you don't lose them?  Or, how do you get these onto the new 6TB drive and back into the Library?

And FWIW, I have been using online cloud storage for my backups... around 9TB total now.  But I am running into the challenge of running out of disk space in about a year.  I currently have 4 HDDs in my HTPC, and thought about upgrading their size... I do not necessarily have the spare room to temporarily store for transfer.... but I could I guess just add an external HDD.

[Z0001]

On the point of hardware to playback for UHD from disc and HDD:

What specific specs do I need for graphics, CPU, mobo, HDMI out etc.

I'm going to buy a UHD drive and just want to be ready.

Also, can I stream from a UHD disc in a server to a client machine? Are there problems with that?

Cheers
Z

[jmone]

@lynes88 - As I use DrivePool, I "add" a new Drive to the Pool in it's GUI, then I "remove" the old drive.  Drive Pool takes care of all the moving of the files for me between the discs.

@Z0001 - You need nothing "special" to play UHD but the following is important:
- Playback: A modern, med to high end GPU on the devices you want to PLAY the UHD on (pending what madVR settings you are going to use)
- Streaming: If you "client" is another MC instance then just see above.  If the "client" is some other device (TV, Tablet, Phone etc) that required MC to transcode the UHD to another format to stream, then your "server" needs a strong CPU.  I have a i7-8700K and it works fine for all but one title (Billy Lynn as it is a UHD@60p).  I know that on the list of things JR is looking at is to make some changes in this area at some point that may help, but for now the transcoding engine is CPU based.

[lynes88]

Makes total sense... just not familiar with Drivepool.  Never used it before.  Can I run DrivePool on a Win 7 machine?  In my setup, I have an SSD main C drive and 4 internal HDDs and one external USB 3.0 drive.  And assuming that I could use it... do I essentially need double the amount of space in order to duplicate files?  My movie files make up around 8 TB I think now... and I have a total of  around 16TB across all 5 drives but I also have other media on these drives too (music) and may not have enough free space to duplicate movies.

It sounds like you use DrivePool to duplicate your movie folders so that is what I assume that you can "swap" drives as needed.  Two of my oldest drives are only 2TB models... I could replace those with 6TB versions too... that would put me in a better position.

[lynes88]

Also, I had an alternate plan too... using Backblaze cloud backup of my media drives.  In the event I need to replace a drive or recover a substantial amount of my media...I have a backup in the cloud.  I don't need DrivePool really unless I want quicker and easier expansion/recovery options? 

[Z0001]

Thanks for all your help guys!

I'm thinking of getting this LG drive today: LG BH16NS55.

It may have firmware 1.03. How easy it it to roll that back to 1.02 so the drive is "friendly"?

Cheers
Z

[jmone]

I've not done the roll back (as mine were already the "right" version) but others have.

[jmone]

Makes total sense... just not familiar with Drivepool.  Never used it before.  Can I run DrivePool on a Win 7 machine?

Yes.  I use it on both Win10 and WinServer 2012R2 just fine.  Others run in on Win 7 / Win 8 etc.

... do I essentially need double the amount of space in order to duplicate files?

There is an option in DrivePool to select what you want duplicated (and the # of Times).  I don't use this feature as I run two pools (one on my Win10 box the Other on my WinServer 2012R2 and sync between them).

It sounds like you use DrivePool to duplicate your movie folders so that is what I assume that you can "swap" drives as needed.  Two of my oldest drives are only 2TB models... I could replace those with 6TB versions too... that would put me in a better position.

When you "add" a drive that space is added to the Pool.  When you "remove" a drive the contents will be moved from that drive to other drives (assuming you have enough space, which you will as you just added a new bigger drive).  That is how I swap a drive out.

[Sparks67]

However, I want to ensure that it will handle 4k video bandwidth requirements so that I have future proofed and don't need to reinvest again (however if 4k movies are going to use 100gb of disk space I guess that begs the question of when will hdd $/gb be low enough).

UHD Bluray is only based on Hevc 5.1 of the main tier.  https://en.wikipedia.org/wiki/High_Efficiency_Video_Coding_tiers_and_levels  As you review the chart, then you will notice more tier levels.  I suspect you will see download format or a larger size disc for UHD bluray for the other tier levels.  The telecommunication companies are developing a large fiber network for HEVC. 

 

I am thinking of using an Addonics hardware solution that allows me to choose a case, install their port multiplier and connect to a NAS adapter via a single USB 3.0, allowing jbod mode over the network. I can then use a USB hub to expand when I need to. And I don't need to build a separate server, get to grips with a NAS software and maintain it.

If you want an expandable system, then you might want to review this article about Sas Expanders.  http://www.sasexpanders.com/  Actually, I have (2) 15 bay hard drive cases, and I just bought a HBA for my HTPC.  Then I just connect it with the Sas Expander cables.   

I just use Windows 10 Pro with SnapRaid. 

[Z0001]

Lets break it into several parts (and these can be separate or all on one PC pending what you want to do).
- File Store : Hosts the actual media and shares it to the Network.  Needs to have plenty of SATA Ports, Bays, with good throughput.  Does not need a high end CPU or GPU.  Could be a PC, NAS, or Direct Attached Storage (eg box with a Sata or USB connection).  From my experience many of the cheaper NAS or Attached Storage boxs have poor throughput or unreliable connections.  Does not

- MC Server : Runs MC and shares the Database to MC Clients.  It will also be the box that will do any of the "Transcoding" if needed by the clients.  If transcoding you will need a strong CPU.  I have a i7-8700K Overclocked and it can transcode all my UHD with the excpetion of Billy Lynn's Long Halftime Walk (which is 4k/60p - eg much higher frame rate than 46/24p on most discs).  Hendrik is currently also looking at adding GPU Transcoding and if this happens in MC then this PC will need a strong GPU.

- MC Client / HTPC : Runs MC as a Client, and will Load the Library from the MC Server, and then play the media from the File Store.  Will need a strong GPU if using ROHQ (madVR) and a reasonable CPU.

As I said you can have all or some of these on One PC, (or two, or three etc).

Looks like my best option is going to be building a MC server housing the discs and building a new "main" HTPC. In a year or so I'd get a second HTPC for the kids.

My questions:

How reliably can the server serve say two files at the same time?

If the server was performing other duties, eg serving to one or two HTPCs and at the same moment playing back or being used as a workstation, would this all run smoothly? What CPU would the server/workstation need?

Thanks everyone!
Z

[jmone]

I've been using my MAIN pc in the config with years without issue.  It it is just serving files you are good to go.  The only limit is the speed of your HDD and NW connection.  If on a 1GBit network you will be able to serve over 10 streams (pending the bit rate of the videos).  There is also very little CPU being used so simultaneous use as a normal workstation also has no impact. 

If you are using MC to transcode video on the fly for consumption by removes, DLNA devices etc then your CPU is going to get hammered.  I tested this a few years ago and with BD could get 3-4 streams going before my PC's CPU started to max out.  Since then I've got a better CPU (i7-8700k) and it is also fine transcoding UHD BD.... with the excpetion of Billy Lynn which is a 60fps UHD (this may improved at some point as Hendrik say this is currently single threaded but may change in the future.  So for now you want a fast CPU)

[Z0001]

All you need is a the "right" drive and SW to rip the UHD.  Once ripped it is just another HEVC Video and there is no constraint on what Mobo/CPU/OS/SW needed.  Have a read of the FAQ on this - https://yabb.jriver.com/interact/index.php/topic,112653.0.html and follow the links for more info.

Just following up on this. If I rip my discs to mkv is there any hdcp 2.2 or other 4k DRM in the way? Any handshakes between screen and graphics I need to know about? Is Microsoft playready 3.0 an issue?

Seems to be a lot to consider when embarking on a 4k capable system!

Thanks all
Z