Topic: Use a 4K capture card to tone map all sources with JRVR(?) Which cards will work (Read 8760 times)

pixelpusher15 · « **on:** January 20, 2022, 09:29:22 am »

I have been slowly planning and building a HTPC that was to eventually be used for an MadVR Envy like system. I was going to use Video Processor + MadVR to do it.
VideoProcessor link: http://videoprocessor.org/

Well I just tested JRVR a bit it it might be good enough if JRiver allows easy use with capture cards. The thread below is what piqued my interest:
https://yabb.jriver.com/interact/index.php?topic=130584.0

If my understanding is correct it looks like it might "just work" with a DirectShow capable card...(??) If this is the case the JRiver is a much simpler solution and may also allow the use of better cards. Right now the only real options with VideoProcessor are BlackMagic DeckLink cards. $200 for a 4K30 capable cards and $550 for a 4K 60 card. What I'm wondering about is cards like the AverMedia LiveGamer 4K, or Elgato 4K60 Pro MK.2. Both of these cards are in the $200-250 range for 4K60 capability. But, there's always a but.... I believe the developer of VideoProcessor said that there's some funny business going on with how these cards work. IIRC it is that they may convert HDR to SDR when sending the stream to another app. I'm not sure about this and I don't have the technical knowhow to figure it out without just buying them and testing.

I'm hoping there are others here that know more than me or that have tested this. I also would find it to be awesome if this thread could be a place where we document what works and what doesn't. Tonemapping external sources well would be an amazing feat in my mind and if it can be done with JRVR it would add a ton of value IMO. As of now it looks like the Magewell cards work, but they are quite pricey.

mattkhan · « **Reply #1 on:** January 20, 2022, 10:03:19 am »

The only way to know is to try it unfortunately

I have a different magewell device and it does not work well with jriver TV feature (but does work fine with other software)

Yaobing · « **Reply #2 on:** January 20, 2022, 12:02:43 pm »

The answer is yes, generally speaking. If the device supports DirectShow, it should work. However, we do have to implement quite some specific code to support some individual devices, and we do have a mechanism for adding new code to support new devices.

We have code to support at least one BlackMagic device. BlackMagic devices are a bit of strange in some aspect.

You need to try your devices and see how well they work. If they don't work, let us know and we might make some changes to make it work.

Yaobing · « **Reply #3 on:** January 20, 2022, 12:35:11 pm »

I will add that the video captured on disk is uncompressed video, unless the capture card itself has hardware compression. The capture is written in JRiver TV format, JTV, which can be converted to another file format after the capture.

pixelpusher15 · « **Reply #4 on:** January 20, 2022, 04:59:02 pm »

Quote from: Yaobing on January 20, 2022, 12:35:11 pm

I will add that the video captured on disk is uncompressed video, unless the capture card itself has hardware compression. The capture is written in JRiver TV format, JRV, which can be converted to another file format after the capture.

This bit tripped me up a bit. But I think get it now. I’m not wanting to record the incoming feed. Just view it and apply tone mapping to it. Is this possible or do I have to record it first? I have zero interest in recording.

JimH · « **Reply #5 on:** January 20, 2022, 06:36:44 pm »

There's little difference between live and recorded, from a user point of view.

Yaobing · « **Reply #6 on:** January 20, 2022, 08:40:26 pm »

JRiver Media Center can either record a show to disk or render the show on screen live. I don't know how Video Processor work, so I can't comment on that.

jmone · « **Reply #7 on:** January 20, 2022, 11:06:14 pm »

I had a bit more of a play using my Blackmagic Ultrastudio 4K to forward a video/audio feed from an NVidia Shield to MC.... and it works..... but there are some limits to think about:
- HDCP: I could not (as expected) play any protected content, you will need an HDCP stripper for such content. No issue with non protected content.
- Video Output Format: You will need to find a capture card to output/passthrough the Video Formats (incl HDR) that you want. My BM WDM video output seems to only offer UYVY, 8-Bit, 4:2:2 (at various resolutions / framerates) regardless of what the source is, looks good but you then don't get any HDR tonemapping in JRVR/madVR etc.
- Audio Output Format: You will need to find a capture card to output/passthrough the Audio Formats you want.
- Changing Refresh Rates: MC will let you change refresh rates of your monitor on starting a playback but, you can not do this dynamically once the video playback/capture has started. This can be an issue if the source device uses one refresh rate for it's menu etc then a different one for the actual video. You will probably want to pre-set the display frequency before commencing.

pixelpusher15 · « **Reply #8 on:** January 21, 2022, 06:42:30 am »

I emailed AverMedia support asking if they supported HDR with other applications and they said no. I think this is a known thing based on some older conversations on AVS. I believe the HDR metadata is stripped out so everything appears washed out. VideoProcessor apparently maintains this when working with BlackMagib cards since it is directly using the SDK. From my understanding VP and JRiver don't work together as of now. But, what I'm now wondering is if I can add back in HDR metadata via an HDFury device. My Vertex2 allows for custom HDR to be injected. I could easily do this but it would be the same for each piece of content. I can also control it with my Harmony remote. I already have to switch picture modes for HDR so this could just be another command, turn the custom HDR on/off based on HDR or SDR. I thought this might work since it is my understanding that JRVR and MadVR pretty much ignore the metadata when doing their DTM goodness.

There are a few assumptions and "my understandings" here so all of this could fail. I do have a BlackMagic DeckLink Mini 4K coming today so I can start testing with that. It doesn't support 4K60 though.

The framerate handling is a concern for me. I'm not sure what to do about that as of now.

bogdanbz · « **Reply #9 on:** January 27, 2022, 08:11:17 am »

I don't believe there are any 4K video capture cards that support capture of HDR content at the full HDMI 2.1 bandwidth of 48 GBps (or at least 40 GBps), so it's likely better to wait for availability of such cards first.

I also considered using a 4K capture card to perform real-time color correction in software, since I switched from a LG OLED to a Sony OLED TV that has no uploadable hardware LUT for this. The plan was to redirect the graphic card output to one of its inputs, do the color correction, and use the capture card output to drive the TV display. Other brands you may look into besides BlackMagic and AVerMedia are Magwell, Elgato, Datapath Vision.

pixelpusher15 · « **Reply #10 on:** July 18, 2023, 08:16:41 pm »

Coming back to this...

I got a BlackMagic Quad 4K HDMI recorder and was able to get it to work with JRiver. I able to open the device and view my sources with minimal latency (audio is in sync). My issue, as of now, is that I'm unable to tone map HDR which is my whole point of wanting to do this. Is it possible for JRVR to work with capture inputs - or - am I missing something?

This is on version 31 which I know this is now in the wrong forum but I figured I'd keep this thread going...

mattkhan · « **Reply #11 on:** July 19, 2023, 04:20:42 am »

Quote from: pixelpusher15 on July 18, 2023, 08:16:41 pm

Coming back to this...

I got a BlackMagic Quad 4K HDMI recorder and was able to get it to work with JRiver. I able to open the device and view my sources with minimal latency (audio is in sync). My issue, as of now, is that I'm unable to tone map HDR which is my whole point of wanting to do this. Is it possible for JRVR to work with capture inputs - or - am I missing something?

This is on version 31 which I know this is now in the wrong forum but I figured I'd keep this thread going...

JRVR is used for capture card playback, what makes you think it isn't being used? you should be able to check the debug osd to see what is going on

perhaps the issue is that it doesn't see the source as HDR? does other software see the relevant metadata on the video stream? or is there some blacklink utility that can show what it is presenting? if it's present, it's possible MC just doesn't read that info from the stream

Hendrik · « **Reply #12 on:** July 19, 2023, 06:47:54 am »

That would be my guess, that either the card doesn't present metadata, or whatever part is reading the image from the card isn't getting the metadata. I think Yaobing is dealing with capture input, maybe he can shed some light if metadata about the image is read at all.

You could make sure JRVR is running by turning on the OSD (Ctrl-J) and then also posting a screenshot of that. If all the input data is unknown, then its likely not reading anything from the card.

pixelpusher15 · « **Reply #13 on:** July 21, 2023, 12:10:43 pm »

The card is presenting the metadata. i can confirm this by using VideoProcessor (see attached). And yup, it looks like MC isn't reading in the HDR metadata but JRVR is being used. (see attached)

claw · « **Reply #14 on:** July 22, 2023, 11:35:13 pm »

Dennis had to write code to get HDR10 metadata for Video Processor to send to madVR. The DeckLink cards don't forward the metadata. Before he did that he created a metadata injector to send hard-coded metadata values. It might be possible to add it as a plug-in filter in JRiver. I assume that JRVR does not rely on accurate HDR10 metadata values.

https://github.com/defl/directshow_metadata_injector_filter

claw · « **Reply #15 on:** July 22, 2023, 11:38:21 pm »

Quote from: pixelpusher15 on July 18, 2023, 08:16:41 pm

Coming back to this...

I got a BlackMagic Quad 4K HDMI recorder and was able to get it to work with JRiver. I able to open the device and view my sources with minimal latency (audio is in sync). My issue, as of now, is that I'm unable to tone map HDR which is my whole point of wanting to do this. Is it possible for JRVR to work with capture inputs - or - am I missing something?

This is on version 31 which I know this is now in the wrong forum but I figured I'd keep this thread going...

I have tried and failed to get JRiver to work with a BlackMagic DeckLink Mini. Could you describe how you did it with the Quad?

saltanar · « **Reply #16 on:** July 23, 2023, 07:03:33 am »

I’m following this thread because I’m also very interested to use JRVR as video processor with HDMI 4k capture card.

pixelpusher15 · « **Reply #17 on:** July 24, 2023, 10:07:46 am »

Quote from: claw on July 22, 2023, 11:38:21 pm

I have tried and failed to get JRiver to work with a BlackMagic DeckLink Mini. Could you describe how you did it with the Quad?

hey claw, this is how I have it setup. I forgot if there were some other things I did. TBH, I had a hard time too and ended up just clicking random things until it worked, lol

Let me know if you have questions about other settings

mattkhan · « **Reply #18 on:** July 24, 2023, 10:32:44 am »

Does it work with timeshifting? ie can you pause and unpause ok

pixelpusher15 · « **Reply #19 on:** July 24, 2023, 12:21:12 pm »

Quote from: mattkhan on July 24, 2023, 10:32:44 am

Does it work with timeshifting? ie can you pause and unpause ok

I'm having an issue where the image is off center (down and to the right) but yes, pause and unpause work well.

claw · « **Reply #20 on:** July 24, 2023, 09:05:22 pm »

Quote from: pixelpusher15 on July 24, 2023, 10:07:46 am

hey claw, this is how I have it setup. I forgot if there were some other things I did. TBH, I had a hard time too and ended up just clicking random things until it worked, lol

Let me know if you have questions about other settings

I was able to get JRiver 31 to recognize the DeckLink card as a Camera device. Sending it 4K23 input and the picture is smooth.

I never got it to work with version 28. Now to figure out the JRVR tone mapping options.

I will probably place another HDfury device between the RTX3080Ti output and the LG OLED I am testing with in order to get better signal information about the output.

Right now it seems that JRiver is converting 4K23 input to 4K60 output.

mattkhan · « **Reply #21 on:** July 25, 2023, 12:55:28 am »

So basically the request to jriver is to add whatever the minimum required metadata is to enable jrvr to see it as HDR and allow DTM to activate? I guess ideally you would want to be able to switch this on the fly (i.e. content could change from SDR to HDR externally) via some keyboard shortcut/mcws call?

jsmith967 · « **Reply #22 on:** July 25, 2023, 01:06:15 am »

I am also very interested to get this working with a capture card to allow JRVR to process the signal and output it.

claw · « **Reply #23 on:** July 25, 2023, 02:41:09 am »

Quote from: mattkhan on July 25, 2023, 12:55:28 am

So basically the request to jriver is to add whatever the minimum required metadata is to enable jrvr to see it as HDR and allow DTM to activate? I guess ideally you would want to be able to switch this on the fly (i.e. content could change from SDR to HDR externally) via some keyboard shortcut/mcws call?

The BlackMagic DeckLink cards do not pass on any HDR metadata.

JRVR status screen reports:
Primaries: Unknown
Transfer: Unknown

Perhaps an option that would force JRVR to see the input as:
Primaries: BT2020
Transfer: PQ

I don't think that luminance values such as Max/Min Luminance, MaxCLL, and MaxFall should be needed since I assume JRVR can tone map HDR content that has zero values. There are quite a few HDR discs that have zero values.

mattkhan · « **Reply #24 on:** July 25, 2023, 02:49:48 am »

Yes that's what I am saying the feature request to jriver is, provide a way to specify just enough info on the stream to force jrvr tonemapping to activate.

I guess that lies in Yaobing's area to deliver rather than Hendrik but they would have to comment on that.

FenceMan · « **Reply #25 on:** July 25, 2023, 04:08:57 am »

Is this currently working with non HDR sources, decklink mini and JRVR?

claw · « **Reply #26 on:** July 25, 2023, 01:58:35 pm »

Quote from: FenceMan on July 25, 2023, 04:08:57 am

Is this currently working with non HDR sources, decklink mini and JRVR?

Looks like the DeckLink cards don't pass Primaries and Transfer Function for SDR either. Although the SDR output looks OK; perhaps defaulting to SDR input. (Note that I had JRVR configured to output BT2020)

Yaobing · « **Reply #27 on:** July 27, 2023, 11:16:29 am »

Quote from: claw on July 24, 2023, 09:05:22 pm

I was able to get JRiver 31 to recognize the DeckLink card as a Camera device. Sending it 4K23 input and the picture is smooth.

The reason why such devices are recognized as cameras is because they do not provide programming interfaces for us to choose which video output line to use. You have to go to the device's own set up or configuration page to set it up. Likewise, might there be a setting there too to let it output metadata so the downstream DirectShow filter JRVR could process? To be honest I don't have much knowledge on this.

claw · « **Reply #28 on:** July 27, 2023, 06:16:21 pm »

Quote from: Yaobing on July 27, 2023, 11:16:29 am

The reason why such devices are recognized as cameras is because they do not provide programming interfaces for us to choose which video output line to use. You have to go to the device's own set up or configuration page to set it up. Likewise, might there be a setting there too to let it output metadata so the downstream DirectShow filter JRVR could process? To be honest I don't have much knowledge on this.

Pretty sure there is not any configuration setting for this. Seems to be an issue with all of the avaliable 4K capture cards. While they may support HDR recording, they don't seem to include HDR metadata in pass through mode.

What I think might work is if JRiver/JRVR supported adding DirectShow Filter plug-ins in Advanced...Custom...JRVR mode. The filter might be able to use the Black Magic API to obtain the metadata, or it could just hard code the metadata which would work with most any capture card. The filter would need to be called before JRVR is.

I previously linked to a Directshow Filter that hard codes HDR metadata including the missing Primaries and Transfer Function.

https://github.com/defl/directshow_metadata_injector_filter

Never mind about the metadata injector filter I linked to. It was written to work with madVR only...

The filter will work regardless. Whatever you tell it, it will inject. It cares not what the source is (but it cares about the sink which has to be madVR).

Hendrik · « **Reply #29 on:** July 28, 2023, 03:12:15 am »

I looked at it, and to me the filter looks like its not dependent on madVR, it was just made for madVR because at the time it was the only one to understand this kind of metadata - but since I defined that metadata more or less for LAV, of course JRVR understands the same.

Technically we allow adding extra filters into the DirectShow chain - if that works with live input however ... I don't have a setup to test.
For MC31.0.40 and up, I added the injector to our video post-processor filter whitelist, maybe we can get that working somehow.

claw · « **Reply #30 on:** July 28, 2023, 02:15:49 pm »

Quote from: Hendrik on July 28, 2023, 03:12:15 am

I looked at it, and to me the filter looks like its not dependent on madVR, it was just made for madVR because at the time it was the only one to understand this kind of metadata - but since I defined that metadata more or less for LAV, of course JRVR understands the same.

Technically we allow adding extra filters into the DirectShow chain - if that works with live input however ... I don't have a setup to test.
For MC31.0.40 and up, I added the injector to our video post-processor filter whitelist, maybe we can get that working somehow.

Thanks.

I did not notice anything madVR specific when I looked at the code. I was just going by the author's comment.

When .40 is available for download I will take a look.

JimH · « **Reply #31 on:** July 28, 2023, 02:17:13 pm »

Quote from: claw on July 28, 2023, 02:15:49 pm

When .40 is available for download I will take a look.

It's here: https://yabb.jriver.com/interact/index.php/topic,136599.msg947065.html#msg947065

jsmith967 · « **Reply #32 on:** July 28, 2023, 05:06:40 pm »

Quote from: claw on July 28, 2023, 02:15:49 pm

Thanks.

I did not notice anything madVR specific when I looked at the code. I was just going by the author's comment.

When .40 is available for download I will take a look.

Please let us know what you find out!

claw · « **Reply #33 on:** July 28, 2023, 08:17:29 pm »

Quote from: Hendrik on July 28, 2023, 03:12:15 am

I looked at it, and to me the filter looks like its not dependent on madVR, it was just made for madVR because at the time it was the only one to understand this kind of metadata - but since I defined that metadata more or less for LAV, of course JRVR understands the same.

Technically we allow adding extra filters into the DirectShow chain - if that works with live input however ... I don't have a setup to test.
For MC31.0.40 and up, I added the injector to our video post-processor filter whitelist, maybe we can get that working somehow.

I was able to get the metadata injector filter to affect the metadata when playing file-based media. But not from the capture card.

I registered the filter in Windows 10 and added the filter as a Video post processor filter in Custom JRVR mode. Screen shot attached.

I took a screen shot of the JRVR statistics when playing back an HDR video in Red October JRVR mode. (I didn't have an SDR media file to test with)

I then took a screen shot of the stats when playing the same HDR video in Custom JRVR mode with the filter.

The setting values I configured in the directshow_metadata_injector_filter.ini configuration file were applied overriding the original values for:

System (BT.709 instead of BT.2020)
Primaries (DCI-P3 instead of BT.2020)
Transfer (2.2 Gama instead of PQ HDR10)
Levels (Full/PC instead of Limited/TV)

I was never able to get the HDR10 Nits value to change. It would be reported as zero when I did not override the transfer from PQ.

Finally, I took a screen shot of the stats when playing an HDR video through the BlackMagic DeckLink 4K Mini Recorder capture card. No metadata values from the metadata injector filter were applied. So either there was no attempt to apply the filter to capture card input, or it failed.

Note that I will redo this test using an SDR file as a more realistic use of the filter.

Hendrik · « **Reply #34 on:** July 29, 2023, 02:03:24 am »

Yaobing will likely have to look into if we can use the custom filters for live input.

saltanar · « **Reply #35 on:** July 29, 2023, 05:52:54 am »

is interesting to know if we can also use Dirac or similar plugins on audio part and let JRiver to sync audio and video

Hendrik · « **Reply #36 on:** July 29, 2023, 06:16:44 am »

With live input maintaining sync with extensive processing is not that easily done. With a file you can just read ahead.

saltanar · « **Reply #37 on:** July 29, 2023, 07:39:45 am »

yes but I know that the Dirac plugin of the Dirac Live processor already informs the host of the latency, if JRVR will do the same we can have a perfect sync.

jsmith967 · « **Reply #38 on:** July 31, 2023, 08:04:27 am »

This is probably a naive question.

In order to accomplish the goal of input HDR signals and applying processor: Instead of trying to get JRiver to work with Decklink capture card, what about getting JRVR to work with VideoProcessor? I suspect JRiver only wants users to use JRVR in JRiver but thought that if it is a direct show renderer, one could just use it in VideoProcessor like Madvr and then JRiver does not have to code to figure out how to get Decklink capture to work properly.

jsmith967 · « **Reply #39 on:** August 04, 2023, 04:54:46 am »

No one? I would appreciate some insight!

mattkhan · « **Reply #40 on:** August 04, 2023, 07:05:58 am »

Quote from: jsmith967 on August 04, 2023, 04:54:46 am

No one? I would appreciate some insight!

all that is needed is for jriver is inject the required metadata in when playing back from a capture card. It makes absolutely zero sense to even think about doing that via some 3rd party app that exists purely to plug madvr together with a filter that injects that metadata which is probably why you got no comments.

jsmith967 · « **Reply #41 on:** August 04, 2023, 07:24:13 am »

Thank you for the reply.

OK. You make that sound easy. To my knowledge, JRiver does not inject this metadata when playing back from a capture card yet (as the posts above indicate). Perhaps you're suggesting that since it is so easy, it will possibly happen in the future? If so, that is good news! I was just suggesting possibly an alternative that would work right now, which sounds like it will not.

Quote from: Yaobing on July 27, 2023, 11:16:29 am

The reason why such devices are recognized as cameras is because they do not provide programming interfaces for us to choose which video output line to use. You have to go to the device's own set up or configuration page to set it up. Likewise, might there be a setting there too to let it output metadata so the downstream DirectShow filter JRVR could process? To be honest I don't have much knowledge on this.

@Yaobing, Any ETA on when JRiver will inject the required HDR metadata when playing back from a capture card?

mattkhan · « **Reply #42 on:** August 04, 2023, 07:39:51 am »

I couldn't say if it is actually easy but it doesn't sound so complicated to implement, to test and verify it behaves is another question. They don't do ETAs though so just have to wait and see.

claw · « **Reply #43 on:** August 04, 2023, 09:30:02 am »

Quote from: jsmith967 on August 04, 2023, 04:54:46 am

No one? I would appreciate some insight!

Your question was asked early in the development of JRVR.

https://yabb.jriver.com/interact/index.php/topic,130657.msg906609.html#msg906609

Quote from: zybex on September 24, 2021, 11:41:09 am
@Hendrik, is JRVR [eventually] usable by other apps like MPC-HC?

No. Its a core component of Media Center.

I would suspect that JVR is not DirectShow since JRiver is a cross platform product, not just for Windows.

Besides, JRiver is a licensed and paid product and JRVR has increased its value. Separating JRVR so that it can be used with other products means JRiver would end up competing with itself in other free-to-use products.

jsmith967 · « **Reply #44 on:** August 11, 2023, 02:33:09 pm »

I was assuming that one would have to pay for JRiver in order to have access to the JRVR. But I suspect you're right and so it is moot, JRVR is probably not a directshow filter anyway so it would not work.

I guess I'll have to wait to purchase until they get working the capture issue in JRiver.

jsmith967 · « **Reply #45 on:** March 05, 2024, 07:14:48 am »

I was wondering if there had been any developments in order to use JRVR with external inputs with the capture card. As this thread indicates, the issue was injecting the relevant metadata so that it would be seen as HDR. I'm sure there are other issues to work out but that was the most recent. I am looking for a replacement for Videoprocessor that the first post mentioned.

Thanks for your work.

FenceMan · « **Reply #46 on:** April 29, 2024, 09:22:35 am »

Just curious if there has been any progress with this or if it is basically a dead issue?

jsmith967 · « **Reply #47 on:** April 29, 2024, 09:44:23 am »

I second this, as I am very interested in using JRiver to do the tone mapping from capture cards.

danbez · « **Reply #48 on:** May 30, 2024, 01:35:38 pm »

Just found this thread. @Hendrik Perhaps it should be moved to MC 32 since 31 is now legacy?

Right now this idea is even more interesting as we can get devices that will output full DV Profile 7 with FEL included. So if we can plug this device to the capture card + JRVR we can achieve the best possible DV output and the best JRVR tone mapping solution.

And yes, right now I can do that with VideoProcessor + MadVR, but I prefer to not get stuck with a super old version of MadVR since all versions after v.113 are blocking the use of Capture Cards. (Concerns with cannibalizing Envy sales...).

All I need is an option that can I use to force HDR/PQ/BT.2020 or SDR/BT.709 as an input into JRVR. (output of the Decklink capture card).

mattkhan · « **Reply #49 on:** May 30, 2024, 02:00:44 pm »

I think MC support for capture cards needs some work above and beyond this, ime it's not reliable and can't handle audio and video in the way other windows apps can out of the box.

INTERACT FORUM

Author Topic: Use a 4K capture card to tone map all sources with JRVR(?) Which cards will work (Read 8760 times)