Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[Yaobing]

Users have requested that we support HDR tone mapping, and Hendrik said it was great if we can do it.

We support these devices using DirectShow, as WDM devices.  Blackmagic Design does not support HDR tone mapping in DirectShow directly, a user mentioned that someone used a "back-door" method to pass the HDR tone mapping to video renderer. I thought we might be able to do that too, as we use JRVR.

On the other hand, the company provides an SDK so we can use that too.  That means we redo part of our code and use the SDK functions instead of Microsoft functions.

Either way, I have not gained enough understanding to proceed further.

jmone said he could document current status of MC's support of such devices, what's working, and what's not.  So I create this thread for everyone to chime in.

[Hendrik]

For HDR, all you really need from the device is some metadata if the source is HDR or not, thats the bare minimum. Additionally, there might be some static HDR metadata that could be useful (brightness etc), but if its not present it would still work ok-ish.
If we can somehow obtain this, I can tell you how to format it to send it to JRVR.

[jmone]

MC Exposes 2 different "Devices" with my BMD UltraStudio 4K Mini:
- Decklink Video Capture, and
- Blackmagic WDM Capture

I have both added in the TV Section.  I "think" the Backmagic WDM Capture is the most recent as looking at the exposed option in the pins, it goes up to 4K, where the Decklink only goes up to 2K.

Let me know what you would like me to test.  eg, I can do short recordings if that helps, but what signal(s) do you want me to feed it?  I normally attach a Video Camera (that could output SDR, HLG, Raw) over HDMI or attach a MC Library Server client (and do SDR / HDR).

[jmone]

Did some testing feeding the BMD a UHD HDR Signal (from a MC Library Server Client). 
- I don't think there is any colour space metadata being sent.  I say this is even with capturing the stream in Resolve it does not tag it's recordings either (you have to manually assign the colour space Gamut and Gamma) to the clip.
- For some reason it is only showing it as an 8-Bit even if sending 10-Bit (in Resolve it can be captured in 10-Bit)
- Watching live playback is smooth (even though JRVR is reporting tons of empty queue counts)
- Recording in MC is not usable (guessing it is trying to stuff Uncompressed Video into the JTV Container without any on the fly compression), so you get maybe 1fps.

[jmone]

Here is the filter graph info:

Code: [Select]

Filter Graph Info (TV graph):

    Filter 'JRiver Audio Renderer'
        CLSID: {A4002F8E-510F-442C-8AD3-F9C7B23FB394}
        Host:
        Input Pin 'In'
            Connected to pin 'Output1' of filter 'Blackmagic WDM Capture C7A65834542C631B'
            Major type MEDIATYPE_Audio  Sub type MEDIASUBTYPE_PCM, Format type FORMAT_WaveFormatEx

    Filter 'JRVR'
        CLSID: {555E6803-14EF-4946-8646-B6EF9E9121E2}
        Host:
        Input Pin ''
            Connected to pin 'XForm Out' of filter 'AVIDecompressor'
            Major type MEDIATYPE_Video  Sub type MEDIASUBTYPE_RGB32, Format type FORMAT_VideoInfo

    Filter 'AVIDecompressor'
        CLSID: {CF49D4E0-1115-11CE-B03A-0020AF0BA770}
        Host: c:\windows\system32\quartz.dll
        Input Pin 'XForm In'
            Connected to pin 'Output0' of filter 'Blackmagic WDM Capture C7A65834542C631B'
            Major type MEDIATYPE_Video  Sub type Unknown GUID Name: {43594448-0000-0010-8000-00AA00389B71}, Format type FORMAT_VideoInfo
        Output Pin 'XForm Out'
            Connected to pin '' of filter 'JRVR'
            Major type MEDIATYPE_Video  Sub type MEDIASUBTYPE_RGB32, Format type FORMAT_VideoInfo

    Filter 'Blackmagic WDM Capture C7A65834542C631B'
        CLSID: {17CCA71B-ECD7-11D0-B908-00A0C9223196}
        Host: c:\windows\system32\ksproxy.ax
        Output Pin 'Output0'
            Connected to pin 'XForm In' of filter 'AVIDecompressor'
            Major type MEDIATYPE_Video  Sub type Unknown GUID Name: {43594448-0000-0010-8000-00AA00389B71}, Format type FORMAT_VideoInfo
        Output Pin 'Output1'
            Connected to pin 'In' of filter 'JRiver Audio Renderer'
            Major type MEDIATYPE_Audio  Sub type MEDIASUBTYPE_PCM, Format type FORMAT_WaveFormatEx

[jmone]

I think there are only two things that would make it "work" correctly as it stands:
- Colour Space:  Let the user in the TV HW Setup page select the colour space for the device (HDR / SDR)... or even let them create two (one for SDR, one for HDR).
- Encoding: Add a new HW accelerated encoding feature for recording.

Apart from that, the current setup using WDM Capture Device works well.  The above two features would also be good for any sort of live HDMI feed.

[jmone]

It would also be good to add in the JRVR Info Box under Input what the FPS is.  I can confirm that the JTV recording was correct (23.98 fps) for this BD capture.

[Yaobing]

Thanks Nathan!

Yes, video capture is done without any on-the-fly video compression, because we don't have a video compression filter.  The video compressor created by Matt more than 20 years ago was good for SD analog TV, but not for HD video, not to mention the 4k or higher videos.

[jmone]

You don't really want to play with uncompressed video , the bitrates are insane.... but we already have avcodec as part of MC's plugins.

I did a quick test and the following worked fine in capturing a live UHD stream from the WDM filter (with the correct frame rate automatically) that then plays just fine in MC.

Code: [Select]

ffmpeg -f dshow -i video="Blackmagic WDM Capture":audio="Line In (Blackmagic UltraStudio 4K Mini Audio)" out.mp4
It would need some tweaking for Format, Bitrate, Buffering and setting the colour space tagging etc, but it works simply enough.  I don't know if you can build it all in one graph, or if you would need to kick off the capture / compression in one thread, and the playback in another.

[jmone]

When I look at the options for the WDM filter using:

Code: [Select]

ffmpeg -f dshow -list_options true -i video="Blackmagic WDM Capture":audio="Line In (Blackmagic UltraStudio 4K Mini Audio)"
...it looks like there is nothing to set in the way of frame rate or resolution for the Video side (it matches what the Ultrastudio is being feed) but you can specify the Audio capture (though I got the 2ch by default anyway).

Code: [Select]

[dshow @ 0000021b1d3fd040] DirectShow video device options (from video devices)
[dshow @ 0000021b1d3fd040]  Pin "Output0" (alternative pin name "0")
[dshow @ 0000021b1d3fd040]   pixel_format=uyvy422  min s=3840x2160 fps=50 max s=3840x2160 fps=50
[dshow @ 0000021b1d3fd040]   vcodec=v210  min s=3840x2160 fps=50 max s=3840x2160 fps=50
[dshow @ 0000021b1d3fd040]  Pin "Output1" (alternative pin name "1")
[dshow @ 0000021b1d3fd040] DirectShow audio only device options (from audio devices)
[dshow @ 0000021b1d3fd040]  Pin "Capture" (alternative pin name "Capture")
[dshow @ 0000021b1d3fd040]   min ch=1 bits=8 rate= 11025 max ch=2 bits=16 rate= 44100

[Yaobing]

ffmpeg -f dshow -i video="Blackmagic WDM Capture":audio="Line In (Blackmagic UltraStudio 4K Mini Audio)" out.mp4

So FFMPEG is handling the capture filter's output internally and convert to desired output.  It appears that we can not take advantage of that in our current system because we are not controlling the output.

We will have to create something new for this.

[jmone]

Pretty much.  It is just an example of what could be done to record the uncompressed HDMI feed to a std rendered H264(or H265, AV1 file) in real time that then use would be used for normal playback / time shifting.  You can also specify encoding settings like resolution, bitrate etc etc.  There are also flags for taging the file with Colour Space info (BT709, BT2020 etc). 

It would be worth chatting to Hendrik, as for all I know this can all be done using the existing avcodec plugin that does the on the fly video transcoding used for streaming to client devices. 

[Yaobing]

One approach would be to create a video compression filter, using the available library, like the reverse of LAV Video Decoder.

Another is to abandon the current Microsoft approach, and use Blackmagic API directly.  They use the COM too, but just not DirectShow.  Not sure if this is going to be easier than creating a stand alone compression filter, as we still need to deal with what to do with the video output we get from the device.

Creating a filter would seem better from the point of being able to use it with other video capture devices that we also support.

Hendrik, any thoughts?

[jmone]

I agree.  It could then be used with any device. 

FWIW, these are a range of these BMD devices (both PCI-E cards and thunderbolt connected boxes) and most of them don't have an on-board Video Encoder, so even with the API you would need to still encode the video somehow.

[Yaobing]

In build 37, just out on beta board, I added this:

 Changed: Added some exploratory logging on Blackmagic DeckLink video capture devices.

Please run the device and provide me a log file.

The relevant log lines contain this: "BlackmagicWDMCapture::ExploreDecklinkAPI"

[jmone]

Tried running it a few times on the "Blackmagic WDM Capture" but MC now just crashes (see log).  I'll rollback and see if it still works with the older version of MC.

[jmone]

Yup it is only Build 37 that crashes.  Build 36 is fine.

[jmone]

...and the crash happens immediately after the new "BlackmagicWDMCapture::ExploreDecklinkAPI" call.

0008847: 13328: TV: BlackmagicWDMCapture::ExploreDecklinkAPI: Start
0008867: 13328: TV: BlackmagicWDMCapture::ExploreDecklinkAPI: decklink device: UltraStudio 4K Mini, model UltraStudio 4K Mini
0008867: 13328: General: TopLevelExceptionFilter: Unhandled exception -- program crashing
0008867: 13328: General: TopLevelExceptionFilter: Message: 514, wParam: 0, lParam: 121701199

[Yaobing]

Is "UltraStudio 4K Mini" the device you tried  to use in MC?  Exactly the same?

[jmone]

I'm not sure I understand the question fully. 

I've setup the device in in MC prior to Build 37 in the devices section of TV and it works just fine, but that profile crashes on Build 37.  I'll try setting it up again in Build 37.

[jmone]

I just tried to do the setup again in Build 37 and that makes MC crash as well.

FWIW - Here is the DirectShow Filter into again from Build 36

Filter Graph Info (TV graph):

    Filter 'JRiver Audio Renderer'
        CLSID: {A4002F8E-510F-442C-8AD3-F9C7B23FB394}
        Host:
        Input Pin 'In'
            Connected to pin 'Output1' of filter 'Blackmagic WDM Capture C7A65834542C631B'
            Major type MEDIATYPE_Audio  Sub type MEDIASUBTYPE_PCM, Format type FORMAT_WaveFormatEx

    Filter 'JRVR'
        CLSID: {555E6803-14EF-4946-8646-B6EF9E9121E2}
        Host:
        Input Pin ''
            Connected to pin 'XForm Out' of filter 'AVIDecompressor'
            Major type MEDIATYPE_Video  Sub type MEDIASUBTYPE_RGB32, Format type FORMAT_VideoInfo

    Filter 'AVIDecompressor'
        CLSID: {CF49D4E0-1115-11CE-B03A-0020AF0BA770}
        Host: c:\windows\system32\quartz.dll
        Input Pin 'XForm In'
            Connected to pin 'Output0' of filter 'Blackmagic WDM Capture C7A65834542C631B'
            Major type MEDIATYPE_Video  Sub type Unknown GUID Name: {43594448-0000-0010-8000-00AA00389B71}, Format type FORMAT_VideoInfo
        Output Pin 'XForm Out'
            Connected to pin '' of filter 'JRVR'
            Major type MEDIATYPE_Video  Sub type MEDIASUBTYPE_RGB32, Format type FORMAT_VideoInfo

    Filter 'Blackmagic WDM Capture C7A65834542C631B'
        CLSID: {17CCA71B-ECD7-11D0-B908-00A0C9223196}
        Host: c:\windows\system32\ksproxy.ax
        Output Pin 'Output0'
            Connected to pin 'XForm In' of filter 'AVIDecompressor'
            Major type MEDIATYPE_Video  Sub type Unknown GUID Name: {43594448-0000-0010-8000-00AA00389B71}, Format type FORMAT_VideoInfo
        Output Pin 'Output1'
            Connected to pin 'In' of filter 'JRiver Audio Renderer'
            Major type MEDIATYPE_Audio  Sub type MEDIASUBTYPE_PCM, Format type FORMAT_WaveFormatEx

[jmone]

Here is the log of it working in Build 36

[JimH]

I'm not sure I understand the question fully. 

I've setup the device in in MC prior to Build 37 in the devices section of TV and it works just fine, but that profile crashes on Build 37.  I'll try setting it up again in Build 37.

Identical name?

[jmone]

Sure, but I'm not sure what string and in what area to compare too?  All I know is that the profile I'd set up in that worked now crashes MC Build 37 and I can't setup a new one in MC37 as it too crashes.

[Yaobing]

Thanks, you answered my question.

In DirectShow, MC picks up this device: "Blackmagic WDM Capture" which is so generic.

In Blackmagic DeckLink API, this device showed up before the crash: " UltraStudio 4K Mini".  There is no match.

That is a problem, as we can not identify the device in an easy way.  That, however, should not be the cause of the crash, which I am still trying to figure out.

[Yaobing]

Figured out why it crashed.  I emailed you an updated DLL.  It should prevent crashing.  It will not give us any other info because we are not able to have matching devices names.

Emailing failed for some reason.  Here is the file:

[jmone]

Crashing is fixed   Log attached of it playing the device.

[jmone]

You may be able to do what you want over dshow anyway?  Had a look at the options that are exposed (using ffmpeg).

I'm using these devices (that work in MC now):

[dshow @ 000001e92068cf00] DirectShow video devices (some may be both video and audio devices)
[dshow @ 000001e92068cf00]  "Blackmagic WDM Capture"
[dshow @ 000001e92068cf00]     Alternative name "@device_pnp_\\?\decklink#avstream#9&1d8ec6ee&0&0#{65e8773d-8f56-11d0-a3b9-00a0c9223196}\decklinkcapture1"
[dshow @ 000001e92068cf00] DirectShow audio devices
[dshow @ 000001e92068cf00]  "Line In (Blackmagic UltraStudio 4K Mini Audio)"
[dshow @ 000001e92068cf00]     Alternative name "@device_cm_{33D9A762-90C8-11D0-BD43-00A0C911CE86}\wave_{D1D418E7-A7DA-47EB-8662-28E939E16C24}"

But don't use these ones as MC does not seem to get a feed (just a blank screen)
[dshow @ 000001e92068cf00]  "Decklink Video Capture"
[dshow @ 000001e92068cf00]     Alternative name "@device_sw_{860BB310-5D01-11D0-BD3B-00A0C911CE86}\{44A8B5C7-13B6-4211-BD40-35B629D9E6DF}"
[dshow @ 000001e92068cf00]  "Decklink Audio Capture"
[dshow @ 000001e92068cf00]     Alternative name "@device_sw_{33D9A762-90C8-11D0-BD43-00A0C911CE86}\{AAA22F7E-5AA0-49D9-8C8D-B52B1AA92EB7}"

[jmone]

...and this is what I see for the cobo that works now in MC:

ffmpeg -f dshow -list_options true -i video="Blackmagic WDM Capture":audio="Line In (Blackmagic UltraStudio 4K Mini Audio)"
[dshow @ 000002cae903cfc0] DirectShow video device options (from video devices)
[dshow @ 000002cae903cfc0]  Pin "Output0" (alternative pin name "0")
[dshow @ 000002cae903cfc0]   pixel_format=uyvy422  min s=3840x2160 fps=50 max s=3840x2160 fps=50
[dshow @ 000002cae903cfc0]   vcodec=v210  min s=3840x2160 fps=50 max s=3840x2160 fps=50
[dshow @ 000002cae903cfc0]  Pin "Output1" (alternative pin name "1")
[dshow @ 000002cae903cfc0] DirectShow audio only device options (from audio devices)
[dshow @ 000002cae903cfc0]  Pin "Capture" (alternative pin name "Capture")
[dshow @ 000002cae903cfc0]   min ch=1 bits=8 rate= 11025 max ch=2 bits=16 rate= 44100

[jmone]

...and here is the loooong list of options for the generic "Decklink" one (that do not work in MC currently).

[Yaobing]

I am still not able to make a name match (or model) between the API name and the DirectShow name.  So I created a DLL to use the first device enumerated by the API for the purpose of exploring its capabilities.

[jmone]

- This seems to work.  Here is the log file that shows not only the device enumeration but also Playback of the two exposed BM "devices" from the UltraStudioMini4K (same as before):
- Blackmagic WDM Capture : Works as usual - shows the image & sound
- Decklink Video Capture : Black Screen

[jmone]

You are still going to need some encoding filter to make timeshifting / recording possible.  Most of these BMD devices (except some of the higher end models) do not have an onboard HW encoder. 

[Yaobing]

Playback of the two exposed BM "devices" from the UltraStudioMini4K (same as before):
- Decklink Video Capture : Black Screen

This is because I never put code in to explicitly support this device. That means it will load up generic code, instead of Blackmagic specific code. Now this is added.



Please test the updated DLL.  I also refined some of the exploratory logging entries.

[jmone]

Here you go, same test as before will a log.  Similar results.

I did get the Decklink Video Capture to move past "Starting TV" on one occasion (did not get a log) and not sure how as I could not repeat it.  On that occasion, I got audio but a black video that was reported as something like 576i (I think), rather than the UHD it was being sent. 

[Yaobing]

Thanks.

DeckLink Video Capture filter could not be rendered, according to the log.  I am still trying to determine why, but there is nothing else that got logged.

Have you used GraphEdit or GraphEditPlus before?  If so and if you still have it on your computer, can you try it? For this device, you need to add two filters, "DeckLink Video Capture" and "DeckLink Audio Capture".  Both filters expose an output pin called "Capture".  See if you can render from these pins in GraphEdit.  The video capture filter also has two other output pins, and I don't know what they are for.

[jmone]

Ahhh, the "good 'ol days" of graphstudio!  Had a good play:

- Decklink:  EDIT: See later posts, it can be made to work!  I could only get Audio to work.  Always a blackscreen.  Tried both 1080p and UHD as I notice in the (see pic) DS options there is no ability to pick anything above 2K.  I know that everything has to be set to the same format (Input Device, UltraStudio, and Renderer) or it fails....

- BMD WDM:  This one just works, and you don't have to set anything on the UltraStudio or in the WDM drivers (infact there is no options at all), it will put it into the correct mode each time the graph is created (eg UHD or 1080p) based on the incomming signal.  It's also one filter for both Video and Audio capture.  Nice.

Can you explain what the issue is of using the BMD WDM filters over the Decklink ones?  The WDM ones "just work" already and seems to be the one for "consumption" rather than the Decklink ones that are more for studio work.

[jmone]

...and you don't have to worry about the other two pins which are:
- VANC (Vertical Ancillary data) : Additional meta data embedded into the SDI stream for specific broadcast and cinema camera setups (you can ignore)
- TimeCode : Used to match a multi camera setup to avoid drift from different clocks in each (you can ignore)

[jmone]

Got "Decklink" working at 1080p, with the following settings:

[jmone]

Still no luck getting Decklink to work in MC even at 1080p.  I think the issue is, even if I manually set the Desklink Filter, it seems to always revert back to the default (see pic) once playback is attempted.

[Yaobing]

That graph looks simple enough. I figured out (sort of) why it is not playing in MC.

Code: [Select]

0034766: 42708: TV: CMainTVManager::PlayDefaultChannel: Start
0034766: 42708: TV: CMainTVManager::PlayDefaultChannel: Finish (0 ms)

I would expect logging lines between the two lines above, but none was found. That means the graph-building is skipped entirely. The only explanation would be that the device exposes an interface that led MC to treat it as an "external device" - "The IAMExtDevice interface controls an external device, such as a DV camera or video tape recoder [sic](VTR)." per Microsoft.

That is a bit surprising. The DLL below should confirm whether that is the case. We had code to deal with DV camcorders, but it was about twenty years ago. Right now I am not sure if it still works, but the code (or part of it) is there. I may have to make a special case not to treat "DeckLink Video Capture" as an external device.

[jmone]

I can't download that new DLL, I get a 404 - File not found.

[JimH]

Try this one:


It had a double JR.  I'm just guessing this is the right one.

[Yaobing]

Sorry, my bad. Link fixed.

[Yaobing]

This DLL forces Blackmagic devices not to be an "External Device":

[jmone]

Nice work, getting there.  The playback graph seems to get built

The log should show in order:
- 1 x BMD WDB Play: Works as usual, got both video and sound
- 2 x Decklink Plays: Works, but the input video format is not set correctly and reverts to 525i @ 59.94 and not the 1080p/50p signal I was feeding it.  I do get the playback window but the video is blank as a result.  Sound works fine.  I tried setting it in the DS Filter's pop up but it does not "stick" between sessions, and changing it once the graph is setup does not work. 

Looking at the Decklink API doco, I think you can check what the input signal is and set the device accordingly before building the graph?

[Yaobing]

Thanks.

It is odd that the format resets to SD video.  The graph dump shows that SD format was used for connection.  I will try setting the format.

[Yaobing]

Looking at the Decklink API doco, I think you can check what the input signal is and set the device accordingly before building the graph?

The IDeckLinkInput interface has methods to iterate through all supported display modes, but I can not find a method that tells me which format it is currently using.  Maybe it expects the apps to set it.

For now I will just see if I can set the capture filter's output format to 1080p/50p, for you to test.

[jmone]

- I know that in Davinci Resolve I have to manually set the exact resolution and frame rate or capture will fail
- The BMD WDM filter auto adjust to the input resolution and frame rate.

BlackMagic have a forum for App Developers you can ask questions with at - https://forum.blackmagicdesign.com/viewforum.php?f=12

There are references in the Decklink API Doco to these calls:

Automatic Mode Detection
 The automatic mode detection feature will notify an application when a property of the video input signal changes. This feature is supported on certain DeckLink devices. For an example of using automatic mode detection, please refer the AutomaticModeDetection sample in the DeckLink SDK.

 To use this feature please refer to the following steps:
 — Call IDeckLinkProfileAttributes::GetFlag with the BMDDeckLinkSupportsInputFormatDetection flag to check that the DeckLink hardware supports the automatic format detection feature.
— Create a callback class that subclasses from IDeckLinkInputCallback and implements all of its methods. The IDeckLinkInputCallback::VideoInputFormatChanged method will be called when a change in the property of the video signal has been detected.
 — Install a callback by calling IDeckLinkInput::SetCallback and referencing an instance of your callback class.
 — Call IDeckLinkInput::EnableVideoInput with an initial video mode and pixel format and set the bmdVideoInputEnableFormatDetection flag.
 — Call IDeckLinkInput::EnableAudioInput.
 — Call IDeckLinkInput::StartStreams to begin capture.

 While the input streams are running:
If a change in a property of the input video signal is detected then IDeckLinkInputCallback::VideoInputFormatChanged will be called in your callback object with the new video properties provided in the parameters.
If the video mode or pixel format has changed, then the following sequence could be used to restart capture with the new settings: IDeckLinkInput::PauseStreams Call IDeckLinkInput::EnableVideoInput with the detected video mode and pixel format.IDeckLinkInput::FlushStreams IDeckLinkInput::StartStreams
 — Call IDeckLinkInput::StopStreams to stop capture.
 — Call IDeckLinkInput::DisableVideoInput
 — Call IDeckLinkInput::DisableAudioInput

[jmone]

This is also these resources - https://www.blackmagicdesign.com/developer/products/capture-and-playback/sdk-and-software

Towards the bottom of this page is a "Introduction to Video Capture and Playback" button that gives you some code samples including one called "DeckLinkInputDevices.cs"  that seems to have what you may need.  I've attached this one.