OK, I got it to work perfectly in the end.
Anyone with this issue, here is what worked for me to get the same performance in JRiver and in MPC-BE:
- Uninstall the standalone version of MadVR
- Keep the standalone version of LAV installed
- Install Red October HQ with custom settings
- Copy all the files from my MadVR installation into the JRiver MadVR folder (users/user/appdata/roaming/jriver/mediacentre24/plugins/madvr), including the files from the latest test build 38. This folder will be modified by JRiver in a later update, so make sure you have a backup somewhere to replace if necessary.
- I simply deleted the settings.bin in the JRiver MadVR install folder to have my registry settings apply (I had backed them up and re-imported them just to be on the safe side).
- Add MadVR renderer, LAV Source Splitter, LAV Audio and LAV Video as custom filters. All the settings are accessible there, as well as in the tray when a file is playing.
Otherwise without the custom filters I had poor rendering times and/or huge clock deviation that caused lots of frame drops.
The only other issue I had was to get HDMI bitstreaming to work, and that was because I was using the default direct sound device (which works fine with other players). I had to select manually the nVidia HDMI device WASAPI in JRiver. The standard nVidia HDMI device one wouldn't work.
I would highly suggest to make a sticky about this HDMI issue as I'm not the only one with this problem and I couldn't find a simple solution to this.
Anyway, I now have the same performance as with MCP-BE, but with BD and UHD BD Menus

Keep up the good work everyone, looking forward to spending more time again with JRiver, and thanks again to SamuriHL for his precious help with this issue.