Isn't the ultimate solution good hardware that doesn't need exclusive mode? I mean, if the hardware can render the video and overlays without dropping frames, why not?
From what I can tell, even pretty modest hardware is capable of this. Both my work machine (old video card) and home machine (new video card) seem to handle 1080p without trouble in non-exclusive mode.
Is this a case of chasing low-end hardware (which should shrink in importance over time), or is there some other argument for exclusive mode?
There are a number of arguments for exclusive mode:
(1) In windowed mode, every Direct3D9 "Present" call blocks until the present has succeeded. The reason for that is that actually "Present" ends up copying/blitting the rendered frame into the render window DC. This must be done during a VSync phase (otherwise there would be tearing), so "Present" can't return until the blitting was executed. In exclusive mode, "Present" goes to a backbuffer (if one is still available) and the "Present" call returns at once, allowing you to start rendering the next frame. This way you can render and present up to 32 frames (Windows 7) into the future. Meaning that even if the PC suddenly gets busy doing some funny stuff, the next (up to) 32 frames are already in the GPU driver frame queue and should show up just fine, without needing any CPU cycles. In windowed mode you can only ever render and present the very next frame. Meaning that if the PC suddenly gets busy and you get no CPU cycles for a couple of milliseconds, there may be frame drops. This problem is intensified if higher refresh rates are used. E.g. some madVR users are driving their CRT projectors with 120Hz.
(2) In windowed mode, the renderer has to use heaps of tricks (e.g. flushing the GPU and waiting for the result by polling the GPU state in a 100% CPU consuming loop) to make sure that no tearing occurs. That costs CPU and GPU performance, and even then, with a bit of bad luck, tearing can't always be avoided. In exclusive mode frames are switched in the driver in the VSync interrupt handler -> no tearing, ever. And better performance.
(3) Only exclusive mode allows the use of more than 8bit output. This may not sound very useful, but it would allow madVR to lower the dithering noise level, which might be visible on really big front projection screens.
(4) I'm not 100% sure, but I think only exclusive mode will support frame packed 3D rendering (HDMI 1.4a).
Is this a case of chasing low-end hardware (which should shrink in importance over time)
FWIW, the current madVR version only offers very basic video algorithms. I do plan to offer higher quality algorithms in the future, and more features. The highest quality algorithms might not run fluidly even with current high-end GPU hardware. E.g. there are scaling algorithms available which need a minute to upscale one 320x200 image, when using a current top-of-the-line CPU. If I implement such a scaling algorithm in madVR, you probably can't have enough GPU power.
VMR9 Renderless mode. For some reason that I have not understood, it does not work for all video cards and I did not try hard to fix it.
Hmmmmm... How do you draw your GUI in your exclusive mode VMR9 renderer? Do you simply not show any GUI at all? Or do you draw it onto the video image before passing it on to VMR9?