Looks like this is not related to hardware acceleration after all - it's still happening even with it disabled. I guess it must be a madVR thing rather than a LAV Video thing?
I'm really not trying to be a broken record on this but...
1. You're experiencing a serious problem that no one else, even people with similar hardware setups, are seeing.
2. You are running the hardware wildly out of spec. I don't know where I saw it, but I remember you posted your specific OC in some thread before, and it was pretty aggressive.
3. You seem to refuse to consider that #1 could be related to #2.
Is there any way you can follow this advice above:
If your BIOS allows you to set and recall profiles easily, it would be handy to save your OC settings, drop the CPU back to defaults, and try again just to see.
I've been doing this a very, very,
very long time. When something smells like hardware to me, it isn't always, but there's at least a very, very good chance that it is.
Otherwise, I'm not much help. I'm not seeing anything similar on a wide variety of machines. The system, as described, should absolutely be able to handle BluRay playback. Clearly, that kind of performance isn't something that most people would just "not notice" (such as occasional single-frame drops and whatnot).
There is no big outcry, and you don't have any components that are oddball. Therefore, it is
very unlikely to be something in the program, but rather, something specific to your system. Not impossible, but very unlikely.