I don't use a receiver, I want to decode directly to active speakers.
In dsp studio I am on 7.1ch and up or downmixing is disabled.
Okay, that's good.
And what do you have your Video Mode set to? Be sure to try one of the two Red October modes, and make sure Bitstreaming is set to none. You said you had upmixing disabled, right? And, just to check, what is
Options -> Audio -> Output Mode set to? (ASIO or one of the WASAPIs is what you want.)
Lastly, to decode DTS-MA in LAV and Red October, you do need to find a third party DTS decoder (dtsdecoderdll.dll) and put it in your system path directory. Have you done this?
EDIT: I see Bryan covered that. It is easiest with MC and Red October to copy the dll into your system path. Otherwise MC removes it when it updates to a new version of Red October, I think. You should also be able to put it in MC's folder itself (right next to the Media Center 17.exe file). LAV follows
this search path to find the DLL. The "executable module for the current process" is MC itself. But I haven't tried this myself. Mine is in my C:\Windows\SysWOW64 folder.
Assuming you have done all of that, and your settings are right...
If you select the first of those two listed DTS tracks (either English or German, depending on what you want I suppose), does sound come out of your rear or front high speakers?
I suppose it could be just that the decoder is reading the info about the streams wrong. I don't have a 7.1 system here, so I don't know how it looks in MC.