Are you really still using MC17?
I have the onboard audio set to exclusive mode on my Workstation. With MC running I can play and hear YouTube videos. If I start playing audio in MC, YouTube errors out. If I have something playing in MC and try to start YouTube, YouTube fails to start. That is what I expected; Exclusive Mode only has an effect if MC is actually playing something.
So something else is wrong or has changed. Maybe something to do with your DAC. Firmware? Driver?
The setting you want is at: Tools > Options > Audio > Audio Device > Device > Device Settings > "Open device for exclusive access" checkbox.
Note that MC takes a little while to fully shut down, so if there is a problem in MC, wait a little while after shutting it down before testing other audio applications.
If you do turn off Exclusive Mode, and check the MC Audio Path when playing Hi Res files to see what it is outputting. If it is CD quality, you have some settings wrong in DSP, or something else is set up wrong.