There are some serious problems with this feature. I wanted to see if the long-standing bug affecting mp3 conversions still exists, and I was able to confirm that it does. If you play a track that doesn't fade out or end in silence you'll notice that the mp3 conversions all end too soon. It's very noticeable some of the time.
But the huge problem with the Listening test is that it can change output settings that it should not. My e22 DAC can play DSD up to 256 and PCM up to 352,800, so in DSP studio I have Sample Rate set to "No change" for all rates. When the Listening test attempts to play the mp3 32 or 96 files MC tells me playback is not possible, and asks if it can change my output settings. For the sake of the test I said yes. When the test finished I checked the output settings and found that it had change EVERY sample rate other than 44.1 and 48 to 48. Obviously not required or desired.