I have the same problem with a NAD DAC1, a MAC and Mediacenter 18. I can get the DAC1 to work with OS-X sounds, but not with Mediacenter, even if I change the bitrate with DSP studio. When I configure sound output to USB Headphones, Use CoreAudio as the player's output, and adjust bitrates with DSP studio, it always complains that the hardware does not support that bitrate.
Did you ever get it to work?