I don't think the CAPS bit makes a difference!
But I think this is a generic problem with using Remote Desktop to access hardware that is local to the remote machine (IYSWIM). The same issue occurs with secondary monitors on the remote PC - Remote Desktop doesn't see them.
I can't remember but there might be an option in the Remote Desktop settings to use USB devices from the host. There is definitely a setting to play audio locally or remotely but I'm not sure if that 's a red herring - the problem is that the session isn't seeing the USB DAC, but once it can see it then you direct output to it via JRiver (unless of course your settings are to send to the default Windows sound device).
What's confusing though is that it seems to work correctly initially but goes wrong when rebooted. How did he get it working correctly in the first place if it's headless and only accessible via Remote Desktop? And how does he reset it to find that the next reboot causes it to go wrong again?