Aaahhhh... DOH! Apologies.
So, first off, you cannot disable the little display window in the bottom left corner. Best you can do is minimise it using the button in the top right corner of that window.
*an open "playing now" tab will, of course, remove the little display window. Nice one Brian.
2nd, the Windows Explorer thing.... you cannot change that either... well you can, but, how that will affect you depends a lot upon your typical workflows.
The only way to change what you're seeing is to remove all of the file associations from Media Center.
If you do that, Media Center will no longer launch when you double click on audio files in Windows Explorer.
You should still be able to select files in Windows Explorer, then right click on them, and look for "Play in Media Center" or similar options, which will work just fine.
If you can live with that, the easiest way to remove the file associations from Media Center is probably to reinstall it. Choose the "Custom" option, then click through, pressing next each time, taking the defaults presented to you, until you see the file association screen...
Click on the "Unselect all" button, then the "Next" button to complete the installation and remove the associations, which should restore the old file type defaults in Windows Explorer. I'm not sure, but you might need to reboot in order for Windows to sort itself out...
-marko