This is by no means any kind of complete answer. Just some thoughts:
1. Depending upon the bitrate of the MP3 file, MANY people (including me) can hear the difference between MP3 encoding and lossless. At lower bitrates (128 to 256) it's pretty reliable hear the difference and to pick out the lossy file. Other people claim that 320 kpbs MP3 files are nearly impossible to tell from lossless. I don't have the motivation to find out. Disk space is CHEAP and lossless files aren't very big. So everything is lossless in my collection unless I have no other choice.
2. In my own limited experience, the differences between lossy and lossless are the details. Reverb trails, and other low level details. On the other hand, the difference between 16/44.1 and higher sample size and sample rate audio were different. It was more of a sense of "correctness" or realism, as opposed to specific details I could pick out. If you're trying to do these types of tests, it's going to be MUCH harder to pick out differences on poorly recorded material. It's going to be more difficult on material that has less realistic sounding instruments. So high energy, low dynamic range rock is going to be harder to distinguish than say folk music with acoustic instruments. Just an example.
3. I've recently learned that short term A/B testing sometimes fails to produce differences, while much longer term A/B testing becomes more reliable. This is an amazing conclusion to me, because it reinforces the idea that the subtle details are picked up by your subconscious brain and may not be immediately obvious in an A/B test that lasts 10 seconds, 1 minute, etc. Then of course there is the psychological pressure that one feels when doing a "test" where we are switching back and forth. With a longer term test, these effects are reduced.
Here's one person's experience with short versus long term AB testing:
http://www.head-fi.org/t/701900/schiit-happened-the-story-of-the-worlds-most-improbable-start-up/7725#post_11921090What I have personally found is that I feel like listening for MUCH longer with lossless music. I think my brain subconsciously tells me "this is correct" versus "there's something wrong" and makes me enjoy the music more. Just a theory based on my experiences. I don't have very many high resolution songs, so I can't comment on those in this context.
Enjoy.
Brian.