I'm not a professional audio engineer, but I have fooled around with recording and reproducing sound. The main advantage I found with higher sample rates for recording is that if you're doing a lot of signal processing, pitch shifting, etc. the sampling rate can make a huge difference in how the final processed clip sounds (higher sampling rate improved the subjective results when doing extended signal processing). I don't know exactly why (as a technical matter) this was the case, but it was obviously different when there was less headroom. I imagine it may relate to data redundancy? But that's kind of a shot in the dark.
I'm not sure what the advantage is of using a higher sample rate for
playback, except perhaps for a desire for "bit-perfect" playback of recordings recorded at higher sample rates. I personally cannot hear the difference between sample rates (above 44k) on otherwise identical program material (i.e. I have the same audio in different sample rates, and cannot distinguish them), but I've known some people who claimed to be able to do so.
I'll second your question as to a good technical explanation of why higher sample rates would sound better for playback.
Edit: Apparently some people think higher sample rate playback is actually harmful due to the risk of ultrasonic sound causing intermod in the audible range:
http://people.xiph.org/~xiphmont/demo/neil-young.html.
As an aside, I can't hear much above 16KHz myself, but I was using a compression driver that had an 8 dB peak at 19KHz for a while. I found I kept getting mysterious headaches whenever I listened to music, often immediately, that went away as soon as I stopped the music. After RTAing the system, I nulled out the 19KHz bump and my headaches went away. So I couldn't hear it, but I could definitely "sense" ultrasonic sound, at least when it was that close to the audible range.