As mentioned earlier, I'm fairly new to MC and am not terribly familiar with playlists and how they work--or anything else, for that matter. It's sounds like you've given the matter some serious thought, and probably have a more fully-developed idea than me.
But here's what I was thinking.
I create a playlist called Playlist-A. It has twenty songs arranged in a particular order.
I create Playlist-B, to which I add 40 other songs.
Just routine playlist business so far.
But now, I grab Playlist-A with the mouse, drag and drop it onto Playlist-B.
Playlist-A is still Playlist-A, and it's still in the list of playlists. But also, it has been placed AS A UNIT in Playlist-B. So now, when I look at Playlist-B, I see the 40 songs I had previously placed in it, and my last entry says "Playlist-A." All of the fields on the line (in Playlist-B) where the track info goes are blank, except possibly "Duration," because none of the other fields have a single value.
Playlist-A, in other words, is now an element, a line item, in Playlist-B. All the other line items are songs, so I guess Playlist-A is a "pseudo-song." In has all the characteristics of a song/track when being processed by the playlist processor, and the user can therefore move it around within Playlist-B (to change where it plays in B). And when the user turns on "shuffle," A gets played in a random order just like any song in B.
But when it comes time to actually play Playlist-A (within B), then control is passed to Playlist-A, and the playlist processor completely ignores Playlist-B while A is playing. Playlist-A has a fixed order, like all playlists, and so is played through from start to finish; that may be five songs or 500. Only when A gets done does control pass back to Playlist-B, which plays the next song after the pseudo-song of Playlist-A.
Note that "shuffle" does not apply when playlist-A is playing within Playlist-B. If you play Playlist-A by itself, then shuffle works as expected. But if you're playing Playlist-B, and shuffle is turned on, then that shuffle mode is suspended while when Playlist-A is played. When A gets finished, regular processing of Playlist-B, including shuffle if applicable, resumes.
The PURPOSE of having Playlist-A (or many such playlists) embedded in Playlist-B is to allow the user to create one big playlist that has "randomizable" elements--all the tracks and playlists in B--combined with smaller groups of songs that occur only in a fixed order. Obviously, if I wanted all the songs to be in the same order all the time, there would be no need for these nested playlists. Equally obviously, there would be no need for them if I wanted no order; just put them all in a big list and turn on shuffle. But the need for fixed seqments within larger "shuffle-able" elements creates the need for the nested playlists.
I hope this is helpful, or at least clear. If not, please send an email to ty/at/microsabio.com. I'll be happy to continue this discussion by email or phone.