I guess part of my point was if we can figure out what triggers it, maybe a better solution is a tweak to the algorithm, rather than a dumb "no dupes" post-processing step (reducing the length of the list). But, to do that, you have to figure out under what conditions the problem occurs, and when it doesn't.
Fair enough. We managed to get very different results with somewhat similar settings (your variety setting was more "variant" than mine it's true). I find that "lots" doesn't give me very good results (all over the map); I really like the results I get with "more variety", except for the dupes. But that's the only obvious difference.
Yeah, I keep the "default" one set to Lots. Perhaps because I felt (long ago at some point, perhaps influenced by beer, when that option got set last) it was too "constricted". Perhaps because I was annoyed by dupes. I can't remember. I don't even know what the defaults are (I've had stuff in there since before there
were options).
I will say, though, that I really like how it works out with Variety set to Lots, and further "constrictions" in the rules. I have a bunch of saved Play Doctor lists, and I use these the most (particularly because I can use them through JRemote with a touch of trickery). I've explained these elsewhere, but for example, I have one that is:
Name: The Girl/Boy List
Search: Aphex Twin
Variety: Lots
Search:[Media Type]=[Audio] -[Media Sub Type]=[Audiobook],[Podcast],[Other] -[Rating]=1 [Genre]=[Electronic],[Electronic\Ambient],[Electronic\Drum And Bass],[Electronic\Dubstep],[Electronic\Electro Swing],[Electronic\House],[Electronic\Industrial],[Electronic\Trance],[Experimental]
And others...
Name: Some New, Some Popular
Search: Blank
Variety: Lots
Search:[Media Type]=[Audio] -[Media Sub Type]=[Audiobook],[Other],[System],[Test Clip] playlistid==643011798,=658284581,=194281342,=398063784 ~mix=1000,20%,{playlistid==643011798},20%,{playlistid==658284581},,20%,{playlistid==194281342},40%,{playlistid==398063784}
Those playlists point to some "top hits" Smartlists of different varieties (by total number of plays, by rating, etc) and to one "recently imported" Smartlist.
These work pretty brilliantly.
One thing I would very much like to see with Play Doctor is a new setting, in addition to Variety, to set the bias for Discovery vs. Hits (which you could get from the Rating and Number Plays fields, of course), something like my "New and Popular" list described above.
iTunes Radio has this setting, and I it works pretty darn nicely. It is a simple three-point slider (kind of like the some, more, lots setting we have now for Variety) but it slides between "popular/hits" and "new discovery" mode. Of course, they're using aggregate iTunes Match and Genius data to consider everyone's statistical information, probably bias from "the Industry", and sales in iTunes. That gives them a much richer data set to mine than any one user's Library (no matter how well tagged). But, I think they probably already have some arbitrary weighting to this kind of "category" built into the Play Doctor algorithm. It would be
very handy to be able to more easily "bias" it, just a touch in either direction, towards more "high-rated" tracks and (in other cases) towards more "unknown" ones.
In any case, with those results and that kind of search, I'd say that it does need some kind of tweak. I guess I just didn't notice because I've been keeping mine set on Lots for my "pick a song and seed" mode Doctor usage.