I don't sort by "similar;" I have a separate Smartlist that pulls similar tracks. What you define as "similar" is really up to you.
For me, I added a rule that Rating is greater than or equal to 4 stars. Then I use Genre and Year as the Limit (advanced) Fields. You can be as restrictive or as open as you want with it. For example, add another rule with BPM is in the range X to Y. Use your Style Field as the Limit (advanced) Field. Limit number to 20 so it only gives you 20-tracks at a time. The options are pretty much endless.
Below is what I use for my Genre Smartlist:
([IsPlayingNow]=[1] or [Media Type]=[Audio]) [Channels]=2 ~sort=[IsPlayingNow]-d ~limit=1,-1,[Genre] ~sort=[Date (year)]-d,[Album],[Description],[Disc #],[Track #]
If you wanted to use this for your Style Field instead, just change the Limited (advanced) from 1,-1,[Genre] to 1,-1,[Style].
One thing of note: My Genre Smartlist does not work "perfectly" for me (and maybe MrC could help with this). Genre is a List Data Type (ie you can add multiple Genres separated by semicolons), and it sounds like your Style Field is the same. The way my Expression currently works, it only brings up tracks that match the Genre exactly. So if the Genre is just "Rock," then I get plenty of results. However, if I have the Genre as "Rock; Indie," it only brings up Tracks with Genre "Rock; Indie." Tracks with just "Rock," just "Indie," or alternatively "Indie; Rock" are omitted. Ideally, I wish it would bring up any of the possible Genres (not sure if that is possible). Or possibly, if it only used the first listed Genre since that is always the predominate one.