Pick the artist. Then at the top, use the 'Play all' button or the plus button to the right of that for more playback choices.
When you add it to web Gizmo, maybe, but for now I'm on an iPhone that doesn't have that widget.
Even if you solve that, though, it wouldn't help with this:
I can't use one Genre/Artist/Album view to browse both by Genre (when I want to) and by "All Artists". To have a way to browse all artists, I have to make a separate Artist/Album view. With "All" as a choice, I can just use the Genre/Artist/Album view for everything. If, instead, I want to dig down and find a specific album, I can just do "All", "All", and then browse my Albums.
Likewise, on my Genre/Artist/Album music views, I actually like to structure them like this: Genre/Artist/Artist/Album. The first Artist Category is Grouped by 2 letters, so that it gives me A-B, C-D, E-F, etc. That's handy because my full Artist list, even subdivided by Genre, is often quite long (especially if I pick "All Genres" on the previous tier). But, sometimes you want to pick a Genre and then browse all of the artists. Maybe you don't have very many of a particular Genre, so it is easier to see the whole list at once. So, when I open my Disco genre, it is usually easier to just see all of the artists at once (unless I'm looking for something specific, like Abba). But, when I open my Indie genre, I need that A-B, C-D, structure or I have like three "pages" of As.
"All" as a choice provides this flexibility. Without it, I would have to choose between never seeing all of my Disco artists listed in one view, and scrolling through an endless set of As, then Bs, then Cs, then Ds to try to get to The Decemberists.
EDIT: I see rjm and gvanbrunt already covered this pretty well. I use the same thing for my movies too, with a grouped Name category.