First, though it is not that important to the discussion at hand, it sounds like you are in "Thumbnail" mode. So for demonstration purposes, try the following:
1.) Launch MC;
2.) In the upper right corner, make sure either the world icon (All Media) or musical note (Audio) icon is highlighted/selected;
3.) In the tree pane, select "Media Library";
4.) [OPTIONAL] Go to the View --> View Settings --> List Style menu and select "Automatic" or "Details";
5.) In the search bar (again, upper right corner), start typing in all or part the name of the track you are looking for (MC will begin filtering the track list as you type);
6.) When enough filtering has been done and you spot your track down in the list pane, right-click on the track and highlight Locate. Once highlighted, a submenu will appear where you can choose what you want to locate (based on the selected track). In your case, you would select Artist.
At this point, you will have a list of all tracks by that Artist, and you can sort this list any way you want. Again, in your case, you will sort by Album. (If Album isn't one of the columns being shown, then right-click on one of the column headers and choose Customize Current View... to add it.)
The above "sorting" business is not really necessary, mind you, since your current view will also have an "Album" pane which lists all the albums.