Embed the singles cover art in the respective file. Then add a very short dummy music file with track # 0 and embed the cover for the entire album in that file. In standard and cover view the cover art for the very first file in an album is shown as album thumbnail. Unfortunately this is not always the case in theatre view where a random cover art from some other music file in the album is chosen sometimes.
Be careful when you embed singles cover art so that you do not overwrite the sole copy of the image file for the entire album. There is unfortunately not an option to save cover art only inside the music file so when you embed a copy is saved also to the cover art folder as set in options, i.e. the album cover art image file is overwritten when you embed your first singles cover art. I handle this by temporarily changing the cover art folder option from my usual folder.jpg to next to the music file named album artist - album when I embed singles cover art. But sometimes I forget.