I'm not a regular here and so am not a emoticon user, sorry. As to your questions, first how do you find out which files have cover art. In any view of the audio files you can right click on the column headings and then select "Customize Current View...." In the dialog that starts and from the "columns to show" list slide down until "Image File" is selectable. Select it. That will add a column telling you whether an image file is associated with the audio file as cover art. It may give an external file reference or, if the cover art was added with the option to store the image with/in the file the column will simply say "insde file". In any case it will show that cover art image exists for this file. Now for the second question, how do you view the cover art while the audio file is playing? Select "View" from the Menu bar at the top of Media Center. Then select either Full Screen View or Theater View. In Theater View some options will display in the middle of the screen and the cover art is the first one. With cover art selected it will be displayed in the lower left of the screen. In Full Screen View the cover art is displayed in the middle of the screen. As you go from one song to another in a playlist the cover art image changes appropriately.
Well there you have how I do it on my laptop. As anyone who workd very long with these confounded machines will note it could be different for you depending on your configuration and setup but, in general, these things should work. Hope this helps you.