Remember to check where MC is looking for the files. You can find this option under Tools - Options - File Location - Cover Art.
Default is in a special folder, and if this option is not changes I imagine you would have problem with more than a few albums.
If you have lots of files named *.jpeg I would add it all to playing now, and use the "Rename, Move & Copy Files" tool (right click all the cover art and choose Library Tools and "Rename, Move & Copy Files". There you select Rename, and then enter jpeg in "Find What" and jpg in "Replace With".
To add all this files to MC's playing now initially, you COULD import only images to MC (don't need to have the files in playing now if you do, as it's in the library), or you could for instance user the windows search tool to find all *.jpeg files within the folder structure and throw it into MC Playing Now.