I would think that is a result of the way Cover Art is managed in MC, the settings you are using for Audio Cover Art, and where you store you audio files in MC. JRemote is then just picking up the Cover Art from MC, which is probably also wrong.
So does the correct Cover Art show up in the "Recently Imported" or if you play a file from each Album, "Recently Played" Smartlists in MC?
What settings do you have in "Options > File Location > Cover Art"?
What directory structure do you use to store audio files? i.e. One sub-directory for all Audio files, or lots of Audio files from various Albums in one sub-directory? That would be defined in the "Options > File Location > Audio (CD rips and Podcasts)" section, unless you change the location of rips.
For comparison, I have two Albums called simply "Anthology" and they have the correct Cover Art. If I play a song from each of them, the correct Cover Art is shown in the "Recently Played" Smartlist in MC.
Let us know those settings and we'll work it out.