How have you set up your Multidisc Sets? As one Album, and then just listed Disc 1, Disc 2, etc. within that? Or have you set up each of the Subsidiary Discs as an Album?
If the former, the Cover Art displayed would be for the Multidisc Set and not the Subsidiary Albums. But you would not have any place to store actual Album covers then, as multiple discs in an Album don't have their own Cover Art. The Subsidiary Albums also would not be identified in MC, which could be a problem for you.
If the latter you would have Cover Art for the Subsidiary Discs, but not for the Multidisc Set. Also the Albums would be identified in MC, but not the Multidisc Set, hence the reason you can't add Cover Art to it.
MC does a similar thing to what you require for TV, with a Series>Season>Episode hierarchy, and Cover Art at each level, but that is enabled by specific functionality.
Okay, I don't know the answer, so maybe a thorough search of the forum first as it must have been discussed before, and also look up how these are handled in internet Music Databases. You may find a clue there.
However, you could look at the Series capabilities in MC, and try;
Adding a [Series] tag to each Album in a Multidisc Set.
Create and/or modify your Views to show [Series] Cover Art, and if there is not series recorded for an Album, show Album Cover Art.
Not sure how that would go though, as Series Cover Art probably only works when [Media Sub Type] is "TV".
You could duplicate that sort of functionality by creating a new tag called perhaps [MultidiscSet], then tagging each Album (and hence all tracks within it) within the Set with the name of the Set, then creating a View that displays Sets if the tag appears, or Albums if not. Maybe.
Search first.