My apologies if there is a more appropriate place for MC11 issues. I looked for a forum dedicated to it and did not see one, redirect me if there is one.
You have come to the right place. Unfortunately, other than a little tidbit (which you may already know), I don't have an absolute answer to your problem.
Under MC11, the degree symbol in the APE tags or filenames display instead as a square.
I can tell you that the "square" you mentioned is a placeholder that Windows uses when it encounters a font encoding that it doesn't understand (or is non-printable). For example, if you open a text file with UNIX (or Mac) line endings on a PC you will see the squares everywhere there should be a line break (and the text file will be all run together into one line).
I can also tell you that I have plenty of files with non-alphanumeric tag characters including accents, umlauts, and a bunch of other wonky (is that a word?) characters. They've never caused me problems.
As a simple workaround, I'd probably try deleting the box character out and then copy/paste a new degree symbol into the tag inside MC (using the Character Map tool as a source for the degree symbol). I work in a Multimedia Services department (graphics, AV, video edting, etc) at a scientific research institution, and I have to work with funky text characters all the time (eg Greek characters). When you move them from one application to another and they cause problems (as they sometimes do), you can spend days banging your head against the wall trying to figure out why, or you can just delete the errant character and re-add it from within the app that you are trying to use.
I hope this helps a little. Really, this would be more of a question for Matt (or for someone else who understands the character encodings used for Tags by the APE encoder).