I would lay my money on an HDMI handshake/EDID issue.
I have a minor problem at the moment in that my 4K TV, which is playing at 1080p, refreshes the screen resolution/frame rate/whatever randomly every now and then, so that I might get a brief black screen or more usually just get the TV's resolution information overlay display when it happens. Annoying, not fatal. This only started happening after I briefly set the resolution to 4K on the PC, just to check out what a certain file looked like on the TV at 4K. Setting the PC back to 1080p has not stopped the problem. I haven't diagnosed the issue yet because unplugging and replacing cables is a bit difficult in my installation at the moment. I suspect it is because the EDID was changed, or Windows/GPU driver/something decided that the HDMI cable couldn't handle 4K.
Anyway, on your issue, my best guesses:
Not the 45ft total length, as long as you are using a powered splitter. Also, the receiver should be powering the signal, and not just passing it through with attenuation. You may be able to check that with the manufacturer.
Not an HDCP issue, as stuff just wouldn't play at all. You couldn't trick HDCP into playing the way you describe. I think you would get a different error message as well, and from Windows not just MC.
Thoughts:
Direct Sound isn't affected as Windows make sure the output matches the capabilities of the target device?
The problem goes away when the projector is disconnected. The projector is stereo only? Windows has decided that the projector is the endpoint of the HDMI chain, so uses its capabilities when in Exclusive Mode to decide if MC can play the file? (Maybe some Windows update, perhaps even related to HDCP or DRM, is more aggressively trying to determine the HDMI endpoint?
What has changed in the setup recently, or is this a new installation? Including Windows updates.
Suggestions?
Hmm. Difficult.
I take it the AVR is set to 5.1 or 7.1 channels? Is the first 4K TV, the one directly connected to the HTPC, also set to the same configuration? Can the projector be set to accept the same configuration?
Do you have Zones set up for playing to the different targets? Do the zones have appropriate channel configurations?
Are all the cables long distance 4K capable? i.e. Are they Active HDMI cables? Attenuation is an issue for longer cable, particularly at the bandwidth required for 4K. Active HDMI cables have a chip in them to boost the signal, apparently.
Use Wireless HDMI? Whether that would work depends on what the problem is. Windows selecting the wrong endpoint could still be a problem.
Did you try unplugging the projector, rebooting the HTPC, playing to each TV, rebooting the PC, and then plugging the projector back in, then test again? Some sequence like that might reset the EDID and/or HDMI endpoint.
Anyway, I'm an amateur at this so just guessing a bit, and out of time. Let us know if you find the problem though, as it may help others, including me!