I generally don't exclude my media files from Windows Defender on my server.
It absolutely positively impacts performance if you do exclude them. However, as tij referenced above, there's a good reason to actually scan your media files before opening them. It isn't just files (like PDF) which can contain scripting. The decoders used to demux and read media files have (AVI, MKV, and JPEG are some examples just over the past year) been subject to a wide variety of different buffer overflow and similar bugs. This can then be used, if you craft a malformed media file just-so, let you use a JPEG to deliver a malware payload when the file is read. There have been many real-world attacks using this vector.
And, the issue is, how many zero day bugs are out there in the hundreds of decoders used to read your media files that we don't know about? ALL software written by humans contains bugs. And, worse to some degree, it is possible for malicious actors to intentionally insert hidden bugs in some sets of open source code. And even if all of your content comes from "trusted sources" (all completely legit online stores, etc) who's to say that they aren't exploited? This is exactly the kind of tactic they've caught state actors using to do bad things in the very recent past. You want to target a wide swath of a particular population? Get a mole in Amazon (or Apple or whatever) to infect specifically chosen movies likely to be used by the audience you're targeting and done and done.
That said, you don't really need to be scanning them in multiple places, and you don't necessarily need to scan them on-the-fly. So, for example:
* If you use a client-server setup of MC, you don't need to have your AV scan the files on-the-fly on BOTH the client and the server. Let the server do it, and keep the paths excluded on the clients.
* If you have a very static long-term Media Library path (where new files never go directly), you can exclude them from on-the-fly scanning and maybe set up a recurring "manual" scan of the path periodically in the middle of the night, and leave the on-the-fly scanning enabled for the "new content" directories.
But, yes, disk access will be slower when AV is scanning ANYTHING on the fly. It puts interrupts around all disk access. That makes all disk access higher latency, and has the potential to introduce additional bugs. And, of course, the file IO readers on the AV application are equally vulnerable to accidentally trigger the malware payloads while they're trying to scan them. There are sandbox escapes for AV file readers all the time.
So, it is a crapshow. AV software is bad, but so is not having it at all in many cases.