I've found that to be the case with TigerVNC as well.
I initially used Tiger but it wouldn't allow shutdowns and reboots. Had to do a "SUDO REBOOT" and there were some other anomalies. Then I tried Microsoft desktop (XRDP) and finally RealVNC (doesn't run on Raspi 5) and it seems the best. I cycled through all of them thinking it was VNC causing the hangs....but they all did it.
I'm thinking there's some handshake problem with the NAS server but every time I use the Bookworm file utility, I can easily and rapidly access the NAS music files.
But it may also have something to do with the origin of the libraries. I saved my Mac's library and restored on the RP4 (and later the RP5) --- and then globally changed all the Mac NAS file "VOLUMES" references to "/media/NAS/" and it seems to work but maybe I missed something?
Both the RP4 and RP5 have the same library.