Interested in MrC's response.
I don't have any inside knowledge about file limits. Folks and devs here have indicated there are not issues with large libraries. MCs db is very fast.
Some things to consider:
1) check your Power Options control panel for disk spin down time. Once you start playing a track, it likely is entirely in memory shortly thereafter; therefore, and long track might allow a disk to go back into low power/sleep mode.
2) disable A/V software, as a test, entirely. It would take volumes to explain just how intrusive this stuff is, and how it can negatively affect Windows. It is mind-bogglingly complex and fickle.
3) defragment the disk(s). A fragmented disk, especially a full one, can wreak havoc on performance.
4) check eventvwr.exe (Event Viewer) for important errors indicating problems, especially with the disk I/O chain (disk, controller, etc.). A bad disk or controller can cause long pauses while it times out. (I just had a CD/DVD drive become flaky, and its long access timeouts cause ALL of Windows to slow to a crawl.)