I can understand why it's annoying you, but it's a problem of your own making.
Luckily it is easy to fix.
If you have already deleted files from your drive, as you have, deleting them from the library is not extreme, it is the correct thing to do.
Use the smartlist described here to identify all the files in your library that are not on the disk:
https://wiki.jriver.com/index.php/Broken_LinksThen delete them from the library.
To prevent this from continuing to trouble you in the future, change your behavior: stop deleting media files from outside JRiver, and instead use JRiver to delete them. You can thereby delete them from the disk and from the MC library in one step.
Good luck!