BTB/WTW aren't particularly useful when you send RGB anyway, but thats more of a philosophical debate then anything. I don't think anyone can prove a visual difference either way, since with proper calibration BTB/WTW are supposed to be invisible. But it makes no difference to the resolution/refresh rate combination that is possible, anyway.
On a PC, especially with madVR, you lose quite a bit of quality when you send sub-sampled YCbCr instead of RGB, so that should be avoided.
For 60Hz RGB at 4K - you need a full HDMI 2.0 chain to be able to do that. Your GPU can definitely do it, so make sure you connected to a port on the TV thats full HDMI 2.0, and if you have an AV-Receiver in between make sure its also HDMI 2.0 capable - and to top it all off, you need a High-Speed HDMI cable. Some TVs sometimes need some special options set to allow HDMI 2.0 unlimited input for some reason that eludes me, might be worth checking if there is something like that for your model.
One thing to note is that 60Hz RGB at 4K is limited to 8-bit only over HDMI for bandwidth reasons, so you might need to specifically set 8-bit in the NVIDIA control panel to be able to see that resolution combination.