Can you check/confirm that you don't have other settings setup to change refresh rate?
For example if you set Settings -> Tree & View -> Fullscreen -> Resolution, this would apply entirely independent of the video, and also ignore the delay. I would recommend to not set this field, or rather ensure its set to "Desktop Settings" (which translates to "leave it at what it is"), so that the video-specific option can run.
If this one applies first, then the Video -> Display Settings mode doesn't need to change anymore, and also not apply its delay.
I can confim that this setting is set to "Desktop Settings".