What are you referring to with displaymode switching? Turning on HDR for you? Frame rate switching?
Are you using any other tools to apply the ICC profile to the full desktop or anything like that?
Also, did you measure the profile yourself, or is that a vendor provided one?
Processing of the ICC profile has become more complete, including getting the black point from the profile, which might cause more issues with not-ideal profiles.