However, don't use Windows Task Manager to report GPU activity. It's not accurate.
Use somethng like GPU-Z
GPU-Z's GPU Load Meter isn't necessarily accurate for this task either.
I can't find any documentation over what "load" exactly GPU-Z is tracking with that meter. A modern GPU is made up of a whole bunch of largely independent parts. Is that load tracking the load on the Shader Cores? (That would be my guess.) Or is it tracking the ROP rasterizers? Or the texture units? Or the memory controllers? Or, perhaps most pertinent to this discussion, the hardware video decode logic processors?
Or is it tracking all of these components and producing some kind of aggregate "score" that is translated (at the GPU-Z designer's whim on "importance") into a 0-100% load meter?
I can't find any documentation one way or the other.
It is entirely possible that a GPU could be fully decoding video, via its dedicated hardware, and if GPU-Z is only tracking the shader cores (which may or may not be used for this hardware logic), they could be sitting totally idle so GPU-Z would show 0 or 1% or something low like that.
I'd consider GPU-Z's load meter a very rough guide at best, unless someone can point to some documentation.
None of this says anything about the OP's problem, of course.
That sounds like a hardware or driver problem to me.