Rudy is about right why "naive" going backwards doesn't work. Videos cannot be decoded backwards (not to mention that you also cannot read a file backwards, you would have to "guess" where the previous frame was, or search for it)
The only way to do this is to go as far back as is needed (ie. to the previous known seekpoint, or something like that), and then slowly move forward from there until you reach the frame you want, but this is riddled with a lot of complexity to get it accurate, because its entirely different to how moving forwards works. Frame Stepping forward has a native API that you can just tell "one more frame" and stuff just works. I believe this is also how MPC-HC works, but due to the complexity of this backwards-stepping was not implemented when we did forward stepping. You can step back one-second using normal seeks if you're in frame stepping mode and then manually move forwards again, though, but its just not going to be as accurate.
Maybe Yaobing sometime gets bored and adds backwards-stepping (he did the forward stepping), but I wouldn't expect it anytime soon.