I'm not an electrician, and would much rather err on the side of caution.
There are multiple potential issues for that, but it does sound like a grounding issue. I've had MacBook Pros (all aluminum chassis) which would have that tingling sensation when you touched them if they were connected with the short two-prong adapter rather than the three-prong (grounded) extension lead.
I would be somewhat concerned if a desktop computer is doing the same thing, since it should only be connected to a properly earthed outlet.
The first thing I'd do would be to get an outlet / receptacle tester ($5-10) and confirm that the outlet it's connected to is safe.
Make sure that the power supply is properly screwed into the exterior of the case.
And check that the computer and any devices it is connected to are all connected to the same outlet to avoid ground loop problems. If it's connected to multiple devices (speakers via analog audio, TV via HDMI etc.) then remove as many of those devices as you can.