Never.
Tunnel view
This is probably THE biggest spoiler for me with 3D. In elementary school we used to create 'looking boxes'. We took a box, for example from a new pair of shoes. We then cut out little people, a house, a sun or a moon, some clouds, colored them and glued them upright in the box. We then cut a hole in the lid and glued some thin colored paper in it so light comes in coloring the scene inside the box. A hole in the side to look through and voila, 3D TV man! The point here is in case you don't get it, the TV image will never exit the tunnel view between your eyes and the TV bezel. This 'tunnel' is the 'looking box'. The image can only exist in this tunnel, there can be no image outside of it. The first time I saw 3D in a show case I thought 'hey this is cool!'. Then some smoke came towards me and it created this sharp border on the edges of this 'tunnel'. I took off the glasses and walked away. I can't imagine anyone wanting to look at something so ugly and so fake.
Forced focus
Although stuff appears in front or behind the focus point, you can't focus on anything else. If the camera has a large depth of field and you see things sharply close and far away, your brain has a tendency to focus, but that object isn't really any farther than what appears close to you. I believe this is where the eye strain comes from, it can even cause headaches and is closely related to the next point.
Nausea
People experience motion sickness with 3D that never had motion sickness before. I do get motion sickness with first person shooters sometimes, especially on bigger screens and its worse with 3D movies due to added realism of the image. This is due to the fact that your brain registers movement through vision but the vestibular does not. It's basically the same thing that causes car sickness but reversed where the vestibular registers more movement than your eyes can see due to the limited view through small windows. This confuses things and causes nausea.
Viewing angle
Although it has improved, the technology simply limits the viewing angle. There is a sweetspot and this limits the fun for people sitting outside of it. If your room allows it you could sit further away and this allows 1 or 2 more persons to enjoy 3D in all its glory, but the closer you sit, the smaller the sweetspot.
Glasses
I wear glasses already. Passive or active is already a no-go. I could try wearing 2 pairs but my nose is too small.
Many of the other downsides mentioned in this thread like less brightness and reduced resolution are only limitations to keep the costs down. In fact there are already TV's being showcased with full 4K 3D and double the brightness of a normal full HD TV - it's only a matter of time before these appear in the shops for decent prices. The glasses won't be an issue once/if the glasses-less passive 3D becomes a more viable option but right now that is far from being the case. My other points will never solve with the current 3D technology.