Many of us have been doing it for a long time with good success in systems with high SQ.
The only difficulty I see with going straight to the power amps is if you have multiple inputs to deal with, instead of the computer being the only source. Not an issue for us.
Rod