Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[Matt]

Overview
A while back I was looking for an A/B switch to compare DACs.

The user 6233638 pointed me to the FiiO four input switcher (for $20).

I found some time today to test a $2500, $900, and $5 DAC each from different manufacturers.

Setup
I did the test on Sennheiser HD 650 headphones.  In my experience, headphones are great for hearing detail and differences in sound.  With headphones, the DACs are running close to 100% so the noise floor (background hiss) is less important than in a loudspeaker setup (where you might run the DAC at -40dB or more with a high gain amplifier).

I set up a zone for each DAC in Media Center.  I linked the zones so they would play the same thing.

I used a decibel meter between the headphones while playing pink noise (Tools > Audio Calibration > Pink noise in MC) to match the levels.

Then I was able to play all sorts of clips and instantly switch between the DACs.  It's fun to be able to switch like this because you can compare quickly and not rely on memory (which isn't all that good for sound details).

I also tried to twist the input cables and insert them randomly so that the testing was as blind as possible (at times getting help from my wife).

I played songs and switched with my eyes closed, ranking the three DACs.  I have a lot of songs I've listened to critically through the years for all sorts of tests.

Results
I expected to be able to hear differences, especially with the $5 analog from a motherboard sound card in the mix.

However, any differences were extremely small.  At times I thought one might have a little better stereo separation, or another might be a little punchier in dynamics.

When I actually ranked the playback, the results were not consistent.

To test myself a little, I made one of the three one decibel louder than the others.  This difference exceeded any difference between the DACs when they were level matched.  The extra decibel made it sound clearer and more dynamic.

Playing silence, all three were completely silent.  As I mentioned above, this would probably change if the DAC were connected to a power amplifier and run much below max output.

There were large differences in the maximum volume the hardware could achieve (important if you're trying to power headphones without an amplifier, etc.).

Conclusion
For me, the main point of this is that once you reach a certain level of goodness with sound, differences become very subtle.

I can't say "there are no differences" but I also can't say "there are clear differences."  I think I prefer one of the DACs, but it's possible this is because I'm friends with the engineers that built it!

It's possible a better listener or better equipment would expose more differences.  However, if somebody with as much listening experience as me with well-respected headphones isn't sufficient to draw the conclusion, it supports my point that differences are pretty subtle.

Another point is that even low-end hardware has gotten better through the years.  I've had motherboard soundcards that had a loud hiss with all output and made sound when the mouse moved.  Clearly things have improved.

I would encourage anyone that likes to compare sound (especially reviewers) to do tests like this.  It's fun, but it also helps you appreciate the degree of differences in sound.  Sometimes reading forums, these differences can get exaggerated.  I think it also underscores that proper testing is challenging.  You can't listen to something and wait very long before hearing the other thing if you want to draw meaningful conclusions.  Variables like our mood, temperature, imperfect level matching, our poor memory for sound detail, etc. are going to make more difference than the "real" differences.

[jmone]

Nice write up Matt.  Who would have thought....  all those years on producing DACs they are now both refined and a commodity.  Not magic, Love the Experience Curve.

[InflatableMouse]

Interesting. Thanks for sharing.

I do think you should repeat the test with a set of properly setup speakers and from an ideal listening position. While headphones are great for the sound and detail, they reveal little to nothing of the soundstage.

So you've established one fact, they sound about the same. But do they provide the same soundstage?

[Matt]

I do think you should repeat the test with a set of properly setup speakers and from an ideal listening position. While headphones are great for the sound and detail, they reveal little to nothing of the soundstage.

So you've established one fact, they sound about the same. But do they provide the same soundstage?

I've always found headphones the most revealing for tests (not to be confused with the best sounding).  I don't think I've ever heard an audible issue that was there with speakers that I couldn't hear more easily with good headphones.  This includes details about the soundstage like timing, stereo positioning, etc.

However, like I say, it's possible better ears or better equipment would yield different results.  I'm just reporting one test I did myself.

I'd love to see other people do tests and share what they learn.

[jmone]

I'd love to see other people do tests and share what they learn.

...but only if there is AB test method or else you will find the expectations of differences will colour any results.

[InflatableMouse]

I've always found headphones the most revealing for tests (not to be confused with the best sounding).  I don't think I've ever heard an audible issue that was there with speakers that I couldn't hear more easily with good headphones.  This includes details about the soundstage like timing, stereo positioning, etc.

However, like I say, it's possible better ears or better equipment would yield different results.  I'm just reporting one test I did myself.

I'd love to see other people do tests and share what they learn.

Fair enough. You're probably better trained with headphones than I am. In my experience positioning gets messed up on headphones which is why I mentioned it but I suppose that isn't necessarily the same for everyone.

Once I get my Teac DAC I'll try to do some tests but the problem is I can't easily switch without considerable downtime and manually skipping to the point I was listening to. That just doesn't work.

Maybe if I create a zone for each device I assume they can play simultaneously. I can then link them and I can configure my AVR to level the inputs. But I'm concerned I won't be able to get the levels correct. Would a phone with a dB app suffice for that?