INTERACT FORUM

Please login or register.

Login with username, password and session length
Advanced search  
Pages: [1]   Go Down

Author Topic: Blind A/B tests  (Read 1981 times)

kstuart

  • Citizen of the Universe
  • *****
  • Posts: 1955
  • Upgraded to MC22 Master using preorder discount
Blind A/B tests
« on: November 22, 2016, 11:01:35 pm »

As I said above, we may extend this feature, but for now, let's see whether you and others can successfully tell the difference consistently between FLAC and 256Kbps MP3.  So far, the record isn't clear.

The following by the creator of the first DAC, Mike Moffat:

Back in the early 1970s, before I founded Theta Electronics, the tube audio products company, I had a busy part time biz rebuilding Dynaco Tube Amplifiers. At that time I had converted to the tube based practice for my own system, convinced that tubes sounded better than the solid state gear of that era. In my ramblings, I met John Koval, a man who had designed a modification for the old Qual ESL loudspeakers which made them sound much better. “The mod gets rid of a 5 db bump in the 200-400 Hz region which makes them much flatter” he explained. I told him that I was enchanted with the sound of tube amplifiers and preamplifiers. He explained that as long as the frequency response was the same and the levels were precisely matched, there was no way anyone could tell any amps/preamps apart in blind A/B tests. He had built a custom box that matched levels and randomized any two amplifiers or preamplifiers with a pushbutton to switch between them. Bullschiit, I thought, what about the solid state A/B box and its sonic signature.

Intrigued, I built a similar box with passive relays and a passive attenuator. Damn, if he wasn't right. It is really difficult to tell differences in an instantaneous blind A/B test between tube gear that I built versus some commercial gear that I was not particularly fond of. I used to bet John beers that I could tell the difference. Usually, I won at 7 out of 10 picks or so – the best I ever did was 9 out of ten. But it was really hard.

This whole deal made me wonder if I was crazy hearing differences between amps. If what John said was true, and many others have said in the passing 40 years or so, there is no point for an audio hobby involving anything other than transducers. WTF?

So I tried something new – I still did the A/B tests, matched levels, but allowed long-term listening to each; at least an hour or two with known recordings. Guess what! Suddenly I knew which was what. I tried it out on John B and Mike and Dave and all my other audio buddies. They called it too – tubes vs a bad solid state preamp. Every friggin' time. My enthusiasm had returned. This taught me that the human ear is an integral, NOT differential device.

So much for the blind A/B instantaneous naysayers. All that matters is frequency response, they say. People can't hear anything much above 20KHz in their prime, less later. The ear has a short memory, it is all bias, blah, blah. They should take up a different hobby, say stamp collecting.

Thanks to Dr. Heil, the inventor of the Heil AMT speaker who shared this experiment with me over 40 years ago, Consider this: I am 67 years old – my high end extends to just under 15KHz (not bad for and old fart). I can play back two pulses 200 microseconds in length separated by 20 microseconds and clearly hear two pulses. Not unusual until one considers that 20 microseconds corresponds to a square wave of 50KHz. And then, there is the time domain – home of spatial cues which audio measurement traditionalists ignore. I believe that in the quest for the best sound, an open mind is the most important asset. I will even listen to cables, even though I believe in my heart that all technology about cables is well known. Who knows, even an old fart like me could be surprised.

Until then, yet another retelling of my old John Koval saga is 40 year old news to me.

GreenMan

  • Recent member
  • *
  • Posts: 16
Re: Blind A/B tests
« Reply #1 on: November 23, 2016, 10:54:29 am »

We can certainly remove the feature if people are offended.  There are plenty of other things we could be doing.
When one states any opinion on the audibility of something, some people will be offended if it does not agree with their experience.  But people do change their opinions as they gather more experience.  By providing a good quality tool, you are not pushing your opinion -- just providing your valued customers a way to gather more experience in a way that they want.

While foobar2000 provides a general purpose AB compare, foobar2000 makes no claim to be bit-perfect.  I have performed a test of MC's bit-perfect claim on 24-bit WAVs with a WASAPI exclusive mode device (using my DAW to record the loopback), and I confirm that MC is indeed bit-perfect.  Some foobar2000 users recognize the sound quality problem and use an unofficial alternate output component: https://sourceforge.net/projects/foobar2000-wasap2-output/
However, I found this component crashes frequently.  Also, on hi-res recordings I have mastered, I clearly hear that MC sounds more accurate (like my DAW) than foobar2000.  So I keep using MC.

My DAW, while bit-perfect, is clumsy as an AB compare tool.  So I see an opportunity for MC to offer a general purpose bit-perfect AB compare feature with a well-designed user interface.  By general purpose, I mean that the user can select A and B files independently.  I paid for MC because of the stated plans to add this feature, and foobar2000 cannot compete because it lacks the bit-perfect aspect.  foobar2000's distortion/muffling biases users toward an opinion of no difference between A and B.

There is already a lot of current MC users in this thread asking for MC to support a general purpose AB compare feature.  I suspect that the addition of this feature would get many others who are using free players to buy MC.  I see no reason why JRiver wouldn't want to increase their market share, demonstrate the superiority of MC over free software, and improve the user's flexibility in performing valid subjective testing.
Logged
Pages: [1]   Go Up