INTERACT FORUM

Please login or register.

Login with username, password and session length
Advanced search  
Pages: [1]   Go Down

Author Topic: Listening Test is Unfit  (Read 4673 times)

mark_h

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 1854
Listening Test is Unfit
« on: December 05, 2017, 01:36:44 am »

Have explained before that the JRiver Listening Test is unfit for purpose.  Once the bit rates increase is *does* become more challenging to hear the difference, but that doesn't mean they aren't there, and so you need the tool to help, not hinder the process  The testing tool has to allow the user to flip to and fro between tracks continuing from exactly where you were - this is the only way to IMMEDIATELY hear the subtlest of differences - and it should ideally be using a loop of only a few seconds, not entire tracks.  Starting the track over , as the JRiver tool does relies entirely memory and that has been shown over and over to be a flawed method...

Something like this is MUCH more credible:

https://lacinato.com/cm/software/othersoft/abx
Logged

JimH

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 72446
  • Where did I put my teeth?
Re: Listening Test is Unfit
« Reply #1 on: December 05, 2017, 01:45:14 am »

So, if you can't reliably hear a difference, or remember it, is it really a difference?  Or is it significant?
Logged

mark_h

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 1854
Re: Listening Test is Unfit
« Reply #2 on: December 05, 2017, 04:11:24 am »

Depends on your agenda...  If it’s to get to the truth, CAN people HEAR differences, then the tools must help, not hinder. 

If the agenda is to prop up your bias then by all means build a tool that reinforces that. 
Logged

Hendrik

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 10944
Re: Listening Test is Unfit
« Reply #3 on: December 05, 2017, 05:13:51 am »

The question is if people can hear differences under real circumstances, not artifically generated scenarios. Only real conditions matter in the long run.
If you can't tell a difference without artifically repeating a very short segment over and over, then is there really a significant difference?
Logged
~ nevcairiel
~ Author of LAV Filters

mark_h

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 1854
Re: Listening Test is Unfit
« Reply #4 on: December 05, 2017, 07:50:52 am »

The question is if people can hear differences under real circumstances, not artifically generated scenarios. Only real conditions matter in the long run.
If you can't tell a difference without artifically repeating a very short segment over and over, then is there really a significant difference?

That's an answer for psychologists to determine under proper test conditions?  Do those "insignificant" differences add up to something psychologically (and maybe even physiologically) significant over time?   It's all well and good to say it doesn't matter, but based on what; I'd argue a "faulty" testing tool that distorts results and (bad) intuition...  not good enough, I'm afraid...

And anyway, that's a separate debate.  My "grievance" is with JRiver's continued stance that the Listening Tool "proves" that most people cannot tell the difference, with the implication that follows that therefore there is no difference.  It's a faulty tool that generates a false conclusion...

Give us a tool that allows us to test this properly and thoroughly, or just be honest and admit bias :P
Logged

rec head

  • Citizen of the Universe
  • *****
  • Posts: 1009
Re: Listening Test is Unfit
« Reply #5 on: December 05, 2017, 08:23:47 am »

I have noticed that while I can't always tell a difference and my ears are far from golden that lower quality bothers me over extended time periods. I have no idea why but it is what it is.
Logged

pschelbert

  • Galactic Citizen
  • ****
  • Posts: 459
Re: Listening Test is Unfit
« Reply #6 on: December 05, 2017, 08:49:41 am »

Hi

a suggestion may be to implement what foobar ABX does in JRiver.
Or just use foobar ABX for that purpose.

There you can:
-compare tracks as short or long you want, loops you define how long
-switch as fast as your muse does
-make statistics to verify if you reliably hear a difference (statistically significant, double blind)

And you may be surprised how unreliable you can distinguish a difference!

Peter
Logged

JimH

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 72446
  • Where did I put my teeth?
Re: Listening Test is Unfit
« Reply #7 on: December 05, 2017, 09:33:59 am »

And anyway, that's a separate debate.  My "grievance" is with JRiver's continued stance that the Listening Tool "proves" that most people cannot tell the difference, with the implication that follows that therefore there is no difference. 
I think you're distorting what I've said a little.  When we built the tool, our intentions were completely innocent.  I was honestly curious what we would find.

What we have found is that very few people get it right.  I've never counted, but it's in the neighborhood of 10%.

That may only mean that the high bitrates are (almost?) impossible to distinguish.
Logged

JimH

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 72446
  • Where did I put my teeth?
Re: Listening Test is Unfit
« Reply #8 on: December 05, 2017, 09:43:38 am »

If the agenda is to prop up your bias then by all means build a tool that reinforces that.
Mark,
I think you're a little over-zealous in your expression of our opinion.  Opinion isn't necessarily bias.
Logged

pschelbert

  • Galactic Citizen
  • ****
  • Posts: 459
Re: Listening Test is Unfit
« Reply #9 on: December 05, 2017, 09:59:31 am »

I think you're distorting what I've said a little.  When we built the tool, our intentions were completely innocent.  I was honestly curious what we would find.

What we have found is that very few people get it right.  I've never counted, but it's in the neighborhood of 10%.

That may only mean that the high bitrates are (almost?) impossible to distinguish.

Hi

yes its true high bitrates cannot be distinguished, other serious professionals have done extensive test (AES-paper) and I tried as well.

Take a highres 350kHz/24bit file and downsample to 44.1kHz/16bit. and compare ( I did with foobar ABX and STAX Headphones)

You can distinguish between two differently mastered files from the same source but not different bitrates for the same master, as long as it is 44.1kHz/16 bit at least.

In most cases mp3 256VBR is not distinguishable! (mp3 done with lame, which is in JRiver)

I see no need to implement all the ABX test-tools in JRiver, okay if its in its nice.

Peter
Logged

dtc

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 3119
Re: Listening Test is Unfit
« Reply #10 on: December 05, 2017, 10:58:12 am »

Hi

yes its true high bitrates cannot be distinguished, other serious professionals have done extensive test (AES-paper) and I tried as well.

Take a highres 350kHz/24bit file and downsample to 44.1kHz/16bit. and compare ( I did with foobar ABX and STAX Headphones)

You can distinguish between two differently mastered files from the same source but not different bitrates for the same master, as long as it is 44.1kHz/16 bit at least.

In most cases mp3 256VBR is not distinguishable! (mp3 done with lame, which is in JRiver)

I see no need to implement all the ABX test-tools in JRiver, okay if its in its nice.

Peter

First, the JRiver test is between various mp3 samples and higher sample rates.  And even in the little sample on this forum there are people who could easily distinguish the different formats.

If you are referring to the AES paper from the Boston group, that experiment has been debunked questioned  many times. It was not a well run experiment.

I have run experiments too. Some people can hear differences and some cannot. I know one person who can distinguish 16/44 and 24/96 routinely, using tracks that I personally created from a common source. There is no issue of different masters, different volume levels, etc. The source was various records (aka vinyls) and they were digitized using a professional A to D converter.

I wish people would stop making categorical statements. A person with perfect pitch hears differently from an average person. There are people who have trained themselves to hear very minute differences, like pre-ringing artifacts.  A professional concert violinist will hear things that the average person will not.  Human hearing is a very complicated process and people have very different hearing capabilities.  And, of course, the playback equipment matters.

This discussion will go nowhere. Some people are totally convinced that people cannot hear differences and some, like me, know people who can hear those differences.
 
Logged

JimH

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 72446
  • Where did I put my teeth?
Re: Listening Test is Unfit
« Reply #11 on: December 05, 2017, 11:32:30 am »

First, the JRiver test is between various mp3 samples and higher sample rates. 
Yes, but the original from which these are made can be anything that MC can convert. 
Logged

dtc

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 3119
Re: Listening Test is Unfit
« Reply #12 on: December 05, 2017, 11:37:51 am »

I have noticed that while I can't always tell a difference and my ears are far from golden that lower quality bothers me over extended time periods. I have no idea why but it is what it is.

The same effect is felt by many after listening to overly detailed speakers for an extended period of time. The speakers seem to sound great, but after a while a fatigue sets in, often accompanied by headaches. It is as if the brain is being overloaded in listening to them. It could be that the brain has to work too hard to interpret the data or it may be that the brain just finds them irritating. Hard to know, but it is a well known phenomenon.

We sometimes forget that the process of hearing is actually translating a set of physical impulses into an electrical signal in the brain, which we experience as "sound". It is a very complex process and differs person to person. There is even some research that very high frequencies do not come through the ear but are somehow received by other parts of the body, probably by the brain directly. Pretty heady stuff.
Logged

pschelbert

  • Galactic Citizen
  • ****
  • Posts: 459
Re: Listening Test is Unfit
« Reply #13 on: December 05, 2017, 12:59:34 pm »

Hi

yes the AES Boston paper. A lot of controvesary, however nobody showed a serious counterstudy, I am still waiting!
I did experiment with high-end, golden ears in Switzerland and Germany. Ten files, same song but different coding (CD, highres, mp3), minimum two exactly the same! People could not distinguish on their equipment (very costly $$$$), nor could I.
Some have been 100% convinced the same file sounded different!
Psychological effect, nothing strange just the ear is not an obejctive device.

Testtool is not the problem, o mak test.

With JRiver you can convert or with other tools (Pro Tools, Soundforge..) to almost any format.

Peter
Logged

marko

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 9143
Re: Listening Test is Unfit
« Reply #14 on: December 05, 2017, 02:25:01 pm »

Dunno about my ears, but my nose works just fine...
https://youtu.be/gM8vOcxatsI

:D
Seriously.... If it sounds OK to you, then it most probably is, and you should just relax and enjoy it.

dtc

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 3119
Re: Listening Test is Unfit
« Reply #15 on: December 05, 2017, 03:23:38 pm »

Hi

yes the AES Boston paper. A lot of controvesary, however nobody showed a serious counterstudy, I am still waiting!


No counter study does not counter the flaws in the study. And BAS never repeated their experiment under better conditions.

Quote
I did experiment with high-end, golden ears in Switzerland and Germany. Ten files, same song but different coding (CD, highres, mp3), minimum two exactly the same! People could not distinguish on their equipment (very costly $$$$), nor could I.
Some have been 100% convinced the same file sounded different!
Psychological effect, nothing strange just the ear is not an obejctive device.


Fine. I have a living, breathing person that can tell the differences in my tests. That is enough for me.

By the way, sometimes making the experiment too complicated invalidates the results. Human hearing is a funny thing. That is one reason I take a lot of these experiments with a grain of salt. Not being able to hear a difference may be due to the design rather than the ears.

I was talking to a well known speaker designer recently about high res music.  On his newest, inexpensive speaker, he was using a a tweeter that topped out at 20 KHz. He then tried a similar model that went to 40 KHz. He liked the 40 KHz one much better. Hard to know what exactly was going on, but he ended up using the 40 Khz one.

As I said, there will be no resolution here. I just wanted to present another opinion, backed by experiment.
Logged

pschelbert

  • Galactic Citizen
  • ****
  • Posts: 459
Re: Listening Test is Unfit
« Reply #16 on: December 05, 2017, 03:58:22 pm »

Hi

the Boston-AES study may be flawed, now its up to others to show different. Its just amazing thousand of discussions and nobody shows that they are wrong.... funny. Sure its bad for the industry that High-Res is not performing up to promises..

However the recording and mastering still is key and art what makes the biggest difference.

Peter
Logged

dtc

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 3119
Re: Listening Test is Unfit
« Reply #17 on: December 05, 2017, 06:29:47 pm »

Back to the original point, any listening test should be under the control of the listener and the listener should be able to switch to an alternate version at the same listening point. Many people have relatively short hearing memory, so, for them,  the differences need to be tested on a short term basic. And, yes, hearing short term differences is relevant. It may not be the way we normally listen, but it does show preferences, which can be relevant to long term listening pleasure. 
Logged

JimH

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 72446
  • Where did I put my teeth?
Re: Listening Test is Unfit
« Reply #18 on: December 06, 2017, 01:18:25 am »

Closing this now.
Logged
Pages: [1]   Go Up