Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[mwillems]

If you compare two files, there is a 50% chance you'll get it right.

Can you reliably rank the FLAC above a 256Kbps MP3, five times in a row?

For all the complaints about what we did, I'm not seeing many reports of real tests.  That suggests that people may not be reporting the results.  Or that there is a complete lack of interest.

I think part of the issue is that much of the interest in this thread is going in a different direction.  Several people in the thread (including me) were hoping for a way to compare two or more files of their choice in an ABX way.  That can be handy for a lot of reasons other than the audibility of sample rates/bit depth, etc. 

For example, I've bought many remasters of albums, and for some of my favorite albums I have five versions!  The sample rate is all the same, but I know the masters are all different.  I would love a way to be able to compare all five versions of 21st century schizoid man, for example, and see which one I really prefer. 

Having a way to test the audibility of FLAC v. mp3 is nice, but, for my part, I'd like a more general purpose tool that could be used to test all kinds of differences.

[mvilla]

Several people in the thread (including me) were hoping for a way to compare two or more files of their choice in an ABX way.  That can be handy for a lot of reasons other than the audibility of sample rates/bit depth, etc. 

Fully Agree

[GreenMan]

We can certainly remove the [Listening Test] feature if people are offended.  There are plenty of other things we could be doing.

When one states any opinion on the audibility of something, some people will be offended if it does not agree with their experience.  But people do change their opinions as they gather more experience.  By providing a good quality tool, you are not pushing your opinion -- just providing your valued customers a way to gather more experience in a way that they want.

While foobar2000 provides a general purpose AB compare, foobar2000 makes no claim to be bit-perfect.  I have performed a test of MC's bit-perfect claim on 24-bit WAVs with a WASAPI exclusive mode device (using my DAW to record the loopback), and I confirm that MC is indeed bit-perfect.  Some foobar2000 users recognize the sound quality problem and use an unofficial alternate output component: https://sourceforge.net/projects/foobar2000-wasap2-output/
However, I found this component crashes frequently.  Also, on hi-res recordings I have mastered, I clearly hear that MC sounds more accurate (like my DAW) than foobar2000.  So I keep using MC.

My DAW, while bit-perfect, is clumsy as an AB compare tool.  So I see an opportunity for MC to offer a general purpose bit-perfect AB compare feature with a well-designed user interface.  By general purpose, I mean that the user can select A and B files independently.  I paid for MC because of the stated plans to add this feature, and foobar2000 cannot compete because it lacks the bit-perfect aspect.  foobar2000's distortion/muffling biases users toward an opinion of no difference between A and B.

There is already a lot of current MC users in this thread asking for MC to support a general purpose AB compare feature.  I suspect that the addition of this feature would get many others who are using free players to buy MC.  I see no reason why JRiver wouldn't want to increase their market share, demonstrate the superiority of MC over free software, and improve the user's flexibility in performing valid subjective testing.

[AndyU]

For example, I've bought many remasters of albums, and for some of my favorite albums I have five versions!  The sample rate is all the same, but I know the masters are all different.  I would love a way to be able to compare all five versions of 21st century schizoid man, for example, and see which one I really prefer. 

Forgive me if I seem thick, but what's stopping you just playing them and seeing which one you prefer? Get someone to disguise the identity in some way beforehand.

[GreenMan]

Forgive me if I seem thick, but what's stopping you just playing them and seeing which one you prefer? Get someone to disguise the identity in some way beforehand.

My experience is that instantaneous switching back and forth between two versions of the same track improves my ability to detect differences between them.  Listening to whole track A and then whole track B puts much more demands sonic memory.  I feel more distraction and stress trying to remember what track A sounded like when I am listening to track B.  My experience is that the scenario of whole track A and then whole track B biases me to conclude that there is no difference.

[mwillems]

Forgive me if I seem thick, but what's stopping you just playing them and seeing which one you prefer? Get someone to disguise the identity in some way beforehand.

Getting someone to disguise the identity before hand is what I do now, but it takes some doing and involves roping someone else with a basic level of technical ability into the process.  So I don't do it very often, and it requires advanced planning, etc.  It would be nice to be able to just "do it" on a whim and to my heart's content.

[MusicBringer]

Fully Agree

Me too.
...and these days who uses empty threes. I certainly don't.

Without doubt we require an ABX type test please.

[mark_h]

With the minimum functionality of something like the Lacinato ABX tester:

http://lacinato.com/cm/software/othersoft/abx

MC has the opportunity to go even further with added functionality.

[dtc]

I would also like to see the ability to compare the same file with and without DSP Studio options applied. That would allow an easy comparison, for example,  between Redbook and an upsampled version, or the results  of equalizer settings, etc.

[imeric]

Some foobar2000 users recognize the sound quality problem and use an unofficial alternate output component: https://sourceforge.net/projects/foobar2000-wasap2-output/

 I clearly hear that MC sounds more accurate (like my DAW) than foobar2000.  So I keep using MC.

foobar2000's distortion/muffling biases users toward an opinion of no difference between A and B.

Good to hear I'm not alone! Last time I did a blind test comparison bwn Foobar and MC I got a 10/10 score using Wasapi for both but that was with my Benchmark DAC1.  I then since upgraded a while back and need to redo this with my Exasound DAC and ASIO...)  (But for this I need someone to assist for the blind test like last time) But even without a blind test I still find MC more accurate than Foobar.

Good to hear I'M not the only one who noticed the less accurate MUFFLED sound (muffled...that is the perfect word for it!!) of Foobar

I need to re-do an A/B with Foobar's new output against MC as I never tried it. I still use Foobar due to the enhanced crossfading and lyrics plugin and still use it for casual listening.  I also love how you can customise its layout...

Totally agree with mcwillens as well. We need to be able to choose the files we want with more flexibility on conversion.

[mark_h]

I deliberately chose a relatively long audio test segment to help reduce/avoid audio memory since it seems to me the value in a test like this is to see what one perceives in absolute terms without a relatively recent audio reference to compare to. Comparing relatively short audio snippets may enable me to be more accurate at the higher bit-rates but its not how I listen to (and enjoy) music. 

As, you say, it depends on your goals.  If the goal is to determine whether you can tell one format from another, then memory is your enemy; the longer the clip, the harder it is especially when the quality of samples become increasingly comparable.  You need to use shorter samples and have the ability to switch live between samples that are tracking each other in time and volume.  Even starting over a short track from the beginning can "confuse" the process.

[dtc]

As, you say, it depends on your goals.  If the goal is to determine whether you can tell one format from another, then memory is your enemy; the longer the clip, the harder it is especially when the quality of samples become increasingly comparable.  You need to use shorter samples and have the ability to switch live between samples that are tracking each other in time and volume.  Even starting over a short track from the beginning can "confuse" the process.

This post below was moved from this thread, but has a direct bearing on these recommendations. It concludes that long term comparisons are far better than short term comparisons. Part of the problem with ABX comparisons is that their is no universally accepted test methodology and the methodology can have a significant effect on the outcome.

http://yabb.jriver.com/interact/index.php/topic,108083.msg749401.html#msg749401

[GreenMan]

I agree with mark_h.  Windows users can try foobar2000's ABX component to avoid the audio memory problem in their experiment.  I find that the crossfade feature of the ABX effectively causes more audio memory to be needed, so I disable the crossfade to better hear differences.  By disengaging the distraction of audio memory, aspects of sound perceived by other parts of the brain become more apparent.  The brain science is similar to how a person can shift their visual attention between central vision and peripheral vision.  Focusing on the peripheral vision requires relaxing, which won't happen when the person is distracted by something in their central vision.

I find 320kbps MP3 to preserve the intellectual aspects of the music, but degrade the emotional aspects (which is why I listen to music).  I find the difference even more apparent compared to hi-res recordings instead of red-book CD.  Different parts of the brain involved in the two aspects.  When I am listening for emotional differences between A and B, I find I need a relaxed training period before I notice them.  Being stressed or distracted causes only the sympathetic nervous system to be active, while I need the parasympathetic nervous system to dominate to perceive the emotional differences.  Once I am trained on these differences, I can focus on them quickly and do well on an ABX test.

Primarily where I hear the emotional degradation of 320kbps MP3 is in the bass.  Make sure to listen with well-balanced loudspeakers or headphones flat down to 20Hz.  This is inexpensive to achieve with in-canal headphones like the Etymotic Research HF5.  Bass is better perceived with the full body (not just the ears), and this is least expensively achieved in car with a subwoofer.  Getting loudspeakers to produce symphony levels flat down to 20Hz is hugely expensive in a living room.

Finally, having JRiver customers reporting on audibility of MP3 distortion is complicated by the variability in how muffled/balanced their playback system and recordings are.  Trained experimental psychologists know to minimize such variables by comparing the results from difference subjects on the same playback system and the same well-recorded music.  If MC is not configured as bit-perfect, then there is the muffling associated quantization distortion of digital volume control and/or digital filtering.  Better to have the volume control in analog after the DAC so MC can be configured as bit-perfect.

Foobar2000 makes no claim to be bit-perfect, so it can be the dominant muffling factor in the experiment.  MC, which can be configured as bit-perfect, doesn't offer instantaneous switching between A and B, so it can't avoid the audio memory effect today.  I propose that MC is doing more harm than good by having a "listening test" feature with such an experimental psychology methodology flaw.

[GreenMan]

Note that ABX tool is very flexible in that it can be used for listening to whole tracks or instantaneous switching.

[GreenMan]

As a mastering engineer, I offer artist clients the options of any distribution format they want: hi-res, red-book CD, 320 kbps and iTunes 256 kbps AAC.  Clients, of course, want to hear the differences before they decide what to distribute.  MC today doesn't offer the ability to compare all these formats.  A general-purpose compare tool with user-specified files would achieve this.  Since my own experience is that instantaneous switching between A and B is necessary to avoid the audio memory problem, I want to refer my clients to a tool that has this feature.  Finally, MC's bit-perfect feature is what would differentiate it from other AB compare tools.  In summary, if MC adds a general-purpose ABX tool, I will be sending clients JRiver to buy MC (who otherwise wouldn't know about or buy MC).

[AndrewFG]

A thought for the budding brain scientists among you..

I would suggest that instead of asking yourselves one question "which one sounds better" you should try to ask yourselves two questions: Firstly "can I hear a difference", and secondly "given that I heard a difference, which one do I prefer"..

The reason is that you use different parts of your brain to answer each question; namely what I would call the "difference engine" and the "preference engine". The human brain's difference engine is stunningly good, whereas its preference engine is actually pretty lousy.

And furthermore the brain's self referential judgment module needed to answer the question "which one do I prefer", is subtly but significantly different to its external referential judgment module needed to answer the question "which is better".

Have a lot of fun everybody!

[mark_h]

This post below was moved from this thread, but has a direct bearing on these recommendations. It concludes that long term comparisons are far better than short term comparisons. Part of the problem with ABX comparisons is that their is no universally accepted test methodology and the methodology can have a significant effect on the outcome.

http://yabb.jriver.com/interact/index.php/topic,108083.msg749401.html#msg749401

Interesting, but doesn't work for me - if you ask me to tell the difference I need short samples to be accurate.  But I get the argument, and I'm sure there is something to it; the longer you listen to something, the more attuned you become to the signature of the sound - and then when you switch you will notice "something" is different.

[kstuart]

For example, I've bought many remasters of albums, and for some of my favorite albums I have five versions!  The sample rate is all the same, but I know the masters are all different.  I would love a way to be able to compare all five versions of 21st century schizoid man, for example, and see which one I really prefer.

The problem is that later, you may find that your improved listening ability might make you prefer a different one.

For some reason, there is an insistence that - while on the one hand, Joshua Bell can play violin better than some neighborhood kid who has played for two weeks - yet - "everyone can hear music equally".

Any studio engineer knows that in the area of listening, "practice makes perfect", and at this point, most of them have stopped posting in forums due to the average consumer thinking "I can do everything just as well as anyone else".

From comments posted on various sites, it seems clear that people who evaluate different releases of the same album, do not even understand what might make one better than another.

While it varies quite a bit between genres, I find that about 99% of "remasters" are worse than the earlier CDs.  They use simple tricks like boosting bass and treble EQ to separate consumers from their dollars one more time.  (Mobile Fidelity has been famous for decades for "U shaped EQ".)