Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[JimH]

MC22.0.41 added a new Listening Test, a comparison feature available from the Tools > Library Tools Menu.  It allows you to listen to different formats of the same track, rate them, and then find out how good you are at distinguishing them from each other.

Here is how it works.

1.  Find an audio track in your library.

2.  Select it and choose Listening Test from the Library Tools menu.

3.  MC will then convert the track to several new temporary files whose names are A, B, C, etc. and automatically put them in Playing Now.

4.  Play them as much as you want, rate them with MC's star ratings.

5.  When you click done, the results will be displayed.

6.  When you press OK on the results page, the temporary files will be deleted.  Your original file is not affected in any way.

[JimH]

Once you've tried it, please post your results.  Most of mine were embarrassing.   

[ssands]

By different formats, do you mean different bit-depths and sampling, and lossy vs. non-lossy?

[JimH]

Right now, it's these:
FLAC 16/44.1
MP3 256Kbps
MP3 128K
MP3  64K
MP3  32K

We may change this in the future, but even now, it's a tough test to pass.

[~OHM~]

Right now, it's these:
FLAC 16/44.1
MP3 256Kbps
MP3 128K
MP3  64K
MP3  32K

We may change this in the future, but even now, it's a tough test to pass.

I Knew it!

[JimH]

The build required will be posted on this board in a few hours.

[DocLotus]

I've been wanting something like this for years to compare various formats.

I assume that for this test to be meaningful I will have to select a track with the highest possible bit depth as a starting point?

[JimH]

You could select anything MC can convert but it should be lossless at least.

[DocLotus]

Got version .41. Ran the test, a pop-up answered my above question (must be at least 44.1 kHz, 300 kb, 16 bit).

GREAT job with the info.

[DocLotus]

Won't convert my tracks. All my tracks are MP3 Extreme which means most are much less than 300 kbps.

Looks like a track has to be at least 300 (or lossless) as the popup stated.

[JimH]

You could rip a CD to FLAC, just to try it.

[Matt]

Coming next build we'll have this:
Fixed: The listening test wouldn't accept low or medium bitrate MP3 files.

[DocLotus]

Ahhh...  YOU GUYS ROCK!!! 

This is going to be one great feature.

[Knick]

Crashed about 4/5 of way through three times using same file.
Windows 10 64 bit
Flac file was "Take Five"-24 bit, 176.4K version
attached a screenshot if that is helpful.

[Matt]

Crashed about 4/5 of way through three times using same file.
Windows 10 64 bit
Flac file was "Take Five"-24 bit, 176.4K version
attached a screenshot if that is helpful.

Coming next build:
Fixed: Doing a listening test with a really high sample rate file could lead to a crash.

Hopefully that solves the problem.

Thanks.

[AndrewFG]

I did not test it, but does it include the original track also in the A, B, C, comparison list?

[Matt]

I did not test it, but does it include the original track also in the A, B, C, comparison list?

It does not include the original, but one of the copies is a lossless FLAC file so it's pretty much the same.

[AndrewFG]

I was thinking of the people with HD audio (e.g. >16bit depth, >48kHz sample rate) who will certainly want to compare their HD originals versus the CD quality Flac transcode; or those who will want to compare their SACD / DSD rips versus the CD quality Flac. ( perhaps I'm thinking a bit ahead here.. )

So I would suggest to include a copy* of the original among the A/B/C selection. ( *I suggest to make a copy of the original so that the real original does not risk being lost by mistake.. )

[dtc]

Any chance for a traditional A-B comparison tool, that allows comparisons between 2 user chosen files with user controlled switch timing , with both a random and a user controlled switch.  There have been requests for that over the years and it seems like a more general purpose tool. Foobar has an A-B comparison tool, but having to switch players changes the test conditions. I would suggest two modes - one with 2 user chosen files and one with 1 user chosen file with and without DSP enabled. An option to switch tracks  either from the start or from the current playing position would be nice. No file copying necessary.

[imeric]

This is awesome! Can't wait to try it out!

Would it be possible to add an option where we could call the Format converter to generate 4-5 lossy files and manually choose our own conversion settings?

I could then use the external encoder option in MC.

I would personally love to blind test Lossless against Lame, Apple AAC (QAAC), fdk_aac and different bitrate settings I could manually change.

Or if easier just let us choose 4-5 files to choose from in a playlist (that I would convert myself manually in MC)...

[JimH]

Did anyone get all the tracks rated correctly?

[JimH]

I was thinking of the people with HD audio (e.g. >16bit depth, >48kHz sample rate) who will certainly want to compare their HD originals versus the CD quality Flac transcode ...

We'll try to add this.

[imeric]

But the huge problem with the Listening test is that it can change output settings that it should not. My e22 DAC can play DSD up to 256 and PCM up to 352,800, so in DSP studio I have Sample Rate set to "No change" for all rates. When the Listening test attempts to play the mp3 32 or 96 files MC tells me playback is not possible, and asks if it can change my output settings. For the sake of the test I said yes. When the test finished I checked the output settings and found that it had change EVERY sample rate other than 44.1 and 48 to 48. Obviously not required or desired.

I just tried with my e22 and had the same thing.  MC is doing what it's supposed to.  The e22 can't play anything below 44.1 PCM.

[imeric]

Crashed about 4/5 of way through three times using same file.
Windows 10 64 bit
Flac file was "Take Five"-24 bit, 176.4K version
attached a screenshot if that is helpful.

ALso crashed with a dsf file

[kr4]

All this is very interesting in theory.  Most of us have specific comparisons in mind and would like to be able to select 2 or 3 original files and run a comparison.  Personally, the formats I would be interested in comparing are all (well) above 16/44.1 and, possibly, in multichannel.  Of course, that can run up against system resource limits.

[imeric]

I can definitely see myself spending a lot of hours on this if it allows for a bit more flexibility in the conversions...No need for the Wife to help out with blind tests anymore

Lossless vs Lossy
LAME vs AAC
Redbook vs Hi-Res
PCM Hi-Res vs DSD
And so on !!!

As Jim pointed out..I maybe in for a little surprise with the results I get...

[kstuart]

Various conditions can affect this:

* State of mind   (e.g. just read another article on election results  )

* Full tracks vs short section

* Headphones vs speakers

* Morning vs night

etc.

[dtc]

All this is very interesting in theory.  Most of us have specific comparisons in mind and would like to be able to select 2 or 3 original files and run a comparison.  Personally, the formats I would be interested in comparing are all (well) above 16/44.1 and, possibly, in multichannel.  Of course, that can run up against system resource limits.

Agreed. I would use a tool that allowed  me to select the tracks to compare, but doing pre-set low sample rates is not of much interest.

[Dawgincontrol]

Tried three different files.  All Lossless format, various bitrates.  One jazz, one rock, and one classical.

Easily picked the FLAC file every time and the highest MP3 was 2nd all three times.  Various success with the next three MP3s.  Took a little time going back and forth through 5 formats.   Started to get listening fatigue by the end.

It's a good start but would love an ability to compare HI-res vs 16 bit cd transfers.  Also, if we could only do a one on one comparison from a choice it would be great.

[JimH]

Thanks.  It's nice to see real data.  It would be good to know how reliably you can pick the best and the worst format if you test 5 or 6 times.

[JimH]

All this is very interesting in theory.  Most of us have specific comparisons in mind and would like to be able to select 2 or 3 original files and run a comparison.  Personally, the formats I would be interested in comparing are all (well) above 16/44.1 and, possibly, in multichannel.  Of course, that can run up against system resource limits.

There is a new build here:
https://yabb.jriver.com/interact/index.php/topic,107927.0.html

It uses the original file as one of the test files. 

[kr4]

There is a new build here:
https://yabb.jriver.com/interact/index.php/topic,107927.0.html

It uses the original file as one of the test files.

And the options are there for the others?

[JimH]

At this time, it only converts to lower resolution formats.  Is that what you're asking about?

[jmone]

I'll fess up on my results - I've used a couple of well listened to tracks on a pair of Grado SR60....

This is very informative.  Another subjective listening test over a few different tracks.  I can easily pick the MP3 32 and 96.... maybe the 128 (but I'm probably kidding myself), and certainly can not differentiate the 256 and FLAC tracks.  So for my ears it looks like the stock std CD / BD lossless Formats are fine and already deliver a sound quality better than I can discern.  That said I do hear a different between differently mastered albums with the later HD tracks much more pleasing (but this presumable is more to do with the re-master rather than the formats being used).

[mwillems]

I can (sometimes) tell the difference between 256 and FLAC on tracks with lots of loud low bass on systems that can reproduce it.  I think that's because one of the ways that mp3 compresses is by attenuating very high and very low frequencies. There's definitely one track I have with some some loud 25Hz rumbles where I can get 4 out of 5 right every time and occasionally all 5 (I can just listen for volume/distortion and it's pretty obvious). 

But it's kind of a synthetic test.  For actual, you know, music, I can't do anywhere near that well.

Also a small "bug"  If you hover over the tracks while you're rating them, the tool tip will pop up and show you the file name, which will tell you whether it's an mp3 or a flac.  The file names should either be less obvious, or the tool tip should be disabled I would think.

[kr4]

At this time, it only converts to lower resolution formats.  Is that what you're asking about?

Too bad.  I want to compare specific formats, e.g., DSD vs. DXD or DSD64 vs. DSD128 vs. DSD256, and nothing below 16/44.1/

[JimH]

Too bad.  I want to compare specific formats, e.g., DSD vs. DXD or DSD64 vs. DSD128 vs. DSD256, and nothing below 16/44.1/

We may do that in the future, but what happens if you use it now?  Can you reliably identify the best format and the worst?

[Matt]

I want a way to compare APE and FLAC files because to my ears, FLAC always sounds a little flat by comparison.  APE files are just perfectly true to the original and FLAC is just lacking that little edge.

Ha!

[JimH]

Also a small "bug"  If you hover over the tracks while you're rating them, the tool tip will pop up and show you the file name, which will tell you whether it's an mp3 or a flac.  The file names should either be less obvious, or the tool tip should be disabled I would think.

The tooltip was supposed to be disabled in the last build or two.  What build are you using?

[imeric]

I want a way to compare APE and FLAC files because to my ears, FLAC always sounds a little flat by comparison.  APE files are just perfectly true to the original and FLAC is just lacking that little edge.

Ha!

Isn't the track fully decompressed before it plays in MC? Just wondering how that would be possible...

[mwillems]

The tooltip was supposed to be disabled in the last build or two.  What build are you using?

I'm using .41, which is the one where tooltips are supposed to be off, but they weren't for me.

[JimH]

On Windows?

[kr4]

We may do that in the future, but what happens if you use it now?  Can you reliably identify the best format and the worst?

From long history, I have no interest in MP3 so testing it is, imho, a waste of my time. 

[JimH]

From long history, I have no interest in MP3 so testing it is, imho, a waste of my time.

I'm not pushing MP3 at all.  I just want to know two things. 

Is the test working for most people?

At what point can most people reliable and consistently hear a difference?

If you don't want to try it, that's fine.  I do expect that we will expand it or add different tests.

[dtc]

 

Is the test working for most people?

As I said above, A-B testing with the user selecting the tracks would be of use, but I have no interest in the current implementation.

In my house, if I put on an MP3 my wife walks in and says "what's that?. She definitely can hear the difference. That's all the test I need.

[mwillems]

On Windows?

Yes .41 on windows.

[psam]

I have tried quite a few 24/192 FLAC files , all with the same result.
MC crashes when the progress bar gets to the end.
Standard 16/44.1 FLAC files work fine. I always make the two lower rate MP3s and the original. The other two bitrates (256 & 128) sound the same to me.

[JimH]

Are you using 22.0.41?

https://yabb.jriver.com/interact/index.php/topic,107927.0.html

[kstuart]

Too bad.  I want to compare specific formats, e.g., DSD vs. DXD or DSD64 vs. DSD128 vs. DSD256, and nothing below 16/44.1/

If you want to do that with someone else's files (and in a non-automated way), you can go to:

http://www.2l.no/hires/index.html

for free downloads.

[kr4]

If you want to do that with someone else's files (and in a non-automated way), you can go to:
http://www.2l.no/hires/index.html
for free downloads.

Yes, I know but I have many others as well.  It is the automated, blind testing that I seek.