INTERACT FORUM

Please login or register.

Login with username, password and session length
Advanced search  
Pages: 1 [2]   Go Down

Author Topic: NEW: Listening Test  (Read 43089 times)

MediaJunkey

  • Member
  • *
  • Posts: 2
Re: NEW: Listening Test
« Reply #50 on: November 19, 2016, 01:36:08 pm »

Nice feature. Little embarrassed I picked MP3 128 as my second favorite on one song but at least I did pick flac as the best. On another I picked flac as the third best, lol. Should get rid of the MP3 32, sounds horrible and is easy to spot. Maybe replace it with MP3 160 or an even harder challenge would be MP3 320.

 ;)
Logged

JimH

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 71348
  • Where did I put my teeth?
Re: NEW: Listening Test
« Reply #51 on: November 19, 2016, 02:09:08 pm »

Should get rid of the MP3 32, sounds horrible and is easy to spot.
32Kbps is there so everyone is a winner on something.
Logged

psam

  • World Citizen
  • ***
  • Posts: 153
Logged

GreenMan

  • Recent member
  • *
  • Posts: 16
Re: NEW: Listening Test
« Reply #53 on: November 20, 2016, 03:08:18 pm »

If JRiver would support letting the user choose the A file and the B file separately, then users can compare any difference in processing they want.  This is the most flexible, and it would make the "Listening Test" feature a true AB compare.  For me, it is only important to compare WAV files of the same sample rate.  If I want to compare against a lossy compressed format, I can decode it to WAV first.  In fact, I suggest that a true AB compare be limited to files with the same sample rate at the feature introduction for ease of implementation.

I do professional music mastering, and I need this to compare different versions of my processing.  I bought MC22 because the "Plans for MC22" thread says that the "AB Comparison for listening tests" feature is included.

While cross fading between A and B tracks avoids a possible click, I find that it interferes with my ability to hear the difference between A and B.  Differences in spacial ambience are much easier to hear when the transition between A and B is as quick as possible.  If MC users insist on cross fading, please make it a defeatable option.

Please also ensure that MC supports letting the user train on the A and B files manually (sighted) for as long as they want before starting the blind testing phase.  foobar2000 was smart to include this in it's ABX comparator.
Logged

hoyt

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 863
Re: NEW: Listening Test
« Reply #54 on: November 21, 2016, 05:26:52 pm »

If JRiver would support letting the user choose the A file and the B file separately, then users can compare any difference in processing they want. 

Agreed.

Did people request the ability to compare lower resolutions?  I record music and often want to compare microphones, preamps, conversion techniques, mic patterns, locations, etc.  Those are different files.  I've never said to myself, "Gee, I wish I could compare a 128k MP3 to a FLAC file."  Maybe other people did, but certainly not the limitations that I anticipated for an AB test.  When is version 2 due? :)
Logged

JimH

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 71348
  • Where did I put my teeth?
Re: NEW: Listening Test
« Reply #55 on: November 21, 2016, 05:55:05 pm »

Agreed.

Did people request the ability to compare lower resolutions?  I record music and often want to compare microphones, preamps, conversion techniques, mic patterns, locations, etc.  Those are different files.  I've never said to myself, "Gee, I wish I could compare a 128k MP3 to a FLAC file."  Maybe other people did, but certainly not the limitations that I anticipated for an AB test.  When is version 2 due? :)
As I said above, we may extend this feature, but for now, let's see whether you and others can successfully tell the difference consistently between FLAC and 256Kbps MP3.  So far, the record isn't clear.
Logged

rec head

  • Citizen of the Universe
  • *****
  • Posts: 1004
Re: NEW: Listening Test
« Reply #56 on: November 22, 2016, 07:36:17 am »

It should probably be called a hearing test or equipment test because it doesn't seem to have the listening features people are looking for.
Logged

AndrewFG

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 3392
Re: NEW: Listening Test
« Reply #57 on: November 22, 2016, 07:46:41 am »

but for now, let's see whether you and others can successfully tell the difference consistently between FLAC and 256Kbps MP3.  So far, the record isn't clear.

Dear Jim, I don't think it should be your role (even if you are the CEO) to try to "prove" that people can or cannot hear a difference between Flac and Mp3. And even if you do "prove" something, your role is certainly not to rant about it. On the contrary I think your role is to deliver the best dang Media Player that money can buy, and deliver features that give your customers the most amount of fun.
Logged
Author of Whitebear Digital Media Renderer Analyser - http://www.whitebear.ch/dmra.htm
Author of Whitebear - http://www.whitebear.ch/mediaserver.htm

JimH

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 71348
  • Where did I put my teeth?
Re: NEW: Listening Test
« Reply #58 on: November 22, 2016, 07:59:48 am »

We can certainly remove the feature if people are offended.  There are plenty of other things we could be doing.
Logged

rec head

  • Citizen of the Universe
  • *****
  • Posts: 1004
Re: NEW: Listening Test
« Reply #59 on: November 22, 2016, 01:27:47 pm »

It should probably be called a hearing test or equipment test because it doesn't seem to have the listening features people are looking for.

Sorry, I should have said that I like the feature. The only thing I would change would be that I would like to switch between files as they are playing and without the next track starting over.
Logged

theoctavist

  • World Citizen
  • ***
  • Posts: 228
  • a bad liver and a broken heart.
Re: NEW: Listening Test
« Reply #60 on: November 23, 2016, 03:10:18 am »

ive never seen *anyone* reliably identify 320kbps(or 256 for that matter) vs redbook/lossless

great feature, yall! thanks!
Logged

KingSparta

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 20048
Re: NEW: Listening Test
« Reply #61 on: November 24, 2016, 02:18:17 pm »

Cross Fade should be turned off, then turned back on after your done with the test..

I was getting a crash(with warning) with 22.0.43, Sometimes, not sure yet what may trigger.

next time it does it i will take a print screen.
Logged
Retired Military, Airborne, Air Assault, And Flight Wings.
Model Trains, Internet, Ham Radio
https://MyAAGrapevines.com
Fayetteville, NC, USA

mark_h

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 1851
Re: NEW: Listening Test
« Reply #62 on: November 26, 2016, 03:54:42 am »

ive never seen *anyone* reliably identify 320kbps(or 256 for that matter) vs redbook/lossless

I've done it many times.  Not with the JRiver tool, mind you, as it's not yet a suitable tool for such testing.
Logged

JimH

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 71348
  • Where did I put my teeth?
Re: NEW: Listening Test
« Reply #63 on: November 26, 2016, 07:02:34 am »

If you compare two files, there is a 50% chance you'll get it right.

Can you reliably rank the FLAC above a 256Kbps MP3, five times in a row?

For all the complaints about what we did, I'm not seeing many reports of real tests.  That suggests that people may not be reporting the results.  Or that there is a complete lack of interest.
Logged

BillT

  • World Citizen
  • ***
  • Posts: 207
Re: NEW: Listening Test
« Reply #64 on: November 26, 2016, 08:09:47 am »

Or that there is a complete lack of interest.

Yes.

By and large the result of comparisons is uninteresting. Done some in the past and now just want to play the music.

There is room for some mildly interesting comparisons. I'd be interested to see the results of a properly conducted, statistically significant comparison of HD audio with the same source properly converted to CD audio quality to find out if the difference was actually detectable (it's almost certainly insignificant). I haven't seen any so far - too much vested interest on the HD side and not enough interest on the rational side.
Logged

JimH

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 71348
  • Where did I put my teeth?
Re: NEW: Listening Test
« Reply #65 on: November 26, 2016, 10:23:35 am »

I've removed some off topic posts.

If you have suggestions on improving this feature, please use this thread.  If you want a different feature, please use another thread.  I realize that you may find this a difficult distinction.
Logged

JimH

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 71348
  • Where did I put my teeth?
Re: NEW: Listening Test
« Reply #66 on: November 26, 2016, 10:32:47 am »

Logged

AndyU

  • Galactic Citizen
  • ****
  • Posts: 363
Re: NEW: Listening Test
« Reply #67 on: November 26, 2016, 02:01:37 pm »


If you have suggestions on improving this feature, please use this thread. 

My suggestion would be a way of testing a file playing normally against the same file with the computer driven close to maximum processing etc.. It is often claimed by a section of the audiophile community that "reducing processing" or some such improves sound quality.

Add to that comparing a file being played back from memory to the same file being read from disc.
Logged

mark_h

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 1851
Re: NEW: Listening Test
« Reply #68 on: November 27, 2016, 02:56:24 am »

If you compare two files, there is a 50% chance you'll get it right.

Can you reliably rank the FLAC above a 256Kbps MP3, five times in a row?

Me?  Yes.  But as I've said before, not with the MC Listening Test as the tool gets in the way of the process.  320 vs FLAC is the real challenge and requires quite a bit of concentration to do, but it's doable and to a statistically significant degree.

At lower bitrates there are obvious "tinklies" (high frequency distortion) and other artefacts that give mp3 away.  At 320, they are mostly absent, but tinklies can still be there in certain instruments, eg tambourines/chimes/cymbals, ie the high-frequency instruments.  But the giveaway differences usually come down to things like the air between the instruments, or the definition of the instruments themselves, with the mp3 smearing the boundaries ever so slightly, presenting a more homogenous sound, and FLAC giving the accurate presentation.  Often subtle, but always there.

The tool is fine for showing that mp3 can be "good enough" for casual listening, eg sat at a computer typing this message.  Maybe that's all you are trying to do.



 

Logged

OverTheAir

  • Recent member
  • *
  • Posts: 43
Re: NEW: Listening Test
« Reply #69 on: December 01, 2016, 05:00:12 pm »

Useful feature IMHO. I only did one test run using an 8 minute Bruch violin CD track (so 40 minutes total test time), where I listed to each track just once before ranking. Clearly I should do more test runs with different recordings on different audio systems. The test was run on my desktop using the motherboard Realtek audio chip and Pioneer BR41 speakers, alternative options are to replace the Realtek with an ODAC USB DAC, and also run tests on my more expensive (higher quality?) home theater audio system to see if I can repeatedly, accurately discern differences in all three equipment sets, just the more expensive system or none at all.

In my one run through, my 58 year old ears detected the the 32k and 64k accurately but didn't rank the 128k, 256k and Flac correctly. For reference, I have no history of exposing my ears to loud concert music but online audio tests played back through the first set of equipment suggest a high end frequency limit of ~13-14k. I deliberately chose a relatively long audio test segment to help reduce/avoid audio memory since it seems to me the value in a test like this is to see what one perceives in absolute terms without a relatively recent audio reference to compare to. Comparing relatively short audio snippets may enable me to be more accurate at the higher bit-rates but its not how I listen to (and enjoy) music. 

Interestingly my perception of 128k classical audio streams has been that they sound "compressed" and "lacking highs" when selecting either higher bit rate streams or playing back my CD rips, but there are so many other differences when doing so that its actually meaningless, so I'll be interested in more rigorously testing the ability to determine between 128k vs 256/Flac .
Logged

JimH

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 71348
  • Where did I put my teeth?
Re: NEW: Listening Test
« Reply #70 on: December 02, 2016, 11:30:04 am »

I moved a few posts.  Please use this thread for general discussion:
https://yabb.jriver.com/interact/index.php/topic,108131.0.html
Logged

Woodchuck#1

  • Recent member
  • *
  • Posts: 18
Re: NEW: Listening Test
« Reply #71 on: December 05, 2016, 07:26:40 pm »

Does it works with a CD track?
Logged

gingellr

  • Junior Woodchuck
  • **
  • Posts: 54
Re: NEW: Listening Test
« Reply #72 on: December 13, 2016, 05:50:45 pm »

Great Idea and a good test thanks for the effort. Interesting to try it out so keep up the good work.

Just did two quick tests, first one i got all 5 in the correct order....so was confident....
Second test got the flac and  the 32 correct the other 3 were the wrong order!

Its late and need to try it out properly when i'm not tired.

The flac stands out by a long way from the others and is obvious as the best quality and 32 is so obviously the worse.

and not to state the obvious but some systems wont be able to pick up the differences and some will, so it maybe relevant for people to state what equipment the test was ran on.

I am using an audiolab dac/pre amp, bryston poweramps and pmc ob1 speakers.
Logged

JimH

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 71348
  • Where did I put my teeth?
Re: NEW: Listening Test
« Reply #73 on: December 13, 2016, 06:26:33 pm »

Great report.  Thanks.
Logged

Blu99Zoomer

  • Recent member
  • *
  • Posts: 46
Re: NEW: Listening Test
« Reply #74 on: December 23, 2016, 11:20:34 am »

Just wanted to say that I like this option!  I have tried it once.  I am embarassed to say that I only got the 32 correct!  Ugh!!  But I will try again and see if I can replicate my results.  I hope NOT!

Best Regards,

Blu99Zoomer
Logged

Rockets71

  • Recent member
  • *
  • Posts: 14
Re: NEW: Listening Test
« Reply #75 on: December 25, 2016, 10:09:51 am »

Thank you for the Listen Test. For my first use of this feature I utilized the track Freddie Freeloader from Kind of Blue by Miles Davis (which is a 192 KHz 24 bit FLAC version purchased from HD Tracks). Not withstanding my need to check the Hardware Buffer box in ASIO4ALL in order to eliminate issues associated with 192 KHz playback in my system (for all other sample rates the box must remain unchecked), I was able to clearly discern all of the lower quality MP3s from the FLAC versions. It does require active listening, but you can really hear the difference. This result is a particularly satisfying confirmation that my decision to build my computer around my HT Omega Claro Halo (outfitted with version 4 Burson Supreme Sound Op Amps) was indeed a good one.
Logged

rossp

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 772
Re: NEW: Listening Test
« Reply #76 on: January 25, 2017, 05:24:32 am »

Nice test Jim, I enjoyed it.

Ross
Logged

tyler69

  • Citizen of the Universe
  • *****
  • Posts: 946
Re: NEW: Listening Test
« Reply #77 on: February 07, 2017, 12:29:10 pm »

Copied this from another thread:

Well I think that it's not impossible to hear a difference when a component is swapped.
But on topic: would it be possible to add functionality to this feature (or implement it as a new one with a different Name) in Order to compare two manual selected tracks from the library? If so, Media Center should try to allign those tracks.

Another related thread:

http://yabb.jriver.com/interact/index.php/topic,109305.0.html

 ;D
Logged

thorsten

  • World Citizen
  • ***
  • Posts: 194
Re: NEW: Listening Test
« Reply #78 on: February 08, 2017, 01:42:28 pm »

This is a great feature! Sounds funny, but this might be the reason of updating from MC21... :o
Logged

Mike48

  • World Citizen
  • ***
  • Posts: 214
Re: NEW: Listening Test
« Reply #79 on: March 15, 2017, 07:18:52 pm »

This is an interesting feature, and I'd like to see it expanded so it could start with higher resolutions than CD quality. I agree with Kal on that.

mp3 is a dead issue to me, and I suspect to many audiophiles. Storage is cheap and plentiful, so why make lossy copies of material? I do use mp3 in my car, whose media player won't do flac, and it's perfectly fine for that. I know that high-rate mp3 can sound good. But so what? I have no need for it.

But what about sound-quality comparisons of lossless material at and above CD quality? Do the 24-bit, 44.1 kHz tracks in my library sound worse if dithered to 16 bits? What happens to quad-rate tracks when downsampled to double rate? Single rate?
Logged

JimH

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 71348
  • Where did I put my teeth?
Re: NEW: Listening Test
« Reply #80 on: March 15, 2017, 07:21:35 pm »

Did you try the test?  Could you post your results?
Logged

rexmcartor

  • Recent member
  • *
  • Posts: 11
Re: NEW: Listening Test
« Reply #81 on: March 23, 2017, 03:31:31 am »

I am not to surprised by my results.  I have done blind tests in the past on websites with songs I am not very familiar with and it is harder to tell.  But I have never had a blind test where I could pick the song.  So I chose Comfortably Numb by Pink Floyd.  Original was a 24/96 high quality vinyl rip.  I chose to do all formats.  Keep in mind it is hard for me to rate this song low.  So Most of my ratings are 3 or 4 stars as an acceptable version if needed. 
But I chose
5 Stars for DSD
4 Stars for FLAC
4 Stars for 256 MP3
3 Stars for 128 MP3
3 Stars for 96 MP3
1 Star for 32 MP3

I may try again with different tracks to see if I can still pick that well.  I do consider myself to have very good listening ears though.

Thanks a lot for this feature.  It was nice to be able to justify the hard drive space.
Logged
Pages: 1 [2]   Go Up