INTERACT FORUM
More => Music, Movies, Politics, and Other Cheap Thrills => Topic started by: Hilton on August 21, 2015, 06:24:54 am
-
Hi All,
I have a small dilemma and don't want to dig a hole any bigger or start a religious flame war so need some advice.
As some of you would know I attended an Audio meet last weekend in Sydney.
JRiver was compared against Audirvarna+ and Pure Audio in a single blind test.
In my opinion the test wasn't run as well as it could have been, but there was limited time and it was done as fairly could be expected for an audio enthusiast group.
Unfortunately for JRiver it garnered only 4 votes in the player comparison test out of about 35 people. This was pretty consistent through the testing with JRiver getting much fewer votes but scoring better in a couple that surprised some.
Tom the club president has suggested my review and leaning to JRiver is biased and I should probably include the low scores in my post... He's not wrong, I am biased and I do try to have an open mind, but not at the expense of JRiver publicity.
How people can rate the Audirvana+ player higher than JRiver when Audirvana+ clearly glitched and stumbled twice on playback is a mystery to me. The glitches automatically disqualified it in my mind, no matter how much I enjoyed it's sound, and it did sound good, but I wouldn't say better than JRiver.
I drafted a massive email to go back to Tom the president of the Audio club that ran the testing, but I just don't think it's going to help anything.
Should I just post the imperfect test results in the thread I made and we should just cop-it-on-the-chin that JRiver didn't get the votes?
Or should I risk a holy war?
Here's where I got to with the email, but I just think its not worth it. Some of it I would have published in the forum but I'm now leaning towards making a very light and humorous admission that it was a fun experiment and JRiver just wasn't up to it on the day and leave it at that.
It's a can of worms and I just think we should let sleeping dogs lay...... What are your thoughts....??
------------------
Hi Tom,
I have Jim’s blessing to use my own judgement. Not to make a big deal out of it, but I do want to be fair and open minded and acknowledge that JRiver didn’t test well in the scenario we were using as I believe that’s the right thing to do.
So I’ll put the results up and the link to Darko’s review. (Was Darko there? It would have been good if he had been there to make it more objective as a review writer, I apologise if he was there as I wasn’t introduced)
I could leave it at that, but I don’t think that would be fair.
I have no wish to belittle the testing as it is very difficult thing to do without some sort of bias creeping in, and under the test conditions I don’t believe with the limited time that you could have done anything differently.
Maybe I’m grasping at straws, but I believe the only thing short of a double blind test that may have influenced the voting was to run the two tracks in three lots, each in different orders (9 times)with voting anonymously (written down) and then discussed at the end.
The reason for this is that there’s a few forms of bias that trickle into these things that can be, for the most part, eliminated with this approach.
These things are also hotly debated as you’re aware and I’m on the more analytical open minded side of things. It’s not a religious objective vs subjective thing for me.
As you know I went in being sceptical but hoping to be able to hear a difference. Many of the attendees would have been in this camp.
I think the testing was about as fair as it could be, and perhaps my suggestions would make no difference, but I do think it’s worth pointing out the forms of bias that can affect such testing.
This still doesn’t improve JRiver’s position in the scoring, but it perhaps more fairly reflects the test conditions at least.
The forms of bias that I believe need to be considered are:
Confirmation Bias
Being pre-disposed to liking a specific thing to confirm your own belief – some of the tests were not blind tests.
There was an element of this, but I think the test was as fair as could be under the circumstances.
Mob Mentality Bias vs Im a Unique Weirdo Individual Mentality Bias
Two sides to this obviously; Following the herd or the mob is as big a problem as Confirmation Bias.
Without full double blind and anonymous testing there will be some bias in the voting, its human nature.
There may well have been an element of this. Anonymous voting and double blind testing is the only way to avoid it.
Memory Bias
Being pre-disposed to a certain sound because the user already extensively uses one of the tested products - I would fall into this category as I could clearly hear my preferred sound and it was evident in my voting. Maybe a fluke. (I picked JRiver 3 times)
What would be interesting is to re-survey the attendees as to what their existing preferred product is and put that in the mix of results to see if there is any correlation. (there’s that analytical side of me )
Systematic Bias
From obvious defects or flaws in the testing – This one is a bit easier, however, for example, had Audivarna+ not glitched I may have given it votes as I quite liked it but I immediately dismissed it because of the glitches.
The other systematic bias I could say existed though this may be controversial, was the order of the players being tested. I heard someone else say during the testing that statistically people will vote less for the 1st tested player as opposed to the 2nd and 3rd where there is no real discernible difference. (JRiver was the 1st player we auditioned with 2 tracks before moving on to the next player - another test was done with a single track and the players all back to back but it was no longer a blind test with Tom telling us which player was which before playing the track - again opening up another risk of bias )
I researched this and it’s hard to find any evidence for this in Audio testing, but I’m sure there will be PHD papers out there from statisticians that will argue the point till blue in the face.. but you know that saying, “Lies Damn Lies and Statistics”
Random Error and Placebo
Audio Memory can be long term yes, but in a situation where users are trying to hear a difference where none or very subtle differences exist, placebo and random error are certain to creep in.
-
Mate I think you are on a flogging to nothing. I enjoyed your reports and sorry I missed it... well sort of. I'm happy that DACs make a difference, but unless the player code is buggy there is 0 difference with any of the "bit perfect" players, or their $1,000 usb cables feeding the DAC. I agree that their methodology is not up to double blind but you are dealing with religious fanatics. I'd suggest that you propose a double blind "Mega Player Test" for the next met.
....and then in the MC DSP setting push the boost up 1/2 db and MC will win 8)
-
I drafted a massive email to go back to Tom the president of the Audio club that ran the testing, but I just don't think it's going to help anything.
Should I just post the imperfect test results in the thread I made and we should just cop-it-on-the-chin that JRiver didn't get the votes?
Or should I risk a holy war?
My Hero. Sally forth. I eagerly await thy return.
(https://chivethebrigade.files.wordpress.com/2013/03/knights-armour-920-12.jpg)
-
My Hero. Sally forth. I eagerly await thy return.
Thanks Jim that doesn't really help.. LOL but it is darn funny.
I just sent you another email :)
-
Mate I think you are on a flogging to nothing. I enjoyed your reports and sorry I missed it... well sort of. I'm happy that DACs make a difference, but unless the player code is buggy there is 0 difference with any of the "bit perfect" players, or their $1,000 usb cables feeding the DAC. I agree that their methodology is not up to double blind but you are dealing with religious fanatics. I'd suggest that you propose a double blind "Mega Player Test" for the next met.
....and then in the MC DSP setting push the boost up 1/2 db and MC will win 8)
Yes I know... :) You can't reason with the devil.. :)
-
On the question of whether our Windows audio is different than our Mac audio, here's a nice test done by user mitchco:
http://www.computeraudiophile.com/content/513-jriver-mac-vs-jriver-windows-sound-quality-comparison/
-
I would argue there are only downsides here :) so publish the facts without commentary/justification/your explanation of the results and leave people to make up their own mind.
-
On the question of whether our Windows audio is different than our Mac audio, here's a nice test done by user mitchco:
http://www.computeraudiophile.com/content/513-jriver-mac-vs-jriver-windows-sound-quality-comparison/
As I just sent in my email.
I’ve read that one before, it’s a very interesting read and why I was questioning Tom about his statement on MAC vs PC.
I still can’t help feeling that something was afoot with the testing, not intentionally mind you, but you can never be sure.
I’m also still not certain that there isn’t more to it than bits in bits out.
Timing is important, there’s Jitter which shouldn’t be audible but can be, there’s that Dith…ing word that we don’t like to talk about.
And there’s truncation and rounding, related to dithering, and buffering differences which Im not convinced either way can or can’t be heard.
Some of the above may not have been measurable in Mitcho’s tests simply due to the resolution of the recording and testing method.
The errors or timing issues could have been filtered or cancelled out as he was recording at exactly the same bit depth and sample rate from memory.
It would be interesting to see Mitcho do the same thing but with say double the bit depth and sample rate on the recording side. (and into a different DAC)
I don’t know!! Sometimes you just have to sit and enjoy the music.
-
I'll see Jim's Image and raise you one.
(http://www.motifake.com/image/demotivational-poster/1404/sacrificial-lamb-lamb-sheep-wolf-demotivational-posters-1398750702.jpg)
-
oohhh its gone public... I'm in for it now.... Just like Nathans Lamb to the Slaughter :) I can see the hordes coming with pitch forks and torches to lynch me now. ;D I better run for the hills.....
-
I would argue there are only downsides here :) so publish the facts without commentary/justification/your explanation of the results and leave people to make up their own mind.
Even the facts will be disputed. A bit is a bit, for example. A PC's timing can't affect a DAC with a buffer and a clock, for another.
-
I don't buy the jitter argument. Sure there can be jitter in the PC but it is just feeding the DAC's buffer. Unless the DACs Bufffer runs out how on earth does it matter the inter packet timing that the packets arrive into that buffer as long it is in the right order? It is the DAC's job to then covert the data from it's buffer into a analogue wave form. Must be a the words worst DAC if the inter packet timing from a PC to its buffer makes the slightest difference in how it processes it's own queue.
-
Even the facts will be disputed. A bit is a bit, for example. A PC's timing can't affect a DAC with a buffer and a clock, for another.
You must be feeling good Jim>? Ready to do some moderating? LOL
-
oohhh its gone public... I'm in for it now.... Just like Nathans Lamb to the Slaughter :) I can see the hordes coming with pitch forks and torches to lynch me now. ;D I better run for the hills.....
Yup - You're rogered
-
I would argue there are only downsides here :) so publish the facts without commentary/justification/your explanation of the results and leave people to make up their own mind.
I agree, but included in the "facts" are the methodology as well as the results. I think you should post the results *and* the entire methodology. One is worthless without the other.
Of particular note, if the players weren't carefully volume matched the results are not particularly meaningful. JRiver tends to attenuate slightly to account for intersample overages to leave headroom, etc. I don't think double blind is absolutely necessary to get good results (an appropriately structured single blind can be almost as good), but failure to match levels (not by ear, with an SPL meter and pink noise) is going to be fatal to any test validity. Less than half a dB volume difference has been shown to be enough to create preference with identical source material and playback chains.
Timing is important, there’s Jitter which shouldn’t be audible but can be, there’s that Dith…ing word that we don’t like to talk about.
And there’s truncation and rounding, related to dithering, and buffering differences which Im not convinced either way can or can’t be heard.
I don't buy the jitter argument. Sure there can be jitter in the PC but it is just feeding the DAC's buffer. Unless the DACs Bufffer runs out how on earth does it matter the inter packet timing that the packets arrive into that buffer as long it is in the right order? It is the DAC's job to then covert the data from it's buffer into a analogue wave form. Must be a the words worst DAC if the inter packet timing from a PC to its buffer makes the slightest difference in how it processes it's own queue.
I have yet to see any actual evidence that two different bit-perfect players can have different jitter profiles. Jitter isn't mystical, it can be measured, and all claims (that I've seen) that different bit-perfect players have statistically significant differences in jitter performance are unsupported by measurements. The closest to evidence the "jitter" crowd has is an article by a DAC designer that explains how it could happen, but includes no measurements or actual proof. If anyone has seen any such studies or measurements, I'd actually like to see them as it would be nice to think this argument wasn't a complete red herring.
Different dithering algorithms can create different sound, but is unlikely to be audible at normal playback levels. But, that at least is a real possibility in that it is something that could legitimately be different between the players. If dithering is happening, truncation or rounding shouldn't be happening, right? The whole point of dither is to prevent noise from truncation or rounding.
It's possible that JRiver may have sounded worse in that environment than the other players. My money is on volume difference; it's possible the lack of double blindness may have contaminated it (especially if the crowd knew the fellow throwing the switches well enough to read his expression/body language), but a difference that significant probably resulted from something actually audible I'd guess.
-
This>>>> :)
but a difference that significant probably resulted from something actually audible I'd guess.
-
I'll see Jim's Image and raise you one.
That got a good laugh here.
-
You must be feeling good Jim>? Ready to do some moderating? LOL
On this forum, a bit is, and forever shall be, a bit.
-
This>>>> :)
Next time, bring an SPL meter and a usb stick with a pink noise clip ;D
-
I better run for the hills.....
Don't forget your horse.
-
Darko's review: >> (which I quite enjoyed)
http://www.digitalaudioreview.net/2015/08/a-stand-off-down-under-puremusic-vs-jriver-vs-audirvana/
-
Don't forget your horse.
OK... :) thanks...... I think....
-
The closest to evidence the "jitter" crowd has is an article by a DAC designer that explains how it could happen, but includes no measurements or actual proof. If anyone has seen any such studies or measurements, I'd actually like to see them as it would be nice to think this argument wasn't a complete red herring.
I did read a very detailed paper on Jitter awhile ago. I may have even posted parts of it on here now I think of it... let me see...
-
Yep here tis...
I too tried xxxx many moons ago with no perceptible improvement, however it did screw up stability of the system and I quickly removed it.
My knowledge has improved since and I'm tempted to have another more technical analysis of it.
I just read a whole bunch of latency and jitter info and found this one from Yamaha particularly useful. It's still sinking in as it's quite a complex subject.
http://www.yamahaproaudio.com/global/en/training_support/selftraining/audio_quality/chapter5/10_jitter/
Particularly interesting is this comment.
From my understanding only the input and output can cause jitter, once in the DSP engine/domain timing is irrelevant.
audibility of jitter
audibility of jitter
Assuming a 0dBfs sine wave audio signal with a frequency of 10kHz as a worst case scenario, a jitter signal with a peak level of 5ns will generate a combined A/D and D/A jitter noise peak level of:
EA/D+D/A = 20.log(2.5.10-92.π.10.103) = -64dBfs
When exposed to listeners without the audio signal present, this would be clearly audible. However, in real life jitter noise only occurs with the audio signal in place, and in that case masking occurs: the jitter noise close to the audio signal frequency components will be inaudible, so the average audio signal’s spectrum will mask a significant portion of the jitter noise.
Note that the predicted level is the jitter noise peak level generated by 0dBfs audio signals. In real life, the average RMS level of jitter noise will be lowered by many dB’s because of the audio program’s crest factor and the system’s safety level margins used by the sound engineer. Music with a crest factor of 10dB played through a digital audio system with a safety level margin of 10dB will then generate jitter noise below -84dBfs.
The audibility of jitter is a popular topic on internet forums. Often a stand-alone digital mixing console is used in a listening session, toggling between its internal clock and external clock. In these comparisons it is important to know that such comparison sessions only work with a stand-alone device. If any other digital device is connected to the mixer, then clock phase might play a more significant role in the comparison results than jitter.
In uncontrolled tests, many subjective and non-auditory sensations have a significant influence on the result. More details on quality assessment methods are presented in chapter 9.
In multiple clinical tests, the perception threshold of jitter has been reported to lie between 10 nanoseconds(* 5U) for sinusoidal jitter and 250 nanoseconds(* 5V) for noise shaped jitter - with actual noise shaped jitter levels in popular digital mixing consoles being below 10 nanoseconds.
and this from that thread...
Jitter is real, audible in some circumstances, and measurable. This is an excellent article from a non-woo-woo engineer on the subject (he also happened to be one of a very few people with instruments actually capable of taking decent jitter measurements): http://nwavguy.blogspot.com/2011/02/jitter-does-it-matter.html
The vast majority of quality modern DACs (including several models I can name under $100) are more or less completely insensitive to jitter: they show jitter distortion well below -100dBFS with normal playback. Nicer DACs will often have jitter measurements below -120dBFS where you're getting into "resistor noise" levels of distortion. If you want to shell out for a Benchmark DAC, they've got it down into the -130's. No one has ever demonstrated that jitter distortion below -100dBFS is even audible in any way.
IMO, it's just not a real problem anymore except on cheap or poorly designed equipment, and becomes less of a problem each year.
Glynor, I'm not a physicist or jitter expert, but my understanding is that those timing measurements are a convention that are effectively "backed out" from the distortion measurements. Nwavguy gets into it a little bit in his article, but there are techniques to identify and measure the jitter distortion of an audio signal (as distinct from other types of distortion), and then that distortion level is referenced to digital full-scale and "translated" into time units based on the sampling rate. So it's a "derived" measurement, and kind of a silly one IMO, since distortion (of which jitter is just one variety) is customarily expressed in dB below fullscale or a percentage, so converting to the time unit seems designed to obscure the issue. Check out this article for some explanation: http://www.anedio.com/index.php/article/measuring_jitter
So he probably wasn't consciously being disingenuous, just following along with a silly convention.
-
Just so we don't misunderstand each other: jitter distortion is 100% real, measurable, and generally accepted as a source of audio distortion. I think it's less relevant than it used to be because DAC design has improved, but there's no question that it exists and is a meaningful parameter in assessing audio quality.
However that's not what I was talking about above. No one questions that jitter exists and can be audible. What I question and what is entirely unproven (as far as I can tell) is that different bit-perfect software players can meaningfully affect jitter. The timing error of packets entering the analog stage of the DAC will be almost entirely the product of the DAC's own clock or adaptive reclocking mechanism, or if it doesn't have one, the clock of last resort (the USB bus clock). The speed at which the software feeds the ring buffer has no logical relationship to the rate at which the DAC or driver empties the buffer, unless the packets can't come in fast enough and then the distortion is unsubtle (and not technically jitter, just a plain old drop out).
I have seen no credible measurements showing that different software players result in different jitter measurements (I'm not even sure I've seen any incredible measurements), and, even conceding that such a thing is real and measurable (but as yet unmeasured) I have also seen no credible studies that would tend to show such a thing is audible.
To be clear, I would welcome a competently performed test that showed that different software players produced meaningfully different jitter profiles. It would answer a lot of unanswered questions. But no one I've talked to about this specific issue related to jitter has offered any evidence other than John Swenson's theorizing (without any supporting experimental data of any kind).
-
I agree, but included in the "facts" are the methodology as well as the results. I think you should post the results *and* the entire methodology. One is worthless without the other.
The "facts" in this case aren't facts.
Statistics aren't statistics with one data point, and you learn absolutely nothing without repeat-ability. There is basically nothing more useless than a single data point.
People commonly think "well, it is better than nothing" but that's completely opposite of the case. It is worse than nothing, because it implies "truth" where none exists. Your sample could be all noise, and if you repeated the exact same test minutes later, you could get the opposite result. It is impossible to know!
-
Just so we don't misunderstand each other: jitter distortion is 100% real, measurable, and generally accepted as a source of audio distortion. I think it's less relevant than it used to be because DAC design has improved, but there's no question that it exists and is a meaningful parameter in assessing audio quality.
However that's not what I was talking about above. No one questions that jitter exists and can be audible. What I question and what is entirely unproven (as far as I can tell) is that different bit-perfect software players can meaningfully affect jitter. The timing error of packets entering the analog stage of the DAC will be almost entirely the product of the DAC's own clock or adaptive reclocking mechanism, or if it doesn't have one, the clock of last resort (the USB bus clock). The speed at which the software feeds the ring buffer has no logical relationship to the rate at which the DAC empties the buffer, unless the packets can't come in fast enough and then the distortion is unsubtle (and not technically jitter, just a plain old drop out).
I have seen no credible measurements showing that different software players result in different jitter measurements (I'm not even sure I've seen any incredible measurements), and, even conceding that such a thing is real and measurable (but as yet unmeasured) I have also seen no credible studies that would tend to show such a thing is audible.
If you read what I posted above in the yamaha training guide, jitter cannot be present in the DSP, only on the input and output. sorry that wasn't meant to sound the way it did... you know what I mean :)
here :)
(https://farm6.staticflickr.com/5780/20767263251_930d270d48_b.jpg) (https://flic.kr/p/xD8DQX)Jitter (https://flic.kr/p/xD8DQX) by Hilton (https://www.flickr.com/photos/133784514@N07/), on Flickr
-
If you read what I posted above in the yamaha training guide, jitter cannot be present in the DSP, only on the input and output.
That's kind of my point. I agree and that's why I think "software player induced jitter" is probably a myth. But the thesis of the "software player jitter" theorists is that something the software does when feeding the DAC creates jitter at the output. My point is that bitperfect software doesn't really have the ability to affect the timing of the DAC's output stage (unless it doesn't provide enough info), which you are confirming. The alternative theory is that electrical activity in the computer screws up the timing of the DAC's output stage (creating jitter), and that different software create different levels of electrical activity. I don't think that's a very good theory either for different reasons.
But regardless of the theoretical explanation, the real point is that there are no measurements (that I know of) showing meaningfully different jitter results wth different software players, which you would expect to be achievable if different software players had differing results on jitter.
Check out: http://archimago.blogspot.com/2013/03/measurements-hunt-for-load-induced.html
http://archimago.blogspot.com/2015/08/measurements-audiophile-sound-and.html#more
I am easy to persaude, show me credible empirical measurements with modern equipment (not crappy old SPDIF interfaces from the 90's), and I will come along ;D I've seen too many people convinced they heard huge differences when deliberately presented with identical playback chains; the ear is easy to fool, instruments are much harder to fool.
sorry that wasn't meant to sound the way it did... you know what I mean :)
No worries, I just want to make sure we aren't talking past each other
-
Here's something I didn't know, is this true? (from computer audio forum)
Audirvana is using iZotope for sample rate conversion and dithering, which offers objectively better performance than JRiver's SSRC resampling and 1LSB RPDF dither if you are processing the signal in any way. (volume control, room correction, upsampling etc.)
However JRiver is far ahead of everything else when it comes to library management and features such as remote control via JRemote, Theater View on a TV, managing playback to several devices at once, and non-audio features like video playback (though the latter is at a very early stage in development on OSX) which is the reason that I will continue to use it instead of anything else for the foreseeable future.
I'm not sure about the Mac side of things, but on Windows you can at least replace their dither with a VST plugin now, which fixes that aspect of performance. If you are not upsampling, that addresses the last 5-10% difference in audio quality between JRiver and other players in my opinion.
You're stuck with their resampling though, which is not the greatest.
It isn't bad, but if you're going to be upsampling everything (e.g. to DSD) there are better options like Audirvana/HQ Player.
But again, JRiver gets you 90% of the way there as far as quality is concerned, and when you consider how many other features it offers, it's still my preferred player by a long stretch.
Personally I don't upsample music at all, only video playback, so it's not a major issue for me - though it is something I'd like to see them improve.
-----
and this
Audirvana has Direct Mode, which bypasses Core Audio. I think this makes a bigger difference than upsampling. And it benefits even with higher resolutions.
When JRiver supports ASIO on the Mac, I think it will likely have matched Audirvana's transparency. But only with exaSound DACs for the time being as only they have released ASIO drivers on the Mac so far AFAIK.
-
PS...
I guess the real question here is, does the dither engine ultimately cause the largest audible difference in the bits in bits out argument? Some of the articles I've just read would indicate so.
If this is the case, I wonder if any of the players had core audio disabled or if there were any up-conversions or down-conversions or resampling happening to make dithering a factor? Tom did say they were all doing pass through or bit perfect (a word he doesn't like).
-
The dither question has been discussed ad nauseum here in the past (and has, in fact, gotten more than one user banned here before). I don't know enough to comment intelligently but I know Matt is positive it is nonsense.
The stuff about bypassing Core Audio is absolutely nonsense, spouted by people who don't know what they're talking about. Core Audio is the OSX audio API. It cannot be "bypassed" unless you use ASIO or re-write all of the audio drivers yourself from scratch.
Their "direct mode" is still through Core Audio, and is the same thing MC does by default on OSX. They're just bypassing the API's DSP and effects, which MC does as well, of course, since they have their own.
-
The dither question has been discussed ad nauseum here in the past (and has, in fact, gotten more than one user banned here before). I don't know enough to comment intelligently but I know Matt is positive it is nonsense.
The stuff about bypassing Core Audio is absolutely nonsense, spouted by people who don't know what they're talking about. Core Audio is the OSX audio API. It cannot be "bypassed" unless you use ASIO or re-write all of the audio drivers yourself from scratch.
Their "direct mode" is still through Core Audio, and is the same thing MC does by default on OSX. They're just bypassing the API's DSP and effects, which MC does as well, of course, since they have their own.
Thanks Glynor for the no nonsense confirmation :) Which colour do you prefer? Black or white? LOL (ps that's a good thing.)
I know about the issues of dithering on the forum (and other forums) which is why it was the last thing to bring up. (and carefully)
Just trying to be objective and analyse all the possible variables.
All other things being equal, with the result JRiver scored it cant of all been bias related even if there were bias. I'm trying to pin it down to where the difference was as there was clearly an audible difference.
If Jitter is out, and the bias doesn't explain the significant difference in the voting then just maybe the Dither plays more of role than previously acknowledged. (puts flame suit on)
I've read lots about dither in the past and again tonight and I understand it very well...... Maybe it's time to put previous opinions aside in the light of these clear test results, even if they're flawed.
If bits are bits, JRiver should have at least scored close to an equal third of the votes.
Dithering is the only thing I can think of that would be different between the players after everything we've discussed. (assuming bit-perfect and volume matched and I have to say I trust Tom on this one)
-
Hilton,
What do you do if you get a bad itch somewhere under all that armor?
-
Dithering is the only thing I can think of that would be different between the players after everything we've discussed. (assuming bit-perfect and volume matched and I have to say I trust Tom on this one)
I don't. If you assume that there was actually a difference that people could identify, then faulty Volume matching is the most likely answer by far. Did you have a decibel meter running throughout the show? If not, then how do you know they were volume matched??
Other possible reasons:
* The winning player is actually not bitperfect. It'd be easy to "win" many of these contests by lying and making your player (because of an "accidental" bug, to be sure) play everything a quarter-decibel higher than you should be.
* The glitches may have actually improved Audionirvana's chances. If everything is equal otherwise, the mind will latch onto what it "remembers" and recall it fondly (the human brain doesn't like it when it is confused). So, the glitches were singular events that separated that playback from the rest of the pack. Making it, therefore, more memorable. To the listeners in their memories, this would seem like "yeah, it glitched, but before and after wow it just sounded great". More great than the others? Who knows. It doesn't matter because that's the event their brain remembers. The difference between the disruption of the glitch, and the otherwise premium quality sound (because, assuredly, everything else in the audio chain was very premium and low noise).
* The Infinite Monkey conundrum (https://en.wikipedia.org/wiki/Infinite_monkey_theorem). I don't know your exact methodology, but I know from your descriptions of everything that you were testing, that it was not very rigorous and there were very few (maybe only two?) tests evaluating three different players. That is an inadequate sample size to allow you to distinguish from random noise, as I mentioned above. Remember, if you capture screenshots of static over and over and over, you can easily get one that looks like this (and eventually will get one like this):
(http://glynor.com/img/screenshots/etc/static_can_look_like_this.png)
-
The winning player is actually not bitperfect. It'd be easy to "win" many of these contests by lying and making your player (because of an "accidental" bug, to be sure) play everything a quarter-decibel higher than you should be.
Bingo. I feel the same about the dithering question.
-
If bits are bits, JRiver should have at least scored close to an equal third of the votes.
I addressed this above, but this statement is absolutely incorrect.
You'd only expect "equal" results once you've done a statistically valid sample. For a single test, or only a couple, you can't draw any conclusion from the data point.
With the Infinite Monkey conundrum I mentioned above, in any single test, there is an equal chance that the first result will be Shakespeare. A tiny chance, but an equal one to all other possible results. If you did this test, and the first monkey typed out a sonnet and then you stopped, it would seem like an amazing result. Of course, it would never happen again.
It seems counter intuitive to people who don't understand statistics, but noise is the expected result of a single test, not "equal results". Just like if you roll a single die, you are not more likely to roll a 3 than any other number.
-
I read the DAR article. I wish these "shootouts" would use an ABX testing methodology (https://en.wikipedia.org/wiki/ABX_test). Say there was a difference observed. The next stage would be the investigation of "why" did MC sound different to the other two players? If identified can some knob tweaking change it? Then retest.
Without a cycle like above you are just going to be clutching at straws, and until then:
(https://encrypted-tbn1.gstatic.com/images?q=tbn:ANd9GcTPHe5g-WUwflrNONcg22Wv3hGjsoU9AxQeQAnC-odj_RugtLGfAA)
-
I've been an audio guy since I was a teenager. I've listened to a lot of systems in my time and heard lots of strange claims about different things that "help" the sound. Lots of these trip my BS meter because I don't believe in magic. I won't say something doesn't exist because I don't understand it. I'll say that I *want* to understand why things work and if I can't, or if there is NO explanation, then I'm suspicious.
The audiophile circles are full of double talk, fake science, and people that don't understand science or engineering explaining how things "should be working". I'm extremely skeptical about differences between players, but I haven't personally done an AB test.
Here's what gets me. This is a big deal, so pay attention:
Data is being sent to a DAC by an audio player. If the data sent is the same from two different players, playing the same song, then the audio coming out of the DAC should be the same right? Why don't we record the data going IN TO THE DAC? If "bits are bits" then we should be able to record those bits and compare player A to player B via the bits they produced TO THE DAC. Right?
If these recorded DAC streams are the same, then the jitter discussion has merit and can continue. But I have a different feeling. I think the bits getting to the DAC are probably different. I *suspect*, with zero proof, that some players are changing something. Subtle eq or volume changes. Something different.
Or this is all just more audiophile snake oil driven by the *desire* to hear differences where none exist. It's really hard to say without testing.
Brian.
-
All of this takes me back to when I was 1st year grad and finally in possession of a bit of moolah (..and kudos to you Hilton for having a crack - nice work).
I trekked all over town listening to various systems in Hi-Fi stores with a few CDs of my preferred music (for some reason they all seemed to like playing "The Race" by Yello :P ). The sales pitches from some were enough to make me puke.
Anyhoo, I made my decision and got my gear home:
- NAD 3240 PE Amp
- Jamo CL30 Speakers
- NAD CD Player of the era (memory has faded the model number from my mind
It's all I could afford and it ended up lasting me the best part of 20 years. I had the speakers reco'd down the track but they were pretty much chopped up at the end. The amp had a loose connection or 5 in there somewhere, so there was some scratchy playback going on in one or the other channel. I ended up "donating" them to a kerbside collection and it all got whisked away before I could blink.
The moral of the story and getting to the point, I realised all the way back then that once I got my gear away from the funky listening studios and the comparisons and the sales pitches, I could just sit back and enjoy the music.
Isn't that what it should all be about? If it sounds good to you, then soak it up and enjoy it. If it doesn't, then do something different. I find the idea of comparisons interesting but I always apply my sense check and invariably end up thinking, "hmmm, interesting".
-
Let me offer an embarassing personal anecdote to explain my point of view about listening tests and the fallability of the ear:
Several years ago I built a pair of home-made bi-amped speakers. They're each the size of a large washing machine and they took me the better part of a year to build (more than a month of Sundays). Because they were entirely home-made and I was trying to do an active crossover from scratch, even after they were structurally complete, they still required quite a bit of tweaking to get the crossovers dialed in and the EQ set.
So I started by just dialing in the EQ that seemed to make sense based on the specifications of the drivers, and taking a couple of quick RTAs with pink noise. That sounded alright, and all of my friends dutifully told me how great it sounded. I kept getting headaches whenever listening to the speakers though, and the headaches would go away right after I turned them off. So I tried tweaking some frequencies, and I'd think I'd made some progress (it sounded better!), and everyone who heard it thought the new EQ sounded better. Eventually, I even started dutifully "blindly" A/Bing the new EQ with the original (I'd switch between them during playback without telling my guests what I was switching, which isn't blind at all), and my guests would invariably swear the new EQ sounded better. And I kept going down this "tuning by ear" method, often reversing previous decisions, backing and forthing and adding more and more convoluted filters.
The most embarassing moment (and something of a turning point) was when I was A/Bing a filter, and a friend and I were convinced we were on to something excellent. After ten minutes of this, we realized that the filter bank as a whole (PEQ2) was disabled :-[. I had been toggling the individual filter, but it wasn't actually even affecting playback. And we had been convinced we heard a difference. And the headaches never went away.
Eventually the headaches (and a growing skepticism) prompted me to stop screwing around and take some real logsweep measurements (which were then a relatively new thing for me), and I realized that there was apparently a huge (10+dB) semi-ultrasonic resonant peak at 18.5KHz that I couldn't even actually hear. So I fixed it. And then my headaches went away.
And then I took an agonizing look at the rest of the measurement and noticed that my "tuning by ear" which I (and my friends) all felt was clearly superior had turned the frequency response into a staggering sawtooth. So I systematically removed the EQ that was pushing things away from "flat," and kept the EQ that contributed to flatness. The result sounded so different, and so much more natural that I was embarassed to have wasted months screwing around trying to use my "golden ears" to tune my speakers. And my wife (who had been encouraging, but politely non-commital about my EQ adventure) came home and asked unprompted if I had done something different with the speakers, and said they sounded much better. And she was right; they did. In a few afternoons, I had done more to move things forward than I had in months of paddling around.
The point of this anecdote is not to try and "prove" that my measurement derived EQ "sounded better" than my ear-derived EQ or that a flat frequency response will sound best [as it happens, I ultimately prefered a frequency slope that isn't perfectly flat, but I couldn't even get that far by ear].
The point is that taking actual measurements had allowed me to:
1) Cure my ultrasonic frequency induced headaches;
2) Improve the fidelity of my system (in the literal sense of audio fidelity as "faithfulness to the source"); and
3) Ultimately find the EQ curve that I liked best.
My ears (and the inadvertantly biased ears of my friends) did not allow me to do any of those things, and in fact led me far astray on 2). My ears couldn't even really get me to 3) because I kept reversing myself and getting tangled up in incremental changes. My ears were not even reliably capable of detecting no change if I thought there was a change to be heard.
Once I realized all this, it was still surprisingly hard to admit that I had been fooling myself and that I was so easily fooled! So I have sympathy for other people who don't want to believe that their own ears are equally unreliable, and I understand why folks get mad at any suggestion that their perception may be fallible. I've been accused by many indignant audiophiles of having a tin ear, and if I could only hear what they hear, then I'd be immediately persuaded. But my problem is not that I am unpersuaded that there's a difference: it's that I'm too easily persuaded! I'll concede, of course, that it's possible that I do have tin ears and other people's ears are more reliable than mine, but the literature concerning the placebo effect, expectation bias, and confirmation bias in scientific studies suggests that I'm not so very alone.
I've seen the exact same phenomenon played out with other people (often very bright people with very good ears) enough times that I find it embarassing to watch sighted listening tests of any kind because they are so rarely conducted in a way designed to produce any meaningful information and lead into dark serpentines of false information and conclusions.
---------------------------------------------------------------
So to bring things back around: if some bitperfect audio players have devised a way to improve their sound they have presumably done so through careful testing, in which case they should be able to provide measurements (whether distortion measurements on the output, digital loopback measurements, measurements of the data stream going to the DAC, or something) that validates that claim. If they claim that their output "sounds better" but does not actually measure better using current standards of measurement, they should be able to at least articulate a hypothetical test that would show their superiority. If they claim that the advantage isn't measurable, or that you should "just trust your ears" than they are either fooling themselves or you.
In a well-established field of engineering in which a great deal of research and development has been done, and in which there is a mature, thriving commercial market, one generally does not stumble blindly into mysterious gains in performance. Once upon a time you could discover penicillin by accident, or build an automobile engine at home. But you do not get to the moon, cure cancer, or improve a modern car's fuel efficiency by inexplicable accident. In an era where cheap-o motherboard DACs have better SNR's than the best studio equipment from 30 years ago, you don't improve audio performance by inexplicable accident either. If someone has engineered a "better than bit perfect" player they should be able to prove it, as they likely did their own testing as part of the design process. If they can't rigorously explain why (or haven't measured their own product), let them at least explain what they have done in a way that is susceptible of proof and repetition. Otherwise what they are selling is not penicillin, it's patent medicine.
Bottom line: if you and a group of other people hear a difference, there may be a difference, but there may not be. Measurements are the way to find out if there is really a difference. Once you've actually established that there is a real, measurable difference, only then does it make sense to do a properly conducted listening test to determine if that difference is audible. Otherwise you're just eating random mold to find out if it will help your cough.
If you want to get to the bottom of it, I'd ask your host if he'd be willing to let you take measurements of the different players with a calibrated microphone, an SPL meter, and some test clips of your own choosing. He'll probably be game, and you'll likely be able to get some really useful information about whether the players sound different and, if so, why.
Or you could just relax and enjoy the music ;)
-
Or you could just relax and enjoy the music ;)
This. Why drive yourself crazy diving into all this stuff? Just enjoy the music!
-
mwillems,
Amazing story, very well written. It should be published in an audiophile magazine. Not that it would make any difference...
Thank you.
Jim
-
I also built a speaker box in 9th grade shop and put it on my headboard of my bed
when sleeping one night it fell off the headboard and hit me square in the forehead where I have a small scar to this day.
it is right next to the scar when I drank a 1\2 gallon of Seagram's Seven, and 1\5th of Smirnoff vodka.
I stopped drinking, and I stopped making speaker boxes.
-
Story amazing written publish somewhere at a time.
-
I've seen so many people saying there are no credible test done to see if there are actually any differences in 'bit perfect' playback.
There are so many intelligent and knowledgeable folks in this forum, who seem to have the 'right' equipment to do such testing, I wonder why no one has designed a test to definitively answer this question.
Perhaps some of the folks around can/should discuss how to perform such testing and maybe an answer can be reached which is repeatable and definitive.
I don't know if the other 2 audio players in the test are free, of if someone has all 3 players, but perhaps they can be used in said testing.
Just a thought ::)
-
Tests have been done. Over and over.
Proof does not dissuade the deluded.
-
I'd be interested in references to tests comparing "bit perfect" players.
Brian.
-
Sorry to say, It's a true story.
-
I've seen so many people saying there are no credible test done to see if there are actually any differences in 'bit perfect' playback.
There are so many intelligent and knowledgeable folks in this forum, who seem to have the 'right' equipment to do such testing, I wonder why no one has designed a test to definitively answer this question.
Perhaps some of the folks around can/should discuss how to perform such testing and maybe an answer can be reached which is repeatable and definitive.
I don't know if the other 2 audio players in the test are free, of if someone has all 3 players, but perhaps they can be used in said testing.
Just a thought ::)
I'm tempted to do it. I have all the measurement gear, software and microphones to do it properly.
I can get OSX running on the NUC without too much effort too.
Only problem with my system is that it might not be good enough gear or setup to resolve the detail to pick if there are any differences, and my room isn't optimal for trying to do blind tests.
It might be a fun experiment to run though and see if any differences can be measured.
-
I'm tempted to do it. I have all the measurement gear, software and microphones to do it properly.
I can get OSX running on the NUC without too much effort too.
Only problem with my system is that it might not be good enough gear or setup to resolve the detail to pick if there are any differences, and my room isn't optimal for trying to do blind tests.
It might be a fun experiment to run though and see if any differences can be measured.
And if you're going to do these things "properly", wouldn't you really need a room like this?
http://www.acoustics.salford.ac.uk/facilities/?content=listening (http://www.acoustics.salford.ac.uk/facilities/?content=listening)
Tests & Standards
This room is one of few test rooms in the UK which can be used to meet the requirements of ITU-R BS 1116-1 for subjective assessments of small impairments in audio systems. This room can also be used to carry out listening tests on loudspeakers to BS 6840-13 / IEC 268-13.
I'd think that the room that you do your test in would be one of the biggest points of variability that a skeptic could point at to say, "not valid". A tough ask to find the right space.
-
If you are going to try to make recordings of the output of the speakers and use that as your basis of comparison, I think it's going to be extremely difficult. I'm betting that you can't make two recordings of exactly the same signal chain with exactly the same song, and have the recordings be identical. If you *can* make recordings that are identical, bit for bit of two different "plays" of the same thing, then you might have a shot at recording the differences between two *different* playback chains.
But I say, why go all the way to the acoustic level anyway? If you're trying to prove whether there's a difference, why not start further up the chain, at the digital level? I don't know how to do it, but I would like to see "recordings" of the digital bit stream being fed to a DAC. Record the output of one player feeding the DAC. Then record the output of the other player. Then compare and see if they are different.
The other way of doing this is pure blind testing. I see you've already said that your listening environment isn't suited to this. Blind testing, done correctly, is pretty difficult, so maybe it's good for you that you can't do it. :) (Meaning that it's a pain to do, so you save yourself the pain by not doing it.)
Brian.
-
It would be possible to do the blind testing using my HD600 headphones though which should be revealing enough.
I'm thinking about getting a new fully balanced DAC and headphone/pre AMP, the Asus Essence III, which does DSD too.
-
Read user mitchco's articles about testing JRiver on computeraudiphile.com.
-
Darko's review: >> (which I quite enjoyed)
http://www.digitalaudioreview.net/2015/08/a-stand-off-down-under-puremusic-vs-jriver-vs-audirvana/
Ah. I read as much of that as I could. It wasn't easy.
One question out of that though:
How can you show that each player produced the same output, if the only voting option was to pick one player which was the "best"?
The whole test was preordained to find difference. That is what we humans do. If we sought to find difference in something measured in the millions, where difference may defined as in the 100,000's, and we saw no difference, we would redefine difference as 10,000's, and if still no difference, in the 1,000's, and so on until even where there is no difference, we will see/hear/measure it, even when we have no instrument that can measure it. (i.e. Pico second jitter.)
The world is full of lots of very different people. I guess that is good, mostly. But that article just made me feel . . . yuck!
-
If you are going to try to make recordings of the output of the speakers and use that as your basis of comparison, I think it's going to be extremely difficult. I'm betting that you can't make two recordings of exactly the same signal chain with exactly the same song, and have the recordings be identical. If you *can* make recordings that are identical, bit for bit of two different "plays" of the same thing, then you might have a shot at recording the differences between two *different* playback chains.
But I say, why go all the way to the acoustic level anyway? If you're trying to prove whether there's a difference, why not start further up the chain, at the digital level? I don't know how to do it, but I would like to see "recordings" of the digital bit stream being fed to a DAC. Record the output of one player feeding the DAC. Then record the output of the other player. Then compare and see if they are different.
The reason to involve analog measurement is that jitter, for example, is measurable on an analog output of a DAC (not necessarily fed to a speaker, you'd want to loop it back to a very low distortion ADC or scope to measure the distortion). Analog output fed to a speaker would also be a really easy way to measure a volume difference.
I think loopback measurements (both digital and analog) are probably the most reliable way to measure and will show a better picture of various distortion components. However, in-room measurements with speakers can show that most of these exotic kinds of distortion are buried in orders of magnitude louder speaker and room distortion that often may render them completely inaudible in practice.
Even quite expensive speakers in treated rooms often have total distortion profiles at or worse than 1% (I've seen speakers that were a good bit better than that, but not orders of magnitude better). A well-behaved DAC might have a jitter distortion component much lower than .001%. It's true that jitter is "more audible" than harmonic distortion, but I'm not sure it's necessarily a hundred times more audible, etc.
If anyone is going down this road, here are two articles on measuring jitter, which is by far the hardest distortion component to measure:
http://www.anedio.com/index.php/article/measuring_jitter
http://nwavguy.blogspot.com/2011/02/jitter-does-it-matter.html
If anyone wants to measure plain old harmonic distortion, Holm Impulse has a distortion analyzer built-in that can provide the THD and also graph the individual harmonics. It can be used for both loopback measurements, and for in room measurements.
-
If you are going to try to make recordings of the output of the speakers and use that as your basis of comparison, I think it's going to be extremely difficult. I'm betting that you can't make two recordings of exactly the same signal chain with exactly the same song, and have the recordings be identical. If you *can* make recordings that are identical, bit for bit of two different "plays" of the same thing, then you might have a shot at recording the differences between two *different* playback chains.
But I say, why go all the way to the acoustic level anyway? If you're trying to prove whether there's a difference, why not start further up the chain, at the digital level? I don't know how to do it, but I would like to see "recordings" of the digital bit stream being fed to a DAC. Record the output of one player feeding the DAC. Then record the output of the other player. Then compare and see if they are different.
The MOTU AVB (http://yabb.jriver.com/interact/index.php?topic=96435.0) audio devices make this kind of testing very easy due to its multi-client driver and excellent mixer. You can reroute the digital input from a media player back to the computer. I do this by routing movies to Room Equalization Wizard for checking the maximum SPL levels on various channels and by routing to Spectrum Lab for checking bass content. You can also reroute back into a DAW or other program and save the file. Compare all files to see if the media players are identical digitally.
You could also have several media players all playing the same content at once to the DAC, and then switch in the mixer which media player is outputting to the speakers. This is using the identical DAC chip and routing for all media players with instantaneous switching.
With a 32-bit DAC, the MOTU AVB audio devices will eliminate dither as being an issue unless 24-bit dither is selected in JRiver in Options > Audio > Audio Device > Device Settings. (Note: The MOTU Ultralight AVB has a 24-bit DAC)
-
I'll make this fairly short: you have an opportunity here. What you need to do is talk it out and see how you can improve another round of tests. Don't argue the results of the current one, just suggest ways to improve methodology next time. Also break it down further than just sound. If it wins on sound, having reliability, features etc. would gain some support for other players.
ABX double blind is the de facto standard. They need to do that. And prior to that, you need to make sure you remove as many variables as possible first. Volume, eq etc need to go unless they are the same for all.
That is what I would press them on. Not on how the test was flawed, but here is how to do it better "so we can really get the ultimate player". Play into their thinking rather than against it.
-
I think gvanbrunt has a good point here.
I've seen tests in the past, back in the old days, where people recorded the SPDIF output of MC and then compared them to the original files, in order to prove lossless output. I looked around briefly, but couldn't find any. They were buried in threads on HydrogenAudio, I believe.
I imagine it would be more difficult, but also possible to accomplish the same thing with a USB DAC, and sniff the USB traffic on the bus. You can do it with Wireshark, but to convince anyone that it was "right at the plug" you'd probably need to use some specialized hardware (http://www.fte.com/products/FTS4USB.aspx).
But, I think gvanbrunt is right here. That won't convince anyone (though it might be interesting to catch "cheaters" if any exist). You'd pretty much need to be an engineer to figure it out, and even with a detailed analysis, to the uninformed, it would just be one more conflicting analysis they don't fully grasp to compare to the other ones from the other "side".
It would be nice to do additional, actually rigorous testing. Because that might actually convert some people. The only way to do it for real is an ABX test. You can ask, for each attempt, for people to choose which they prefer, but you need to do the X test to determine if they know what they're talking about.
You also cannot tabulate the results where the participants can see them "as they're collected" (by a show of hands, marking on a board up front, or whatever) because we're pack hunting animals and the psychology of "going with the herd" (or in some cases with some participants, the urge to "go against the herd") is way too strong. You also need to do multiple "rounds" of testing, with a variety of different samples, to get good results.
But, in the end, psychology is so strong, I don't know if it matters and if you'll convince anyone. We think we are rational machines, but we're not. We operate nearly entirely on a kind of subconscious "auto-pilot", and those tendencies are especially strong with things like this, that are ephemeral (and, to some people, basically spiritual).
If you're interested, and you get iPlayer or can otherwise dig it out, BBC's Horizon did a fantastic show about how the mind works and how we make decisions a while back: Horizon - s2014 e5 - How You Really Make Decisions (http://thetvdb.com/?tab=episode&seriesid=74379&seasonid=574904&id=4805148&lid=7) (and the episode right before it, was The Power of the Placebo, which was also good).
-
Glynor sums up the technical details well. Also keep in mind to "play at their own level" by motivating them add additional criteria to improve the results. All in the interests of finding the best sounding results of course. :)
I should point out that it is a very interesting situation you are in where you can influence their beliefs and possibly see some interesting results. The key is not to whack them over the head. They love experimenting with stuff and trying things. So build on that.
-
I agree with all the above. However...
I just want to point out I wasn't trying whack anyone or be overly critical. As we have all agreed, testing is a very complex process and from the discussions here it's clear a lot of things can be done differently which may or may not alter the outcome of testing. All I was trying to achieve is point out the variability of the test conditions.
Maybe it wasn't fair testing but the point still remains JRiver didn't test well under those conditions. There may have been a number of uncontrolled and unknown variables and biases that led to the result.
I'll consider all of the commentary and perhaps discuss with Tom, if he ever speaks to me again! :)
Even if we don't get the opportunity for a "re-match" I still think it would be an interesting experiment to do more detailed testing and comparison at the bit level and sample level as well as some more controlled listening test conditions, with at least blind ABX testing - double blind is very difficult under these circumstances.
I still think it's possible and quite likely that the human ear and brain is able to resolve more detail than can be measured in digital samples which means scientific analysis of "data" wont necessarily prove they sound different or "better".
Here's some very interesting reading on USB Audio.
http://www.thewelltemperedcomputer.com/KB/USB.html
A highspeed realtime hardware based USB analyser is about $500. (enough for up to 12mbit for class 1 USB Audio - ie upto 96/24)
-
Maybe it wasn't fair testing but the point still remains JRiver didn't test well under those conditions. There may have been a number of uncontrolled and unknown variables and biases that led to the result.
See, this is where I vehemently disagree, and is the biggest problem with many of these kinds of "for fun" audio tests (which try to cloak themselves in science without actually practicing it).
It wasn't an audio test. It was test of some kind, but not an audio test between players. You're saying (and have said a few times) "maybe it was a bad test, but"...
The "but" is... It wasn't a test. If you don't account for some of the factors you didn't account for, rigorously, you have meaningless results. You may have seen some "difference" but you have no way to gauge why you saw the difference, which is pretty much the definition of testing between different audio players. If the effect was "real" (and not driven by psychology), the most likely explanation for your test results, by far, is that volume levels were not properly matched.
It was a waste of time, except for the joy of doing it. Any document published from the results, is not serious and should be disregarded.
-
I still think it's possible and quite likely that the human ear and brain is able to resolve more detail than can be measured in digital samples which means scientific analysis of "data" wont necessarily prove they sound different or "better".
This is a religious statement, not a scientific one.
If that is your hypothesis, then prove it. ;) If you're saying it is "beyond science" and "beyond proof" then it is either religion, science woo (http://rationalwiki.org/wiki/Science_woo), or grifter-speak. You can believe whatever you want, but then you don't get to wear the trappings of science (http://rationalwiki.org/wiki/Pseudoscience) (at least, and not have people call you on it).
-
This is a religious statement, not a scientific one.
If that is your hypothesis, then prove it. ;) If you're saying it is "beyond science" and "beyond proof" then it is either religion, science woo (http://rationalwiki.org/wiki/Science_woo), or grifter-speak. You can believe whatever you want, but then you don't get to wear the trappings of science (http://rationalwiki.org/wiki/Pseudoscience) (at least, and not have people call you on it).
LOL That's too true. I stand corrected... I'm not sure that I'm qualified to prove such a hypothesis. :)
)
-
I agree with glynor. That kind of "belief" is passed around in some audiophile circles until eventually it becomes "fact".
We used to call this "hand waving", as in "you understand, don't you?".
-
I still would love to see another round of tests with more rigorous criteria. I don't believe you need to spend money on a USB analyzer, and perhaps some of the technophiles here could come up with a few simple ways to match volume eq etc for the tests? Anyone have a good way to do ABX that is also simple and cheap? I would bet you can't tell the difference if you even get them close.
-
Our friend, mwillems, had a modified version of his post (http://yabb.jriver.com/interact/index.php?topic=99580.msg689733#msg689733) published on Computer Audiophile this weekend.
http://www.computeraudiophile.com/content/663-my-lying-ears/
Thanks, mwillems.
-
Our friend, mwillems, had a modified version of his post (http://yabb.jriver.com/interact/index.php?topic=99580.msg689733#msg689733) published on Computer Audiophile this weekend.
http://www.computeraudiophile.com/content/663-my-lying-ears/
Thanks, mwillems.
I'm pleased and humbled by the reaction over there. I wasn't sure what to expect, but the comments have been very constructive so far. Some of the discussion over there about the difficulty of describing what different types of distortion or audio filters sound like almost inspired me to try and work up a primer with sample audio clips, but then I remembered that I have a 1 year old who is not very interested in waterfall plots or linear phase filters and who thinks microphone cables are great fun to yank on ;D
But if I were a younger man with more free time...
-
A nice (but no doubt brief) period of clarity.... Now lets get back to Hospital Grade Power Points!
-
Now lets get back to Hospital Grade Power Points!
Oh god no. If I have to hear that one again I'll install one in my home just so I can "insure maximum power transfer" when I stick my tongue in it.
-
If you're interested, and you get iPlayer or can otherwise dig it out, BBC's Horizon did a fantastic show about how the mind works and how we make decisions a while back: Horizon - s2014 e5 - How You Really Make Decisions (http://thetvdb.com/?tab=episode&seriesid=74379&seasonid=574904&id=4805148&lid=7) (and the episode right before it, was The Power of the Placebo, which was also good).
Chased up both docos. Thanks for the tip. Well worth the watch.
While not targetting audio listening tests, both programmes show the power of the mind in convincing the brain of truth. Loved the work with monkeys and currency. Revealing.
-
Our friend, mwillems, had a modified version of his post (http://yabb.jriver.com/interact/index.php?topic=99580.msg689733#msg689733) published on Computer Audiophile this weekend.
http://www.computeraudiophile.com/content/663-my-lying-ears/
Thanks, mwillems.
Thanks Jim, Thanks mwillems. Nice story/piece. :)
-
The AES looks into this apparently.
1. http://www.aes.org/events/136/papers/?ID=3894. Some of the resulting papers are relatively cheap (approx. 20 USD) and very interesting IMO.
2. The site provides names of people in the academic world who could be asked about testing methods and methodology.
George