Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[Ken Brookings]

We have a local group of audiophiles and wannabe audiophiles that get together every so often to compare notes, new music, audio equipment and so on. Last Friday we conducted an encoding format test.  

I am a frustrated scientist/statistician, impassioned long term audiophile and Media Jukeb.. uh I mean Media Center devotee.  These tests were conducted using Krell Electronics, an awesome pair of Eggleston Andra IIs, a Meridian 8000 CD player, Cardas cables and JRiver Media Center 9.088 (oh, yeah and a reasonable amount of sushi and Saki).

Four songs were picked, they were:
Instrumental: She by James Newton Howard (great audio test cut).
Voice: Bird on a Wire by Jennifer Warnes
Symphony: New World Symphony by Antonin Dvorak (a 30 second sample that included very light thin passages followed by full on thunder passage..
Recovered from Vinyl: Fire by Arthur Brown; Using Acid Foundry for clean up.

Each was encoded:
MP3 at 128kbs
WMA at 192kbs
APE lossless.

There were 9 listeners, filling out secret ballots.  Each music sample was 30 seconds long.  Each song sample was played twice thru in each encoding format, then the next sample rotated thru twice.  Not too much Saki had yet been ingested.

Summary of Results:
4 Listeners could detect no difference in the formats.
1 Listener liked the MP3 best
2 Listeners liked WMA best
2 Listeners liked APE best.

Observations:  
The two that liked APE best were classical music fans.
T
he one that liked MP3 best is a rock and roll fan and, you should be warned, has been known to say things about wine like "it has a certain humor" . .

Of the five that had a preference they said there was very, very little difference.  

Attempts to describe their preferences included words like "richness" "depth" and "character" (or were they talking about the Saki?).

For whatever its worth,
Ken

[JimH]

Great report!  Sounds like it was fun.

I'm wondering why WMA was given 192Kb, but MP3 was only allowed 128.

I would also guess that most of the sushi drinkers were male.  We have generally lousy ears.  And I'll bet they were 50ish (no kids can afford that gear).  Age also hardens the hearing.

It would be interesting to invite the wives and daughters next time.

Thanks for sharing this.

[Ken Brookings]

Jim, you're generally right about the listeners.  However, there were three women.  Ages of testers 54 45 (my wife), 50, 22, 39, 15 (my son), and three in the 36 to 45 range.   My wife who has incredible hearing and is a pianist could find no differences.  My 15 y/o son and music enthusiast preferred APE.  The other two women were split between APE and WMA.  The MP3 affecianado is only 36 but a Grateful Dead fan and probably has fried ear drums.

No real reason to chose 128 for MP3s except that is the rate at which most emusic files are sampled.  Plus the participants seemed to have a lot of MP3 stuff in their databases at that rate.

Enjoy your Sunday.
Ken

[MachineHead]

Ken,

You should download Nero from the Hydrogen Audio site and play with the 50 free mp4 encodings. There is a difference, but on the stuff I'm using right now it isn't as dramatic as I thought. Maybe on your test equipment it would make a (bigger) difference between mp3 and mp4 (and other formats). Just a thought.

Last encoding I had used was with Audiophile setting (VBR). The sound is really clear, but as I said, isn't as pronounced as I figured it should be.

Longing for a new soundcard...

[Ken Brookings]

Thanks for the Nero tip, machinehead.  Even though we went thru all that testing I'll still continue to encode with APE.  My old ears probably can't here the difference between APE and more econimical (disk space wise) formats but memory is cheap and someday I'll leave all this to my kids with young ears.  Regardless of what tests show, mine or otherwise, there is something satisfying about creating a big library encoded in a lossless format.
Ken

[Matt]

It's weird how the more you listen, the more you can pick little differences out.

A few times when I've played with making a lossy APE mode, I'll start the night thinking "wow, I'm on to something -- this sounds perfect."  However, by the end of the night, after hours of careful listening, I'll think "yikes, this sounds like garbage!" (even though it's more tuned than what I started with)

I guess it's like how some silly noise like a clock ticking won't bother you until someone points it out.  After that, it's all you hear.

[plas]

MP3 and WMA are not very good encoding formats.
The best encoders are OGG Vorbis and WMA 9 VBR.

[JimH]

Better than those is anything lossless.  APE (Monkey's Audio),  for example.

[fluffybunny]

Ken, great work, but you don't say (or maybe I missed it) whether the audience knew what format they were listening to each time, or whether it was a 'blind' test?

The cynic in me wonders, for example, whether your son picked APE based purely on its sound quality?

Also, can you clarify how the samples were played? Were they all burnt to CD then played from there?

[Ken Brookings]

fluffybunny, thanks for the kind words.  The participants did not know what format they were listening to.  My son is a classical music fan and he claims that with APE the instruments were better individually rendered and with the other formats the instruments were less individually defined.
I burned a CD with the music samples on it so I could just play the cd, pausing after each run thru.
Ken