INTERACT FORUM

Please login or register.

Login with username, password and session length
Advanced search  
Pages: [1]   Go Down

Author Topic: Decoding before caching in memory  (Read 5935 times)

-JK-

  • Junior Woodchuck
  • **
  • Posts: 67
Decoding before caching in memory
« on: June 14, 2013, 02:33:44 pm »

Hi there audiophiles!

After reading http://thewelltemperedcomputer.com/KB/WAV-FLAC.htm and http://www.audiostream.com/content/further-adventures-flac: would it be possible to implement the feature of decoding before caching in RAM? So that the cached file is an ad-hoc created WAV/PCM file of the FLAC (or some other lossless codec's) file?

Regards, JK
Logged

jmone

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 14463
  • I won! I won!
Re: Decoding before caching in memory
« Reply #1 on: June 14, 2013, 04:44:50 pm »

Like this feature?
Logged
JRiver CEO Elect

-JK-

  • Junior Woodchuck
  • **
  • Posts: 67
Re: Decoding before caching in memory
« Reply #2 on: June 15, 2013, 01:05:14 am »

Yes, with additional conversion before caching, as mentioned. :-) It could of course be that MC is already doing this, but I don't think so, I rather guess that it caches the FLAC (or whatever) file and then starts decoding it from RAM.
Logged

rayooo

  • World Citizen
  • ***
  • Posts: 171
Re: Decoding before caching in memory
« Reply #3 on: June 15, 2013, 06:11:27 am »

I've had "play from memory" enabled for ages now. Although, I seem to recall a mention that if playing from a NAS, "Play from Memory" is redundant because the data received via NAS is buffered in RAM anyway, and I've been using NAS for audio storage for years now.

I do still have it selected though.  I've experimented dozens of times with FLAC vs WAV playback and I just don't have the keen hearing needed to detect any difference. I imagine differences all the time, but there is simply no way I'd be able to tell them apart in any kind of real testing.

'not having a clue what goes on under the hood in MC, I'd presume that "play from memory" would load up the FLAC in RAM and decode on-the-fly for playback.
If nothing else this would seem to need a bit less RAM, and thus less buffering time.

Ok, now turn off science and logic...

I'd love to have a "pre-decoded play from memory" option!  just to play with, as food for my inner-audiophile child.  :)

Logic back on...

Something tells me that there just might be one, or two, or 25 things the developers would like to get into MC of higher importance... :) :)  
Logged
MC-27-28> Meitner MA 1V2> CJ-GatV2> CJ ART 300s> Magnepan 20.7

-JK-

  • Junior Woodchuck
  • **
  • Posts: 67
Re: Decoding before caching in memory
« Reply #4 on: June 15, 2013, 10:38:54 am »

Something tells me that there just might be one, or two, or 25 things the developers would like to get into MC of higher importance... :) :)

Sure thing, but I guess it would be one of the quicker implementations. I'd love to see (and test) it anyway! Decoding my entire library to WAV/AIFF is out of the question unfortunately...
Logged

rayooo

  • World Citizen
  • ***
  • Posts: 171
Re: Decoding before caching in memory
« Reply #5 on: June 15, 2013, 12:53:04 pm »

Sure thing, but I guess it would be one of the quicker implementations. I'd love to see (and test) it anyway! Decoding my entire library to WAV/AIFF is out of the question unfortunately...

+1 Yes!
Logged
MC-27-28> Meitner MA 1V2> CJ-GatV2> CJ ART 300s> Magnepan 20.7

Vincent Kars

  • Citizen of the Universe
  • *****
  • Posts: 1154
Re: Decoding before caching in memory
« Reply #6 on: June 15, 2013, 01:42:54 pm »

I know this has been discussed before.

I do think it’s a pity JRiver doesn’t have a true memory playback feature.
Not because I believe in memory playback but because it would make JRiver a nice test bed.
If all processing including decoding has been done, one has the right test environment.
Anybody arguing that there is still a difference after all the decoding has been done will have a very hard time to proof that LPCM sounds different compared with LPCM  ;)
Logged

-JK-

  • Junior Woodchuck
  • **
  • Posts: 67
Re: Decoding before caching in memory
« Reply #7 on: June 15, 2013, 01:57:59 pm »

I know this has been discussed before and Jim won’t have it.
I do think it’s a pity JRiver doesn’t have a true memory playback feature.
Not because I believe in memory playback but because it would make JRiver a nice test bed.
If all processing including decoding has been done, one has the right test environment.
Anybody arguing that there is still a difference after all the decoding has been done will have a very hard time to proof that LPCM sounds different compared with LPCM  ;)


Right, that's just what my intention was. I'm not sure if I should believe in the FLAC vs PCM discussion, but I sure as hell would want to give it a try. :-)
Logged

retro

  • World Citizen
  • ***
  • Posts: 116
Re: Decoding before caching in memory
« Reply #8 on: June 15, 2013, 02:04:30 pm »

Right, that's just what my intention was. I'm not sure if I should believe in the FLAC vs PCM discussion, but I sure as hell would want to give it a try. :-)

+1
Logged

rayooo

  • World Citizen
  • ***
  • Posts: 171
Re: Decoding before caching in memory
« Reply #9 on: June 15, 2013, 03:33:04 pm »

I know this has been discussed before.

I do think it’s a pity JRiver doesn’t have a true memory playback feature.
Not because I believe in memory playback but because it would make JRiver a nice test bed.
If all processing including decoding has been done, one has the right test environment.
Anybody arguing that there is still a difference after all the decoding has been done will have a very hard time to proof that LPCM sounds different compared with LPCM  ;)


Well said! I agree 100%
Logged
MC-27-28> Meitner MA 1V2> CJ-GatV2> CJ ART 300s> Magnepan 20.7

6233638

  • Regular Member
  • Citizen of the Universe
  • *****
  • Posts: 5353
Re: Decoding before caching in memory
« Reply #10 on: June 15, 2013, 06:09:13 pm »

From my testing a while ago, what Media Center appears to do with the Play files from memory option enabled, is store the entire compressed file in memory.
So a 5MB MP3 will take up 5MB RAM, a 50MB FLAC file will take up 50MB RAM, a 500MB DSD file will will increase memory usage by 500MB during playback, and so on.

In addition to that, memory also increases by about 40MB or so during playback whether this option is enabled or not, which appears to be a buffer for decoded audio.

So if Media Center is idle at say 10MB, playback of a 50MB FLAC file will result in around 100MB of RAM usage. (10MB Media Center, 50MB FLAC file, 40MB buffer)
This means that Media Center is always playing audio that has been decoded to memory during playback from that ~40MB buffer.
It may be decoding later parts of the track into memory during playback, but not the part that is currently playing.


If you think about it, to have "full" memory playback, Media Center would have to decode your entire playlist into memory before playback, otherwise it's still going to be decoding other tracks into memory during playback - even if it's not the currently playing track.

Converting a 10-track SACD to Uncompressed WAV took about a minute on my system, resulting in almost 2.5GB of files.
Converting a 10-track 192kHz ALAC album to uncompressed WAV took about 30 seconds, and resulted in 3GB of files. (ALAC is faster to decode than DSD)

And that's what happens with only 10 tracks - think about how many tracks you might have in a large playlist.
Media Center's Play Doctor uses 100 tracks in its playlists for example. Do you have more than 25-30GB of RAM free in your system, and want to wait 5-10 minutes before playback can start?

Not that you could use Play Doctor with this hypothetical feature anyway, because as soon as one track is finished, Play Doctor would normally add another to the bottom of the list, so that there are always 99 upcoming tracks. This would cause Media Center to start work on decoding the track during playback, which removes the "benefits" of "full" memory playback. (no audio decoding during playback)

Audio decoding is a trivial task for a modern processor, and there have been a number of tests which have shown that even very high CPU usage has no impact on audio that is being played through external devices. (USB DACs etc)
People that dispute these facts, are trying to sell you something. (or have been misled by someone that is) And conveniently they will be unable to provide measurements, or may even claim that this is not something that can be measured. ::)
Expect a lot of comments from these people about how it's not only important to have "bit-perfect" playback, but to also get the timing of this bit-perfect playback right. But don't look at jitter measurements! Oh no, they don't show the "timing" errors we're talking about - those can't be measured!


To add this feature would just be a waste of time for the developers at JRiver, a waste of resources on your system, and make the program far less user-friendly during playback.

I've had "play from memory" enabled for ages now. Although, I seem to recall a mention that if playing from a NAS, "Play from Memory" is redundant because the data received via NAS is buffered in RAM anyway, and I've been using NAS for audio storage for years now.
I'm not sure about it being redundant, but the issue is that it may impact playback on slower networks because it will read the whole file from the NAS and place it in memory, then there's no more network traffic until the current song is finishing. If it can't download the next song into memory in time, or the network is slow to respond to new connections, there may be a break in playback.

With memory playback disabled, the song is streamed from the network, rather than the whole thing downloaded into memory at once, so it should be more seamless.
Logged

rayooo

  • World Citizen
  • ***
  • Posts: 171
Re: Decoding before caching in memory
« Reply #11 on: June 15, 2013, 08:23:20 pm »


I'm not sure about it being redundant, but the issue is that it may impact playback on slower networks because it will read the whole file from the NAS and place it in memory, then there's no more network traffic until the current song is finishing. If it can't download the next song into memory in time, or the network is slow to respond to new connections, there may be a break in playback.

With memory playback disabled, the song is streamed from the network, rather than the whole thing downloaded into memory at once, so it should be more seamless.

In my case (fast NAS and wired gigE network) I notice very little added "buffering" time when playing randomly selected tracks from memory vs play from memory disabled. A non issue in my case even with 192k 24bit WAV or FLAC.  "CD" bit rate FLACs buffer and play essentially instantly, High-Rez WAV may take a half second.

I can surely see where decoding and buffering-up a big play list would be silly, I'd hope that if, or when, or ever a "play from memory decoded" mode was added, it would only decode and "buffer-up" a track at a time, which again in my case would be of little concern, others with slow networks and or slower CPUs may find "play from memory decoded" - UN-usable.  Having said that, I wouldn't expect anything to change in MC until (if ever) this WAV vs FLAC issue moves from Voodoo.... where someone figures out a quantifiable explanation and proves there actually is an issue.
Logged
MC-27-28> Meitner MA 1V2> CJ-GatV2> CJ ART 300s> Magnepan 20.7

6233638

  • Regular Member
  • Citizen of the Universe
  • *****
  • Posts: 5353
Re: Decoding before caching in memory
« Reply #12 on: June 15, 2013, 09:49:58 pm »

I can surely see where decoding and buffering-up a big play list would be silly, I'd hope that if, or when, or ever a "play from memory decoded" mode was added, it would only decode and "buffer-up" a track at a time
But then you have to be decoding the next track while the current track is playing, and the claims about this improving audio quality are based around the fact that the CPU is not decoding audio during playback.

The other way to do it would be to have playback to stop in-between each track to decode the audio. My system is relatively fast, so decoding a DSD track might only take 6 seconds - which still completely breaks gapless playback - and on a slower system, that might be 20-30s per track.

And I still don't see the point, because Media Center is already playing from a buffer of decoded audio, and it has been shown that CPU usage has no impact on audio being sent to external devices.
Logged

-JK-

  • Junior Woodchuck
  • **
  • Posts: 67
Re: Decoding before caching in memory
« Reply #13 on: June 16, 2013, 04:33:22 am »

I'd recommend reading the two articles, at least that's what changed my opinion on this subject (and I'm usually rather sceptical when it comes to things like this).

The issues that seem to affect the perception of sound the most in this case are supposed to be the relatively high overhead of the decoding process with FLAC and the asynchronicity of the codec.

I'm still not convinced though as I have to hear and see everything first myself before turning into a believer, but the last paragraph of the second part of the second article (http://www.audiostream.com/content/cut-flac) really made me want to give it a try:

Quote
Thanks to John DeVore for recommending the 1 track over and over test because I was ready to give up after the whac-a-mole approach which gave very different results. That reminds me of another thing I'd take away from this—people who make music or who make things that make music (or both) and listen for a living are worth listening to.

So, if MC doesn't get the feature, does anyone know of a Windows player that already has it? :-)
Logged

Vincent Kars

  • Citizen of the Universe
  • *****
  • Posts: 1154
Re: Decoding before caching in memory
« Reply #14 on: June 16, 2013, 04:47:32 am »

Foobar2000

File> Preferences> Advanced> Playback > Full file buffering up to (kB)

Specify a value large enough to contain the entire track
Logged

JimH

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 72438
  • Where did I put my teeth?
Re: Decoding before caching in memory
« Reply #15 on: June 16, 2013, 06:12:55 am »

The issues that seem to affect the perception of sound the most in this case are supposed to be the relatively high overhead of the decoding process with FLAC and the asynchronicity of the codec.
On a modern PC, the load of decoding a FLAC file would be minimal.  And even if it were not, you wouldn't be able to hear anything unusual unless there was something else wrong with the machine.
Logged

JimH

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 72438
  • Where did I put my teeth?
Re: Decoding before caching in memory
« Reply #16 on: June 16, 2013, 06:21:46 am »

To add this feature would just be a waste of time for the developers at JRiver, a waste of resources on your system, and make the program far less user-friendly during playback.
Agreed.
Logged

-JK-

  • Junior Woodchuck
  • **
  • Posts: 67
Re: Decoding before caching in memory
« Reply #17 on: June 16, 2013, 08:21:52 am »

On a modern PC, the load of decoding a FLAC file would be minimal.  And even if it were not, you wouldn't be able to hear anything unusual unless there was something else wrong with the machine.

How unfortunate! Didn't think that it would be a great deal implementing this feature. I've found some software players that have it built in in the meantime (XXHighEnd, Audirvana), so I'm gonna give it a try and report back.

As for "waste of resources": I think it's a waste to have large amounts of unused RAM, I'd rather do something with it. :-) And I couldn't think of a reason why user-friendliness should decrease.
Logged

JimH

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 72438
  • Where did I put my teeth?
Re: Decoding before caching in memory
« Reply #18 on: June 16, 2013, 08:36:12 am »

I think it's a waste to have large amounts of unused RAM, I'd rather do something with it.
Are you an expert on computing? 

Some of the audiophile discussion of computing is just the repetition of stories that have no basis in fact. 

It is JRiver's job, in part, to sort out the truly relevant improvements possible from the noise of speculation.  We don't make changes just because xxplayer does it.
Logged

Vincent Kars

  • Citizen of the Universe
  • *****
  • Posts: 1154
Re: Decoding before caching in memory
« Reply #19 on: June 16, 2013, 08:55:36 am »

On a modern PC, the load of decoding a FLAC file would be minimal.  And even if it were not, you wouldn't be able to hear anything unusual unless there was something else wrong with the machine.

That is exactly what I’m thinking.
Reported differences between FLAC and WAV are expectation bias (WAV is more “pure” than FLAC so it must sound better) or a problem with the system.
If it is a system problem, how to make sure?

Indeed using true memory playback. You have eliminated the difference in I/O (WAV is about 2x FLAC in size) and the differences in CPU load.
If all these possible differences are eliminated by loading bit identical data in memory, you have the option to ascertain that it is indeed a system error.

Of course, I expect that even with true memory playback people will report audible differences and again in favor of WAV. You can’t beat a sighted listening test!
Logged

6233638

  • Regular Member
  • Citizen of the Universe
  • *****
  • Posts: 5353
Re: Decoding before caching in memory
« Reply #20 on: June 16, 2013, 09:15:30 am »

As for "waste of resources": I think it's a waste to have large amounts of unused RAM, I'd rather do something with it.
As you fill up your memory, the likelihood of electrical interference ruining the playback of cached audio greatly increases, and the more ram you have, the more your system is affected by cosmic rays.
I can hear a big difference between 16GB in my system when it is nearly all used instead of just one 4GB stick of ram that has as much free as possible. Anything that takes ram away from the player has to be disabled.
Audio becomes less coherent, soundstage gets narrower, and timing is wrong as memory usage is increased and it gets split over multiple sticks of ram. I have tested this repeatedly and every time I pull out the other three sticks of ram, the listening experience is transformed.

I recommend only using one stick of ram - the smallest amount that Windows allows - to reduce the noise impact it has.
Windows 8 is best for playback because it uses the least amount of memory once you disable everything.
Slower memory also reduces the high frequency noise produced by a PC, as it isn’t stressing the memory chips as much. I recommend buying the fastest rated ram you can find, and then running it at the slowest speed your motherboard supports, so that it is not working hard at those speeds.


See how easy that was? That’s completely fabricated, but if you don’t know a lot about the inner workings of computers, you might think that sounds plausible, try it, and for whatever reason (placebo, expectation bias, the fact that our audio memory is so small etc.) you thought things did in fact sound better that time.
So you leave your system running with one stick of ram, and post about how it really did take your listening experience to the next level. If enough people try it, someone else will eventually report the same thing.

The less convenient a tweak is, the more audiophiles are likely to believe in it - especially when the majority of people refute it, but there is a small but vocal group that says it really did make a world of difference for them, and that people saying it won't do anything haven’t bothered to try it.
Logged

Hendrik

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 10935
Re: Decoding before caching in memory
« Reply #21 on: June 16, 2013, 09:33:12 am »

See how easy that was? That’s completely fabricated

And i was thinking you lost your mind. :D
Logged
~ nevcairiel
~ Author of LAV Filters

-JK-

  • Junior Woodchuck
  • **
  • Posts: 67
Re: Decoding before caching in memory
« Reply #22 on: June 16, 2013, 12:30:53 pm »

Are you an expert on computing? 

Some of the audiophile discussion of computing is just the repetition of stories that have no basis in fact. 

It is JRiver's job, in part, to sort out the truly relevant improvements possible from the noise of speculation.  We don't make changes just because xxplayer does it.

Well yes, I would consider myself rather experienced in fact. :-) But even with no experience at all it wouldn't seem logical to me not using what is already there. Most people probably have gigabytes of RAM available when listening to music, so why not make it optional to fill this ram with decoded audio? What would be the disadvantage of doing it this way? Would it really be that much of a change code-wise to justify not having it as another potentially customer acquiring item on your feature list?
Logged

DarkPenguin

  • Citizen of the Universe
  • *****
  • Posts: 1921
Re: Decoding before caching in memory
« Reply #23 on: June 16, 2013, 01:29:00 pm »

Cost of dev plus the cost of support for the [redacted] with an overloaded system who decides to enable it all for 0 audio benefit.  I wouldn't spend an hour on it.
Logged

Matt

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 42373
  • Shoes gone again!
Re: Decoding before caching in memory
« Reply #24 on: June 16, 2013, 07:52:36 pm »

I'd actually like to make this change because:

1) It's a little cleaner technically, at least for ranged playback (CUE, SACD ISO, etc.)
2) I'm sick of the zombie argument (ie. it won't die, no matter how many times we try to kill it)
3) It would be fun to see what new argument people have for WAV sounding best after making the change

One consideration is that some formats require a ton of space decoded.  For example, a 20-minute 5.1 DSD SACD track would take around 19 GB of memory (since we use lots of precision, even if it's overkill).  A more normal CD track would take around 50 MB.
Logged
Matt Ashland, JRiver Media Center

-JK-

  • Junior Woodchuck
  • **
  • Posts: 67
Re: Decoding before caching in memory
« Reply #25 on: June 17, 2013, 02:45:43 am »

One consideration is that some formats require a ton of space decoded.  For example, a 20-minute 5.1 DSD SACD track would take around 19 GB of memory (since we use lots of precision, even if it's overkill).  A more normal CD track would take around 50 MB.

I guess you would have to add an exception handle for the rare kind of a situation where there is not enough RAM to cache the entire song(s) (should it not already be there), but to be honest I haven't stumbled across DSD files in my rather lengthy computer audio life yet. Where does one get these? Can SACDs be ripped directly onto hard drives?
Logged

Vincent Kars

  • Citizen of the Universe
  • *****
  • Posts: 1154
Re: Decoding before caching in memory
« Reply #26 on: June 17, 2013, 03:03:17 am »

For some reason or other DSD has become the audiophile flavor of the month.
New DACs now offer DSD over USB (DoP)
The DSD catalog is rather limited.
You can find a couple of download sites here: http://www.audiostream.com/content/dsd-ready-dacs-short-list
Logged
Pages: [1]   Go Up