INTERACT FORUM

Please login or register.

Login with username, password and session length
Advanced search  
Pages: [1]   Go Down

Author Topic: On-the-fly real-time conversion  (Read 2577 times)

marin849

  • Recent member
  • *
  • Posts: 12
On-the-fly real-time conversion
« on: May 23, 2007, 09:37:07 am »

When streaming an audio file with the UPNP server of Media Center12, it encodes the whole track and store it as a temporary file, from which it then streams the audio. Wouldn't it be possible to encode in real-time in memory in stead? Like encode a few seconds at a time, and buffer in memory? To avoid bursts of 100% cpu-load and a lot of disk writing at the beginning of playing a new track.
Logged

John Gateley

  • Citizen of the Universe
  • *****
  • Posts: 4957
  • Nice haircut
Re: On-the-fly real-time conversion
« Reply #1 on: May 23, 2007, 10:34:31 am »

What's wrong with 100% CPU (given that thread priorities will ensure that processes that need the CPU get it) and Disk I/O (since it helps ensure that there is data to stream when it is needed)?

j

marin849

  • Recent member
  • *
  • Posts: 12
Re: On-the-fly real-time conversion
« Reply #2 on: May 23, 2007, 10:39:18 am »

If i sit in the living-room and browse some tracks, and maybe just listen to the first 5-20 seconds of some tracks before deciding, the Media Server PC will be at a constant 100% load, instead of a steady 5% or whatever is required when doing it in real-time. It's just unnecessary, that's all. And it generates extra heat in my server closet. :)
Logged

JimH

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 72413
  • Where did I put my teeth?
Re: On-the-fly real-time conversion
« Reply #3 on: May 23, 2007, 10:50:46 am »

Virus checker?

It's also possible that Windows is swapping memory and disk if the PC's memory is insufficient.
Logged

marin849

  • Recent member
  • *
  • Posts: 12
Re: On-the-fly real-time conversion
« Reply #4 on: May 23, 2007, 11:01:32 am »

No. I checked the behavior with a file system monitor. When I select a track on my receiver and Media Center decides that it's necessary to convert the audio (flac->mp3 for example), what happens is that Media Center starts converting the source file to a file c:\temp\streaming encoder.mp3 and this encoding process is done as fast as possible, i.e. 100% cpu usage and the whole track is perhaps converted in 15 seconds or so. But if I decide to change track after listening to it for, say, 14 seconds, what happens is that Media Center throws that "work" away and starts converting another track, thus 14/15=93% of the conversion work was unnecessary since I never listened to it. What I mean is that it would be smarter to do the encoding at real-time speed.
Something like:
The client requests to read/play a track
Allocate a buffer in memory
Decode the first 5 seconds or so, store in buffer,
Begin serving data to the client
As the client reads data from buffer, more of the source file is decoded to the in-memory buffer as needed, in real-time.

Ideally, uncompressed wav-format could be sent to the client instead of mp3/wma/mpc, to remove the need for re-encoding, and to maximize quality when playing over a LAN. It must be easier to implement this with WAV as well, since no encoder is needed, except formatting the data as WAV.

/Martin

Logged

Matt

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 42344
  • Shoes gone again!
Re: On-the-fly real-time conversion
« Reply #5 on: May 23, 2007, 01:58:44 pm »

UPnP devices often have a large buffer.  We may only be fulfilling the requests from the device.
Logged
Matt Ashland, JRiver Media Center

marin849

  • Recent member
  • *
  • Posts: 12
Re: On-the-fly real-time conversion
« Reply #6 on: May 24, 2007, 12:16:55 am »

Perhaps. But the buffer size would have to be at least 12Mb then, and that is really quite huge. The device is a Denon AVR-4306 receiver. Maybe I should try with another client just to check the behavior.
Logged

benn600

  • Citizen of the Universe
  • *****
  • Posts: 3849
  • Living: Santa Monica CA Hometown: Cedar Rapids IA
Re: On-the-fly real-time conversion
« Reply #7 on: May 26, 2007, 11:05:51 am »

I like the instant conversion feature.  Lol, how does it generate additional heat?  15 seconds of 100% cpu usage will not raise the temperature too much!  That is probably a high estimate, too.  When I get my new Xeon server built in two weeks or so, I'm betting it will be 5-10 seconds!  Woo hoo!

I mean honestly, this seems like an unnecessary complaint considering that from a technical point of view, telling the LAME encoder to only encode short pieces would introduce a lot of additional potential problems--like it crashing from continually activating the LAME thread.  Plus, when LAME is going, no matter what priority it's at (like LOW), it will take 100% if it can.  So you'll have 100% for 1 second, ~2% for 1 second, blah blah blah....blah blah.
Logged

marin849

  • Recent member
  • *
  • Posts: 12
Re: On-the-fly real-time conversion
« Reply #8 on: May 26, 2007, 11:40:04 am »

I like the conversion feature too!
And if you listen to the whole song every time it doesn't make any difference at all, only that the cpu usage comes in short 100% bursts instead of a steady <5%. The problem is if you sit in the sofa and change track 5-10 times in a row to find a good one. The track change time is a bit longer and you waste a lot of cpu cycles and use a lot of disk i/o (several mb/s).
Logged

horse

  • Regular Member
  • World Citizen
  • ***
  • Posts: 212
Re: On-the-fly real-time conversion
« Reply #9 on: May 26, 2007, 03:46:28 pm »

Why convert?

This highlights the current dilema for what format should your source library be?
I went round and round and really liked FLAC, it was lossless, open, could be streamed and had tags. The major downside was client support.
For lossless, the only common format is WAV, however despite the fact it can support tags, nobody does, or they use proprietary ones.
A few years back I had ripped my entire collection in MP3Pro thinking it would catch on. I learned :-)
My priority was good streaming, so my solution was to buy a bigger disk and rip the entire collection again using WAV (500GB for less than $150 supports over 500 CD's), MC12 provide all the information regarding artist, title etc. to my Denon 4806CI. Browsing is almost instant, I get truely lossless because I didn't go via an MP3 codec, ripped HDCD's even work.
If I want to load music onto my MP3 player, MC converts on the fly (OK, convert then download :-) ) and it doesn't really matter if it takes a few extra seconds.
If one day there is a universally supported format which does all I want, I can be 99.99% sure there will be a way to convert from WAV.

OK, so that sorts out my audio, now if I could just find a universal video format with: -

Good client support
Subtitles
Menus
Multi Audio track support
Good compression like H.264, MP4 or xvid/Divx

On the fly transcoding of video is really going to warm up my server room, it's warm enough as it is!
Logged

marin849

  • Recent member
  • *
  • Posts: 12
Re: On-the-fly real-time conversion
« Reply #10 on: May 27, 2007, 01:17:01 am »

Hehe, I did the same thing. MP3 first, bought a better stereo, threw MP3 away. Found FLAC to be optimal (HDCD works great with FLAC as well, since it's bitperfect). :)
However my Denon 4306 doesn't play FLAC. WAV is kind of a solution, as you describe. How doec MC12 provide artist/title info? From the filename? That would be looong filenames.

I still think that with the right software, it could work exactly as you describe with WAV, but with FLAC-files (lossless compression, proper tags stored in the FLAC-file).
 It wouldn't consume any more CPU than playing the FLAC-file on the computer (almost nothing at all), since it should be streamed in PCM/WAV-format to the receiver! No encoding, only decoding of FLAC. But this should be done in real-time without decoding everything to a temporary wav-file. It would be NICE!
Logged

horse

  • Regular Member
  • World Citizen
  • ***
  • Posts: 212
Re: On-the-fly real-time conversion
« Reply #11 on: June 02, 2007, 07:23:52 pm »

Couldn't agree more regarding FLAC nothing to actually decompress, more of a re format and stuff some extra bits in (Good point re HDCD as FLAC is bit perfect) Wish more stuff supported it. just about everything supports WAV - except my Tivo that needs me to convert on the fly to MP3 if I wanted to use it for music.

MC12 uses the information from the library to send to the Denon. The directory and filenames seem to have no affect. It is the library properties that are used to deliver Album, Artist Name, Title, Date etc.
My guess is if I convert all the WAV files to FLAC it will add the tags using the information from the library properties. That is how it seems to work when I add files to my MP3 player using the WAV files as the source. IDv3 tags are populated correctly.

Using the library propeties I have the following:

Album Artist (allows me to manage my various artist albums how I want to)
*Artist
*Title / Track name
*Album Name
Track number
Duration
Date
Disc #
Genre
Useage (party, relaxing, mine, the wife's , childrens etc. Almost another genre)
Mood (another variation for additional genre type classification)
Rating
Comment

The items with * are the only ones displayed on the Denon when playing. All the other I can use for selecting using the menu's on the Denon
Logged

marin849

  • Recent member
  • *
  • Posts: 12
Re: On-the-fly real-time conversion
« Reply #12 on: June 03, 2007, 01:42:02 am »

FLAC still has to be decoded properly, but there is nothing to ENCODE/COMPRESS, as wav is simple pcm-data with a header in the beginning.

It is always the Media Server who sends the browsing information to the device. Theoretically, the directory list function on the client could be used to change settings on the Media Server, execute a script on the computer, or whatever. :)
Logged
Pages: [1]   Go Up