INTERACT FORUM

Please login or register.

Login with username, password and session length
Advanced search  
Pages: [1]   Go Down

Author Topic: Downmixing VSTs  (Read 10574 times)

6233638

  • Regular Member
  • Citizen of the Universe
  • *****
  • Posts: 5353
Downmixing VSTs
« on: May 31, 2014, 04:53:31 am »

I was wondering if there are any plans to support VSTs which downmix the incoming signal.
 
Currently Media Center runs the Output Format DSP first, so any multichannel audio gets downmixed before it reaches the VST plugin.
 
I've just found out that Blue Ripple Sound offer a VST plugin which will do a binaural downmix of any multichannel audio source for headphone use, but it requires a multichannel input, and then I need to send a stereo signal to my DAC.
 
As far as I can see, there's no way to currently do this in Media Center.
 
There are a lot of other headphone downmix VSTs out there as well, and it would be nice if they could be supported. The stereo downmix Media Center offers is rather basic and really just pans the sound across the left and right channels rather than creating a 3D sound field.
Logged

JimH

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 71365
  • Where did I put my teeth?
Re: Downmixing VSTs
« Reply #1 on: May 31, 2014, 07:34:23 am »

Currently Media Center runs the Output Format DSP first, so any multichannel audio gets downmixed before it reaches the VST plugin.
It wouldn't do that unless you told it to.
Logged

6233638

  • Regular Member
  • Citizen of the Universe
  • *****
  • Posts: 5353
Re: Downmixing VSTs
« Reply #2 on: May 31, 2014, 08:44:51 am »

It wouldn't do that unless you told it to.
Right - but if I don't, it tries to send a 5.1 signal to my stereo DAC.
 
What I'm wanting to do is send the 5.1 input to the headphone VST, which converts it to a binaural signal, and then output that 2.0 signal to the DAC.
Logged

mwillems

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 5174
  • "Linux Merit Badge" Recipient
Re: Downmixing VSTs
« Reply #3 on: May 31, 2014, 11:11:22 am »

Right - but if I don't, it tries to send a 5.1 signal to my stereo DAC.
 
What I'm wanting to do is send the 5.1 input to the headphone VST, which converts it to a binaural signal, and then output that 2.0 signal to the DAC.

If you have a dummy sound card output (Mobo, etc.) that supports 5.1 or 7.1, you can do this by setting up two zones with loopback.  I've done it before when I needed to do additional processing or wanted to evaluate VSTs. 

It works like this:

Zone 1 is set to output using the MoBo soundcard with a non-exclusive output mode (WASAPI with exclusive turned off works for me). Output format set to source number of channels and the Binaural VST is enabled.  Make sure the MoBo soundcard is set to the windows default audio device

Zone 2 is set to output to your DAC using your normal settings.  Output format is set to stereo.  It should be irrelevant whether you enable downmixing or not in the second zone if the VST is doing it's job in the first zone.

Start playback in zone 1, and then enable loopback in zone 2 (or vice versa, order isn't super important).

It's not an easy or perfect fix, but it will allow you to do this kind of thing with JRiver, and evaluate how you like the sound of the VST, etc.
Logged

6233638

  • Regular Member
  • Citizen of the Universe
  • *****
  • Posts: 5353
Re: Downmixing VSTs
« Reply #4 on: May 31, 2014, 04:14:40 pm »

Unfortunately, I don't have such a device and loopback has not proven reliable enough for me to consider it a workable solution full-time. (I need a fixed latency and immunity to CPU usage for example)
Logged

J-a-k-e

  • World Citizen
  • ***
  • Posts: 117
Re: Downmixing VSTs
« Reply #5 on: May 31, 2014, 04:41:36 pm »

I run a similar setup for my prefered hrtf headphone plugin "JB Isone Surround"

For this very purpose I have a "Surround" zone which runs in Direct Sound mode with output channels set as default under device settings.
In DSP studio you need to then either disable the output channel plugin or leave the channels option set to "source number of channels."
Now the final step involves having the parametric equalizer plugin set after the multichannel headphone vst plugin, you need to mute all but the two front stereo channels otherwise the resulting two channel output will have the other channels blended in on top.
Logged

6233638

  • Regular Member
  • Citizen of the Universe
  • *****
  • Posts: 5353
Re: Downmixing VSTs
« Reply #6 on: May 31, 2014, 05:11:34 pm »

Unfortunately, I think your device accepts a multichannel input.
When I try to do that using JB Isone Surround I get the same error caused by sending a 5.1 signal to a stereo-only device.
The audio is downmixed to stereo, but the output to the DAC is still 5.1
 
The Output Format DSP is required to set the device output to stereo, but this happens first in the processing chain, preventing other plugins access to the original multichannel data, rather than Media Center's downmix.
Logged

mojave

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 3732
  • Requires "iTunes or better" so I installed JRiver
Re: Downmixing VSTs
« Reply #7 on: May 31, 2014, 10:29:25 pm »

When the JRiver asio driver was first released you could play to it in one zone and use Open Live in another zone to receive the stream. I'm not sure if it still works like that, but you could try.
Logged

J-a-k-e

  • World Citizen
  • ***
  • Posts: 117
Re: Downmixing VSTs
« Reply #8 on: June 01, 2014, 12:47:08 am »

Unfortunately, I think your device accepts a multichannel input.
I know what you're saying, my headphones connect to an O2 and ODAC combo & the ODAC is definitely only a two channel usb dac.

The key is that you must set your audio device to Direct Sound as to quote from the Microsoft msdn link here http://msdn.microsoft.com/en-us/library/windows/desktop/ee418679%28v=vs.85%29.aspx
Quote
If a system is configured for fewer physical speakers than the number of channels specified in a multichannel WAV file, the audio data is mixed appropriately and output to the existing speakers.


In some situations you can have issues with latency and Direct Sound, however I've been using this method for years and I've found it works just fine for watching movies and tv.
Logged

6233638

  • Regular Member
  • Citizen of the Universe
  • *****
  • Posts: 5353
Re: Downmixing VSTs
« Reply #9 on: June 01, 2014, 03:42:32 am »

The key is that you must set your audio device to Direct Sound as to quote from the Microsoft msdn link here http://msdn.microsoft.com/en-us/library/windows/desktop/ee418679%28v=vs.85%29.aspx

In some situations you can have issues with latency and Direct Sound, however I've been using this method for years and I've found it works just fine for watching movies and tv.
Ah, I see. I have to use ASIO rather than DirectSound.
I don't know if it's the driver or something else, but ASIO is the only output I have found to be completely reliable and not subject to interference from high CPU usage. ASIO also lets me set a fixed latency for the DAC.
 
When the JRiver asio driver was first released you could play to it in one zone and use Open Live in another zone to receive the stream. I'm not sure if it still works like that, but you could try.
Again, I appreciate the suggestions, but loopback is not a workable solution here. It's too unreliable. This would primarily be for video playback rather than audio.
Logged

Frobozz

  • Citizen of the Universe
  • *****
  • Posts: 626
  • There is a small mailbox here.
Re: Downmixing VSTs
« Reply #10 on: June 03, 2014, 10:01:56 pm »

This is a feature I'd also like to be able to do in MC.
I use headphones and would like to explore and experience the VST plugins that can downmix multi-channel to headphone binaural.  The plugins to do this can be expensive.  It would be a bummer to buy a plugin and then not be able to enjoy it in MC.  So it's something that I'd very much like to see MC support this through the VST plugins and using WASAPI or ASIO if it's reasonably possible.

The alternatives to trying to do this sort of headphone processing in software or VST plugins would be to do it in specialized hardware like the Smyth Realiser.  The hardware approach is really expensive and makes the VST options really reasonably affordable.  The Smyth Realiser is out of my price range, so if I'm going to do this sort of processing it will need to be in software.
Logged

6233638

  • Regular Member
  • Citizen of the Universe
  • *****
  • Posts: 5353
Re: Downmixing VSTs
« Reply #11 on: June 04, 2014, 07:58:42 am »

This is a feature I'd also like to be able to do in MC.
I use headphones and would like to explore and experience the VST plugins that can downmix multi-channel to headphone binaural.  The plugins to do this can be expensive.  It would be a bummer to buy a plugin and then not be able to enjoy it in MC.  So it's something that I'd very much like to see MC support this through the VST plugins and using WASAPI or ASIO if it's reasonably possible.

The alternatives to trying to do this sort of headphone processing in software or VST plugins would be to do it in specialized hardware like the Smyth Realiser.  The hardware approach is really expensive and makes the VST options really reasonably affordable.  The Smyth Realiser is out of my price range, so if I'm going to do this sort of processing it will need to be in software.
I probably would have bought a Smyth Realiser if it did not require you to take a measurement of an ideal room (i.e. pay for some studio time) and use Stax headphones for optimal results.
The box itself is expensive, but when you consider that, it becomes significantly more expensive. (and a hassle)
 
As I mentioned in my first post, I was considering the Blue Ripple Sound Headphone Surround VST which costs about $500.
It's a lot of money, but I'd consider paying this if it worked with Media Center as I use their Rapture3D OpenAL driver with games and it creates very convincing binaural sound with headphones.
Hopefully there will eventually be a consumer version at a reduced price rather than only offering a pro version. (like the $50 Rapture3D User compared to the $1000 Rapture3D Advanced)
 
But there are a handful of other headphone VSTs which try to do a similar thing too.
It would also be nice if there was a Media Center plugin which supported Dolby Headphone like there is for Foobar. (using dolbyhph.dll similar to the way MC currently uses dtsdecoderdll.dll for DTS-HD)
 
Dolby Headphone is not ideal, but it does create convincing positional audio, which is something that Media Center currently lacks for headphones users.
 
 
P.S. Razer Surround is free until the end of June, works via a virtual sound device (so it will work with MC) and offers up to 7.1 downmixing.
It's focused more on gaming, so I'm not a huge fan of it with movies, but you might want to try it.
Logged

Matt

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 41937
  • Shoes gone again!
Re: Downmixing VSTs
« Reply #12 on: June 04, 2014, 10:52:33 am »

This is a feature I'd also like to be able to do in MC.
I use headphones and would like to explore and experience the VST plugins that can downmix multi-channel to headphone binaural.  The plugins to do this can be expensive.  It would be a bummer to buy a plugin and then not be able to enjoy it in MC.  So it's something that I'd very much like to see MC support this through the VST plugins and using WASAPI or ASIO if it's reasonably possible.

The alternatives to trying to do this sort of headphone processing in software or VST plugins would be to do it in specialized hardware like the Smyth Realiser.  The hardware approach is really expensive and makes the VST options really reasonably affordable.  The Smyth Realiser is out of my price range, so if I'm going to do this sort of processing it will need to be in software.

I would just use MC to downmix.  It does it right.
Logged
Matt Ashland, JRiver Media Center

6233638

  • Regular Member
  • Citizen of the Universe
  • *****
  • Posts: 5353
Re: Downmixing VSTs
« Reply #13 on: June 04, 2014, 02:47:58 pm »

I would just use MC to downmix.  It does it right.
JRSS does a to-the-letter stereo downmix.
This is fine for speakers where the sounds only pan left and right.
 
Processing like Dolby Headphone, or headphone-focused VST plugins create a binaural downmix rather than a stereo mix.
This uses the properties of headphones to create positional information in a field around you (like 5.1 speakers do) rather than simply panning the sounds left and right.
Logged

Frobozz

  • Citizen of the Universe
  • *****
  • Posts: 626
  • There is a small mailbox here.
Re: Downmixing VSTs
« Reply #14 on: June 05, 2014, 03:28:34 am »

I would just use MC to downmix.  It does it right.

MC does do downmixing to stereo right.  The idea here though is to do a specialized downmix for headphones that creates a binaural 3D style of sound from the multi channel mix.  There are VST plugins being created that can do that sort of headphone specific downmixing.  It could be the ultimate headphone crossfeed.  There's a lot of potential in that idea for very good headphone sound.  But you need VST hosts that allow you to use a VST that takes multi channel and outputs stereo.
Logged

Frobozz

  • Citizen of the Universe
  • *****
  • Posts: 626
  • There is a small mailbox here.
Re: Downmixing VSTs
« Reply #15 on: June 05, 2014, 04:14:19 am »

I probably would have bought a Smyth Realiser if it did not require you to take a measurement of an ideal room (i.e. pay for some studio time) and use Stax headphones for optimal results.
The box itself is expensive, but when you consider that, it becomes significantly more expensive. (and a hassle)
 
As I mentioned in my first post, I was considering the Blue Ripple Sound Headphone Surround VST which costs about $500.
It's a lot of money, but I'd consider paying this if it worked with Media Center as I use their Rapture3D OpenAL driver with games and it creates very convincing binaural sound with headphones.
Hopefully there will eventually be a consumer version at a reduced price rather than only offering a pro version. (like the $50 Rapture3D User compared to the $1000 Rapture3D Advanced)
 
But there are a handful of other headphone VSTs which try to do a similar thing too.
It would also be nice if there was a Media Center plugin which supported Dolby Headphone like there is for Foobar. (using dolbyhph.dll similar to the way MC currently uses dtsdecoderdll.dll for DTS-HD)
 
Dolby Headphone is not ideal, but it does create convincing positional audio, which is something that Media Center currently lacks for headphones users.
 
 
P.S. Razer Surround is free until the end of June, works via a virtual sound device (so it will work with MC) and offers up to 7.1 downmixing.
It's focused more on gaming, so I'm not a huge fan of it with movies, but you might want to try it.

The Smyth Realiser recommends using Stax headphones, but does not require them.  You just need to use very good headphones to be able to be able to achieve the proper sonic effect.  Audeze and other headphones have been used with it successfully.  I've read the manual for the Smyth Realiser and setting it up optimally would be daunting.  Getting the full effect does require measurement time and setup time in a high-end surround sound room and high-end stereo room.

The idea of getting what the Smyth Realiser does but implemented in a VST has me very interested.  It could be the ultimate headphone "crossfeed" for music and movies.  A lot of possibility for better headphone sound.   But we need media player host software that can support such plugins if this style of headphone processing is to happen.  Needing to play your music using a multi track pro-audio editing program isn't the way I wan't to play audio.  And we need support for this sort of processing for video playback as well since a lot of multi-channel source is video.
Logged

6233638

  • Regular Member
  • Citizen of the Universe
  • *****
  • Posts: 5353
Re: Downmixing VSTs
« Reply #16 on: June 05, 2014, 08:48:52 am »

It looks like there's a relatively new piece of software - "Out of Your Head" - that replicates what the Smyth Realiser does, using a dummy head for the measurements rather than being personalized for you. It uses a virtual audio device for processing, so it will work with Media Center.
InnerFidelity's article posted yesterday is what tipped me off to it.
 
Current pricing is $99 with $25 for each speaker preset, but there's a trial that lets you use it for about two minutes at a time (changing the preset gives you another two minutes) and some pre-rendered demos on the website.
 
I would say that it does a very good job at taking the sound "out of your head" as the name suggests, but for me, I just couldn't find one of the rooms which I liked the sound of enough to use it all the time.
 
While I won't be buying it, it was a very interesting experience.
I expected to prefer one of the studio presets using Genelecs, but those rooms sounded completely dead and lifeless to me.
It made it clear to me in a way that I hadn't really been aware of before, that the room is so important to the sound of speakers. Some of the recordings were of speakers that cost hundreds of thousands of dollars (in a 7.1 setup) yet they sounded terrible to me - there's one in particular which had so much reverb it sounded like I was listening in a small tiled bathroom.
 
Even discounting how the rooms sounded, the other thing which stood out to me is that there were obvious problems at certain frequencies on most of the speaker setups - LFE in particular. The Focal: Scala Utopian setup was probably my favorite overall, but I still found that highly specific to what was being played, with some tracks not sounding "right" at all.
 
 
While software like this which uses a virtual sound device for processing works with Media Center, I'd really like to see support for Dolby Headphone via dolbyhph.dll, and support for VST plugins which downmix from 5.1/7.1 to a binaural signal.
 
MC does do downmixing to stereo right.  The idea here though is to do a specialized downmix for headphones that creates a binaural 3D style of sound from the multi channel mix.  There are VST plugins being created that can do that sort of headphone specific downmixing.  It could be the ultimate headphone crossfeed.  There's a lot of potential in that idea for very good headphone sound.  But you need VST hosts that allow you to use a VST that takes multi channel and outputs stereo.
To be clear, crossfeed and binaural downmixes are very different things.
Media Center offers crossfeed via its "Headphone" DSP, and I use a different VST for crossfeed that I prefer the sound of.
 
But crossfeed is only a way of making a stereo signal sound more natural on headphones.
When you listen on speakers you hear some of the left channel in your right ear, after a delay and shift in frequency response, and vice-versa.
This is what crossfeed simulates, and it can do a good job making headphone listening sound more natural and less fatiguing - but it only applies to stereo audio.
 
A binaural downmix takes a multichannel input, does some sophisticated processing, and gives you true positional audio, rather than simply panning the audio left and right as a "standard" downmix to stereo does.
I won't say "3D" positional audio, because 5.1 and 7.1 formats only describe a ring of sound around you on a flat 2D plane, there are no height channels.
With games, where the sound is being rendered in realtime and not limited to 5.1 or 7.1 channels, there are ways of getting true 3D sound with height information when using headphones.
Logged

Frobozz

  • Citizen of the Universe
  • *****
  • Posts: 626
  • There is a small mailbox here.
Re: Downmixing VSTs
« Reply #17 on: June 05, 2014, 03:07:19 pm »

To be clear, crossfeed and binaural downmixes are very different things.
Media Center offers crossfeed via its "Headphone" DSP, and I use a different VST for crossfeed that I prefer the sound of.

I intended to put the word crossfeed in quotes because binaural and crossfeed are very different things.  I do think that the binarual style of processing could be the ultimate headphone "crossfeed".  The benefits of crossfeed only better and more natural sounding to the ears and brain.

This is a new area and likely a niche area of audio processing.  I don't know if the VST plugins or the virtual device style options or some other method is the best way to implement this style of processing.  I don't know the dirty details for the pros and cons of the various implementation methods.  The problem with the VST solution is that there are no good media player hosts that can support that style of VST plugin.  The VST option is going to be DOA if it can't get any support beyond professional audio mixing applications.  It needs support in consumer playback software for music and movies if it is going to have a chance.  It's like the chicken and egg problem.  The VSTs won't get developed if there is no good host software to use them.  The host software won't add that support if there are no useful VSTs that need that support.
Logged
Pages: [1]   Go Up