Your answer suggests that the Windows version would do the trick.
Is that true?
(Though some Apple users have the "Nach Canossa gehen wir nicht"-attitude versus Windows, I am more a "Paris vaut bien une messe" type of guy)
Thanks
The windows version can be configured to support certain line inputs, but you might have trouble using toslink/SDPIF for your intended application. Specifically, SPDIF doesn't have the bandwidth to support uncompressed 5.1; it can only output 5.1 with lossy encoding. So it's not ideal from a sound quality perspective at step one, but there's a more pressing problem:
While JRiver for windows can be configured to accept a line input from an ASIO device (or with some extra software, can be made to playback from non-ASIO live inputs as well), I'm not sure that it supports decoding on any of those inputs (I think it's just expecting an audio stream, not a DTS or DD encoded stream). So you might need an additional piece of software as an "interpreter."
And then finding an affordable external sound device with a TOSLINK *input* (which are much, much less common than TOSLINK outputs) might be a challenge. For example there are external devices with SPDIF outputs for $20 or $30, but I'm not aware of any quality products with SPDIF inputs that are less than $100. And even with such a device, you may run into issues identified above with the encoding. I don't have anything I could test with (I migrated away from SPDIF), so I can't confirm, but I would expect trouble with your proposed signal path.
Is there a reason that you can't just use an external USB DVD drive connected directly to the computer? No need to rip, just play the disc back directly in JRiver and get lossless 5.1 (not sure if DVD playback is supported on the Mac side, but it definitely is supported on the Windows side). That would require many fewer steps, and would almost certainly be cheaper and higher quality than trying to route TOSLINK in through an as yet to be determined device (external DVD drives cost $15).