Leave the channel offset at 0. You only change it if you are using a different set of outputs on an audio device. For example, channels 0-7 on my Steinberg UR824 go through the its own mixer while channels 10-18 bypass the mixer. Channel 19+ are routed to digital outputs.
In JRiver you set the output to 5.1 in the Output Format DSP You can either use JRiver's bass management in the Room Correction DSP or you can route manually using the Parametric Equalizer DSP.
If you use JRiver's Bass Management, you will still need to copy the subwoofer channel to the center channel. This will give you bass output on channels 2 & 3 on the RME. Activate the Parametric EQ DSP, add Mix Channels, and copy the subwoofer to the center channel. See the attached picture. You will need to make sure to move the Parametric Equalizer DSP after the Room Correction DSP by grabbing and moving it with the mouse.
If you want to have stereo subs, you will need to do that entirely in Parametric Equalizer. You copy the Left and Right channels to the Center and Subwoofer respectively. Then you add high and low pass filters and possibly an adjust the volume filter. I've attached a picture to show you what to do.
Ask if you have more questions.