INTERACT FORUM

Linux => JRiver Media Center 31 for Linux => Topic started by: danrien on September 17, 2023, 09:27:17 pm

Title: JRiver Panel/MCWS stops serving over HTTPS after a large amount of requests
Post by: danrien on September 17, 2023, 09:27:17 pm
When running MC31 on Linux, if I access a large playlist containing a few hundred to 1000 items over HTTPS in Panel and scroll quickly through it, MCWS stops serving requests until I restart MC. Over HTTP this works as expected. My host PC is running Ubuntu 22.04.3 LTS.
Title: Re: JRiver Panel/MCWS stops serving over HTTPS after a large amount of requests
Post by: Hendrik on September 18, 2023, 01:12:02 pm
Can you post a log of this happening?
Title: Re: JRiver Panel/MCWS stops serving over HTTPS after a large amount of requests
Post by: danrien on September 18, 2023, 02:29:16 pm
Sure... where can I find these log files?
Title: Re: JRiver Panel/MCWS stops serving over HTTPS after a large amount of requests
Post by: Hendrik on September 18, 2023, 02:54:53 pm
Enable logging in the Help menu, then reproduce the issue and you can find the log in ~/.jriver/Media Center 31
Title: Re: JRiver Panel/MCWS stops serving over HTTPS after a large amount of requests
Post by: danrien on September 18, 2023, 11:01:19 pm
There's a lot of initialization things going on, but I think the interesting piece is here:

0018053: 139774794913472: Sharing Plugins: JRWebService::Process: Finish (157 ms)
0018053: 139774794913472: Sharing Plugins: VHTTPMessage::Write: Wrote 2460 bytes
0018064: 139774794913472: Sharing Plugins: CHTTPRequestMessage::ReadPreamble: Failed to read Method
0018064: 139774794913472: Sharing Plugins: CHTTPListenerWorker::HandleConnection: Finish (218 ms)
0018064: 139774794913472: General: CReferenceCountedSocket::Close: SOCKET_DEBUG: closesocket() closing 26
0018333: 139775296251840: General: CReferenceCountedSocket::Close: SOCKET_DEBUG: closesocket() closing 25
0018585: 139774794913472: Sharing Plugins: CHTTPListenerWorker::HandleConnection: Start
0018635: 139774794913472: Sharing Plugins: CHTTPListenerWorker::HandleRequest: TCP: 192.168.1.73: GET: https://192.168.1.2:52200/MCWS/v1/File/GetInfo?File=6693057
0018637: 139774794913472: Sharing Plugins: JRWebService::Process: Start
0018637: 139774794913472: Sharing Plugins: JRWebService::Process: URL: /MCWS/v1/File/GetInfo?File=6693057
0018638: 139774794913472: Sharing Plugins: JRWebService::Process: Finish (1 ms)
0018638: 139774794913472: Sharing Plugins: VHTTPMessage::Write: Wrote 2330 bytes
0018660: 139774794913472: Sharing Plugins: CHTTPRequestMessage::ReadPreamble: Failed to read Method
0018660: 139774794913472: Sharing Plugins: CHTTPListenerWorker::HandleConnection: Finish (74 ms)
0018660: 139774794913472: General: CReferenceCountedSocket::Close: SOCKET_DEBUG: closesocket() closing 17
0018731: 139774794913472: Sharing Plugins: CHTTPListenerWorker::HandleConnection: Start
0018743: 139774794913472: Sharing Plugins: CHTTPListenerWorker::HandleRequest: TCP: 192.168.1.73: GET: https://192.168.1.2:52200/MCWS/v1/File/GetInfo?File=6725319
0018745: 139774794913472: Sharing Plugins: JRWebService::Process: Start
0018745: 139774794913472: Sharing Plugins: JRWebService::Process: URL: /MCWS/v1/File/GetInfo?File=6725319
0018746: 139774794913472: Sharing Plugins: JRWebService::Process: Finish (1 ms)
0018746: 139774794913472: Sharing Plugins: VHTTPMessage::Write: Wrote 2572 bytes
0018773: 139774794913472: Sharing Plugins: CTCPSocketReader::Read: Read failed: Connection reset by peer (104) (socket: 25)
0018773: 139774794913472: Reader: CTLSSocketReader::Read: Read failed: The TLS connection was non-properly terminated.
0018773: 139774794913472: Sharing Plugins: CHTTPRequestMessage::ReadPreamble: Failed to read Method
0018773: 139774794913472: Sharing Plugins: CHTTPListenerWorker::HandleConnection: Finish (41 ms)
0018773: 139774794913472: Sharing Plugins: CTCPSocketReader::Write: Write failed: Broken pipe (32) (socket: 25)
0018773: 139774794913472: General: CReferenceCountedSocket::Close: SOCKET_DEBUG: closesocket() closing 25
0018814: 139775121405632: Sharing Plugins: CTCPSocketReader::Write: Write failed: Broken pipe (32) (socket: 26)
0018814: 139775121405632: Sharing Plugins: CTCPSocketReader::Write: Write failed: Broken pipe (32) (socket: 26)
0018814: 139775121405632: Sharing Plugins: CTCPSocketReader::Write: Write failed: Broken pipe (32) (socket: 26)
0018814: 139775121405632: Sharing Plugins: CTCPSocketReader::Write: Write failed: Broken pipe (32) (socket: 26)
0018814: 139775121405632: Sharing Plugins: CTCPSocketReader::Write: Write failed: Broken pipe (32) (socket: 26)
0018814: 139775121405632: Sharing Plugins: CTCPSocketReader::Write: Write failed: Broken pipe (32) (socket: 26)

The broken pipe message then repeats ad infinitum... I've attached the full log (sans the repeated broken pipe messages, which quickly goes into the many megabytes in size).
Title: Re: JRiver Panel/MCWS stops serving over HTTPS after a large amount of requests
Post by: Hendrik on September 19, 2023, 09:14:11 am
I couldn't spot any issues on my end so far, but I did add some extra logging to see if it indicates anything in the future.

Does this happen repeatedly for you?
Title: Re: JRiver Panel/MCWS stops serving over HTTPS after a large amount of requests
Post by: bob on September 19, 2023, 09:22:06 am
Do a netstat while this is happening and see how many open connections you have.
Perhaps you are running out of a system resource.
Title: Re: JRiver Panel/MCWS stops serving over HTTPS after a large amount of requests
Post by: danrien on September 19, 2023, 11:38:40 am
I couldn't spot any issues on my end so far, but I did add some extra logging to see if it indicates anything in the future.

Does this happen repeatedly for you?

Yes - was just able to reproduce it again...

Do a netstat while this is happening and see how many open connections you have.
Perhaps you are running out of a system resource.


I did this before and after reproducing the issue, at a glance everything looks about the same, anything in particular I should look at?
Title: Re: JRiver Panel/MCWS stops serving over HTTPS after a large amount of requests
Post by: mattkhan on September 19, 2023, 12:11:15 pm
I would look at the browser console as well to see what it reports
Title: Re: JRiver Panel/MCWS stops serving over HTTPS after a large amount of requests
Post by: bob on September 19, 2023, 01:50:14 pm
Yes - was just able to reproduce it again...

I did this before and after reproducing the issue, at a glance everything looks about the same, anything in particular I should look at?
Something like if there were thousands of sockets in time_wait state or similar.
Title: Re: JRiver Panel/MCWS stops serving over HTTPS after a large amount of requests
Post by: danrien on September 22, 2023, 10:51:31 pm
Something like if there were thousands of sockets in time_wait state or similar.


I do not see anything related time_wait, the vast majority (if not all) of the connections are in the "Connected" state.

I would look at the browser console as well to see what it reports

The browser eventually just reports a connection timeout error...

I recorded a video if you are looking for some exciting Friday night entertainment... top left is the Firefox developer tools, top right is a `tail -f` of the media center logs, bottom right is just the top command running, and bottom left is me crazy scrolling through the playlist view. Interestingly, loading large resources such as images seems to trigger it sooner.

https://photos.app.goo.gl/p38XvpGvwKWBwwVs6

Happily, I got SSL working through an nginx reverse proxy to the Media Center HTTP port, so this is now a non-issue for me, but if there's a fix it would definitely simplify things for me.
Title: Re: JRiver Panel/MCWS stops serving over HTTPS after a large amount of requests
Post by: mattkhan on September 23, 2023, 02:45:24 am
running debian testing/kde

I have reproduced this using firefox (but only the 1st time I tried this, subsequent attempts didn't fail)
I have not reproduced it using chrome at all

IMO this is a panel issue in the first instance, I think its throttling/debounce code is not working efficiently, which leads to a massive number of completely unnecessary requests & the number of these varies significantly between chrome & firefox.

My highly scientific way to test this is as following

open large playlist (mine has ~15k items in it)
open it in browser
hold down the page down key until you get to the bottom of the list

in chrome, this consistently results in ~8k requests of which I actually needed just the 1st and last page
in firefox, this consistently results in about 50% more requests (~12k)

once I get to the bottom of the list, if I scroll back up then it starts making the same requests again so there must be zero caching in here (i.e. the viewport is just the visible area) and if there is a throttle, it's really not working properly.

MC completely falling over is a different problem  but probably you could test this properly by taking panel out of hte equation and writing a load script that batters MC with GetInfo?File=XXX requests from multiple threads and see if it you can break it

Title: Re: JRiver Panel/MCWS stops serving over HTTPS after a large amount of requests
Post by: bob on September 24, 2023, 12:02:52 am
Also please verify that the issue is only with https.
Title: Re: JRiver Panel/MCWS stops serving over HTTPS after a large amount of requests
Post by: mattkhan on September 24, 2023, 02:55:56 am
the main difference between chrome and ff is that chrome is visibly faster to scroll so makes fewer requests, there's definitely some sort of missing debounce on scroll though (or similar)

I can't reliably reproduce this on FF so probably the OP has to confirm https vs http
Title: Re: JRiver Panel/MCWS stops serving over HTTPS after a large amount of requests
Post by: danrien on September 24, 2023, 05:57:31 pm
Also please verify that the issue is only with https.

I have only reproduced this on HTTPS.