Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[tiberiuspv]

In the past, I have seen some complaints about MCWS being slow (this is why the CSV mode for MCWS SetInfo was added). I have been playing with MCWS from Javascript in a browser, and I am getting excellent speed except for one caveat.

My test does 116 MCSW SetInfo (simple, not CSV) on 116 different tracks. Elapsed time: 409 ms, which is pretty impressive - close to 300 SetInfo/sec.
The caveat is that almost all the time is spent waiting for the HTTP connection to open: 303 ms for the first reply. If this is removed, we now get over 1000 SetInfo per second.
I suspect the connection opening time was really the core issue making MCWS feel slow.
From a modern browser, the HTTP connections are in keep-alive mode and get reused for many operations, so you pay the startup time only once for many transactions.
I think almost all the earlier examples were Perl-based, and I don't think Perl uses keep-alive connections without some specific work - huge difference when you do lots of small transactions.
If you need to reopen the HTTP connection for every operation, those 116 transactions take almost 40 seconds.

In the same vein, a Current/Files MPL read on the same 116 tracks takes about 54 ms, plus 305 ms of connection setup. For 430 tracks (my entire test library), 68 ms plus setup - excellent.
Pushing it to 10,000 tracks takes 1.5 seconds, roughly 1 second inside MC and 0.5 seconds to transport the 19.6 MB of data. Still excellent.

So I'm real happy with the performance  .

The only intriguing point is the session open time. My suspicion is that MC starts the 52199 web service on-demand only, and shuts it down when all connections are closed (probably to save memory footprint).
So the ~300 ms are the time to start the web server process from scratch (which sounds about reasonable).
Please note that this is not an improvement request since it's fast enough for my purposes. But it might affect the responsiveness of the various MC remotes like JRemote if they use the same type of mechanism.

[millst]

The server has to always be listening for the incoming connections so it can't be completely shut down. It's just sleeping.

Perhaps the 300ms is the establishment time for initial HTTP connection. Once it is established, it would be reused for the other requests, which is why there is no further delay. Is everything on the same network? Same machine? That seems like a long time for a hard-wired connection.

I suppose some power saving options on the NIC could come into play, too. A lot of cards go into low power mode when idle (especially wireless).

-tm

[tiberiuspv]

Everything is on a single Windows 8.1 PC, fairly high-powered (i7 4 cores 4 GHz, 32 GB DRAM, all SSD). No physical network involved.

I just repeated the measurements.
Any time I establish a fresh connection to MC port 52199, it takes about 300 ms. If the connection is already established, everything is lightning fast.

So yes, this is tied to the TCP/HTTP connection establishment with the MC20 52199 server. Since this is entirely on a single machine, there is no NIC or network in the middle.

MC must be doing something costly when it receives its first 52199 connection - perhaps the Windows equivalent to a fork+exec [I know Unix internals well, but almost nothing about Windows internals].
I don't think waking up a thread takes anywhere this much time. For example, some of the page contents are served from an IIS server on the same machine, and the connection time is in the 5 to 10 ms range.

[Hendrik]

Odd that it would take this long. I'll poke the code a bit, lets see what falls out and where the time exactly goes to.

[Hendrik]

I'm afraid I cannot reproduce this. The connection to MC takes 2-3ms for me, local or remote over the LAN. All other time is spent actually downloading the response from the server (ie. 40ms or so for my test)

[tiberiuspv]

Odd.
I just retried with the very simple test I posted in the CORS thread http://yabb.jriver.com/interact/index.php?topic=91013.0 using Chrome. Just commented out the debugger statement.
Authenticate immediately after loading the page: Authenticate time of 2.7 ms (perfect).
Wait a bit, then Authenticate: Authenticate time of 304 ms

The test page was served from the MC webserver (52199) by copying it to data\Library Server\Gizmo\Default\MCCT\CORSTest.html (to get around the CORS issue).
No options set in the test itself - load the page a first time to get the password memorized in your browser. Chrome and MC 20 running on the same high-perf PC (i7 4 cores 4 GHz, 32GB DRAM, all SSD), no network involved.

MC20.0.115, authentication turned on (MCCT/MCCT). Windows 8.1 stock & up-to-date with all standard patches, Google Chrome 43.0.2357.124 m (evergreen).

Attached are two images: slow and fast.
For the slow case, I hit Authenticate about 20 seconds after loading the page and you can see a 306 ms response time.
For the fast case, I hit Authenticate immediately after loading the page and you see a 2.5 ms response time. Note that the initial HTML load from the MC webserver is about 300ms, you can see it in the first line of the network list. That connection is still alive when the Authenticate happens, so no delay.

Then I tried again with IE (up to date IE11) and the behavior is similar but quite odd. 16 ms response time for the fast case (when the connection is already opened), 1 second (!!!) in the slow case when the TCP connection needs to be re-established. The Start time value really corresponds to the TCP establishment time. The keep-alive timeout is much longer with IE than with Chrome (> 1 minute) so you need to take a coffee break when testing... Corresponding images attached. I don't really understand the 1 second value, someone must have been asleep at the wheel - but it's repeatable.

I have not tried with Firefox.

So I'm puzzled you're not seeing the same thing. But please don't waste too much time on this, you probably have more interesting/useful things to do with your time...

[Hendrik]

I managed to kind of reproduce it, but I don't see any delay in our code. Logging indicates that from accepting the connection on the socket to delivering the result, only 2ms or so pass.
So either Winsock's accept() function is slow at accepting new connections (which we use in blocking mode on a dedicated thread), or I don't know. Unfortunately googling has not revealed anything smart yet.

[tiberiuspv]

  - I need a much bigger embarrassed emoticon...

I did some more testing after you mentioned this was a dedicated thread blocking on the socket connection, since the behavior did not make any sense.

It turns out the source of the problem is that I use the address 'localhost' which somehow Windows takes forever to resolve (probably tries WINS broadcast, LDAP, Domain controller, DNS, tea leaves, and a few other things I have no idea of...).
If I replace localhost with 127.0.0.1 in my example, voila, the 300 ms disappear totally. Down to a couple of ms. I realized the issue when I measured the delay connecting to an instance of MC on another machine, and it was much better. Hint, hint...

Some days, I really, really hate Windows for its obscure & unexpected behaviors.

So, abject apologies for making you waste your time down that rat-hole, and thanks for taking the time to look at it.

[Hendrik]

How odd, oh well. Windows be windows.

[millst]

Most of that is pretty normal. Any time you use an IP address instead of a DNS name (yes, localhost included), you're going to skip the lookup part and gain performance. You give up compatibility, since hard-coding 127.0.0.1 will probably fail on an IPv6-only system.

According to the hosts file, it looks like Windows is going through DNS instead of the hosts file now:

# localhost name resolution is handled within DNS itself.
#   127.0.0.1       localhost
#   ::1             localhost

That's Windows 7, BTW. All the Unix systems likely go to the hosts file and get the address there and avoid the DNS penalty. That seems smarter to me, not sure what the MS thinking is there. Perhaps they are trying to avoid rewriting that file when systems transition away from IPv4.

-tm

[jmone]

Wow - I'm glad I found this thread.  I've been working on a script and found some combinations are fast - and some very slow, and it is to do with both Localhost VS IP address and if Authentication is ON or OFF in MC.

Tested doing 100 MCWS/v1/GetInfo calls using COM.
- 850ms : IP Address / Authentication OFF
- 1,250ms : IP Address / Authentication ON
- 2,200ms : LocalHost / Authentication OFF
- 202,000ms : LocalHost / Authentication ON

Yup - it is 100 Times slower if using LocalHost VS an IP Addy if Authentication is ON in MC

Edit - Using these com calls

Code: [Select]

Loop, 100
{
  MC_Call = http://localhost:52199/MCWS/v1/File/GetInfo?File=%ID%?Action=Serialize
  WinHTTP := ComObjCreate("WinHTTP.WinHttpRequest.5.1")
  WinHTTP.Open("POST", MC_Call)
  WinHTTP.SetCredentials(UserName,Password,0)
  WinHTTP.Send()
  Result := WinHTTP.ResponseText
  Status := WinHTTP.Status
  ID:=ID+1
}

[Hendrik]

Luckily for you there is an easy fix.

[jmone]

Yup - but v.odd