INTERACT FORUM

Please login or register.

Login with username, password and session length
Advanced search  
Pages: [1]   Go Down

Author Topic: [RESOLVED by the A Team!] Library Size Too Big (big memory usage)  (Read 4223 times)

Arindelle

  • Citizen of the Universe
  • *****
  • Posts: 2772

Hi everybody. I’ll try to be as clear as I can …. And sorry for the wall of text but I thought more detail the better.

Basically I have JRiver commiting lots RAM -- actually using between 1go and 1.6go of memory on my main Win7 pc (server) and between 500k and 1.5go in memory on an older XP machine (client). The more zapping, the more resources – if I just listen to a full album it settles down to about 500 to 750k after about 15 minutes. Thought it was a problem with the XP client, but it seems to me its on the server PC – the Win7 PC which uses a similar amount of memory when not on the network (500K when not playing anything)



Win7 Memory


Win7 Network

Can anybody help me figure this out?  :-[ I’ve tried for over a week just about everything, basically making client and network unusable

Setup:

JRiver version 18.0.193
Client (“Upstairs”) Win XP/4go RAM/CPU dual core 3.0khz – (crappy PC speakers only)
Connected by CPL (85 Mbps^^ --  I know that's slow, but it worked fine before
Server (Downstairs) Win7 Pro 64bit/i5 2400/8go Ram (has nothing to do with it I’m sure but …optical connection to external DAC=> preamp=>active cards=>amps=>speakers w/passive crossovers by-passed)
PCs with 1go Ethernet cards (client connected direct to router, btw)

I have a medium to large size collection I guess? (2.2 tb about 5700albums/82K flac tracks located on the server PC, including a small quantity of videos which I never watch from the client). In the past I had library servers running on both machines, I’m using it only from the server now.

The “Downstairs” library (36.1mb indicated in pop-up) takes almost 5 minutes to load on the client! – 6 months ago it took maybe 30seconds, and I have only added about 20 cds since then. It virtually floods the network too. I have to explain a little bit though because I haven’t been using the client for a number of months.  I noticed a big lag on my network reusing the xp machine and JRiver -- first I noticed it was on version 17 because it was updating itself to the last build. The server was already on 18. So of course I put MC18 on the client machine. At first I was convinced it had nothing to do with MC because playback was always flawless on my stereo. And there haven’t been any posts here of a similar problem for years. So, I started checking everything … a couple of days ago, with no computers connected to the network except the server pc, and listening to music with no problems, I decided to check what processes were running and I noted that jriver was taking so much RAM without even being solicited from the network at all. I just never noticed it before, probably because my i5 and 8gbRam are a little overkill just for listening to audio and internet usage (basically the only programs I have running are JRiver and Chrome and an av)

What I’ve already checked:

-   router settings/firewal
-   antivirus and firewall settings that might interfere (Bitdefender 2011/2013 – haven’t uninstalled however)
-   client and server settings in JRiver (I messed with those so much I’m not sure if its the right config anymore) – because I use the client to tag to the server I have deactivated the automatic library sync, too no change either way. Of course, I played with the buffer setting, but there are so many new server options that I don’t know what is right anymore (looked everywhere here for an explanation btw not in wiki either. Also I noticed in DLNA Audio=>Advanced I can’t un-check volume levelling or stereo downmix to see if that would make a difference. (never even saw these options before)
-   I reverted both PCs back to version 17 – no real difference; but loading version 16 there was a significant one – was only using 136kb, compared to 1.2-1.6 gb that’s a big difference.
-   Ran deep av and malware scans on both machines just to be sure

I’m attaching system logs; for more exact details … one thing strange with those is that I upgraded both machines to 18.0.193 yesterday but it still indicates I’m using 18.0.189 in the log?!

Thanks to everybody in advance who actually took the time to read all this. Pulling my hair out, it’s probably just something stupid on my part. ?

best,

Steve
Logged

6233638

  • Regular Member
  • Citizen of the Universe
  • *****
  • Posts: 5353
Re: Memory Leak ?! – Can anyone help ?
« Reply #1 on: June 02, 2013, 10:36:49 am »

Are you using Memory Playback with large files? (Tools > Options > Audio > Play files from memory instead of disk)
Any third-party plug-ins?
Logged

Arindelle

  • Citizen of the Universe
  • *****
  • Posts: 2772
Re: Memory Leak ?! – Can anyone help ?
« Reply #2 on: June 02, 2013, 11:06:23 am »

Are you using Memory Playback with large files? (Tools > Options > Audio > Play files from memory instead of disk)
Any third-party plug-ins?

No, I am not using the memory playback option and no third party plugins (other than a skin called "LP" but I don't think a skin would cause an issue would it?)
Logged

JimH

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 72438
  • Where did I put my teeth?
Re: Memory Leak ?! – Can anyone help ?
« Reply #3 on: June 02, 2013, 01:57:50 pm »

Virus checker?

MC will use more memory for a while when it is getting cover art and building thumbnails, but it should settle down after a while.

Logged

Arindelle

  • Citizen of the Universe
  • *****
  • Posts: 2772
Re: Memory Leak ?! – Can anyone help ?
« Reply #4 on: June 03, 2013, 08:14:01 am »

Virus checker?
I looked into that first .. there "should be no av or firewall scanning in real time on my internal network. At least its not showing up (see network scrren shot). But anti-virus do wierd stuff ... trying to avoid to completely uninstall them. and Look at how much network usage with mediacenter.exe is that normal?

Quote
MC will use more memory for a while when it is getting cover art and building thumbnails, but it should settle down after a while.
These are not spikes, and even if they were, more than a 1go? My collection is not small, but it doesn't sound large either compared to what I've read here.  As I mentioned, it does settle down (to between 500 and 750k) if you are not tagging or searching in anyway but that still pretty high, non?

How much RAM are people with medium to large librairies acutally using? Even without being networked.

Has there been any significant changes (later patches version 17 and ver. 18) which could account for a signifcant memory usage? I sort of doubt it.    Maybe its the way my thumbnails are being treated  -- aiiee  :'(
Logged

rjm

  • Regular Member
  • Citizen of the Universe
  • *****
  • Posts: 2699
Re: Memory Leak ?! – Can anyone help ?
« Reply #5 on: June 03, 2013, 09:25:07 am »

I have 283,000 items in my library and MC uses about 70 MB of memory.
Logged

Arindelle

  • Citizen of the Universe
  • *****
  • Posts: 2772
Re: Memory Leak ?! – Can anyone help ?
« Reply #6 on: June 03, 2013, 09:45:11 am »

I have 283,000 items in my library and MC uses about 70 MB of memory.
seems reasonble .. so with my 82,000 items there is as I thought a big leak somewhere
Logged

Matt

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 42372
  • Shoes gone again!
Re: Memory Leak ?! – Can anyone help ?
« Reply #7 on: June 03, 2013, 01:51:52 pm »

Try a simple view.  Genre / Artist / Album, for example.

Complicated expressions, especially ones that use regex or make OS calls can be dramatically slower.  The same applies to calculated fields using these expressions.
Logged
Matt Ashland, JRiver Media Center

Arindelle

  • Citizen of the Universe
  • *****
  • Posts: 2772
Re: Memory Leak ?! – Can anyone help ?
« Reply #8 on: June 04, 2013, 09:25:46 am »

thanks for the reply

Quote from: Matt
Try a simple view.  Genre / Artist / Album, for example.

I created a simple view like that -- no effect; But maybe I misunderstood. You mean create a new library just with that one view??

Quote
Complicated expressions, especially ones that use regex or make OS calls can be dramatically slower.  The same applies to calculated fields using these expressions

Ouside of smartlists only one view has what "might" be considered complicated (compared to other people's it is NOT) but does use regex. No calls to the OS, unless you are talking about tags being written to files which is activated (important for me, that).  All of my views do have 4 or five calculated fields, however. But I have bee using them for years .. simple ones like this - no concantanation (think that's the word :) ) or anything combining or parsing multiple fields. ... just individual fields like this  =>
Code: [Select]
[genre]&datatype=[list]
Logged

Matt

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 42372
  • Shoes gone again!
Re: Memory Leak ?! – Can anyone help ?
« Reply #9 on: June 04, 2013, 09:43:08 am »

If you'll mail a library backup to matt (at) jriver [dot] com, I'll load it and see how it performs here.

Maybe there's something simple we could improve.  Or maybe it'll be fast here, and it'll point to a machine issue.

Regardless, we'll probably learn something.

Thanks.
Logged
Matt Ashland, JRiver Media Center

Matt

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 42372
  • Shoes gone again!
Re: Memory Leak ?! – Can anyone help ?
« Reply #10 on: June 04, 2013, 06:16:12 pm »

Thanks for the library.  It might be a couple days before I get a chance to look at it.

But your email made me wonder if you're loading the library from the server using DLNA instead of Library Server?  This might explain the performance difference.

Try connecting directly to the server using the Access Key (or IP address) on the client.  Any difference?
Logged
Matt Ashland, JRiver Media Center

Matt

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 42372
  • Shoes gone again!
Re: Memory Leak ?! – Can anyone help ?
« Reply #11 on: June 05, 2013, 09:37:01 am »

An interesting thing in the library backup is that there are about 5000 playlists under the 'Car Radio' group named:
01. Alt/Indie Not played (1)
01. Alt/Indie Not played (2)
01. Alt/Indie Not played (3)
...
01. Alt/Indie Not played (4916)

Each one has about 6000 tracks.

So the total number of tracks in playlists is 5000 * 6000 = 30 million tracks.

Since each track takes about 4 bytes to store in a playlist, that's 120 MB just for that playlist group.  And sure enough, your playlistx.jmd file in the library is 113MB.  This will slow down connecting to the library, loading, sending changes, etc.

Could you help us figure out how these playlists are getting created?  Is it possible it happens each time you play that Car Radio button?
Logged
Matt Ashland, JRiver Media Center

Matt

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 42372
  • Shoes gone again!
Re: Library Size Too Big
« Reply #12 on: June 05, 2013, 10:34:19 am »

Next build will auto-clean these playlists:
Changed: Car Radio automatically cleans up duplicate static playlists that used the same index.

However, I'd still like to understand how they got there, because it seems like it must be a bug.
Logged
Matt Ashland, JRiver Media Center

Arindelle

  • Citizen of the Universe
  • *****
  • Posts: 2772
Re: Library Size Too Big
« Reply #13 on: June 05, 2013, 10:45:44 am »

OMG -- that's incredible. 30 million?? And that playlist is actually a smartlist composed of a number of calculated fields too. I'll look into it right away when I get home. Seems promising Matt.  Thanks :)
Logged

Arindelle

  • Citizen of the Universe
  • *****
  • Posts: 2772
Re: Library Size Too Big
« Reply #14 on: June 06, 2013, 07:42:22 am »

Hey you guys are GREAT!!! Matt, you found the problem.  I erased manually all the Car Radio generated playlists (there were thousands, took awhile :P) and ==> 1.6 gigabytes in memory went down to 55K!! Under load – theatre view 3-d generated display views etc. couldn’t even get to 250 K on the server.   Client loads practically instantly, and library size decreased from 356Gb to around 6!!!!  Some lag loading 24bit and high res tracks on the network - but thats to be expected from cpl I think.

THANK YOU GUYS :D This also explains why when I reloaded version 16 it had a normal low memory usage --- No Car Radio buttons at that time.

Quote
Could you help us figure out how these playlists are getting created?  Is it possible it happens each time you play that Car Radio button?

Well I created a smart list using calculated list field “Alternative/Indie” (from Genres) which is also made up of multiple genres for this particular list, and filtered by tracks not listened to 99 years^^. When setting up the “Radio Button” I was playing this smart list and chose to enter it as a static playlist.  I am guessing here, but seems logical that every time I’d listen to a track, the smartlist would update itself, regenerating a new playlist.

Unfortunately, I can’t provoke this happening again – so can’t debug on my end, sorry.  I’m thinking you guys might have tweaked something in version 195 which loaded this morning??? There is only one playlist showing now for each button. For info, the “smart” part of the list is no longer functioning, meaning that it is no longer removing the “played” tracks ….  A static playlist like the description says.  [By the way, when creating a button via the Play Doctor option) this creates about three days of tracks which does increase the library size on loading the program (it spits out some strange choices though Sarah Vaughn through Play Doctor and a Nirvana track and some James Brown?!  ;D )] If there is anything I can do to help debug though? let me know. (maybe restoring from an older backup??)

Quote
But your email made me wonder if you're loading the library from the server using DLNA instead of Library Server?  This might explain the performance difference.
Having this problem, I looked just about everywherre to fingure out the problem. And you know their are options now that I have no clue about. I don't see how to force using the library and not DNA.

here's what showing up from "the client", I don't know how to tell if I'm using the library server or the DLNA server?  I really would like to know the optimal configuration though.


so could anybody tell me if I should change my config? (or should I start another post?) ....
I could easily do 2 zones if 2 config are needed (ex. when using JRemote)



A


B


C


Anyways, thanks again for all your help Jim and Matt and too al who help on this forum.

Logged

syn-ack

  • Junior Woodchuck
  • **
  • Posts: 82
Re: [RESOLVED by the A Team!] Library Size Too Big (big memory usage)
« Reply #15 on: June 06, 2013, 07:49:47 am »

If you have a problem, if no one else can help, and if you can find them, maybe you can hire the A-Team.
Logged

InflatableMouse

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 3978
Re: [RESOLVED by the A Team!] Library Size Too Big (big memory usage)
« Reply #16 on: June 06, 2013, 08:12:01 am »

If you have a problem, if no one else can help, and if you can find them, maybe you can hire the A-Team.

You don't have to find them, they will find you;D
Logged

Matt

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 42372
  • Shoes gone again!
Re: Library Size Too Big
« Reply #17 on: June 06, 2013, 08:28:07 am »

I erased manually all the Car Radio generated playlists (there were thousands, took awhile :P)

Now that you're done, you can multi-select and delete in the tree.  Or delete the 'Car Radio' group (although you would lose any good presets doing this).  Or wait for the new build which cleans duplicates automatically.


Quote
Unfortunately, I can’t provoke this happening again – so can’t debug on my end, sorry.

Hopefully it was a bug that's since been fixed.  Or maybe your cat sat on the keyboard a while back :P


Quote
Having this problem, I looked just about everywherre to fingure out the problem. And you know their are options now that I have no clue about. I don't see how to force using the library and not DNA.

On the client, under Playing Now > Playing From, pick the library you're using.  On the right, it should state if it's a Library Server connection or a DLNA connection.
Logged
Matt Ashland, JRiver Media Center
Pages: [1]   Go Up