Topic: Memory Leak? (Read 19700 times)

jmone · « **on:** December 13, 2013, 01:45:06 am »

The original thread is here http://yabb.jriver.com/interact/index.php?topic=84814.0 and I've now rebuilt my PC twice but still over a few days all memory gets consumed and the PC gets sluggish. In the latest test I removed the WDM Driver (hence why I'm posting here as this driver is not "public") but as you can see MC slowly grows in memory usage (as does other things) from an initial sub 50K till 20x this.

Suggestions?

Yours
"Driven Nuts"

Edit here is a Pic after a boot, loading stuff up as normal and even while ripping a BD in MC.

Matt · « **Reply #1 on:** December 13, 2013, 08:09:45 am »

Memory is allowed to grow for thumbnail caching, memory playback, etc. So 800MB of usage doesn't necessarily mean there's a leak.

If you minimize MC, does it reclaim some?

Can you find an action that causes it to always grow?

What if you turn off Media Network and Auto-Import and let it run?

What if you don't play?

Sorry for all the questions, but basically we need to figure out what one tool or action might cause the problem you're seeing.

jmone · « **Reply #2 on:** December 13, 2013, 01:28:14 pm »

I don't know if this helps, but last night I rebooted at just left MC Server running but did nothing with it.
- Process "Media Center 19.exe" had grown in 150K Commit / 115K Working Set in about 7hours
- This was reclaimed when I closed MC Server

The overall mem use of the PC had grown from 17% to 27% and after I closed MC Server it dropped to 22%. Almost all available memory not "in Use" had been allocated (to what I don't know).

I'll Try to answer some of the questions over the weekend, and I also have a couple of days away so I can let it run undisturbed in any state.

Thanks
Nathan

jmone · « **Reply #3 on:** December 13, 2013, 04:25:59 pm »

- Tested on building a couple of thousand Audio Thumnails. Memory use grew to 122(c)/66(ws)MB during this then dropped back to 100(c)/20(ws) a short time after finishing. After just sitting doing nothing it is now 134(c)/62(ws)MB

- Auto Import is off (always is).

- About to test turning off Media Network

glynor · « **Reply #4 on:** December 13, 2013, 04:33:49 pm »

I heard today from a tech at the office that Windows 8.1 has some kind of default backup process that runs regularly that causes all sorts of memory and performance trouble.

Do you see any transient processes in task manager that reference backup and Microsoft, that appear/ disappear regularly?

jmone · « **Reply #5 on:** December 13, 2013, 04:55:05 pm »

I don't (think so)... I do use WHS to backup this box (and all others every night). What I don't get is that even if I add up all the Memory being used by the Processes in Resource Monitor it is far less than what is being shown as the Total in use (eg have a look at the Pic in the OP).

I'm not seeing this on my other Win8.1 box however, just this one.

Thanks for all the suggestions by the way!

jmone · « **Reply #6 on:** December 13, 2013, 05:02:06 pm »

FYI - I also can run MS's Sysinternals RamMap though I'm not sure if it helps be diagnose this any better. The only thing that looks big under File Summary (that does not appear in the Process Tab" is the Windows.EDB (used by the search indexer) that is 1.6GB of Standby Memory. I'll see if that one grows.

glynor · « **Reply #7 on:** December 13, 2013, 05:08:49 pm »

Just a question...

Have you done a Memtest86+ run or five on that bad boy anytime recently?

jmone · « **Reply #8 on:** December 13, 2013, 05:31:03 pm »

Mmm good idea - I give that a crack overnight as it is OC'ed (inc mem) by ASUS Tune.

Edit - got through one and a bit tests OK (had to do some work then). Will run longer later.

jmone · « **Reply #9 on:** December 14, 2013, 01:27:27 am »

Quote from: Matt on December 13, 2013, 08:09:45 am

What if you turn off Media Network .....

Early Days but with Media Network OFF - Memory has stayed around 53(c)/14(ws) MB for the last six hours. It grows if I play items but then shrinks immediately I stop it.

jmone · « **Reply #10 on:** December 14, 2013, 05:22:51 pm »

So after almost 1 day - MC memory looks stable. Overall memory use is up about 5% from a clean boot. I'll now turn back on Media Network and see what happens.

jmone · « **Reply #11 on:** December 15, 2013, 12:33:55 am »

So 6hrs later with Media Network On - MC useage is up 100MB but I've now used an additional 1TB (6%) of my Physical Memory to something but it is not showing what in the Resource Monitor

jmone · « **Reply #12 on:** December 15, 2013, 03:48:00 am »

I did another test:
- Reboot PC (MC Server running) = Memory use was: System 2,221 GB in use / MC was 36(c) / 13(ws) MB
- Started Gismo on a Tablet and played a Video (BD Rip) = Memory use was: System 3,381 GB in use / MC was 511(c) / 310(ws) MB
... and over 15 minutes of doing nothing else = Memory use was: System 3,640 GB in use / MC was 513(c) / 312(ws) MB
... the rise in system memory use seemed to co-incide with high CPU activity (eg the bursts when MC was transcoding the next bit of the video).

So by playing one video in Gismo my System it used 1.2GB of which 500(c) / 300(ws) MB was reported as MC

I then tried various other activities like stopping playback in Gismo, Playing other Videos even then started MC and did a video convert but the max I got was = Memory use was: System 3,922 GB in use / MC was 1,338(c) / 527(ws) MB

After closing MC/MC Server = Memory use was: System 3,160 GB in use. So I've lost 1GB since startup in 30mins.

FYI - I tried a few more close/open MC start playback in Gizmo Cycles but I did not see any more 1GB lumps disappear.

jmone · « **Reply #13 on:** December 15, 2013, 03:49:45 am »

I'm away for a couple of days - so if there is any longer running test/environment you want me to do, please let me know!

Edit - Overnight, I'll reboot the PC without MC running at all, load the usual Mail and Web and see what happens.

Arindelle · « **Reply #14 on:** December 15, 2013, 09:18:30 am »

This is probably not the case, but last spring I think it was, I had a huge memory problem. What I did to provoke was sticking a smartlist as a fixed playlist in one of my radio buttons.

Quote

Matt -- An interesting thing in the library backup is that there are about 5000 playlists under the 'Car Radio' group named:
01. Alt/Indie Not played (1)
01. Alt/Indie Not played (2)
01. Alt/Indie Not played (3)
...
01. Alt/Indie Not played (4916)

Each one has about 6000 tracks.

So the total number of tracks in playlists is 5000 * 6000 = 30 million tracks.

He said that maybe my cat had sat on my keyboard

Can't hurt to check ? If something is looping somewhere in a smartlist or expression -- oh well probably dumb idea

jmone · « **Reply #15 on:** December 15, 2013, 01:45:56 pm »

Thanks for the idea Arindelle.

Quote from: jmone on December 15, 2013, 03:49:45 am

I'm away for a couple of days - so if there is any longer running test/environment you want me to do, please let me know!

Edit - Overnight, I'll reboot the PC without MC running at all, load the usual Mail and Web and see what happens.

Mem usage grew from 2.8GB to 3.6GB in 8 hours (pics attached) without MC Running at all.... I'll leave it like this for the next couple of days and see how it goes.

InflatableMouse · « **Reply #16 on:** December 15, 2013, 02:33:02 pm »

Did you keep a list of what you changed after a clean install?

I've never seen such a persistent issue before, I'm thinking 3 things:
1. Windows 8.1 borks up on your pc (bug in windows). Then its likely you're not the only one. Search Technet forums and other Windows forums.
2. You repeatedly change/install something after a clean install that is causing this
3. You have some persisting virus/malware

The only way I believe to tackle it is to be pragmatic and systematic. Keep a written log, make a checklist, take your time. No shortcuts and no hasty decisions.

1. The issue might happen on a clean install. Don't update anything, not even drivers. Check it out. If nothing happens after a day or two, proceed to 2.
2. Make a list of things you need to change and install, drivers, software, system configuratation. Things you know you've done before as well. Do 1 thing at a time, and only really 1 thing. Even something seemingly trivial as page file settings. Install a single driver or make a single system configuration change and see what happens (in your case, wait at least 1 day). Mark it on your list and move to number 2. This is time consuming, but I think its the only way to find out which step is introducing your issue.

Number 3 I think is unlikely, you've been scanning so unless its something obscure and not detected, you won't find out unless you know how to actually find it. I suppose if this is the case, it would reveal itself during step 2.

Another thing I want to bring to your attention is what I've suggested before. Let it crash and make sure you get a memory dump (>300MB page file on C: and kernel dump or small memory dump enabled). This dump can be analyzed online at osronline.com. It might show which module is responsible for eating your memory.

If you think this is going to be too time consuming, consider how much time you've already lost. You could take a gamble and reinstall again, or go for Windows 8.0 instead of 8.1 and take your chances with that. At this point though, I think you're only going to be wasting more time, I'm pretty sure its in something you're doing or installing.

jmone · « **Reply #17 on:** December 16, 2013, 08:22:12 pm »

So after a couple of days without MC running the Memory Usage has not really changed. Looks like my problem is with MC & Media Network on that is causing the leaking.

Where to from here?

THanks
Nathan

Matt · « **Reply #18 on:** December 16, 2013, 08:56:58 pm »

Media Network does several things that use memory to help performance (file conversions, browse tree, etc.). They could cause memory to grow, but not technically be a leak.

It would help if you could narrow down the network action that causes the trouble. My guess is that video conversion is related, but network chatter (SSDP, etc.) could also be the explanation.

What if you leave the network features enabled, but don't play from it? Or what if you play a bunch of videos right away?

jmone · « **Reply #19 on:** December 16, 2013, 09:17:48 pm »

Thanks Matt - My tests were not the most scientific but,
- The 3rd post in this thread showed what happened when the PC was rebooted and no progs run but MC Server and showed memory disappearing even without any playback (that I know of anyway).
- The 10th/11th Post in this thread showed little change when MC running but Media Netwrok Unchecked
- The 12th Post showed just turning on Media Network and I lost 1TB of memory but it was not allocated to MC.
- The 13th Post showed when agressivly using the Media Network I lost 1TB of memory (but it was not allocated to MC) in 30mins
- The Final Post showed that with MC not running the PC's memory use did not really change after a few days (it went up a bit)

jmone · « **Reply #20 on:** December 16, 2013, 09:18:23 pm »

I've got a couple of days to test any particular combo(s) you want me to try - I just need some ideas.
THanks
Nathan

Matt · « **Reply #21 on:** December 16, 2013, 09:38:05 pm »

Quote from: jmone on December 16, 2013, 09:17:48 pm

- The 12th Post showed just turning on Media Network and I lost 1TB of memory but it was not allocated to MC.
- The 13th Post showed when agressivly using the Media Network I lost 1TB of memory (but it was not allocated to MC) in 30mins

Is it possible you're not looking at the right thing to decide about free system memory?

The OS can use memory for cache. The cache may appear as used memory, but it will be released when the memory is needed for something else.

jmone · « **Reply #22 on:** December 16, 2013, 10:08:09 pm »

Any pointers would help. All I know is after 1 week all the memory is gone and the Win UI/Mouse starts slowing down. I've been trying to track down what causes the memory growth and from the shorter duration tests above it seems to only occur when MC/Media Network is running.

Matt · « **Reply #23 on:** December 16, 2013, 10:15:40 pm »

Is the extra memory usage concentrated in "svchost.exe (netsvcs)"?

jmone · « **Reply #24 on:** December 16, 2013, 10:18:45 pm »

Nope - it memory usage does not appear to be allocated to any process at all - The reported Total Memory in use just keeps growing.

jmone · « **Reply #25 on:** December 16, 2013, 10:21:11 pm »

eg - If you have a look at the two pics in the first post, you will see that after 4 days, Win is reporting that 70% of Physical Memory is used (and Windows is now sluggish) but the total of the the memory used by the Processes is no where near this.

InflatableMouse · « **Reply #26 on:** December 17, 2013, 12:46:59 am »

Quote from: jmone on December 15, 2013, 01:45:56 pm

Mem usage grew from 2.8GB to 3.6GB in 8 hours (pics attached) without MC Running at all....

Quote from: jmone on December 16, 2013, 08:22:12 pm

So after a couple of days without MC running the Memory Usage has not really changed. Looks like my problem is with MC & Media Network on that is causing the leaking.

Sorry, I'm lost. I replied under the assumption your memory was growing wthout MC running at all ...

jmone · « **Reply #27 on:** December 17, 2013, 01:28:37 am »

Sorry - This was the only example where the memory changed without MC running and it was relatively small (800K) from what I was seeing with MC running. So I repeated over the last three days without MC running and it did not really change. With MC running I can lose 1GB in 30mins, or many GB's over a few days.

jmone · « **Reply #28 on:** December 17, 2013, 01:34:25 am »

Thanks said - I need a better way of isolating and recording what happens as I use the PC most days for work (but we were all away for the last couple of days).

jmone · « **Reply #29 on:** December 17, 2013, 05:28:40 pm »

Here is a 3min (100mb) video hopefully showing what I'm poorly explaining.
1) - Clean Boot with no foreground apps running and just MC Server = 2.2GB of Physical Memory used
2) - 12 Mins of Video Playback from Gizmo (I speed this up 10x so don't worry)
3) - Closed MC Server so it releases all the Mem it used = 2.9GB of Physical Memory used.

So 700MB has gone in 12minutes. After a week nothing is left!

https://dl.dropboxusercontent.com/u/59011278/Produce.zip

Thanks
Nathan

jmone · « **Reply #30 on:** December 17, 2013, 11:56:48 pm »

FYI - reading up on MSDN I've been using PerfMon to track what grows and it seems to be "Pool Nonpaged Bytems" which apparantly indicates a "Kernal Mode Memory Leak" but now I need PoolMon to first isolate "which kernel-mode driver or component is responsible for the leak"... Of course this could all be a read herring as what do I know about this stuff!

InflatableMouse · « **Reply #31 on:** December 18, 2013, 12:07:11 am »

Quote from: jmone on December 17, 2013, 11:56:48 pm

Of course this could all be a read herring as what do I know about this stuff!

You may not know much now but you'll be the expert everyone turns to to solve their memory leaks once you're done!

jmone · « **Reply #32 on:** December 18, 2013, 12:49:04 am »

Not Funny!

It seems the tags I've noticed growing are:

AleE - tcpip.sys - ALE endpoint context
Thre - nt!ps - Thread objects
File - File objects

I'll have to do a clean test tomorrow (friends are over for dinner in 10mins) but I'm leaning towards the AleE as it has doubled in size and since it appears to be network related would explain why I see it with MC streaming media and why Outlook gets slow (I use IMAP so it is always polling).

Just guessing but

jmone · « **Reply #33 on:** December 18, 2013, 01:07:49 am »

Interesting. When closing and restarting MC/MC Server all of three of the above tags reduce but AleE drops to 1/40th of the size, Thre to 1/3rd and File by only 20%.

InflatableMouse · « **Reply #34 on:** December 18, 2013, 01:09:08 am »

I think those are just buffering and closing the application with the open connections clears them?

jmone · « **Reply #35 on:** December 18, 2013, 01:52:34 pm »

So I rebooted and did not run an foreground apps (MC Server was running) and according to Poolmon Commit Memory grew from 2.8GB to 3.6 in 8 hours. The following are the Tags that grew:
- Thre : 4,563,066 --> 152,554,816 bytes and is now the biggest user of memory at 145MB
- AleE : 1,020,768 --> 87,031,392 bytes
- File : 5,169,424 --> 61,472,048 bytes

Also, AfdE, UdpA, FMsl, Ntfx, IPss, MmCa (see pics)

Unless the are suggestions today, I'll do a clean boot then play something from Gizmo and see what the counters do. Unless there are better suggestions?

InflatableMouse · « **Reply #36 on:** December 18, 2013, 01:58:14 pm »

I have no idea what I'm looking at, sorry.

jmone · « **Reply #37 on:** December 18, 2013, 02:03:42 pm »

I don't either! - http://msdn.microsoft.com/en-us/library/windows/hardware/ff560135(v=vs.85).aspx

jmone · « **Reply #38 on:** December 19, 2013, 12:42:58 am »

I think(?!?!?!) I have it sorted..... but time will tell!

Drum roll.... it is the supposedly high quality "Intel PRO/1000 PT Dual Port Server Adapter" card's driver under Windows 8.1 that leaks as you push data through it. So it shows up quickly when using MC Server.

It was originally suggested checking these drivers when I first posted my problems Oct, and I had tried both the ones that Windows Auto Installs (an early plain driver from Intel) + the latest Teaming from Intel directly a couple of times. When using the latest Intel drivers I even tried turning on/off the teaming feature I use and ended up with just the stock drivers on a single port. I've now disabled them in Windows (card still installed) and using the one on the Mobo.

I'm not sure that Intel even support this card on Windows 8.1 as while you can download the drivers from http://www.intel.com/support/network/sb/cs-006120.htm and install them just fine, this other page seems to indicated that it no longer has "Full Support" - http://www.intel.com/support/network/sb/CS-034443.htm

Back to square one on what NIC to get.... A couple of years ago Glynor (I try to follow his HW tips) suggested this card to help spread the demand when both playing content from this PC + at the same time syncing to the WHS box (it was causing playback to stutter). Looks like it is time to move on and find a new card....

Thanks for all your suggestions and patience...
Nathan

InflatableMouse · « **Reply #39 on:** December 19, 2013, 12:55:25 am »

File a bug report with Microsoft that their stock drivers have a memory leak?

jmone · « **Reply #40 on:** December 19, 2013, 01:00:40 am »

Good idea - but according to Device Mgr, the "stock" ones installed during the Win8.1 install are from Intel and dated 14/10/2011 and useless to be as they don't support teaming anyway.

I'm soooo over it I'm ready to just chuck the thing in the bin and get something else.

InflatableMouse · « **Reply #41 on:** December 19, 2013, 01:18:28 am »

I understand your frustration but don't do anything hasty, you'll regret it later. There are no better network cards than Intel's. This problem is just .... bad luck.

Try an older version, for Win7 or 8.0. Support or no support doesn't really matter unless you plan on calling Intel and talking to their support staff. You just want it to work.

Now that you know how to see the problem it should be relatively easy (considering what you've been through already) to try a few different driver versions.

Also, you could try posting on the Intel forums, there are some clever people there which are generally quite willing to help.

jmone · « **Reply #42 on:** December 19, 2013, 02:13:08 am »

You talked me into it (what do I have to lose but more time!) - https://communities.intel.com/thread/47388

JimH · « **Reply #43 on:** December 19, 2013, 07:12:44 am »

A few years ago, we had a problem with an Ethernet driver that was solved by disabling Jumbo Frames. That, and a few other network device problems, might be found in the "Weird Problems" thread.

Hendrik · « **Reply #44 on:** December 19, 2013, 07:18:38 am »

I use the Intel PROset drivers as well, but for a different card (a I217-V onboard chip), so its either a card-specific thing, or I'm just lucky. Never seen anything like this.

Or i'm just oblivious to the issue since its a desktop PC and i usually turn it off in the evening.

InflatableMouse · « **Reply #45 on:** December 19, 2013, 09:12:43 am »

He's also teaming, not sure if the issue is occurring with it disabled?

Hendrik · « **Reply #46 on:** December 19, 2013, 10:02:01 am »

He said it does.

glynor · « **Reply #47 on:** December 19, 2013, 10:05:16 am »

I'm running them as well, and haven't seen any serious issues with 8.1. I have an onboard Intel NIC in that box, and a single-port standalone card in my HTPC.

My firewall runs the same exact PT card he has, but it doesn't run Windows, of course.

I read the problems elsewhere, and it does seem to be real, but I think it depends on a variety of conditions.

jmone · « **Reply #48 on:** December 19, 2013, 01:29:39 pm »

Well it is a good morning! PC's memory is right where it should be

so it seems to confirm it is the Intel NIC (and thanks for the posts

- I've only hit issues on Win 8 64 bit - (no issues on Win 7 64 Bit)
- I've tried it with teaming On/Off (and even just one port) using the Intel PROWin64 Drivers
- I've tried just with one port with the old drivers
- The box runs 24x7
- I run a second 1000 PT run in my WHS 2011 just fine (not teamed - just one port but it is in standby 22hrs per day)

We are off to Whistler for a White Christmas and NY for NYE to see the ball drop (kids are growing up so there may not be too many opportunities for family holidays).

...anyway, I'll be away for a couple of weeks so will work more methodically through the options to see if I can narrow it down when back. At least I'll be able to remote into the box while travelling and it will not have locked up!

JimH · « **Reply #49 on:** December 19, 2013, 01:35:48 pm »

Enjoy your trip. Sounds like fun.

INTERACT FORUM

Author Topic: Memory Leak? (Read 19700 times)