INTERACT FORUM

Please login or register.

Login with username, password and session length
Advanced search  
Pages: [1]   Go Down

Author Topic: Resource Drain with 1.4 million file library [Solved -- virus checker problem]  (Read 2287 times)

Zootsuit

  • World Citizen
  • ***
  • Posts: 139
  • too cool for school

I know this topic has come up before, and I've tried to research anything helpful in other threads, but the closest I came was a posting by Laserwolf in the bug-reporting thread for 18, so at Jim H's suggestion, I'm starting a new thread..

In previous builds, I've noticed some drain on resources, but nothing that disrupted my use of MC. I installed 18.0.78 about a week ago, and immediately began experiencing a major drain on the memory every time MCwas open. It began immediately, decreased somewhat when I initially minimized it, but then began ramping up again, to the point where both the computer and MC became unusable.That required  either that I terminate the program, or in several cases, the program froze and I had to reboot.

My primary computer has 6 gigs of memory, and taskmaster would routinely showing 1 g or more of usage by MC. There was no corresponding spike in processor utilization. The problem was particularly acute when I was working in Chrome with MC in the background, but it wasn't limited to that. I haven't experienced a similar problem on the two clients that connect to the MC server; quite often, the server remains open for lengthy periods of time, when I'm traveling or working, but the problem occurs even before I open the server..

I've tried running MC headless, but nothing changes. I took note of the suggestions about antivirus conflicts, and ultimately removed Avast, or as much of it as I could (with Revo Uninstaller). Without Avast, MC's memory usage remained between 250-325 gs, but I couldn't leave the computer naked (obviously), so tried installing MSE and began experiencing the same issues.

My system info follows. As you can see, there are lots of external drives, although I'm not sure that's relevant. Also, I have a very large library scattered amongst the drives, something on the order of 500 K files totaling 10 TB - lots of duplicates. I've excluded information on the external drives - too lengthy - but can provide that if relevant:

Media Center
    Install Path: C:\Program Files (x86)\J River\Media Center 18\
    Interface Plugins: TiVo Server (running)
    JRMark: never run
    Library: 1386929 files, 180 MB in library, 6.2 GB in thumbnails
    Resources: 198 MB memory, 320 handles
    Version: 18.0.78.0 Registered
Bluetooth Radios
    Broadcom Bluetooth USB with AMP (driver 6.3.0.2500)
    Microsoft Bluetooth Enumerator (driver 6.1.6002.18457)
Bluetooth Virtual Devices
    Bluetooth AV (driver 6.3.0.2400)
    Bluetooth Hands-free (driver 6.3.0.2400)
    Bluetooth Headset (driver 6.3.0.2400)
    Bluetooth L2CAP Interface (driver 6.3.0.2200)
Computer
    ACPI x64-based PC (driver 6.0.6002.18005)...
Display adapters
    NVIDIA GeForce 9500 GS (driver 7.15.11.7556)
DVD/CD-ROM drives
    TSSTcorp CDDVDW TS-H653Q (driver 6.0.6002.18005)...
Human Interface Devices
    Bluetooth Remote Control (driver 6.3.0.2400)...
Logitech USB Wheel Mouse (driver 6.1.6002.18005)
    Microsoft eHome Infrared Transceiver (driver 6.1.6002.18005)...
IEEE 1394 Bus host controllers
    AGERE OHCI Compliant IEEE 1394 Host Controller (driver 6.0.6002.18005)
Imaging devices
    EPSON Perfection V33/V330 (driver 3.9.1.3)
Keyboards
    HID Keyboard Device (driver 6.0.6002.18005)
    HID Keyboard Device (driver 6.0.6002.18005)
    Microsoft eHome MCIR 109 Keyboard (driver 6.0.6002.18005)
    Microsoft eHome MCIR Keyboard (driver 6.0.6002.18005)
    Microsoft eHome Remote Control Keyboard keys (driver 6.0.6002.18005)
Memory
    Free Physical Memory: 3.5 GB
    Total Memory: 6.0 GB
Mice and other pointing devices
    HID-compliant mouse (driver 6.0.6001.18000)
    HID-compliant mouse (driver 6.0.6001.18000)
    Logitech USB Wheel Mouse (driver 6.0.6001.18000)
Modems
    PCI Soft Data Fax Modem with SmartCP (driver 7.74.0.0)
Monitors
    Generic Non-PnP Monitor (driver 6.0.6001.18000)
Network adapters
    802.11n Wireless PCI Express Card LAN Adapter (driver 2.0.4.0)
    Bluetooth Device (Personal Area Network) (driver 6.0.6001.18000)
    Realtek RTL8168C(P)/8111C(P) Family PCI-E Gigabit Ethernet NIC (NDIS 6.0) (driver 6.203.214.2008)
Network Infrastructure Devices
    Microsoft Wireless Router Module (driver 1.0.0.3)
Operating System
    Microsoft Windows Vista 64-bit
Portable Devices
    USB CF Reader    (driver 6.0.6002.18112)
    USB MS Reader    (driver 6.0.6002.18112)
    USB SD Reader    (driver 6.0.6002.18112)
    USB xD/SM Reader (driver 6.0.6002.18112)
Printers
    Canon Inkjet PIXMA iP4000 (driver 6.0.6000.16386)
Processors
    Intel(R) Core(TM)2 Quad CPU    Q6700  @ 2.66GHz (driver 6.0.6001.18000)
    Intel(R) Core(TM)2 Quad CPU    Q6700  @ 2.66GHz (driver 6.0.6001.18000)
    Intel(R) Core(TM)2 Quad CPU    Q6700  @ 2.66GHz (driver 6.0.6001.18000)
    Intel(R) Core(TM)2 Quad CPU    Q6700  @ 2.66GHz (driver 6.0.6001.18000)
Sound, video and game controllers
    Bluetooth Hands-free Audio (driver 6.3.0.2400)
    Hauppauge WinTV HVR-1800 (Model 78xxx, Combo ATSC/QAM) (driver 1.27.26077.0)
    Realtek High Definition Audio (driver 6.0.1.5591)
Storage controllers
    Intel(R) ICH8R/ICH9R SATA RAID Controller (driver 7.6.0.1011)
    Microsoft iSCSI Initiator (driver 6.0.6002.18005)
Logged

Matt

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 42379
  • Shoes gone again!
Re: Resource Drain in MC 18
« Reply #1 on: December 11, 2012, 01:14:02 pm »

    Library: 1386929 files, 180 MB in library, 6.2 GB in thumbnails

With 1.4 million files in the library, I think you're going to have to expect heavy memory usage.  Other programs would die long before this.

If there are specific actions that are slow, we might be able to work to improve them.

Logged
Matt Ashland, JRiver Media Center

Zootsuit

  • World Citizen
  • ***
  • Posts: 139
  • too cool for school
Re: Resource Drain in MC 18 with 1.4 million file library
« Reply #2 on: December 11, 2012, 01:43:56 pm »

That's odd - MC reported there are @520 K files in library, including duplicates. Never noticed the discrepancy.  :o

But to the point - why would this behavior manifest itself only when an antivirus program is running? I suppose I could dump MSE and try AVG or Pandora.

Logged

Matt

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 42379
  • Shoes gone again!
Re: Resource Drain in MC 18 with 1.4 million file library
« Reply #3 on: December 11, 2012, 01:49:23 pm »

@File count
Create the smartlist ~d=a and take a look.  What are all the extra entries?  Adding the expression column FileDBLocation() may help.  If something isn't getting cleaned up properly, it'd be good for us to find and fix it.


@Anti-virus
They hook the process and Windows API, so can, at least theoretically, screw just about anything up.

Make sure you exclude your library folder from any real-time scanning.  If the virus checker wants to virus check the *.jmd library files every time they're touched, it's going to be bad.
Logged
Matt Ashland, JRiver Media Center

Arcturus

  • World Citizen
  • ***
  • Posts: 106
Re: Resource Drain in MC 18 with 1.4 million file library
« Reply #4 on: December 11, 2012, 04:34:39 pm »

It's easy in MSE to simply have it ignore all of JRiver's processes. So no need to dump anything. IF that really is causing you any issue at all its trivial to fix.

Logged

HTPC4ME

  • Regular Member
  • Citizen of the Universe
  • *****
  • Posts: 2760
  • LIFE IS A RADIO... CRANK IT UP TO 11!
Re: Resource Drain in MC 18 with 1.4 million file library
« Reply #5 on: December 11, 2012, 05:21:36 pm »

zoot, i have 780,000 in mine never had problems with virus checkers im using MSE. But last few months I have had 2-3 minute lock ups when searching audio files (version 17), and when clicking on Year tag... basically making jriver server and all clients useless. last night i installed 18, and rebuilding whole library so far at 764,000 files when i goto Audio and do a search files are showing up instantly, and when i click on year that is allowing me to click on it without any lockups... My scenario i'm leaning towards was a corrupt library. i did try restores BACK to where i knew lockups were not happening and that did not fix the problem.

I've always had MANY runtime errors with jriver when doing a full import throughout the years, but i must admit on 18 and being this is my first library build on it im quite impressed only 1 runtime error so far, but i've yet to hit thumbnail building, it will be interesting to see how many runtime/BSOD i get once thumbnail building kicks in... Typically for 780,000 files i get 8-10 Runtime/Lockups/BSOD's when building thumbs.

also just any fyi. i have 6 pc's that all use essentials and connect to server... no files on clients. and have 16GB ram

Quote
It's easy in MSE to simply have it ignore all of JRiver's processes. So no need to dump anything. IF that really is causing you any issue at all its trivial to fix.
could i please get insturctions on this easy process? maybe it will stop my runtime and BSOD when building... be an interesting test at least.

After writing this... thumbnail building has started and im at 6936 of 780371 and my processes for media center is at 1,086,396k, cpu is at 18%
Logged

StFeder

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 1493
  • Fight! You may win. If you don't, you already lost
Re: Resource Drain in MC 18 with 1.4 million file library
« Reply #6 on: December 11, 2012, 05:33:09 pm »

For Windows8:
Hit the windows button and start typing "Windows Defender" at the start screen. You'll get a link to Windows Defender. Open it.

For Windows Vista/7:
Double click the MSE Icon (the small green/red house) in the taskbar to open MSE.

In MSE:
Go to settings. You'll see entrys named "Excluded files & locations", "Excluded file types" and "Excluded processes".

I've added my library location, the MC Program Folder and the MC subfolder at users to my excluded locations. If added all exe files in the MC program folder to excluded processes plus MC18.exe at C:\Windows\System32.

Not sure if this is true for Windows Vista/7, but for Windows 8 be sure to hit "Add" after typing/selecting something in the above box. It should be listed in the lower box than. Hit "Save changes" after adding all exclusions.

I did this, because longer time ago (perhaps MC14) I had some serious problems connected to my AntiVirus tool.
Logged

Matt

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 42379
  • Shoes gone again!
Re: Resource Drain in MC 18 with 1.4 million file library
« Reply #7 on: December 11, 2012, 05:36:35 pm »

xtacbyme, if you have a crash, please start a thread with details and provide a log from Help > Logging.

Thanks.
Logged
Matt Ashland, JRiver Media Center

HTPC4ME

  • Regular Member
  • Citizen of the Universe
  • *****
  • Posts: 2760
  • LIFE IS A RADIO... CRANK IT UP TO 11!
Re: Resource Drain in MC 18 with 1.4 million file library
« Reply #8 on: December 11, 2012, 05:44:55 pm »

StFeder Thanks.

Matt will do... first runtime error while adding files logging wasn't enabled :( so i've enabled it and will report when thumb crash happens. Thanks.
Logged

Zootsuit

  • World Citizen
  • ***
  • Posts: 139
  • too cool for school
Re: Resource Drain in MC 18 with 1.4 million file library
« Reply #9 on: December 12, 2012, 07:00:38 am »

After substituting MSE for Avast, I rebooted and now usage seems normal, stabilizes between 275 and 325 and plays very smoothly. Search still shows a little hesitation, but less than before. :)

StFeder, thanks for suggestion, I've already excluded all possible audio-video file types, will now do it by location to be doubly sure.

xtacbyme, I'll be interested to see what happens with the thumbnails. 

Matt, will 2X the file count. That's an odd discrepancy.
Logged
Pages: [1]   Go Up