INTERACT FORUM

Please login or register.

Login with username, password and session length
Advanced search  
Pages: [1]   Go Down

Author Topic: Slowdowns  (Read 2687 times)

JustinChase

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 3273
  • Getting older every day
Slowdowns
« on: November 08, 2011, 07:23:49 pm »

log sent to Matt

After browsing for a bit (v.31) it all seemed good, considering it was building thumbnails in a new library, then it hung for a few minutes, then came back and seemed fine again.  ?
Logged
pretend this is something funny

Matt

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 41956
  • Shoes gone again!
Re: JustinChase: Slowdowns
« Reply #1 on: November 09, 2011, 10:46:32 am »

Thanks for the log.  There are a few interesting things in it.

First, I see us thumbnailing files like this and it appears we're reading the whole file (which the server meters so it takes minutes):
http://feedproxy.google.com/~r/HdtvPodcast/~5/Qv4bCLYAJd4/HDTV-2010-11-02.mp3

What files in your library have a value like that in the filename field?  Are they Podcasts?  Normally a downloaded podcast has a local filename.

Next, at one point we try to check for an R: drive, and it might have been really slow (like 5 minutes):
0511683: 2956: General: CFileSystemManager::GetDriveExists: Drive does not exist (drive: R:, filename: R:\Rome\)

What is this drive, and do you have any idea why it's so slow?

Finally, the drive that contains your library (J:\MC\libraries\MC17Server\) seems slow.  It routinely takes several seconds to write the library to it.  I wouldn't expect this for a 140,000 file library.  Is J:\ a slow network drive?  Could you move your library to a fast drive?
Logged
Matt Ashland, JRiver Media Center

JustinChase

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 3273
  • Getting older every day
Re: JustinChase: Slowdowns
« Reply #2 on: November 09, 2011, 01:27:52 pm »

Wow, that's a lot of stuff :)

Thanks for the log.  There are a few interesting things in it.

First, I see us thumbnailing files like this and it appears we're reading the whole file (which the server meters so it takes minutes):
http://feedproxy.google.com/~r/HdtvPodcast/~5/Qv4bCLYAJd4/HDTV-2010-11-02.mp3

What files in your library have a value like that in the filename field?  Are they Podcasts?  Normally a downloaded podcast has a local filename.

 I have NO idea what that is, nor how to delete it.  Okay, just checked my podcasts, and I have this one actually downloaded, all the rest in this feed are "In Queue" or "Deleted"...

NAME: HDTV & Home Theater
Feed URL: http://feeds.feedburner.com/HdtvPodcast
Episode URL: http://feedproxy.google.com/~r/HdtvPodcast/~5/6nXaP_dA67g/HDTV-2010-11-12.mp3

But it's not the same one as you saw.  I had no idea I had this downloaded, but upon checking the folder it's located in, and the feed itself, I cannot find the file you refer to above.  I have no idea why MC is trying to thumbnail it.  how can i remove this from the library, or whatever is telling MC to thumbnail it?

This is the only media file in the podcast subfolder for this feed, plus a folder.jpg, and in the folder it's called

2010-11-12 Podcast #452_ Logitech Revue and Google TV.mp3

Next, at one point we try to check for an R: drive, and it might have been really slow (like 5 minutes):
0511683: 2956: General: CFileSystemManager::GetDriveExists: Drive does not exist (drive: R:, filename: R:\Rome\)

What is this drive, and do you have any idea why it's so slow?

I use to have an R:\Drive (Recorded TV), but a couple of weeks ago, I deleted it and now just use a subfolder on my V:\Drive instead.

I just created a new view with no rules, which I think should show me all files in my library.  When I add Location to the view, it shows all the drives I am using, but the R:\ drive is not shown.  I don't know how/why it's still trying to do something with that folder ?

If I do a search for "Rome" while in this view, I only get results on the M:\ drive, weird

Finally, the drive that contains your library (J:\MC\libraries\MC17Server\) seems slow.  It routinely takes several seconds to write the library to it.  I wouldn't expect this for a 140,000 file library.  Is J:\ a slow network drive?  Could you move your library to a fast drive?

This is an issue I was having last week, slow writes to the unRAID server, but MC has always been SUPER fast in writing to it, at least I thought so.  If I move/rename files in MC, it will "move" HD movies in a few seconds (likely just updating the file tables on the disk vs. actually re-writing the file to a new location).  I wonder what I can do to eliminate the lag you're seeing there?

The log file I sent was created on my Desktop, which is still connected via WiFi-g, and all data tasks take longer on this machine, since I assume it's actually using this physical processor to do it's thing.  I wonder if this is the lag you're seeing?  Browsing around the file structure is mostly fast enough, but could always be faster.

Is there any chance that eliminating the files and directories that MC is trying to scan would free up some processing capacity and speed this up?  I'm going to run Cat6 one of these weekends, but likely still a few weeks away.

When on a Gig-E wired computer, I can write to the array at 31MB/s sustained writes while copying a 30GB movie, with bursts up to 90MB/s upon starting.  This seems like it should be plenty fast enough.  It's MUCH slower when I try to initiate using the WiFi machine.

So, how do I remove the podcast and non-existant network drive from the tasks MC is doing for me in the background?  I don't have any idea where it's getting this information from.

If you need/want me to do some more testing/logging, please just let me know what you need, I'm happy to do it, since this seems like it would help speed me up :)

Thanks Matt!
Logged
pretend this is something funny

JimH

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 71439
  • Where did I put my teeth?
Re: JustinChase: Slowdowns
« Reply #3 on: November 09, 2011, 01:36:13 pm »

Check your settings for R: something under File Location.

Search for R: in each view.
Logged

Matt

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 41956
  • Shoes gone again!
Re: JustinChase: Slowdowns
« Reply #4 on: November 09, 2011, 01:41:00 pm »

Create a smartlist: ~d=a r:
Logged
Matt Ashland, JRiver Media Center

JustinChase

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 3273
  • Getting older every day
Re: JustinChase: Slowdowns
« Reply #5 on: November 09, 2011, 04:23:59 pm »

Check your settings for R: something under File Location.

Search for R: in each view.


Create a smartlist: ~d=a r:


Will check again when I get home, but I thought I kinda tried that already...

I just created a new view with no rules, which I think should show me all files in my library.  When I add Location to the view, it shows all the drives I am using, but the R:\ drive is not shown.  I don't know how/why it's still trying to do something with that folder ?

If I do a search for "Rome" while in this view, I only get results on the M:\ drive, weird

In this new view, there are literally no rules, no exclusions, not even media type; and there was no R: drive.  I will try both things suggested and report back later this evening.  I *just* created this as a new library in a blank, new location yesterday (J:\MC\libraries\MC17Server\).  I then did a Library Restore (not including settings) to populate it.  I just don't understand how/why MC is searching for files that I have no record of anywhere.  ?

What about the podcast that doesn't seem to exist either?


I've felt that the server wasn't as snappy as others brag about for a LONG time, so maybe this has been an issue for a while now?  That podcast has a date from (my birthday actually) last year, so I wonder how long MC has been trying to deal with it in the background like this  :P

Will report back in a few hours :)

Thanks again

Oh, if it turns out that these files are "stuck" in MC somehow, I wonder if there would there be any way to find any other problem files like these, and eliminate them too?  I only ran MC for about 10 minutes to get that log, so it's not a very long period to discover more/all problems like this.

One thing at a time, I know :)
Logged
pretend this is something funny

JustinChase

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 3273
  • Getting older every day
Re: JustinChase: Slowdowns
« Reply #6 on: November 09, 2011, 06:39:18 pm »

Create a smartlist: ~d=a r:

Done.  Hit OK, hung MC for about 5 minutes, then returned 79 results, but they all seemed to be like this Law & Order:

I tried all variations I thought reasonable

r:    - 79 results
"r:" - same results
"R:" - same results
r:\   - zero results
Rome - 64 results, similar to the above, mostly in Music

I did see 3 more weird results though in this last search; none of these file results below show a letter in their path...

Unknown Album (14)\Track10.cda
Various (1)\Track13.cda
http://media.theworld.org/pod/tech/WTPpodcast285.mp3

If I r-click and locate...  both choices result in "The file '(0x021C83DB:)\Track13.cda' does not exist."  the other files also do not exist.  ?

I searched the settings for anything that might want a location, and nothing had R: that I could find.  I did a search here too, but I assume it only searches the options, not the choices we make, so I didn't expect any results, and got none.

I guess I might just need to create another new library, then just import my files, but I don't think all my tags are correct, and I'm not sure saving them all to the files would be good right now, for other reasons to do with my lack of good organization right now, so I really don't know what to do next.

I re-ckecked my smartlist from earlier (with NO rules at all) and all results show a drive letter under location, so this is not finding the 3 weird files above.

Also, Matt said my library contains 140,000 files, but the results of this smartlist show 128814 files.  I know Matt rounded, but that's still a difference of 11,000 files, which is more than I'm guessing he rounded.  I must have some other files not showing up somewhere/somehow, no?
Logged
pretend this is something funny

Matt

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 41956
  • Shoes gone again!
Re: JustinChase: Slowdowns
« Reply #7 on: November 09, 2011, 07:17:08 pm »

Could you send me a full log showing some slowdowns?  The last one had been reset, so I couldn't trace back to see what thread was using R:\Rome, etc.
Logged
Matt Ashland, JRiver Media Center

JustinChase

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 3273
  • Getting older every day
Re: JustinChase: Slowdowns
« Reply #8 on: November 09, 2011, 09:45:08 pm »

New log sent. 

Thumbnails finished building this afternoon, and it's been pretty quick all evening so far.  While browsing around waiting for another big hangup, it instead just crashed :(
Logged
pretend this is something funny

Matt

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 41956
  • Shoes gone again!
Re: JustinChase: Slowdowns
« Reply #9 on: November 10, 2011, 08:02:42 am »

I don't really see slow stuff in the last logs you sent.

Loading the view 'File Type/Genre/Exists' is slow, but using the FileExists(...) expression will be slow since it talks to the file system.  I wouldn't use that expression in a view I look at often.

You might also change the auto-import option for removing broken links to 'Yes' (instead of 'Yes (protect network files') to flush entries that really aren't there.  Since network drives come and go, we don't do this by default.


Logged
Matt Ashland, JRiver Media Center

JustinChase

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 3273
  • Getting older every day
Re: JustinChase: Slowdowns
« Reply #10 on: November 10, 2011, 08:38:07 am »

I don't really see slow stuff in the last logs you sent.

Loading the view 'File Type/Genre/Exists' is slow, but using the FileExists(...) expression will be slow since it talks to the file system.  I wouldn't use that expression in a view I look at often.

You might also change the auto-import option for removing broken links to 'Yes' (instead of 'Yes (protect network files') to flush entries that really aren't there.  Since network drives come and go, we don't do this by default.

Okay, thanks for looking.  I assume you didn't see anything about any of the crashes either?

Very weird that the first one showed all that stuff after just one hang, but all that didn't reveal anything, darn computers.

I'm running the updated auto-import now, but for some reason, it wants to analyze audio for 28000 files, which I might as well let it do while I'm at work.

Hopefully this will help.  If not, I'll send another log.
Logged
pretend this is something funny
Pages: [1]   Go Up