INTERACT FORUM

Please login or register.

Login with username, password and session length
Advanced search  
Pages: [1]   Go Down

Author Topic: Background scanning delay on large folder  (Read 2729 times)

FrankP1953

  • Recent member
  • *
  • Posts: 27
Background scanning delay on large folder
« on: May 18, 2015, 04:49:24 am »

I have a Synology NAS with two folders: an M drive for Music and an O drive for Other Music. The smaller drive (O) has 450 Gb, 2200 folders and 23,000 files and the background scanning works with a delay of a minute or three when I am ripping new discs. The larger drive (M) has 1.2 Tb, 6000 folders and 75,000 files and the background scanning seems to be delayed anywhere from five minutes to two hours (I usually get impatient and run Auto-Import manually). Is this normal? Is there no way to accelerate the background scanning on the larger drive?

Thanks,

Frank
Logged

Arindelle

  • Citizen of the Universe
  • *****
  • Posts: 2772
Re: Background scanning very, very, very delayed on large folder
« Reply #1 on: May 18, 2015, 05:39:37 am »

I have a Synology NAS with two folders: an M drive for Music and an O drive for Other Music. The smaller drive (O) has 450 Gb, 2200 folders and 23,000 files and the background scanning works with a delay of a minute or three when I am ripping new discs. The larger drive (M) has 1.2 Tb, 6000 folders and 75,000 files and the background scanning seems to be delayed anywhere from five minutes to two hours (I usually get impatient and run Auto-Import manually). Is this normal? Is there no way to accelerate the background scanning on the larger drive?

Hi Frank

short response, yes this is normal, the drive needs some "activity" from the OS. If this was constant it would become a resource hog like other programs. When you write new data (ripping) it will wake things up after the initial minute wait. After big tagging sessions, and any moving around of files externally or other external manipulations, I always run a manual "autoimport" -- could be a lot more than 2 hours btw ;) I put an icon up in the menu bar, so its painless. Just a button click. Granted if the NAS or your network is slow this could be slightly tedious .. why its a good idea to sandbox your ripped files to a temporary folder and import that - using the move rename tool after you verify/retag the new rips. 4tb manual import takes me less than 2 minutes from local drives. On a Nas that would be slower of course.

If you want to be really couch potato-y -- rip to a temp folder on a local drive, config import to watch that folder. After you are ok with the tracks, set up a preset in the Move, Rename library Tool, and just move the new files to the NAS internally. This will be fast
Logged

FrankP1953

  • Recent member
  • *
  • Posts: 27
Re: Background scanning very, very, very delayed on large folder
« Reply #2 on: May 18, 2015, 05:48:52 am »

Thanks, but I'm not unhappy about the time needed for a manual Auto-Import - I'm unhappy that hours after I rip a disc, it still hasn't shown up in Media Center. Last night before bed I ripped a dozen discs, and left the machine on and MC20 running overnight. By the morning, I woke up but MC had not: Arindelle refers to an "initial minute wait" but this was eight hours...

How do I get MC to be more attentive for background scanning?

Thx
Logged

Hendrik

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 10941
Re: Background scanning very, very, very delayed on large folder
« Reply #3 on: May 18, 2015, 05:54:08 am »

MC relies on the OS to report changes to the filesystem and will only then trigger an import. Its quite possible that these events are unreliable when coming from your NAS.
Logged
~ nevcairiel
~ Author of LAV Filters

mwillems

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 5234
  • "Linux Merit Badge" Recipient
Re: Background scanning very, very, very delayed on large folder
« Reply #4 on: May 18, 2015, 08:02:02 am »

Specifically  it relies on the inotify protocol (see https://yabb.jriver.com/interact/index.php?topic=96935.msg669539#msg669539), which is supported by modern versions of SMB (the filesharing protocol used by most NAS devices), but not by older versions of SMB. 

I don't own a synology device, but googling around it looks like their inotify implementation may be off by default so you may need to tweak a configuration option or two.  Media Center automatically runs a scan on watched folders every two hours, so it's especially odd that eight hours can pass without MC seeing a change (unless the NAS is hibernating and MC doesn't wake up drives just to scan them).  I have a drive with about 7000 folders on it (probably only 60K files), and auto-import works perfectly and within about 60 seconds, so there's no generic issue with "large drives."  The issue is a "MC is not getting inotify events from your NAS for some reason" issue. 

You will probably find it most profitable to do some digging about in your NAS's firmware/configuration.  Some avenues for exploration:

1) Try to identify any other differences between the drives (filesystem?  sleep settings? etc.)
2) Consider if there's any difference about the way you put files on the two drives (i.e. filesystem copy and paste vs. ftp vs. something else).  I've seen some reports that ftp uploads don't send inotify events correctly, whereas SMB copying does send them (i.e. copy and paste in explorer).
Logged

glynor

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 19608
Re: Background scanning very, very, very delayed on large folder
« Reply #5 on: May 18, 2015, 08:13:48 am »

There is a ton more about this, and an alternative, in the Auto-Import article on the wiki.
Logged
"Some cultures are defined by their relationship to cheese."

Visit me on the Interweb Thingie: http://glynor.com/

mwillems

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 5234
  • "Linux Merit Badge" Recipient
Re: Background scanning very, very, very delayed on large folder
« Reply #6 on: May 18, 2015, 08:32:04 am »

There is a ton more about this, and an alternative, in the Auto-Import article on the wiki.

Glynor, I have some proposed changes to that section; for example, filesystem events seem to work fine on MC for Linux for me with normal linux filesystems (ext3/4).  It might also be worth clarifying that (per Bob) the system relies specifically on the inotify protocol (at least in the linux context, but maybe generally).  

Do you have any objection to my making those changes?
Logged

Hendrik

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 10941
Re: Background scanning very, very, very delayed on large folder
« Reply #7 on: May 18, 2015, 08:33:52 am »

inotify is just show Linux calls that, its not a generic concept. On Windows it just relies on Windows' file system events, which works perfectly locally, and also through the SMB/CIFS protocol, if the remote server sends them.
Logged
~ nevcairiel
~ Author of LAV Filters

mwillems

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 5234
  • "Linux Merit Badge" Recipient
Re: Background scanning very, very, very delayed on large folder
« Reply #8 on: May 18, 2015, 08:36:37 am »

inotify is just show Linux calls that, its not a generic concept. On Windows it just relies on Windows' file system events, which works perfectly locally, and also through the SMB/CIFS protocol, if the remote server sends them.

Got it, so Bob was talking about Linux specifically over here: https://yabb.jriver.com/interact/index.php?topic=96935.msg669539#msg669539

That makes sense, given the context (And the wiki for inotify: http://en.wikipedia.org/wiki/Inotify  Learn something new every day)

Regardless, auto-import does seem to work fine in MC for x86 Linux and MC for ARM, so it's probably worth adding that, as the current entry says that MC for Windows or Mac is required.  
Logged

glynor

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 19608
Re: Background scanning very, very, very delayed on large folder
« Reply #9 on: May 18, 2015, 09:32:27 am »

Regardless, auto-import does seem to work fine in MC for x86 Linux and MC for ARM, so it's probably worth adding that, as the current entry says that MC for Windows or Mac is required.  

I left Linux out when I wrote that article because I don't use the Linux version (I think I have it installed in a VM somewhere still, but I haven't used it in forever).

If it works, add it up.
Logged
"Some cultures are defined by their relationship to cheese."

Visit me on the Interweb Thingie: http://glynor.com/

glynor

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 19608
Re: Background scanning very, very, very delayed on large folder
« Reply #10 on: May 18, 2015, 09:33:09 am »

inotify is just show Linux calls that, its not a generic concept. On Windows it just relies on Windows' file system events, which works perfectly locally, and also through the SMB/CIFS protocol, if the remote server sends them.

Thanks, Hendrik.  I was going to mention the same "correction" but needed to check to make sure I wasn't crazy first.
Logged
"Some cultures are defined by their relationship to cheese."

Visit me on the Interweb Thingie: http://glynor.com/
Pages: [1]   Go Up