INTERACT FORUM

Please login or register.

Login with username, password and session length
Advanced search  
Pages: [1]   Go Down

Author Topic: How often should auto-import be running?  (Read 2799 times)

skarsol

  • Regular Member
  • World Citizen
  • ***
  • Posts: 202
  • Change this by choosing profile
How often should auto-import be running?
« on: October 23, 2014, 09:09:03 am »

From the pages I saw it seemed like auto-import was supposed to run every 2 hours (from http://wiki.jriver.com/index.php/Auto-Import). It looks to be running way less often than that for me. Here's my log that's been running since I restarted MC:

# grep -i "Starting auto-import" Log.txt
14536516: 2408: Import: CSynchronizeDatabaseWatchThread::ProcessUpdateMainThread: Starting auto-import
128493172: 2408: Import: CSynchronizeDatabaseWatchThread::ProcessUpdateMainThread: Starting auto-import
128934483: 2408: Import: CSynchronizeDatabaseWatchThread::ProcessUpdateMainThread: Starting auto-import

A relevant import:
128538053: 1660: Import: ImportNewFiles: Adding: W:\The Flash (2014)\Season 01\the.flash.2014.103.hdtv-lol.mp4

The log ends at 166551332.

Delay between file creating, SB creating the nfo, and MC Import's creation of the sidecar:
Oct 21 16:40 the.flash.2014.103.hdtv-lol.mp4
Oct 21 21:34 the.flash.2014.103.hdtv-lol.nfo
Oct 22 22:45 the.flash.2014.103.hdtv-lol_mp4_JRSidecar.xml
Logged

Hendrik

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 10944
Re: How often should auto-import be running?
« Reply #1 on: October 23, 2014, 09:32:48 am »

Auto-Import watches the folders and gets notified by Windows if a new file appears, and then automatically runs an import a minute or so later.
It only uses a timed import if setting up the file system watch fails.

Sometimes this doesn't work reliably on network shares, if the share is from some NAS Box, they don't always properly notify of changes.
The bad thing is, they do let you setup a directory watch, but then don't send you notify events, so its impossible to know if its going to work in the end.
Logged
~ nevcairiel
~ Author of LAV Filters

mwillems

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 5234
  • "Linux Merit Badge" Recipient
Re: How often should auto-import be running?
« Reply #2 on: October 23, 2014, 09:39:28 am »

Auto-Import watches the folders and gets notified by Windows if a new file appears, and then automatically runs an import a minute or so later.
It only uses a timed import if setting up the file system watch fails.

Sometimes this doesn't work reliably on network shares, if the share is from some NAS Box, they don't always properly notify of changes.
The bad thing is, they do let you setup a directory watch, but then don't send you notify events, so its impossible to know if its going to work in the end.

Glynor has said in the past that auto-import does a full scan at intervals in addition to watching for file events; That didn't jive with my own experience, but I assumed he might have superior knowledge.  Is that the case or no?

My own experience has been that file system events normally work, but about one in twenty times there will be some kind of miscue and the file system event doesn't successfully trigger the import.  In those cases the file won't show up until I do a manual run, even if I wait a day.  

If that's not how things are supposed to be working (i.e. if there is supposed to be interval scanning), that would be good to know.
Logged

Hendrik

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 10944
Re: How often should auto-import be running?
« Reply #3 on: October 23, 2014, 09:41:00 am »

There is no scheduled run if file system events could be setup properly.

But maybe we should do one to make sure we don't miss any due to bugs or something?
Logged
~ nevcairiel
~ Author of LAV Filters

Matt

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 42388
  • Shoes gone again!
Re: How often should auto-import be running?
« Reply #4 on: October 23, 2014, 09:42:28 am »

But maybe we should do one to make sure we don't miss any due to bugs or something?

It always runs once on program startup for that reason.
Logged
Matt Ashland, JRiver Media Center

mwillems

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 5234
  • "Linux Merit Badge" Recipient
Re: How often should auto-import be running?
« Reply #5 on: October 23, 2014, 09:46:44 am »

There is no scheduled run if file system events could be setup properly.

But maybe we should do one to make sure we don't miss any due to bugs or something?

There have been some threads complaining about stuff not showing up, but the consensus seemed to be that there was something wrong with people's setup because everyone seemed to be assuming that scheduled runs were already occurring in all cases.  Since that's how folks seem to assume things are working, it might make sense to set it up  ;D

If it's going to happen I'd vote to make the interval customizable or scheduleable; a full auto-import takes a bit of time and can use significant system resources (especially with analyze on import checked).  That way, folks who want everything to show up regularly and don't care about system load can make it run on the hour, but folks concerned about system utilization could have it run once a day at 2am.

It always runs once on program startup for that reason.

Some of us might go two or three weeks without restarting (especially on servers, which is where the auto-importing happens in my house).  That's good to know about the auto-import on startup, though.
Logged

Matt

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 42388
  • Shoes gone again!
Re: How often should auto-import be running?
« Reply #6 on: October 23, 2014, 09:52:01 am »

Well it runs every two hours if there are unwatchable folders.  It doesn't run on a timer if all the folders watch normally, since we should be notified of any change.  I suppose we could run it on a four hour timer even if all the folders watch successfully, but it shouldn't be necessary...
Logged
Matt Ashland, JRiver Media Center

skarsol

  • Regular Member
  • World Citizen
  • ***
  • Posts: 202
  • Change this by choosing profile
Re: How often should auto-import be running?
« Reply #7 on: October 23, 2014, 09:53:41 am »

*something* is causing auto-import to run because I didn't restart MC during this time. The drives in this case are Samba shares coming from a FreeBSD box running Samba4.1 with FAM enabled (using gamin since FreeBSD doesn't have inotify), but I've never seen MC actually get a prompt notification about a file (nor with Samba3).

I'm aware I can set a scheduled task to run auto-import, I just wanted to know what MC was actually supposed to be doing.
Logged

BryanC

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 2665
Re: How often should auto-import be running?
« Reply #8 on: October 23, 2014, 09:54:42 am »

Some of us might go two or three weeks without restarting (especially on servers, which is where the auto-importing happens in my house).  That's good to know about the auto-import on startup, though.

You can trigger a silent autoimport using the MC core command:
Code: [Select]
MC20.exe /MCC 23020,1
...and run it as often as you wish using the scheduler.
Logged

mwillems

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 5234
  • "Linux Merit Badge" Recipient
Re: How often should auto-import be running?
« Reply #9 on: October 23, 2014, 09:59:35 am »

Well it runs every two hours if there are unwatchable folders.  It doesn't run on a timer if all the folders watch normally, since we should be notified of any change.  I suppose we could run it on a four hour timer even if all the folders watch successfully, but it shouldn't be necessary...

I can only offer the observation that even watching folders on local drives on a windows box, I've seen auto-import miss filesystem events sometimes.  It wasn't super common, but I saw it happen more than once.  

When watching samba shares on a linux fileserver over the network, the situation is less good, but still pretty good; I see filesystem events work about 95% of the time, but once a week or so something will not import and I'll have to run a manual auto-import to sweep it up.  It's not a huge issue for me because filesystem events mostly work right for me, but there have been other folks who had much spottier detection (that still, frustratingly, worked some of the time).

You can trigger a silent autoimport using the MC core command:
Code: [Select]
MC20.exe /MCC 23020,1
...and run it as often as you wish using the scheduler.

That's a good solution, I didn't realize one could trigger a silent one like that.  Part of my problem was that I thought it was supposed to be happening at intervals already so I didn't investigate scheduling.  Thanks!
Logged

~OHM~

  • Citizen of the Universe
  • *****
  • Posts: 1825
  • "I Don't Play The Music The Music Plays Me"
Re: How often should auto-import be running?
« Reply #10 on: October 23, 2014, 10:01:06 am »

I just put a button in the top left corner and all i have to do is push the button and it runs....Run Auto-Import Now
Logged
“I've Reached A Turning Point In My Life. I Now Realize I Have More Yesterdays Then Tomorrows”

Arindelle

  • Citizen of the Universe
  • *****
  • Posts: 2772
Re: How often should auto-import be running?
« Reply #11 on: October 23, 2014, 10:08:43 am »

Maybe the question should be "when should I let my media be imported automatically and when should I just click the button?".

I personally never have anything running like autotagging  or auto cover art which significantly adds the time to complete an import.  More times than not the meta data is not good enough and I have to manually tweek it anyways. I never do audio or video sound analysis during import -- very time and resource hungry, I do it later using a simple smartlist.

If people are recording tv programs while they are at work for 8 hours that of course is something else again. Otherwise, any time I  rip or add media to my watched folders, I do a manual autoimport -- its fast. Any time I spend a lot of time retagging from a client, I manualy sync AND run autoimport too. No big deal. And you can listen or watch whatever right away. The import a particular folder option does no reindexing, no fixes for external changes that syncing might have missed, no removing dead links - so I never use that.

However I realize there are a lot of people who just want everything automated and are not so picky with there artwork or genres, so they are given those options.

In the past, I had this set to off, and would only do this manually. Now it works quite well IMHO with little system overhead or network drain.

-- 5 new posts before I finished so a PS fro Matt and Hendrik :)

As long as I an get it to work like it does now that ok; but not every x hours. I also didn't realise it starts silently on each launch.

BUT, what is so bad about just clicking the import button (put in in the main menu bar even like TA0 said) - never did understand why people want this so badly except for recording TV in one's absence (or making it easier for "torrenting" people out there^^).

That being said, would it be a good idea to separate "folder watching" and the autoimport process? For people moving around files externally or doing admin remotely,  I can see where that would be a good thing to have on full auto, but not necessarily the complete import process.
Logged

BryanC

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 2665
Re: How often should auto-import be running?
« Reply #12 on: October 23, 2014, 10:16:30 am »


That being said, would it be a good idea to separate "folder watching" and the autoimport process? For people moving around files externally or doing admin remotely,  I can see where that would be a good thing to have on full auto, but not necessarily the complete import process.


That would be cool, sort of a sandboxed "To Be Imported" section that is excluded from search, default views, and smartlists where the metadata can be altered before a 'real' import commences. As an option, of course.

It could also be used to auto-organize the files on the disk (a feature that many past iTunes users seem to request). This would keep the renaming/moving action safe from the rest of the library which I think is the major reason the auto-organize on disk feature hasn't been implemented.
Logged

mwillems

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 5234
  • "Linux Merit Badge" Recipient
Re: How often should auto-import be running?
« Reply #13 on: October 23, 2014, 10:18:22 am »

BUT, what is so bad about just clicking the import button (put in in the main menu bar even like TA0 said) - never did understand why people want this so badly except for recording TV in one's absence (or making it easier for "torrenting" people out there^^).

I'm not the only person who adds media to my library, but I am the only one with access to Media Center on the server (for administrative control reasons).  The other folks in my household who add media know they need to put the files in a special network location (which is accessible to them), and then the files will show up in their client instance of MC auto-magically after the server imports them.  Except sometimes the files don't auto-import, and because the media-adders only have access to clients which can't ordinarily initiate an import, I get a tech support call.  Being able to say "just wait half an hour, it'll show up" would be handy, instead of "wait till I get home".

BryanC's automation solution is the answer to my use case.
Logged

glynor

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 19608
Re: How often should auto-import be running?
« Reply #14 on: October 23, 2014, 10:29:28 am »

There is no scheduled run if file system events could be setup properly.

But maybe we should do one to make sure we don't miss any due to bugs or something?

Someone told me that it did do both once.

My file systems aren't junky so I can't test it well.   ;)
Logged
"Some cultures are defined by their relationship to cheese."

Visit me on the Interweb Thingie: http://glynor.com/
Pages: [1]   Go Up