INTERACT FORUM

Please login or register.

Login with username, password and session length
Advanced search  
Pages: [1]   Go Down

Author Topic: Auto-import not finding new files?  (Read 6209 times)

6233638

  • Regular Member
  • Citizen of the Universe
  • *****
  • Posts: 5353
Auto-import not finding new files?
« on: March 21, 2014, 12:27:26 pm »

Recently I've been having problems with auto-import not finding files in certain directories any more.
The main thing affected is my main videos folder - it's no longer finding new films that I have ripped.
 
I ripped a handful of new discs this morning before heading out, came back tonight and still had to run auto-import manually for Media Center to find them.
This is not the first time it's happened, but it seemed to sort itself out last time. (I think around the time I upgraded to MC19)
 
 
Is there anything I can do to troubleshoot this? I have removed and recreated the import rule, and it doesn't seem to have helped.
These are just folders on local drives, not external disks, networked drives, virtual drives (drive pooling software) etc.
Logged

MrC

  • Citizen of the Universe
  • *****
  • Posts: 10462
  • Your life is short. Give me your money.
Re: Auto-import not finding new files?
« Reply #1 on: March 21, 2014, 12:30:47 pm »

Does this only happen with large files that are actively being ripped?
Logged
The opinions I express represent my own folly.

6233638

  • Regular Member
  • Citizen of the Universe
  • *****
  • Posts: 5353
Re: Auto-import not finding new files?
« Reply #2 on: March 21, 2014, 12:38:38 pm »

Does this only happen with large files that are actively being ripped?
No - they were ripped about 8 hours ago. The problem is that auto-import isn't finding them once ripping (via AnyDVD) has finished.
 
The video folders on other drives seem to be working, just not this folder for some reason, and it's the one marked as the system's "My Videos" folder. (my music folder is also the one marked as "My Music" which seems to work fine - though I wonder if I should be changing that?)
All the rules are identical (copied and pasted) for each folder, but for some reason this one requires me to run auto-import manually before it finds the files.
 
Other folders on this drive seem to work correctly. (though it can be a bit slow to find them)
Edit: and there are no "bad" files being skipped in the summary. (not that there should be any)
Logged

MrC

  • Citizen of the Universe
  • *****
  • Posts: 10462
  • Your life is short. Give me your money.
Re: Auto-import not finding new files?
« Reply #3 on: March 21, 2014, 12:42:07 pm »

Sorry, what I meant was...

Auto-import runs all the time, and new files trigger file system notifications.  MC has to react to these.  But the state of the file is in flux.  I think MC has some smarts, some recently added (MC18?) to try to deal with ever-changing/growing files.

So might the files be placed in the Bad database?  When you run auto-import manually, I believe this database is used for re-trial.  8 hours later, when your rips are finished, MC can successfully import.

This is my current theory...
Logged
The opinions I express represent my own folly.

6233638

  • Regular Member
  • Citizen of the Universe
  • *****
  • Posts: 5353
Re: Auto-import not finding new files?
« Reply #4 on: March 21, 2014, 12:45:42 pm »

Well the rips only took 20 minutes as it was a handful of DVDs. What I meant was that I intentionally left MC alone after it had finished and 8 hours later it had still failed to import them.
 
Ripping directly to the \Videos\ folder on all my other drives seems to work correctly with auto-import - it just seems to fail on this disk for some reason.
Logged

MrC

  • Citizen of the Universe
  • *****
  • Posts: 10462
  • Your life is short. Give me your money.
Re: Auto-import not finding new files?
« Reply #5 on: March 21, 2014, 12:47:47 pm »

I'll assume My Videos is the stock Windows My Videos folder and not a junction or Library.  Anything else curious or relevant about that filesystem?
Logged
The opinions I express represent my own folly.

6233638

  • Regular Member
  • Citizen of the Universe
  • *****
  • Posts: 5353
Re: Auto-import not finding new files?
« Reply #6 on: March 21, 2014, 01:09:18 pm »

I'll assume My Videos is the stock Windows My Videos folder and not a junction or Library.  Anything else curious or relevant about that filesystem?
Yes, a standard Windows folder on an NTFS drive.

The folder structure is fairly basic. Everything goes into X:\Videos\ and the subfolders are either New, Seen, TV, Downloads
The only rules I have are to set the Media Type to Video, and a folder-based one to set the sub-type: If(IsEqual([Filename (path)],TV,8),TV Show,If(IsEqual([Filename (path)],Downloads,8),Other,Movie)) but that's it.
 
The only thing different about this one, is that this drive is where Media Center is saving downloaded videos. (YouTube/RSS)
But I have a similar folder structure for audio - X:\Audio\ Podcasts, Music etc. which is on the same drive, and new music seems to be detected correctly.


I've just tried ripping another disc (to ISO, as I always do) and it's not being put into the bad files list either during or after ripping.
It just seems like Media Center is not finding changes in this folder until I manually run auto-import.
Logged

glynor

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 19608
Re: Auto-import not finding new files?
« Reply #7 on: March 21, 2014, 02:35:55 pm »

It is very odd that it isn't finding them eventually.

MC's Auto-Import system DOES do periodic full scans (they're just very occasional, something like every 4-6 hours or so, if I remember right).  So they should be eventually picked up, even if Filesystem Events are totally broken.

Any services on the machine disabled from default, or anything like that?

Permissions on the folder causing issues weirdo?

CHKDSK?

sfc /scannow?
Logged
"Some cultures are defined by their relationship to cheese."

Visit me on the Interweb Thingie: http://glynor.com/

6233638

  • Regular Member
  • Citizen of the Universe
  • *****
  • Posts: 5353
Re: Auto-import not finding new files?
« Reply #8 on: March 21, 2014, 03:04:15 pm »

It is very odd that it isn't finding them eventually.
MC's Auto-Import system DOES do periodic full scans (they're just very occasional, something like every 4-6 hours or so, if I remember right).  So they should be eventually picked up, even if Filesystem Events are totally broken.
Well it's working for X:\Audio\… and all the other \Video\ folders (four other drives) just not this one. :-\

Any services on the machine disabled from default, or anything like that?
No, all the Microsoft services are at the stock Windows 8.1 settings.
Permissions on the folder causing issues weirdo?
Everything appears to be fine. (I've never changed the permissions)
CHKDSK?
sfc /scannow?
No problems reported.
Logged

glynor

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 19608
Re: Auto-import not finding new files?
« Reply #9 on: March 21, 2014, 03:12:05 pm »

Darn.

This is getting into crazy-territory, but... Can you try it?

1. Close MC.
2. Change the name of the folder on your disk to something else.
3. Make a new, empty folder with the old name (so create a new one).
4. Move all the crap from the old directory into the new one.
5. Open Properties in Windows Explorer on the new folder, Security tab, and push permissions down the stack to make sure there isn't anything weirdo in there.
6. Try to reproduce.

I can report that I use Handbrake and MakeMKV pretty regularly to rip stuff, and they rip directly to a monitored directory, and MC auto-imports them without issue, even if the encodes are long running.  I've had issues in the past where stuff got sucked into the Bad Files list, but I've never seen it just ignore them altogether.

My SageTV recordings don't ALWAYS get picked up by Filesystem Events.  Sometimes (particularly long-running recordings, like those 4+ hour long ones from the Olympics), they don't get imported until MC does its periodic "full scan" (these are easy to ID looking at [Date Imported] because they'll be all in a big batch) but mine always come in eventually.
Logged
"Some cultures are defined by their relationship to cheese."

Visit me on the Interweb Thingie: http://glynor.com/

MrC

  • Citizen of the Universe
  • *****
  • Posts: 10462
  • Your life is short. Give me your money.
Re: Auto-import not finding new files?
« Reply #10 on: March 21, 2014, 03:15:09 pm »

Antivirus monitoring that one folder tree and not the others?

I'd focus on file system as opposed to disk.  MC and Windows here are operating at the file system level, not disk level.
Logged
The opinions I express represent my own folly.

glynor

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 19608
Re: Auto-import not finding new files?
« Reply #11 on: March 21, 2014, 03:17:39 pm »

I'd focus on file system as opposed to disk.  MC and Windows here are operating at the file system level, not disk level.

Agreed.

I suggested the directory swap trick because I've seen directories get borked before where Windows API calls on them start to fail, and "making a new folder and deleting the old one" is often the easiest solution.
Logged
"Some cultures are defined by their relationship to cheese."

Visit me on the Interweb Thingie: http://glynor.com/

6233638

  • Regular Member
  • Citizen of the Universe
  • *****
  • Posts: 5353
Re: Auto-import not finding new files?
« Reply #12 on: March 21, 2014, 03:35:28 pm »

OK - I changed the system "My Videos" location to C:\Users\username\My Videos\ (without moving the files)
Renamed the original folder.
Created a new X:\Video\ folder, and moved the files into it.
Ripped another disc.
 
Still not being picked up by auto-import. :-\
No problem importing files from X:\Audio\Music\ or X:\Images\
Logged

6233638

  • Regular Member
  • Citizen of the Universe
  • *****
  • Posts: 5353
Re: Auto-import not finding new files?
« Reply #13 on: March 21, 2014, 04:46:23 pm »

OK - I changed the system "My Videos" location to C:\Users\username\My Videos\ (without moving the files)

Still not being picked up by auto-import. :-\
It seems that I managed to overlook an important point here - it turns out that Media Center treats the system folder differently from others in the auto-import config, following Windows' location rather than the path specified. (I only specified the path before, but I guess it picked up on the fact that it was the system "My Videos" folder)
 
I had to enter the auto-import configuration and change it from C:\Users\username\My Videos\ back to X:\Video\
And it seems to be working now!
 
I've just tried ripping a few more discs (tip: when you're doing tests like this, just mount an ISO in a virtual drive rather than using a physical DVD - the rips go at 100MB/s+) and it seems to be picking them up almost instantly now.
 
 
I'm unsure about whether or not I should reassign X:\Video\ as the system "My Videos" folder again or not, or whether I should also set "My Music" away from the X:\Audio\Music\ folder that Media Center is currently watching.
There don't seem to be any problems automatically importing music files, but I wonder if I should be keeping Windows away from that location just in case I launch WMP by mistake and it starts messing with my files?
Logged

MrC

  • Citizen of the Universe
  • *****
  • Posts: 10462
  • Your life is short. Give me your money.
Re: Auto-import not finding new files?
« Reply #14 on: March 21, 2014, 04:49:41 pm »

I avoid the My * folders as much as possible for exactly the reason you mention.
Logged
The opinions I express represent my own folly.

6233638

  • Regular Member
  • Citizen of the Universe
  • *****
  • Posts: 5353
Re: Auto-import not finding new files?
« Reply #15 on: March 21, 2014, 04:54:43 pm »

Aha - it wasn't me being an idiot.
 
I just relocated "My Music" to C:\Users\username\My Music\ and checked the auto-import config. Media Center was still pointing at X:\Audio\Music\ and it was no longer listed as being the system audio folder.
It was not until I restarted Media Center that the auto-import config changed over to C:\Users\username\My Music\ and I had to manually switch it back over to X:\Audio\Music\
Logged

MrC

  • Citizen of the Universe
  • *****
  • Posts: 10462
  • Your life is short. Give me your money.
Re: Auto-import not finding new files?
« Reply #16 on: March 21, 2014, 05:06:14 pm »

There is magic with certain special Windows folders.  You may notice, if you look at a non-logged in user's profile, the music folder is just named "Music".  Once the user logs in, it becomes My Music.

Probably the safest thing to do after you change some special folder is to log-off, minimally, and sometimes rebooting might be necessary.
Logged
The opinions I express represent my own folly.

6233638

  • Regular Member
  • Citizen of the Universe
  • *****
  • Posts: 5353
Re: Auto-import not finding new files?
« Reply #17 on: March 21, 2014, 05:56:13 pm »

There is magic with certain special Windows folders.  You may notice, if you look at a non-logged in user's profile, the music folder is just named "Music".  Once the user logs in, it becomes My Music.

Probably the safest thing to do after you change some special folder is to log-off, minimally, and sometimes rebooting might be necessary.
Yes, the desktop.ini file controls this - I made sure to delete it from my folders.
The issue was with how Media Center handles the auto-import path.
 
When it is set to a "system folder" it automatically follows that location - but it seems it only updates this when you relaunch the program.
So it appeared to be pointing at the correct directory until I restarted MC and it updated the location.
 
After it had done this, I was able to point it back at the correct folder.
No need to log off or restart. (at least on Windows 8.1)
Logged

glynor

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 19608
Re: Auto-import not finding new files?
« Reply #18 on: March 21, 2014, 08:46:59 pm »

I'm not sure I get it, but I think I see what it is doing now.  I'll have to test it out to be sure I fully understand (I also don't use the system folders much at all, and ignore them).  But, anyway, I'm glad you figured it out.



Logged
"Some cultures are defined by their relationship to cheese."

Visit me on the Interweb Thingie: http://glynor.com/

6233638

  • Regular Member
  • Citizen of the Universe
  • *****
  • Posts: 5353
Re: Auto-import not finding new files?
« Reply #19 on: March 21, 2014, 09:11:39 pm »

I'm not sure I get it, but I think I see what it is doing now.  I'll have to test it out to be sure I fully understand (I also don't use the system folders much at all, and ignore them).
Well I still don't know why auto-import hasn't been working correctly for the last week or two - I suspect what fixed it was creating a new folder and moving everything in there.
 
The issues I had with system folders is simply the way that Media Center tracks their location as you reassign them in Windows.
 
 
Originally I added X:\Video\ to my import rules (which was also assigned as the "My Videos" folder) and Media Center automatically detected that this location was the "My Videos" folder. This folder has been working fine for most of the time I've been using MC19.
What I did not notice earlier this evening, was that when I changed the folders as a result of troubleshooting this issue, Media Center had changed my import rule from pointing at X:\Video\ to C:\…Video\ automatically when I changed the location for "My Videos".
So when I changed the auto-import rule back to X:\Video\ it worked again (likely as a result of being a new folder)
 
If X:\Video\ had never been assigned as the "My Videos" folder, simply creating a new folder and moving the files there as you suggested would most likely have worked.
 
 
Going through the process a second time as I wanted to keep my music out of the "My Music" folder, I changed the system location from X:\Audio\Music\ to C:\…Music\ while Media Center was running. (without moving any files - simply assigning "My Music" to a new folder in Explorer)
I opened up the auto-import configuration after making the change and checked that my rule was still pointing to X:\Audio\Music\ - which it was, and it no longer said [System Music folder] next to the location. So everything appeared to be fine.
However, once I restarted Media Center, the rule automatically changed to C:\…Music\ (the new "My Music" location) on its own - so that's how I had managed to miss the change before, and was left wondering why auto-import still wasn't working.
Logged

glynor

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 19608
Re: Auto-import not finding new files?
« Reply #20 on: March 21, 2014, 10:00:09 pm »

Originally I added X:\Video\ to my import rules (which was also assigned as the "My Videos" folder) and Media Center automatically detected that this location was the "My Videos" folder. This folder has been working fine for most of the time I've been using MC19.
What I did not notice earlier this evening, was that when I changed the folders as a result of troubleshooting this issue, Media Center had changed my import rule from pointing at X:\Video\ to C:\…Video\ automatically when I changed the location for "My Videos".
So when I changed the auto-import rule back to X:\Video\ it worked again (likely as a result of being a new folder)

Now I totally get it.  That's actually fairly cool.  It is being helpful.

1. You added the X:\ drive folder to Auto-Import, everything is cool.
2. You moved your system's Environment.SpecialFolder.MyVideos directory to match this folder at some point (that's C#-speak, but the C++ version of this)
3. When you closed MC (or, I suppose, the next time you ran it), it saw that X:\Video was now == Environment.SpecialFolder.MyVideos, so it reassigned itself to %USERPROFILE%\Videos

That way:
1. You have a user who gets a new bigger hard drive, so they plug it in and it gets a new drive letter.
2. Then, they open Windows Explorer (after googling "how to move my music folder windows") and do this (or this guys regurgitation of the same).
3. That moves their "My Music" and "My Videos" folders via the Windows way (where it then asks you if you want to move everything to the new location).  They say yes.
4. Normally, their links in the Library would all be broken.  But in this situation, it Just Works.  Initially they will all be broken (because they did it externally from MC), but Auto-Import will fix the broken links (retaining all metatadata and playlists and stuff) as long as it is monitoring the new folder.

Yeah, that makes sense, and is actually a pretty nice touch.

You moved the special folder in Windows, and didn't want it to move your files (said no to that prompt, certainly, or did it another way).  So, for you, it just messed you up!  ;D
Logged
"Some cultures are defined by their relationship to cheese."

Visit me on the Interweb Thingie: http://glynor.com/

6233638

  • Regular Member
  • Citizen of the Universe
  • *****
  • Posts: 5353
Re: Auto-import not finding new files?
« Reply #21 on: March 21, 2014, 10:39:35 pm »

Yes, it's great that Media Center does this - it's just a pain when you're trying to troubleshoot an issue like this and don't realize auto-import is now pointing at the wrong directory. ;D
 
It would certainly make things easier for most users.
Logged

glynor

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 19608
Re: Auto-import not finding new files?
« Reply #22 on: March 21, 2014, 11:09:51 pm »

Yes, it's great that Media Center does this - it's just a pain when you're trying to troubleshoot an issue like this and don't realize auto-import is now pointing at the wrong directory. ;D

Certainly!  It had me stumped.

But, I'm kinda glad it happened (to you, natch)... It revealed a cool little feature.  I might even use it for recommending things in the future.  Certainly for a basic user, with MC installed on a laptop, no external drive (yet), and a music library from iTunes and a few videos in the %USERPROFILE%\Videos folder.  Just import it in place, and maybe re-organize it out of the iTunes subfolder system later (learn a little about the Rename, Move, and Copy tool as you go).  But if you ever need to move to an external disk, just do it this way.

More Learn To Stop Worrying and Trust Auto-Import fodder.
Logged
"Some cultures are defined by their relationship to cheese."

Visit me on the Interweb Thingie: http://glynor.com/
Pages: [1]   Go Up