Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[Don W]

I just discovered that two days ago almost 2/3 of my entire library was re-imported. That's nearly 10,000 files. This changed the import date and reset the number plays and last played date values for all of these files. Luckily I caught it before too much time went by and could restore from an auto library backup from 4 days ago. But that's just luck that I caught it.

Can someone please explain just what can trigger a re-importing like this? The only thing I can think of that I might have done to trigger this myself is I fixed a bug in one of my import rules which write a value to a custom tag. Would this cause a re-import? If so, how can I fix an import rule without triggering an entire re-import?

This has happened to me a few times before. The last time it happened I posted a question about it but got no useful reply.

The play stats are nice to have but they are useless if MC keeps re-importing and resetting them.

[Don W]

Its frustrating that no one will answer. It's important to me and seems like a straight forward question: What triggers re-importing (and losing plat stats) and how can it be prevented?

[MrC]

I believe that a file is not imported if its Filename (a full path) matches an entry already in MC.   So one might suspect that your full path names are different somehow.  Can you elaborate on your paths, now and in the recent past?

[Don W]

My file names have not changed in months. Months. This happened a couple days ago. I am talking about 10000 files. 2/3rds of my library. The other 1/3 DIDN'T get reimported, for some reason. They are all in the same root directory, with the same naming convention.

If this was caused by some change I made to my library, there is only one thing I can think of. I have a rule in auto import where I set the value of a custom field named AcquireDate. I created this field because the built-in DateImported data is lost when ever the library is re-imported. The rule sets AcquireDate = Now(), but in order to strip the time of day, it first formats Now to a string and converts that back to a Date. If it already has a value, then it is not changed. Well, I had a bug in the rule. It was missing a parentheses. I fixed the bug. Two days later I noticed my play stats were wrong.

I do have a rule for naming files when they are imported. But 1) that rule wasn't changed and 2) it is applied to every file in the music collection, not just the 2/3 which were re-imported.

[JimH]

Sometimes Windows can change the drive letter of an external drive.  If that's the problem, see the wiki topic called "Moving Files" for  some of the tools you can use.

MC makes automatic backups that might be useful.  Go to File/Library to see them.

[Don W]

Hi Jim,

I'd be very surprised if the drive letter was changed by Windows. I have a pretty good understanding of what can trigger that. For example, if you plug a USB drive into the computer, and that drive hasn't been assigned a letter yet, then Windows might assign it the letter of a mapped network drive, if the network is offline. But that didn't happen. Besides, if that somehow were the case, then I would expect that all the files would have been re-imported, not just 2/3 of them, since all my files are in the same sub directory on the same drive. Unless Windows changed the drive letter and then unchanged it while MC was doing the import... And if that happened, then the newly imported files would not play, because the drive letter in their path would be wrong. But they did play.

Thankfully for the automatic backups, I didn't loose more than 2 days of play statistics. But I was lucky to have noticed before losing more than that.

Is there any other scenario where MC might re-import files? Not, by chance, if an import rule which writes a tag value, changes?

[MrC]

Import rules are only triggered when a file is actually imported.  In otherwords:

   1. Import file...
   2. Apply auto-import rule

Are the files on a NAS, or external USB HD, or other file server?

[Don W]

Hi MrC,

The files are on a NAS. The shared folder of the NAS is mapped as a network drive in Windows. Like I said in response to Matt, I'd be surprised if Windows changed the drive number. There are lots of reasons that is unlikely, which I brought up in my last post. I suppose instead of changing the drive number the drive could just be missing somehow, but I do have the setting to protect files on missing drives selected, which should guard against that. Is there any other reason why it might be relevant that the files are on a NAS?

[MrC]

I suspected a NAS.  So you're using a Windows MAP network drive.  After login, it can take a little time for the mount and share mapping (if you have the share set to remap upon login).  I wonder if you started MC, the share wasn't available, and MC started marking the items as non-existent.  This might explain a re-import.  Or maybe MC started use UNC paths instead of Drive paths.

[Don W]

If the share wasn't available, the protect files on missing drives option should have protected the files, I would expect.

The share is mapped to drive D. I have auto import set up to import audio files from D:/Media/Audio. That is the only auto import for audio. If the share wasn't available, drive D wouldn't exist, as there aren't any other drives in the system to take its place. Except possibly a thumb drive. I suppose if there was a thumb drive plugged in the system AND the NAS was unavailable, then drive D could exist and MC might assume files were non-existent. That in itself seems unlikely, but lets say possible. It still doesn't explain how only some of the files were marked non-existent and not all of them. I guess the thumb drive could have been removed while MC was in the process of marking files as non-existent, making it so drive D was missing so the protect files from missing drives rule kicked in at that point. I don't recall using a thumb drive in the system any time lately, but that doesn't mean 100% for sure I didn't. It all seems unlikely, but does seem to be the only explanation possible...

I'll be watching carefully to catch it as soon as possible, if/when it happens again, and try to get more data.

[MrC]

When things are not working as expected, it doesn't make much sense to use those very rules to exclude what might be going wrong.  Your proposition is that something seems like it might not be working correctly.  My questions and suggestions are meant to help ferret out what is going wrong.  To do this, we try to use the basic premises of what the system tries to do, and then test those hypothesis.

This situation is difficult to diagnose, because the only thing to go on is your statement that your files are being re-imported.  And unless a reproducible case can be made, it will be nigh impossible to find and fix.

I can come up with several possible explanations for why a subset of your files appear "re-imported", but I don't think that's the interesting factor, so I've ignored that.  It is sufficient enough that some files seem to have been re-imported.

[Don W]

My proposition, which is not a proposition but a simple statement of fact, is that 10000 files were re-imported, causing me to lose all the play stats for those files, and I don't know why.

In my previous post I put forth my own personal theory of what could have been the cause of this. Did you notice that my theory doesn't in any way say that MC did something wrong? I don't think there is a any other explanation possible which doesn't in some way "blame" MC.

If you can think of "several" reasons why 10000 the files were 1) removed from the data base and 2) imported again into the data base with 3) exactly the same file names as before, then I REALLY would like to hear them. It might not seem important to you, but it is to me, as I would like to prevent this from happening again.

[JimH]

Cosmic ray
NAS bug
Unexpected shutdown
Virus
Disk controller
Other hardware or driver problem
Act of God
NSA

Back up your library regularly and restore it if you have a problem.

MC makes library backups automatically.  You might be able to restore one now.

[MrC]

@ Don W, from your post last night (since edited), I see I've offended you.  No offense intended, and so I'll apologize.

My aim was to re-frame the discussion towards devising steps to reproduction.  In the end, if JRiver can't reproduce it, the matter will wither on the vine.

[joechip]

I've been having this same issue.  I have video files on a remote laptop drive which I have mapped to my main computer which is running MC19 (as drive Q).  I've set the auto-import function up, and configured to protect network files. I've been a bit a bit perplexed occasionally when MC forgets the files on this mapped drive, usually after the laptop has been in a sleep or shutdown state for a time, and I have to reimport all of those files. My solution, which I'm testing right now, is to configure MC to access that drive through the network directly rather than using the mapped drive (Q, on my system). I think MC just sees the mapped drive as another local drive (even though it is obviously a network location) and treats it as such, which is why the protect network files is failing. Will let you know how it goes.

[nitephlight]

same issue here, number of plays is always suspiciously low but didn't pay heed till i had had to start re rating items i could have sworn were done (file I've had for years).

i take very regular backups, short of restoring, How would i selectively update the current library where the backup copy has > plays or tags?

[joechip]

So the test failed. With the auto-import configured to protect files on missing drives I still lost all of the files on the network location while it was unavailable and had to re-import.

[Matt]

So the test failed. With the auto-import configured to protect files on missing drives I still lost all of the files on the network location while it was unavailable and had to re-import.

Could you describe your setup in a little more detail, including a full filename as seen by MC?

Thanks.

[joechip]

Could you describe your setup in a little more detail, including a full filename as seen by MC?

Sure.

I'm running a desktop PC (Windows 8.1) as the primary machine/server with the latest build of MC19. It's connected via HDMI to a 24" Benq monitor (primary display) and a 40" Samsung TV (secondary display). I have a laptop running MC19 as a client. The server accesses quite a few local drives on the desktop, and one shared drive on the laptop that I store some video files on.  The laptop isn't on all the time. 

Here is a full filename, including path, of a typical file on the laptop (as seen by the MC19 server) :  \\MECHAGODZILLA\Movies\The Mindy Project\Season 1\the.mindy.project.mp4

I have configured the auto-import function to include the shared drive on the laptop, to update for external changes, and to fix broken links ("yes: protect files on missing drives").

As I mentioned, when the laptop is off the server removes the files from the library that are on the laptop drive. After the laptop is back on the server reimports the files.

I originally had the laptop drive mapped as a local drive on the desktop, and had the server access the drives through that mapped drive, and had the same experience.

[joechip]

Any ideas here?