Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[rjm]

I upgraded from 12.0.147 to 12.0.151 today.

The first thing I always do after upgrading is run Auto-Import Now which is the only method I know of for checking the integrity of my database. If it reports that nothing has changed except my DVDs (which is a known bug) then I know everything is ok.

In this case it reported 17,000 of my 99,000 files had changed. And as a consequence, for all those media types that do not store internal tags (like docs and videos), I lost all of my database information for them.

I immediately assumed a major bug in 12.0.151 but before reporting it I thought I should do some due dilligence.

I first compared all my media files against my backup and confirmed that none of them had changed.

I then reinstalled 12.0.147 and confirmed it also thought 17,000 of my files had changed.

I then restored a database from about a week ago and the problem was corrected. Auto-Import now reports that nothing has changed, as it should.

So it appears that my database was somehow corrupted over the last week. I use MC every day and it worked fine over the last week and I did not notice anything strange. Whatever got corrupted only seems to have affected the database's understanding of when a file changes on my system. In other words, I think the field data, views, etc. were ok.

I inspected the details of the Auto-Import log on the 17,000 files that supposedly changed and I do not see any patterns. It appears to be a random mix of all files types (MP3s, docs, video, images).

I am extremely careful and cannot figure out what, if anything, I did wrong. I did have one nasty BSOD a couple days ago but do not think (but not sure) that MC was running at the time.

Any ideas from the gurus out there on why MC would suddenly think 17,000 of my files had changed when all else seemed to be ok with my database?

Thanks, Rob

[hit_ny]

I am extremely careful and cannot figure out what, if anything, I did wrong. I did have one nasty BSOD a couple days ago but do not think (but not sure) that MC was running at the time.

I'm inclined to think this might have been the cause, provided MC was running and was busy at the time. Do you remember if at the time, there was any music playing in the background

I've had library corruptions happen usually if the above is true. If MC was busy updating a field and a crash occurs, sometimes all entries for that field in the library will be blank.

This is not neessarily a bug per se, when a crash occurs (for whatever reason) there isn't any time to close gracefully, MC is no longer present. So MC saves frequently, "Saving Database" in the status bar. I don't even rely on this, instead preferring to make my own deliberate backups at certain milestones, ie i just finished an import verfied everything went ok. Thing is with auto-import i no longer am in control of when these events occur so i disabled it completely. Not knocking auto-import, its certainly useful, but have not figured out a way to work it into my routine as yet.

If i have crashes, i restore library from backup.

I have smartlists that check for basics, like file count etc, presence (ie not empty) of essential tags etc.

Interesting idea about auto-import to check library integrity, out of curiosity, how long does MC take to auto-import nearly 100k already present files ?

[rjm]

hit_ny, thanks for reply.

I do not think MC was running when my system crashed but it may have been. If indeed it was, I can understand why the database may have been corrupted.

Like you, I also have turned off Auto-Import. I follow a very deliberate workflow and nothing gets added to my database until all the tags and directory/file names are perfect. I do not like anything changing my database in the background. I have many thousands of hours of work invested that I am unwilling to risk.

I back up my database at least once a day by synchronizing it to another network drive. The problem with this approach is that if I do not detect an error, as occured in this case, the problem just gets replicated. I think I will start doing more of what you do and create periodic snap-shots using the built-in backup feature.

I really like the auto-import method of checking database integrity because it also confirms that none of my media files were accidentally changed. With MC 12 the speed is amazing. It takes only 51 seconds to scan my 99,000 files (assuming all thumbnails are already built).

[hit_ny]

I think I will start doing more of what you do and create periodic snap-shots using the built-in backup feature.

File->Library->Backup Library

(preferably to a different partition on a different HD, (optionally a different machine even) to the one that MC resides on).

If you do several saves a day, the earlier copy for that day will be overwritten by default unless you manually rename the file.

I really like the auto-import method of checking database integrity because it also confirms that none of my media files were accidentally changed. With MC 12 the speed is amazing. It takes only 51 seconds to scan my 99,000 files (assuming all thumbnails are already built).

This will work provided :
- MC is the only app that tags/modifies your files.
- you save tags back to the files.

Reason i think its so fast is that MC only compares file modification times, if there is a mismatch it imports the files.

i stopped saving tag updates to files a while back (easier to verify file integrity via checksum files), so i doubt i would be able to make use of this technique.

[rjm]

Yes I am sure it is only comparing file (or more probably) directory modification dates.

I like having tag info in the files. If JRiver goes out of business some day and I have to switch to a different media manager it will be much easier to move the media if all the tags are imbedded. This only works if you don't frequently change the tags, otherwise you end up with a media backup nightmare that you are trying to avoid. My approach is to edit the tags so they are perfect prior to import and then never change them. Won't work of course if you use transient tags such as Rating.

[jgreen]

RJ--
Just for giggles, what are your settings under "tools/auto-import options/options . . ."?

Also, is this affected data on a firewire/usb drive?  Or is your network syncing so extra-fancy that it might have represented this affected data as new or removed?

Just running with this, could your BSOD have wiped some network info that would have had the same result?

[rjm]

My settings are:
run auto-import in background = off
build thumbnails = on
fix broken links = yes
update for external changes = on

All the data (library + media files) is on local drives.

I confirmed that none of my media files had changed by comparing them against a backup copy I keep on a network drive.

It is possible the BSOD corrupted my library, assuming MC was running at the time. But it is suspicous since everything else in the library (except info MC uses to determine when a media file has changed) seemed to be ok.

[jgreen]

FWIW, it doesn't sound like your MC library became "corrupted".  I've had a corrupted library in the past, and it simply refused to load.  Rather, I think that your issue is traceable to this setting:

Fix broken links = yes

It sounds like MC suddenly saw those files as detached, and removed the links.  Possibly this is related to your BSOD, in that in recovering afterwards MC was unable to link to those files. Somebody smarter than me may know.

For the life of me, I don't understand what the benefit is to automatically fixing broken links, whereas the downside of it is readily apparent:  done wrong, all your related tag info goes out the window with the link.  Fortunately, MC autobacks fairly regularly.  FREQUENT manual backups, as you've now vowed to do, are the best insurance, IMO. 

[rjm]

Thanks jgreen.

The reason that I have fix broken links turned on is not because I want MC to fix my broken links, but rather I want import now to tell me if anything has changed (such as an accidental change to my media files, or a problem with my media drive, or something weird happening to my library).

In other words, I am using run auto-import now to check the integrity of my library and media files. If it reports nothing has changed I know all is well.

Perhaps I do not understand how fix broken links works. I am assuming that if I turn it off, and I accidentally delete some of my media files (or my media drive develops errors) import won't tell me about the problem.

[jgreen]

Well, this is really the point at which you want to hear from someone who knows what they're talking about, but since none of those guys have posted lately, I'll keep at it.

AFAICT, it was "fix broken links=yes" that removed the library entries (and tag info), and it was auto-import itself that alerted you to it.  So you could change to "fix broken links=no" and still periodically run auto import, as you've been doing.

I run import every now and then just to check, and I usually pick up a few orphans or dupes.  I'm sure there's a better approach for verifiying library integrity, so would somebody please post it?

[hit_ny]

I'm sure there's a better approach for verifiying library integrity, so would somebody please post it?

..yeah i'd be interested to know bout that as well, but to date RJM's method (with caveats i mentioned above) appears to be the fastest way to do it.

But what happens if there is a tag mismatch, between the file's tags and its corresponding library tags. Conceivably, in this case the file tags would update the ones in the library during an import.

So which version of tags is *correct*, the ones in the files or the the ones in the library  ?

What if for any reason a file tag update failed ?

For instance, MC can't write to a file if its playing (filesytem locks the file), instead MC defers the tag update to when the filesystem releases the file (ie the next file in PN comes up or you stop playback). If at this point MC crashes for whatever reason then the value in the library will be more current than the one in the file.

I stopped using fix broken links a long time back. All it does is remove missing items from the library.

A safer way i feel is to use a smartlist to show the missing items and after you can decide whether you want to remove or investigate/fix the missing links (if any).