Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[glynor]

Recently, fitbrit and I were having a discussion about some issues getting MKV files to build their thumbnails and the topic shifted a bit to reference a serious problem I've been struggling with for quite some time (effectively since MC12's Auto-Import tool reached its current design).  Here's my new thread detailing this problem... I've mentioned this before but never got much traction. 

The problem is that if all of these things are true:

(1) You do Rename Files From Properties on a large batch of files.
(2) The result of the Rename will move the said files from one drive to another (meaning it is more appropriately a "move" operation than a "rename" operation).
(3) The tagging operation (move) takes a long time to complete.  This can be caused by a number of factors:  Either the source or the destination drive are slow (external or network). OR you are simply moving a whole bunch of big files.  Both of these factors are extremely common with my video files (which are often big, and often need to be moved from one drive to another).
(4) Auto-Import is enabled and is monitoring the destination folder.

The result is:

Everything gets all messed up.  A seemingly random selection of the files will get "Auto-Imported" by Otto in the new Destination directory, while the Rename From Properties tool is still running in the background.  This causes large numbers of the files to lose ALL of their tag data, because it effectively re-imports them from scratch (and MC does not support saving tags in-files for videos).  When this happens, only some of the files get broken.  I think the ones "early" in the move operation work fine, but all of those that take longer to finish moving than whatever the Auto-Import trigger "time out" is get re-imported and therefore "broken" (all tag data erased).

There is no way to undo this, once it happens (because even restoring a backup library is useless because the files in question were moved).  Often, when you first realize that it is happening, you can sit there and watch the problem get worse and worse.  I've had Auto-Import "import" 50 or so files, realize that it was happening and messing my files up, and not being able to do anything because the Rename Files From Properties tool still had 400 more files to "move", and I know it is going to break them too!  I've only personally experienced this using the Rename Files from Properties tool, but I'd guess that it would apply equally to any slow Move operation, including manually tagging the [Filename] field, or using the Library Tools --> Move/Copy File tool.

This CONSTANTLY is a major pain.  The only work-around is to manual disable Auto-Import before I use the Rename Files from Properties tool.  I try, but it never fails that when I'm tagging my video files, and I finish tagging a whole series full of TV shows and then go to move them to their "final destination" using Rename Files from Properties, I forget and I end up having to start essentially all over again.

The solution seems simple.  MC should automatically suspend Auto-Import whenever a tagging operation that affects the [Filename] tag is still in progress, re-enabling it only after the tagging is completed.  This would solve this problem completely.

[Alex B]

I don't have that problem because my Auto-Importer is switched off. But that's me. I understand your pain.

[Matt]

We'll take a look.

Auto-import should step-down during any tag or move operations.

[rjm]

Auto-import is like UAC on Vista. Turn it off and life is sweet.

[Matt]

Feel the love.

[rjm]

No disrespect intended. Anything of substance that runs in the background is bound to cause problems from time to time. JRiver's auto-import is much better than most background stuff provided by other vendors.

[Matt]

Next build:
Fixed: Improved handling of concurrent import (or auto-import) processes running during large, slow file move operations.

The solution had to be a bit smarter than just disabling auto-import, since you could also run a manual-import during a big file move operation.  Instead, it tracks files that in the process of being tagged (both their old and new filenames) and nicely handles them in the import code.

[glynor]

The solution had to be a bit smarter than just disabling auto-import, since you could also run a manual-import during a big file move operation.  Instead, it tracks files that in the process of being tagged (both their old and new filenames) and nicely handles them in the import code.

Boom-shaka-laka!  Thanks a lot!  This solves one of my huge long-standing problems with Otto.

Oh and...

Feel the love.

I didn't at first, but you've really won me over.  I rarely disable it.  It works quite amazingly well.  To everyone: If you haven't tried it, put your fear aside (but back up the library first) and give it a whirl.  It works quite well if used appropriately.

[rjm]

I seem to be an abnormal member of our species. My philosophy is that nothing gets added to my library unless the tags/filenames are perfect. Otto conflicts with my religion.

[glynor]

I seem to be an abnormal member of our species. My philosophy is that nothing gets added to my library unless the tags/filenames are perfect. Otto conflicts with my religion.

See... that is odd (from my religion's point of view).  What do you use to tag and fix the filenames?  That's what MC excels at above all else!

Or do you just "view and edit the new files in MC" but not actually "import" them until they're all tagged and ready?  I've never understood the distinction between these two ideas... I import everything and then use MC's tools to fix everything.  I mean... Importing just "tells MC it exists"... Nothing more special than that.  What difference does it make if you "import" before or after (and it's actually easier my way because you can manipulate the files in more ways in MC's UI if they're "imported" -- like drag-drop and whatnot).  I abhor pulling up Windows Explorer to look at my new media if I can avoid it!

If you have a large back-log of files you haven't gotten to yet (don't we all) and you wanted to exclude these "untagged" files from all your regular View Schemes, you could easily add a User Defined Field called [Tagged] and set it to 1 when you've finished. (And add Step 4 [Tagged]=1 filters to your other Top-Level View Schemes so you won't see the untagged files except for in your specially designed tagging View Schemes...)

[rjm]

Sorry, I should have been clearer...

I of course use MC for tagging and renaming. No other tool comes close to MC's power.

My workflow is:
- acquire new media and put in Work In Progress folder
- manually import media from Work In Progress, anaylze audio, and tag to perfection
- double check previous step
- rename from properties moving files into my media library using standard directory/file name schemes
- backup media to 2 other drives (one on network, one external USB)

With Otto I would have some impure dirty sinful stuff in my library.

[glynor]

I do essentially the same exact thing, only I have Otto monitor the "in progress" folder (which is M:\incoming\ or N:\new\ on my system depending on what it is).  That avoids having to do the import step.  When I open MC, the files are just "there".

That works well for me since I do also use MC for watching video, and sometimes I want to watch the show before I've bothered to take the time to tag it (as soon as it finishes encoding or downloading or whatever).

[rjm]

I understand. I have the same need from time to time and simply navigate to Work In Progress via Drives and Devices where you can play media without it being imported into the library. My approach avoids having any bad/incomplete tags in the library at any time.

[JimH]

Should be improved in 12.0.400.

[hit_ny]

I seem to be an abnormal member of our species. My philosophy is that nothing gets added to my library unless the tags/filenames are perfect. Otto conflicts with my religion.

That's pretty similar to the way i do it, but its getting tedious to do.

So...

If you have a large back-log of files you haven't gotten to yet (don't we all) and you wanted to exclude these "untagged" files from all your regular View Schemes, you could easily add a User Defined Field called [Tagged] and set it to 1 when you've finished. (And add Step 4 [Tagged]=1 filters to your other Top-Level View Schemes so you won't see the untagged files except for in your specially designed tagging View Schemes...)

This is becoming more appealing and trying to move towards it.

But i don't need auto-import for this. Just drop the new unfinished stuff in a to-process folder and run a manual import. Then move to a to-listen folder and schedule AA or tagging whenever appropriate via viewschemes.

What i wish was possible was a way to save album mpls with relative paths to the respective album folders once i'm done, doing it manually takes a while.

[glynor]

True, you don't need the auto-import, but it works just as well.  Then when you first open MC, or as soon as you drop new files in there, they just "appear".  If you have View Schemes limited to [Tagged]=0 (and maybe with sort=[Date Imported],d) the files would just always "appear" in your "Incoming" View Scheme.  When you get to them you get to them.  When you have the time to work on tagging new files, you just open MC and work on it.

The other nice thing about this method is that you don't have to make sure that ALL your new files go to the same incoming folder.  For example, when I encode my own video files (from my TV recordings) they're always already named exactly how I want them (I set the filename properly with the encoding application).  So there's no reason not to just move them directly into the proper place in my media directory.  I do that, Otto finds them and then I do Fill Properties from Filename, fix a couple of other tags, and they're done (I don't need to do Rename From Properties).

It certainly doesn't work for everyone's work-flow.  However, if you haven't tried it "just because it doesn't feel right", it is worth reconsidering your pre-conceived notions about the work-flow.

[hit_ny]

The other nice thing about this method is that you don't have to make sure that ALL your new files go to the same incoming folder.  So there's no reason not to just move them directly into the proper place in my media directory.  I do that, Otto finds them..

So Otto has to go through all your stuff to find them ?

stuff that might be spread on several partitions and possibly external drives.

Background processes, whilst i'm already listening and possibly simutaneously tagging something and what happens if MC crashes for some reason.

hmm  ?

i don't know whats been imported or not.

Re-run import and if Otto missed it then this time around it will defnitely get it.

[JimH]

So Otto has to go through all your stuff to find them ?

stuff that might be spread on several partitions and possibly external drives.

Background processes, whilst i'm already listening and possibly simutaneously tagging something and what happens if MC crashes for some reason.

You can specify as many or as few folders to search (or exclude) as you want.  It happens in a background thread.  It's not even noticeable.

[glynor]

So Otto has to go through all your stuff to find them ?

I was very skeptical too, but yes, it works.  I have stuff spread all over different disks, removable and network and local...  Otto is way in the background and doesn't consume much resources.  I think it exploits the Windows File Change Windows Messages to only "search" in a targeted manner when it is notified that something has changed in a specific folder.

Either way, it is nothing like doing the full manual scan every 5 minutes or whatever you might be thinking of.  It is much, much slicker than that.

[hit_ny]

As i play with this mass import idea, i think i've hit a snag.

There's no way to get a meaningful list of recently imported albums !!!

I can use the [Last Played] field to mimic this to an extent as usually listen to new stuff in sequence so this is fine. Course if i ever listen to an older album then the order will be off.

So i thought why not do a Move/Copy field

[Last Played] --> [Date Imported]

[last Played] does not show up in the pull down of the Move/Copy field dialog as a selectable field and [Date Imported] field properties show it can't be edited.

Its possible to do this with mpl but its a hassle i'd prefer to do without.

If you import a bunch and they get put in a to-analyse folder, the import date is when they were imported. Then AA and then listen, which might happen at anytime and in any order. I tried the modify date but then i don't save tags back to the file so the modify date is when the file was created and that may be weeks or months ago. Even if you did save tags to the files the modify date won't always be a reliable indicator of what was recently added to the library.

All the filter in the viewschemes does is show them as added or not based on their location, if they are on this partition then hide else display as part of the library.

I can't as of yet think of a way to show something like a recently imported view scheme that i have when i used to import a small bunch at a time.

[glynor]

There's no way to get a meaningful list of recently imported albums !!!

I'm not sure what you are looking for at all.  Could you explain further exactly what you want to list, and in what order?

[hit_ny]

A viewscheme that shows recently imported with thumbnails in the file list view, in the order that the albums were imported in reverse order. So the first thumbs show the last albums imported.

I usually only import into the library, a small bunch after i finished listening to them so the import date always shows when they were imported/added to the library.

My point was if you do a mass import and listen later, a recently imported viewscheme won't really work as the import date may be very old by the time you get around to listening to them. And if you only move to final destination after listening there is really no way to tell when the album was 'added' to the library.

its already imported, moving it from the to-listen folder to the final destination will show it in the library viewscheme as those areas wont be filtered out

[hit_ny]

*bump*

[glynor]

Sorry... I've only just gotten back from my "vacation" for the holidays.  I say vacation only in the loosest sense of the word though, as I'd have much preferred to have been home and at work (and I'd be much better rested if so).

To top it off, I'm nursing a wicked hangover (in its second day) and I have yet another foot of snow to deal with in my driveway (and I only cleared out the previous 9 inches or so yesterday morning which didn't do much to help the hangover).

So.... Bump this again if I don't respond in the next day or so.

[benn600]

Excellent solution glynor!  I'd suggest trying it hit_ny!

[hit_ny]

What i'm getting at is this...

Say you have a bunch of albums to import
a-album
b-album
c-album
d-album
.
x-album
y-album
z-album

I think if you import them all, then MC will be import them in alphabetical order.

After listening, albums are moved out of the to-import folder to their final destination and become visible in the album audio viewschemes (as the to-import folder is filterd out).

Now if you listen to albums x,y,z, then e,f,g and a few days/weeks later, you get to albums a,b & c. Moving them out as you go along..

..the recently imported viewscheme will show, in order of most recently imported

z,y,x......c,b,a

(it too has the filter mentioned previously).

But you listened to albums a,b,c after x,y,z so why are albums x,y,z being displayed as most recent.

So the recently imported viewscheme in a way does not reflect anymore, the sequence how you added to the library & listened to those albums. So i tried the [Last Played] field to get the same order that i was used to have when i imported only a few albums at a time and that too only after i listened to them.

By importing only a few albums as you go along, the [Date Imported] field does double work
- showing the most recently imported albums
- as well as the order you listened to them. (close enough)

Importing a bunch of albums in one go changes this. The recently imported viewscheme i think becomes less useful as you might have imported 10s of albums in one go if not more.

Using [Last Played] to do the sorting isn't always optimal as you might choose to listen to an older album and then it will appear in that scheme as well.

So i'm left with no reliable indicator to show the order in which i listened to newly imported albums  ?