Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[glynor]

In another thread, maid just asked the obvious question.  I'd been planning to write this up since they introduced the wonderful new Tag on Import feature, and had never gotten around to it.  There's no time like the present.

sorry to be a newb but can you please explain how to insert the rule for Hawaii Five-0

No worries.  It is a bit weird.

I'm going to start with a simpler example.  Instead of Hawaii Five-0, I'm going to use an episode of CSI recorded by SageTV as an example.

Side Note: I used SageTV recordings as the example files in these instructions, but the ideas behind this are really generalized.  Any time you know something ahead of time about the filename of future files themselves, or about how Carnac will interpret those types of filenames, you can use a similar system to get whatever resulting metadata you want.

Tip: Carnac is the name of the new "logic" in MC 17's auto-import system that looks at the filename of files it imports, and it tries to "figure out" what the file likely is based on that filename.  It knows common naming patterns like "s06e19" mean the file is Season 6, Episode 19 of some TV Show.

CSI Example (simple):

So, assuming your files come in with filenames like this:
CSICrimeSceneInvestigation-S12E10-GeneticDisorder-7091620-0.ts

"Carnac" (which is MC's new auto-tagging feature) will set the following:
[Media Sub Type]=TV Show
[Series]=CSI Crime Scene Investigation
[Season]=12
[Episode]=10

All of that is right, except that it is silly that the episodes of CSI are called "CSI Crime Scene Investigation" (a bit redundant, like when people go to an ATM Machine).   So, you want to turn the [Series] tag automatically into "CSI" instead.  What you need is a rule that replaces the [Series] tag, with something you can calculate using the stuff you already know at import time.

So, open MC's Auto Import options, and edit the settings for the folder where you want these rules to apply.  In the lower section of the dialog, add an "Apply these tags" rule:

1. Choose Custom as the type to open the editor.
2. In the Field box, you want to choose Series.  That's because the goal is to replace whatever MC would normally decide [Series] should be, with the results of an expression instead.
3. In the Value box, add the following expression:

Code: [Select]

If(IsEqual([Series],CSI Crime Scene Investigation,1),CSI,[Series])


That will do it.  To be clear, this does NOT go through your existing library and fix files already there, it only impacts new files that Auto-Import picks up (though that same expression would work if you applied it to your whole library).  Also note, it has to be Auto-Import that does it (though you can use "Run Auto-Import Now" from the Tools menu).  If you import files by drag-dropping them onto MC itself, or from the context menu in Windows Explorer, these rules don't get triggered.

Why does it work?

The first part says If(<TEST>, <is True do this>, <Otherwise do this>).  So, this breaks down as:

1. Test: IsEqual([Series],CSI Crime Scene Investigation,1)
That just tests if the [Series] tag matches "CSI Crime Scene Investigation".  The number 1 at the end, tells IsEqual to work in "case-insensitive string compare for equality" mode.  There are 9 possible modes for the IsEqual command.  You can do numeric greater than and less than tests, substring searches, and other things too.

2. True Result: CSI
That's pretty simple, if your test comes back positive, you want to fill the [Series] tag (the field you picked in the top box) with "CSI", so you put that in that spot.

3. False Result: [Series]
This is important, these rules apply to every file you import from that directory.  If, instead of putting an expression in that Value box, you just typed "CSI" in the value box, then no matter what the file MC imported from that directory was called before, the rule would override it and set [Series] to "CSI" every time.  So, in this case, you want the result if your test is false, to just keep [Series] set to what it was set to before and move on.  To do this, you put the tag in the False spot of your If() statement.

Hawaii Five-0 Example (escape characters):

So, moving on to the Hawaii Five-0 example.  I saved this because the text we want to put in the True Result part of the If() statement happens to contain parenthesis.  Parenthesis alone will mess up the expression.  So, assuming you have files like this one:
HawaiiFive0-S02E12-AlaheoPauole-7091569-0.ts

Carnac will set the following:
[Media Sub Type]=TV Show
[Series]=Hawaii Five 0
[Season]=2
[Episode]=12

You want to turn the [Series] tag automatically into "Hawaii Five-0 (2010)".  So, add a [Series] Tag on Import rule with its Value set to:
If(IsEqual([Name],Hawaii Five 0,8),Hawaii Five-0 /(2010/),[Series])

The "/" character inserted before those parenthesis are escape characters.  That tells the expression engine "this next character is just a text character, don't treat it as anything special".

[glynor]

Essentially all of my Tag on Import rules use some variation of that same expression.

They start with "What field do you want to change?" = the Field box.
Then, some variation of this in the value box:
If [This Field] contains/is equal to "this text", then set the value of the field I specified above to "this instead", otherwise, leave it alone.

That is written like this:

Code: [Select]

If(IsEqual([Series],CSI Crime Scene Investigation,1),CSI,[Series])

[glynor]

Files that Don't Already Have a Good Series (setting more than one value):

Okay, I figured I'd explain one other handy tip.  One thing I often want to do is override the [Name] field when a show often might end up tagged kind-of stupidly by Carnac.  This is particularly useful for things that don't really have meaningful episode names, but where you can fill it with something useful yourself (usually a date), like for a News program.

For example, I have Sage record The Daily Show every night there is a new airing.  These sometimes come in with episode titles set to the interview guest's name, and season and episode numbers.  Then Carnac can do a decent job (though the guest name isn't always very useful).

But, just as often, the filename comes in with no episode title or season/episode info in it at all.  When this happens, Carnac can't extract the other metadata properly, so you just end up witha [Name] field filled with whatever the filename was (like the old way).  So, a filename like this:
TheDailyShowWithJonStewart-6935715-0.ts

Carnac spits out:
[Media Sub Type]=TV Show
[Series]=blank
[Season]=blank
[Episode]=blank
[Name]=The Daily Show With Jon Stewart

Carnac doesn't know if that is the name of the show, or the name of a Movie or the name of the episode.  There just isn't enough information there in the filename.  So, it just dumps what it finds into the [Name] field.  But, I know that any recording that says "The Daily Show" anywhere in the filename, is a recording of The Daily Show.

So, for these, I have two rules.  The first one is a [Series] rule, much like the ones above:

Code: [Select]

Rule Field: Series
Value: If(IsEqual([Name],The Daily Show,8),The Daily Show With Jon Stewart,[Series])
That makes sure that the [Series] field is always set correctly.  It says "any file that comes in with a [Name] tag containing "The Daily Show", change the [Series] tag to "The Daily Show With Jon Stewart" instead.  The number 8 at the end of the IsEqual does a substring (case insensitive) search.

Then, I have a second rule to actually set the [Name] tag:

Code: [Select]

Rule Field: Name
Value: If(IsEqual([Series],The Daily Show With Jon Stewart,1),FormatDate([Date Created,0],Date,Unknown Date),[Name])
This one is a bit more complex, but basically, it just sets the [Name] field for any "Daily Show" recording to a date like this: 11/20/2011.  The unknown date bit isn't used, that would only happen if there was no valid Date Created on the filesystem, and the [Name] at the end is a If False passthrough of the existing Name information for all of the non-Daily Show files, just like above.

One last thing... Order matters!  It is important to have the second rule come after the first rule in the list.  These are evaluated from top to bottom.  So, before you can check if the [Series] field contains exactly "The Daily Show With Jon Stewart", you have to first make sure that's what it is set to.

[maid]

Thank you very good explanation.

[glynor]

I thought I'd add two other little general expression building tips:

Readable Expressions and The Expression Editor:

Whitespace and carriage returns around "components" of an Expression are ignored.  This is nice because it makes it easy to build the expression using something that looks more like a C program, which is a bit more readable.  In most places in the MC UI where you can enter an Expression, you can click on the little "disclosure triangle" doodad and pick the Expression Editor.  This is a nice big, resizable window, so you have lots of room to design your expression.

So, when you're typing these things out and tweaking them, it can be helpful to type it like this instead of all on one line:

Code: [Select]

If(
   IsEqual([Series], CSI Crime Scene Investigation, 1),
   CSI,
   [Series]
)
Or, for a more complex example:

Code: [Select]

If(
   IsEqual(
      [Series],
      The Daily Show With Jon Stewart,
      1
   ),
   FormatDate(
      [Date Created,0],
      Date,
      Unknown Date
    ),
   [Name]
)
MC treats these the same as the all-on-one-line versions.  It makes the expressions seem less like Perl/Magic and more readable for normal humans.

[glynor]

Secondly...

Using an Expression Tester Custom Field and View:

Something I found extremely helpful while building my Tag on Import rules was having a way to test the expression quickly before I actually put it into the Auto Import settings dialog.  This is pretty easy to do with a custom Expression Tester field and view.

1. Go to Options -> Library & Folders and click the Manage Library Fields button.
2. Add a new field and call it Expression Tester
3. Choose Calculated data as the Data type for this field and click OK.

Then...

4. Make a new view somewhere in the tree.  It would probably be useful to just duplicate one of your existing views that has nice sorting and panes and whatnot.  I used my "All Media" view which has the File Location as one of the available panes.
5. Call this view "Expression Tester".
6. Add a new column to the Details view, all the way over to the Left, and choose your new Expression Tester field as the column.

Then...

7. When you want to test an expression, enter it into the Calculated Data section of the Expression Tester field (just go back into the Manage Library Fields dialog again and select Expression Tester from the list on the left).
8. Edit your expression as you want in the Expression Editor for that field.
9. Check the results in the Expression Tester column of the view you just made.
10. Once you get it "right" then just copy/paste the expression itself over where you want to use it.

EDIT:  An alternative way to accomplish the same thing is to use an Expression Column instead of a calculated field and then displaying that field in a view.  I use the Expression Tester field because I have a few different views, with different panes, for my Expression Tester, and I want them to all use the same expression in all places.

[MrC]

I started to reply to the original question last night, but could not auto-tag to change Series, so didn't want to post a response w/out my test working.

The expression I had, which worked fine in an expression column, simply would not be changed upon import:

  If(IsEqual([Series], Hawaii Five 0,1),Hawaii Five 0 /(2010/), [Series])

I imported a movie file (mpg, wma), tagged Series with "Hawaii Five 0", updated the file tags (sidecar for mpg, in file for wma), and then deleted the file from MC.  Unchecked was Skip for files that already have a value in this field.

Upon re-import using Tools > Import, the Series never changed.  The expression column was correct.

Any ideas why these didn't work?

[glynor]

Did you not delete the auto-created JRSidecar.xml file from the import directory?

I had trouble with that when I was originally building mine.  If you don't delete the JRSidecar.xml files, it will just keep re-using the data from the sidecar file when it re-imports the test file.  To test them using the actual files in the import directory, you have to:

1. Disable Auto-Import's "Auto" mode (or else it'll grab them again before you can do anything else).
2. Implement your new Tag On Import rule.
3. Right-click on the "test file" and choose Locate -> On Disk (External).
4. Delete the file's JRSidecar.xml file from disk.
5. Remove the test file from the MC library.
6. Kick off Auto-Import manually using the Tools -> Import -> Run auto-import now thingy.

I have the exact Hawaii Five-0 rule I posted in my Tag on Import rules already and it works fine.

A side note: I've been recording that show for 1 1/2 seasons now, but I've only ever actually watched the Pilot.  The Pilot was silly, but pretty fun, so I've let Sage keep recording it, figuring that some day it might be a fun show to watch when you're in the mood for some explosions and bikinis.  Has anyone watched it and has it continued in the basic vein of the Pilot, or did it go down the tubes like CSI?

[glynor]

I imported a movie file (mpg, wma), tagged Series with "Hawaii Five 0", updated the file tags (sidecar for mpg, in file for wma), and then deleted the file from MC.

Ahh, I didn't see this.  That's why.  In-file tags and sidecar files (the same thing as far as MC is concerned) always "win".

The order MC applies the tags when importing is like this:

1. Import happens.
2. Carnac is applied.
3. Any Tag On Import rules are applied (top to bottom in the list).
4. In-file and Sidecar tags are applied.

So, what was probably happening was that MC was importing the file, applying Carnac, applying the Tag on Import rules, and then overwriting all of that using your sidecar/in-file tags.

This is wrong (and Matt already told me that once).  See below.

[MrC]

Ok, so the only way to test this would be to import something that ensured Carnac found a Series of Hawaii Five 0.  Otherwise, if there is no Series (db or in-file), the rule couldn't succeed.  Something about this seems oddly uneasy.

I have a file and I don't know what tags are in it, but I know what it is, and I want auto-import rules to set/overwrite.  But according to rule 4 (regardless of setting Skip for files that already have a value in this field) that can't happen?

[glynor]

The order MC applies the tags when importing is like this:

1. Import happens.
2. Carnac is applied.
3. Any Tag On Import rules are applied (top to bottom in the list).
4. In-file and Sidecar tags are applied.

I simplified for brevity.  It works like that in-practice.  I don't actually know what order the things happen in, though I suspect that isn't right because of the "skip for files that already have data" option.  In the end, in-file tags (which includes sidecar files) always win any conflict.  I guess that is the simpler rule to remember.  Order of processing isn't strictly relevant.

Actually, after consideration, I know it doesn't really work like that.  Because you CAN use the contents of the file tags as part of your Tag On Import rules, if you can "trust" that they'll be there.  You just can't overwrite something that is in an actual in-file tag.

So, instead of processing order, the better way to look at it is this:

File Tags and Sidecars > Tag On Import Rules > Carnac

So, it would work if you modified a Field that didn't exist in your tags (a Library-only field, for example), instead of trying to modify the [Series] tag with the rule.

Long-Distance EDIT: This isn't even quite true. Embedded file tags can be modified by Tag On Import rules, but data in JRSidecar.xml files cannot be. See the new and improved Tag on Import wiki article for details.

[MrC]

Ok, I got it to work trivially now.  The only thing that changed was that I bumped up one (beta) version, but there are no changes in the change log that indicate this would have been the cause.  I'm not going to worry about it...

Big Thanks.

[glynor]

Another Tip for the thread...

How To Auto-Generate JRSidecar.xml Files Using a Tag On Import Rule:

This is another handy tip.  Because I whined incessantly, they set the feature up so that whenever you import a file that "triggers" a Tag On Import rule, those tags are written out to the JRSidecar.xml file (and one is created if it doesn't already exist).

This is handy if you want to make sure that all of your files that get imported get a JRSidecar.xml file created right away.  An easy way to do this is just to make a custom field and then write a value to it for all files that get imported.  For example, in my Library, I've added a MakeSidecar custom field (integer type, but it doesn't really matter).

Then, I have a Tag On Import rule for all of my watched folders where videos might show up that is very simple:

Code: [Select]

Rule Field: MakeSidecar
Value: 1
Since you've written a value to a field, MC will create the JRSidecar.xml files for videos as needed.  This is very handy if you want to use those sidecars for external processing, or if you just want to be able to move the files to a different system, with their auto-generated metadata intact, without having to remember to manually "Update Tags (from Library)" first.

[jgreen]

Nobody likes a know-it-all.

[darichman]

Glynor - I've only just noticed this thread, and wish to say: thanks!

There's at least a dozen new and useful insights and tidbits I'll be using for sure.

[glynor]

Nobody likes a know-it-all.

Hah!

One thing I don't know how to solve is how to automate the Fill Properties from Filename style of parsing from this new tool.

Say you have your music files all organized like this:
M:\audio\music\The Black Keys\El Camino\1 - Lonely Boy.flac
M:\audio\music\The Black Keys\El Camino\2 - Dead and Gone.flac
etc

In other words, in the Rename, Move, and Copy tool, you used a rule that generated M:\[Media Type]\music\[Artist]\[Album]\[Track #] - [Name]

How do you "reverse this" like you can in the Fill Properties from Filename tool, using the Tag on Import rules system?  I don't think it would be easy if you don't know ahead of time what specific values you are searching for, like in my examples above.  I don't have a real reason to do it for music, per-say, but I do have a bunch of places at work on my network shares where it would be convenient to parse a directory/filename structure for metadata on a on-going basis.

MrC might be able to figure out some kind of RegEx magic voodoo to solve that problem, though.

Basically, I have a few big network shares at the office where different recording systems (like DVRs, but for live seminars and events) save their videos when they're done recording them.  I have MC auto-importing these UNC shares.  But, the filename structure the "DVRs" use contains a bunch of useful metadata, like [Location], [Date], and stuff like that.  If I could easily, and automatically, parse this info, I could include even more metadata in the file/folder structure.

But the problem is that I don't always know ahead of time what the possible value lists for the fields might be, so I don't have anything too look for with my IsEqual() function.  Location is easy, since the systems (usually) don't move.  But, I could conceivably include a [People] tag in the filename structure, and then have that field populated in MC.  I could do this manually with the Fill Properties from Filename tool, but I need it to happen automatically on import (or I'll never remember to do it, and then I can't count on that metadata being useful).

[MrC]

MrC might be able to figure out some kind of RegEx magic voodoo to solve that problem, though.

I can, I can!  What are you willing to trade? :-)

It is a little repetitive currently, as you have to do it piecemeal, one field at a time.  Although I'm bound by silence, that's what request #1 is for.

To do it now, as long as your filenames have a relatively consistent, detectable pattern, you'll just pull in the pieces you need using the same regex() expression in each auto-tag rule area, but outputting only the pieces you want for any given tag.

See this thread for basically the same help request.  Instead of output all the [R#] values, output only the one you want for a given field in a single auto-tag rule.

Do you need help writing the expression?

[rick.ca]

It is cumbersome currently, as you have to do it piecemeal, one field at a time.  Although I'm bound by silence, that's what request #1 is for.

I vote your vow of silence be revoked. And while this would be handy for the Fill Properties from Filename tool, it already has an alternate method for setting multiple fields. Auto-Import Rules do not. Carnac might be displeased, however, if made redundant by a regex expression.

How might this work for rules? I suppose it would have to be a special kind of rule (in that it fills multiple fields), but could be handled within the same UI as for the existing rules. Or if the variables of the Regex() remain available, the expression could be entered first (as a "Regex" instead of a particular field), with subsequent rules assigning variables to individual fields. Either way, that would be preferable to somehow providing for a regex expression separate from the existing rules.

[jgreen]

FWIW Glynor, if you check the Google translator the phrase "Nobody likes a know-it-all"  translates into "Thank you for your contributions, your wit and humor, and I am even further in your debt."

[MrC]

How might this work for rules?

Let's create a rule that pulls from [Filename (name)], say to auto-tag some scheduled recordings for which we have useful names, but no tags yet.  They are in the form:

   Program Name - DJ - Year

The expression, simpler than the one this other thread,

   Regex([Filename (name)], /#^(.+) - (.+) - (\d{4})#/, 1)

would capture all three values, and outputs the first (argument 3 set to 1).  Use this rule to create 3 auto-import rules, one for Name, one for Artist, and one for Year.  But change Regex()'s output of the captures from 1 to 2 to 3:

   [Name]: Regex([Filename (name)], /#^(.+) - (.+?) - (\d{4})#/, 1)
   [Artist]: Regex([Filename (name)], /#^(.+) - (.+?) - (\d{4})#/, 2)
   [Year]:  Regex([Filename (name)], /#^(.+) - (.+?) - (\d{4})#/, 3)

btw. It occurs to me it would be very useful to be able to copy/paste all three rules above into the auto-tag rule area, and have it enter 3 separate rules.  This would allow for easy entry of a bunch of rules in one fell swoop.

[MrC]

FWIW Glynor ... and I am even further in your debt.

I understand beer donations are appreciated.

[glynor]

I understand beer donations are appreciated celebrated.

T, FTFY.

[fitbrit]

Just adding another 'thank you, glynor' to the thread.

[rick.ca]

Use this rule to create 3 auto-import rules, one for Name, one for Artist, and one for Year...

Yes, I realize the same rule could be entered for as many fields as necessary. My question had to do with the possibility of accommodating one expression and many fields. This would be particularly slick for the situation where all the files in the import folder are consistently named (e.g., a \Video\Series folder). But with most users relying on Carnac, and probably few being comfortable with regex anyway, it doesn't make sense to depart from the existing UI.

[MrC]

In some other thread, I suggested some syntax such as:

  (var1,var2,var3)=val;val2;val3

which is essentially a list-context assignment.  I could see generalizing field assignment from single field to single field/multiple field.  Since there is no UI location where this can be entered, one possibility could be a command-entry area that can be opened and command entered (perhaps similar to Directory Opus' > or ? command entry area).  Then, single field or multiple field assignment could be done something like:

  field1=val
  (field1,field2,field3)=val;val2;val3

Of course,

  =val

in a field in the tag action window, or pane field, would work as usual.

With this type of syntax and capability, then auto-tag rules could be specified in like-fashion.

    (Name,Artist,Year)=Regex([Filename (name)], /#^(.+) - (.+) - (\d{4})#/, -1)[R1];[R2];[R3]


This then marries the idea of the Templates or proposed Regular Expression mode in Fill Properties From Filename with Auto-tagging rules.

But these are just ideas...

[rick.ca]

Since there is no UI location where this can be entered...

I suppose there could be a Field List... item at the top of the field pick list that would then allow multiple fields to be selected (maybe just by converting the list to a check-box list). Then the Rn list in the expression would be put in a matching order. It would be safe to leave it up to the user to get the matching right. Nothing is going to blow up if it's not.

As an import rule, the Skip for files that already have a value would have to be an all or nothing thing. That's fine, as separate rule can be used if not all the fields are to be handled the same way.