Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[fitbrit]

I'm helping someone whose library is a bit of a mess in that using different programs to 'clean up' tagging has made things worse and inconsistent.
The audio files from the same albums - all of them classical music- are correctly named and located consistently within the same folder as the other tracks from the same album IN WINDOWS. However, the album name that we want displayed in MC is not necessarily the same as the folder name the tracks are in.
The problem is that within MC, the tracks that should belong to the same album have been split into two groups - those that are correctly tagged, and those that are not (including having empty tags for album, artist track number etc.. This means that, depending on the view, each album is showing up as two different albums in MC, which is to be expected.

What I'd like is some kind of expression for [album], say, that does the following:

If the tracks have the same [filename (path)], and some of the tracks have a non-empty [album] tag (and it is the same entry in those non-empty cases), and some have no entry in [album], tag the empty entries with the same [album] name as the non-empty ones.
This would have to work over multiple different [filename(path)]s at once to be truly useful.

I could then modify that expression to fill out [genre], [artist], [album artist], [year], [composer], [conductor], etc.

Is such a thing possible?

There are 27,000+ files so doing the steps semi-manually is highly undesirable to say the least!

[blgentry]

I understand what you're asking for.  I also suspect that you're rather sophisticated with MC.  But just in case...

You mentioned that the files themselves (file names) are "correct".  What would happen if you simply used the Fill Properties From Filename tool on the ones that have no album tag?  Would it get the same Album tag as the ones that *are* tagged properly?

There's probably a smarter way to do this, as you've described your requirements.  But here's one other idea if the one above doesn't work.  You'd want to experiment with this method on an album or two to see if it works:

1.  Select the tracks from an album (or many) that are tagged properly.
2.  Use the Rename, Move, and Copy tool to write the Album tag into the Directory structure.  I'd just make the lowest level directory the album name, but you might want to do something with Disc # as well if that's an issue.
3.  Select files from the same album (or albums) that are not tagged.
4.  Use the Fill Properties From Filename tool to extract that album tag from the directory structure and write it to the tags of the song.

I suppose if you need song Name, Artist, etc, you could also use the Rename tool to encode that into the file and directory properties, and then extract it back out for the files that don't have the correct tags.  It seems like this would work pretty automagically if you set up your templates for both ends of the operation correctly.

Brian.

[fitbrit]

Thanks, Brian!

You mentioned that the files themselves (file names) are "correct".  What would happen if you simply used the Fill Properties From Filename tool on the ones that have no album tag?  Would it get the same Album tag as the ones that *are* tagged properly?

That is the first thing I thought of, of course, but then determined:

The audio files from the same albums - all of them classical music- are correctly named and located consistently within the same folder as the other tracks from the same album IN WINDOWS. However, the album name that we want displayed in MC is not necessarily the same as the folder name the tracks are in.

[fitbrit]

There's probably a smarter way to do this, as you've described your requirements.  But here's one other idea if the one above doesn't work.  You'd want to experiment with this method on an album or two to see if it works:

1.  Select the tracks from an album (or many) that are tagged properly.
2.  Use the Rename, Move, and Copy tool to write the Album tag into the Directory structure.  I'd just make the lowest level directory the album name, but you might want to do something with Disc # as well if that's an issue.
3.  Select files from the same album (or albums) that are not tagged.
4.  Use the Fill Properties From Filename tool to extract that album tag from the directory structure and write it to the tags of the song.
Brian.

This is the second thing I thought of, but the snag is that MC doesn't just rename the folder the files are in - it will actually create a new folder and move the files - which would ONLY move the selected files, and not solve the problem at all. 

[glynor]

I suspect this will be next-to impossible to do with the Expression engine, because it works at the file level. It is difficult to evaluate items as "groups" (look at the characteristics of all files in an [Album] or [Artist] and do something if X files meet Y criteria).

The new ~expand modifier might be able to help you out, with some multi-smartlist trickery, but it is making my brain hurt thinking about it.

Can you provide concrete examples (filenames, actual tags, etc)? That might help.

[glynor]

The more I think about it... I think you can do something, but I'll need some actual concrete examples to give you a tool.

It still will be somewhat manual (it won't just spit out "tags" and fix them), but I should be able to make it generate an ordered list of items to fix, which you should be able to fix pretty quickly with Pane tagging.

Kinda like this, but with multiple Smartlists to drive the results:
http://yabb.jriver.com/interact/index.php?topic=98324.msg679822#msg679822

[fitbrit]

Thanks. I will simulate some samples for you later, after putting the (sick) little one to bed.

[fitbrit]

Attached is an Excel spreadsheet screenshot where I have attempted to simulate the issues in the library for two albums.
Note:

• The filenames are perfect: All the
• Some fields are empty
• The Album Name may not be the same as the folder name that the files are in. The album tag should be preserved ultimately, and the files not have their filename or path altered if possible
• Some [Artist] tags do not match with the others within the album. This is very rare.

I can provide the spreadsheet via Dropbox if needed.

[glynor]

So the metadata included in the Filenames is abbreviated, but you want the tags to be "expanded". Is that basically it? Can I ask... Why?  How did it get into this state, and why do they want to both:
* Preserve the file struture
* Not use the metadata included in the file structure

Because, obviously, extracting the metadata from the file structure, is much easier. And if that is what this user has been using to browse their system in the past, why not replicate it, and then let them modify it over time?

If it isn't what they've been using, then a better strategy would probably be to get it from what they were using. If iTunes, you can import iTunes XML files, with some elbow grease. You might be better off doing that, if they didn't hose their whole iTunes Library somehow at the start of all of this.

But, if you insist...

Fixing Track Numbers

The first, and easiest step, from your example, is to fix the track numbers.  You can do this with a simple Library Tools > Fill Properties from Filename on the whole set (at least assuming they all match your example).  Just enable only the Filename template, and enter: [Track #] -

That should fix those, which is good, because the individual items like [Track #] and [Name] are going to be the nightmare to do manually.

Fixing Albums:

This uses a modified version of the trick I use to make my Current Shows view, which I've intended to explain how to make in a detailed tutorial, but haven't finished writing it up yet.  You're going to make a Smartlist, expand it.  And then make a view that filters against that smartlist, and expands it again. This will effectively find files where:

* At least one of the files in the folder has an [Album] tag.
* At least one of the other files in the same folder is missing an [Album] tag.

And then display them together in a View so you can quickly fix them with Pane Tagging.

1. Make a new Smartlist, with this search:

Code: [Select]

[Media Type]=[Audio] [Album]=[] ~expand="Filename (Path)" ~sort=[Filename]
This will search for:
* All audio files where the album field is blank.
* Then, it will take those files, and expand the list to include all the other files in the same exact path as those files with missing [Album] tags.

So, it will result in a list, assuming the [Album] tag is missing from lots of the individual albums, of basically all of the files within the Library.

2. Make a new Panes Style View and add three categories, in this order:
* Location (at root, so it shows a directory tree).
* [Filename (Path)]
* [Album]

You may also want to add other fields that you'd like to be able to quickly "fix" at the same time after these (like [Artist], [Composer], etc) as categories.  But stick them at the end, if so.

3. In your View, you probably want to:
* Uncheck Populate Tree
* Check Advanced > Filter in Both Directions

You might not need the latter, but it could come in handy.  And the former just won't be useful for this task, so just disable it.

4. Add a Search to the view with this search:

Code: [Select]

[Media Type]=[Audio] -[Album]=[] playlistid==661116148 ~expand="Filename (Path)" ~sort=[Filename]
Where the PlaylistID points to the Smartlist you made in Step 1.

This will re-filter the list to show only folders that contain at least one file with a filled [Album] tag. It will still expand any "found" album out to the whole "folder" full of tracks.

Then, you should be able to go through the [Filename (path)] category in this view, one at a time, and the list below will fill with just the contents of that folder.  The [Album] category you added, will fill with at least two values:
* The filled Album name (or names if the folder contains more than one value for this field)
* Unassigned

Click down in the file-list area, Control-A to select them all, and then tick the box next to the Album value you want to keep. You'll probably want to disable the warning before you enable Tagging Mode, or every time you leave the view for a second, you'll have to re-enable it (you can always re-enable this warning when you are done).

While you are at it, you can use the other checkboxes (if you added them) to fix other relevant fields, if desired.

This should be reasonably quick, as you'll usually only have two or three choices in the [Album] category (and one of those will be Unassigned every time).  I had you add the Location category so you can pre-filter and group the task up into "hunks" by selecting only portions of their filesystem "tree" if needed.

It'll look something like:

click to embiggen

Then, when you are all done, you can go back to your original Smartlist, and it should be filled ONLY with the list of files which contain NO valid [Album] tags of any kind (they're all blank).  These you can fix with Fill Properties from Filename because there is no good tag metadata that you're overwriting.

Repeat for Artist and other Fields

Then, you can tweak the list and View searches to do the same thing for [Artist]. This is a little more difficult because you want to get the "parent of the parent" folder, and not just each individual "album folder" (which are the last ones in the path, and so you can "cheat" by just using the full path).

You cannot use ~expand to directly expand on an Expression.  So, you have to cheat and make a Calculated Field that spits out what you want.  This should be pretty easy.

Make a new calculated field.  So you can re-use it, call it something generic like [Expand Folder].  Add this expression:

Code: [Select]

FileFolder([filename], 1)
Now, you can use this field instead of [Filename (path)] in the entire example above, and substitute [Artist] (or [Composer] or whatever field is most accurate for that "level" of the filesystem hierarchy) for [Album] in the searches. I added this field to my example when testing, and it is shown as a column in the screenshot above.

So, this time, make the original list:

Code: [Select]

[Media Type]=[Audio] [Artist]=[] ~expand="Expand Folder" ~sort=[Filename]and the search in the View:

Code: [Select]

[Media Type]=[Audio] -[Artist]=[] playlistid==661116148 ~expand="Expand Folder" ~sort=[Filename]
And do it again.  Work your way up the file tree.  If there is another valuable level of the filesystem hierarchy, then modify the [Expand Folder] to:

Code: [Select]

FileFolder([filename], 2)
And do the next level.  Where you could get hosed on these later steps is if some of the folders are non-unique, and there are multiple folders with the same names, but which are for different artists or whatever (perhaps in different parts of the filesystem directory structure). This is also why I had you add the Location field as a Category.  That way, you can pre-filter it to make sure you're only looking at files that are really part of the same "composer" or "artist" or whatever as you do these later steps.

If you do add the extra fields in step 2 above, and then fix them as you go through the albums, these later phases should be quicker and easier.

[blgentry]

Wow, that solution from Glynor is crazy... very detailed and involved.  If that works, awesome.

But how about something more simple.  What happens if you throw a group of these files into MusicBrainz Picard?  It's whole job is to figure out stuff like this.  Or are you going to tell me that the tags your friend has chosen are specially tweaked and Picard's new tags aren't good enough?

To do *exactly* what you've asked for screams for custom programming.  Someone skilled with a language that has a library for dealing with ID3, OGG, (and whatever other types of tagging formats that are applicable) would be able to write up a short program to do what you wanted.  The biggest issue I can foresee (haven't not tried this) would be trying to make these rules apply across all different sound file formats.

Hey, at least you've now got some options and some things to think about. 

Brian.

[glynor]

Yeah... I thought about the custom programming route. The issue is that this assumes that the issues in the library data are regularized. In other words, that they are exactly (not mostly) as fitbrit has described. I'd be willing to bet you a six-pack of beer that this is NOT the case, and that fitbrit's description (and example) is mostly right, covering the majority of the issues, but that the 27k+ files are strewn with outliers.

The only way to fix that, is to have a human eye, I think.

But how about something more simple.  What happens if you throw a group of these files into MusicBrainz Picard?  It's whole job is to figure out stuff like this.  Or are you going to tell me that the tags your friend has chosen are specially tweaked and Picard's new tags aren't good enough?

If this can fix it, or even a huge portion of the issues, it would save a ton of time.  The big issue is if the person really is dedicated to those specific tags and tagging structure.

Also, in my experience, the automated taggers don't do a great job with Classical music, and from fitbrit's example spreadsheet, I suspect that's a lot of what we have here.

So...?

But, my solution is not actually that complex, once you get going.  The View pointing at a Smartlist with ~expand functionality is SUPER useful.  Like I said, that's what my ultra-cool Current Shows view is based on.  I'll really, really try to finish that write up in the next few days.

Just been busy...

[fitbrit]

Firstly, let me thank you so much for all that work, glynor. If you are coming up to Osheaga this summer, I will definitely buy you some drinks!

I really appreciate your procedure. There are some things in it that I am not familiar with doing at all, and this will be a HUGE learning experience for me - it may be a kind of watershed from which I don't look back. I will go through the steps again on my own library to see what they do. It appears that it's still a very manual procedure in the end, with the human having to eyeball and make decisions. It would be ideal if it could somehow choose the non-unassigned value if the there is only one (which it almost always is). However, this is still a massive help over grouping all the files by [filename (path)] and then inline editing all the different fields. I'd much rather check a few boxes!

Yes, all the music is classical.
I do not know if the MusicBrainz Picard solution will produce satisfactory results.
All the affected files are in a folder not too deep from the root level, just above what normal users would keep their artist folders in.
The "album" folder names in my spreadsheet are just to indicate that they are related but not exact matches to the [album] tag. There is no set rule that I could figure out. One just contains more info than the other.

[fitbrit]

BTW, the first thing that I did was to fix the track numbers by doing =left([filename (name)],2) on all the blank track numbers. At least that part was consistent in the file naming.
Then I fixed the [name] fields by removing all the track numbers, so at least those parts of the library are ok now.

[glynor]

A few things:

* Fill Properties from Filename is usually better than doing =left() type operations, because it'll parse the numbers correctly. The way you did it, your [Track #] field is probably not trimmed (it probably has padded zeros). It's okay, and maybe too late now, but... You probably did that because you are confused about how the Templates work in the Fill Properties from Filename system works.

You DO NOT need to "complete the template"  I said type:

[Track #] -

With the Dash.  Just like that.  This will work on those kinds of Filenames.  It will just read the filename from left to right, and fill [Track #] with whatever it finds before the first " -".  You don't need to "account for" the rest of the filenames. The rest will be ignored.  It only cares about what you type, and looks for the first instance, from left to right.

* When you use the Fill Properties from Filename Directories Template, it works the same. The only weird thing is that it reads the "directories" from "right to left" (meaning, it starts with the file's immediate parent directory, and if there are any slashes, then uses that folder's parent, and if there is another slash, uses it's parent, and so on and so forth.

If you want it to "skip a level of folders" you can use a double slash: "//"  I believe.  I'm not right in front of it, but there IS a way to skip levels of directories.  If it isn't double slash, then it is double slash with a space in between like "/ /".

The Preview will show you the results. Play with it a bit and you'll see what I mean.

* Don't run your test on your own Library. If possible, get a Library Backup from your user, and restore THIS into your computer, into a separate "test" Library.

This is what I always do when I'm helping people, if possible.  MC will do EVERYTHING (pretty much, except IsMissing() searches and Rename, Move, and Copy Files) on the Library even if ALL of the files are "broken" and missing.  The only thing you need to do is turn off Tag Writing in Options, so it doesn't constantly complain and error about being unable to save tag changes.

Otherwise, everything will work EXACTLY like it would on his real system, except it won't mess anything up as you play with it (because you can't even see or access his real files, obviously).

Just make a separate, blank Library in your copy of MC.  Then restore the Library Backup you got from him/her to THAT, making sure to UNCHECK the Restore Settings option.

If you are helping people, STEP ONE should always be to make a Library Backup.  And Step 2 should always be for you to get a copy of that Backup, so that if they hose everything and lose the backup and the dog pees on their hard drive, you have a copy of the backup and can save them.  Plus, it can come in handy for stuff like this.

[fitbrit]

Thanks, glynor.

The Fill Properties from Filename function is actually something I am familiar with, and I made myself experiment with it when fixing someone else's library in a marathon 10-hour session. I will save those tips of yours though, as I will undoubtedly have to 'relearn' them with experimentation next time - again!
In this case, though, the 'left' command was simpler to access, and to my surprise it automatically removed the padded zero. I was prepared to go back and use FPfF if needed.

[glynor]

Ahh, cool.  I didn't think it would.

By the way, if ever needed, you can now fix number padding issues with =PadNumber([Field], 0).  I had Hendrik or Matt fix that a while back.  This puts all padding back to defaults.

[glynor]

PS. I don't think we're going to Osheaga this summer.  But I really want to... The Lineup looks fantastic.

Unfortunately (well, fortunately), we're going on a big trip to Hawaii this fall, and gotta save our pennies.  But, if you ever come down to Bar Harbor Maine to visit our lovely park, please hit me up, and we'll get that beer (or twelve).

Plus, I can give you a tour of JAX.

[fitbrit]

PS. I don't think we're going to Osheaga this summer.  But I really want to... The Lineup looks fantastic.

Unfortunately (well, fortunately), we're going on a big trip to Hawaii this fall, and gotta save our pennies.  But, if you ever come down to Bar Harbor Maine to visit our lovely park, please hit me up, and we'll get that beer (or twelve).

Plus, I can give you a tour of JAX.

Too bad about Osheaga, but I think you will have an amazing trip in its place! A tour of JAX would be amazing one day, too.
I just got the library from the guy, and restored it on my PC. I shouldn't have done that and should have used the laptop instead. Oh well.

[fitbrit]

Fixing Albums:
2. Make a new Panes Style View and add three categories, in this order:
* Location (at root, so it shows a directory tree).
* [Filename (Path)]
* [Album]

You may also want to add other fields that you'd like to be able to quickly "fix" at the same time after these (like [Artist], [Composer], etc) as categories.  But stick them at the end, if so.

3. Add a Search to the view with this search:

Code: [Select]

[Media Type]=[Audio] -[Album]=[] playlistid==661116148 ~expand="Filename (Path)" ~sort=[Filename]
Where the PlaylistID points to the Smartlist you made in Step 1.

I'm not quite sure how to do this bit, specifically what "add a search" means.

[blgentry]

I'm not quite sure how to do this bit, specifically what "add a search" means.

I think what Glynor means is to go to Customize View on this view you just created.  Then select Settings > Included Files > Set Rules For File Display .  In the dialog that comes up you can specify rules in different sections.  But Glynor probably wants you to cut and paste his line.  So click the button at the lower left labeled "import/export".  Then you can paste in his line, save and apply it to your view.

I'll be following along on this thread.  I think it's a very interesting problem.  Please let us know how you progress. 

Brian.

[glynor]

Yes. That's it. Media Views have searches just like Smartlists.  The main differences between a Smartlist and a View are:

1. Media Views can have Categories that let you filter them on the fly.
2. Media Views inherit the searches of their parent Views.

The latter quality is why the views under Audio in the tree only show audio files. Read the Media Views article on the wiki for more information.

You edit the searches there, as Brian indicated.  Otherwise Views are Smartlists, just fancier ones.

When you paste in my search, the Playlist is any part will be broken, because you don't have that same playlist ID in your library.  So, after pasting it, go back to the wizard and update that search item to point at the Smartlist you made in step 1.

[fitbrit]

Thanks, Brian and glynor.

I figured that I could create the rules one by one, but then Brain showed how I could paste in your whole line by going to import.

Everything is working like you said it would, glynor! Thanks ever so much.
I've added a bunch of categories in the panes so that I can fix as much as possible in one go. I think it's at the stage where I can tell the person I am helping:
"This is how you do it - so you can fix things at your leisure."