Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[timwtheov]

Maybe blgentry or someone else can help here.

As I have a huge classical music collection, most of which isn't tagged like I want it to be, I'm always looking for ways to shorten the process via automation.  One thing I'd like to do is get a composer's country from my already finished tags in my main classical view and somewhat automatically fill in the country for new albums whose composers are already in that well-tagged part of the database; so for example, if I'm adding yet another disc (or 100) of Beethoven, I can quickly either put an expression in [Country], have it search the [Composer] strings, find the [Country] for that particular composer (Germany for Beethoven in my scheme), and then put that string in the new disc (or 100's) empty [Country] field.  

Screen shots are in separate messages below, as they're too big to upload with this post. I did create a view with a Listbuild expression so I could get the composer + country together (delimited by a semicolon) for most of the files I'd want to get the country field filled with; and it seems like a Regex could be used here but I'm not sure of the syntax for it.  Or maybe there's a simpler way overall?

Thanks for any and all suggestions!

[Edit] Can't get the screenshots to upload, as the files are too big.  Not sure why.

[blgentry]

This is actually really easy.  Lets use your Beethoven example.  Just tag your new files with [Composer]=Beethoven .  Now, use a view that shows composer and select all files that have Composer=Beethoven.  You'll have a bunch of older files that have the correct Country tag, and some new files that do not have the correct country tag.  Now, open the Tagging Window:  Edit > Tag .

Over on the left in the Tagging Window, you'll see fields displayed.  Make sure your Country tag is showing.  You might have to pull down the gear icon to show it, by showing all fields, or all fields in current view or something.  Once you can see Country, you'll see that it's value says "Varies".  Click on it and you'll see that one of the choices is the correct one (Germany).  Just click on the correct choice.  Now ALL of the highlighted files will have that value.

You can do this with any field.

This looks like a lot of steps in writing, but it's actually quite fast when you do it in real life.

Good luck!

Brian.

[timwtheov]

Yes, I should have thought of this simple way of doing things: duh!

You're making me feel like a novice here!  It's not like I haven't had the program since version 15 or anything!  It's just that since I've discovered the power of expression language, I want to use it all the time, though I'm still learning the syntax for complex functions, like Regex.   

Anyway, thanks!

[RoderickGI]

Once again Brian's method is simple and fast, although unfortunately not automatic. 

But the premise of linking the [Country] field to the [Composer] field falls down a bit if a track lists more that one Composer.

[Composer] is a list type field, and so is allowed to have more than one value in it, separated by semi-colons. I certainly have a few Classical tracks that list more than one Composer, so whose Country would you put into [Country]? The [Country] field is also a list type field, so I guess you could list the Country of each Composer in the same sequence as the [Composer] field. But that does get a bit messy.

It is a shame that MC doesn't have separate tables for things such as Composer, so that all the details of a Composer could be recorded once, and then just linked to again the track using the Composer's name. But alas, MC works based solely on tags in or assigned to files, with only one record per file.

The closest thing MC has is relational fields, but they only work with respect to Artist, Album, or Series, and they wouldn't work with List type fields anyway. At least I don't think they would, although interestingly the [Artist] field is a List type field, so maybe they do? Relational fields are all one-to-one relationships, such as one Series has one and only one [TheTVDB Series ID]. If the [Composer] and [Country] fields were just String type fields this would be a perfect application for a relational field. I have used a custom relational field to flag a whole series as a favourite, with one click. It works very well.

Note that as there is only one standard [Country] field I would be tempted to add a new field called perhaps [ComCountry] for Composer Country. That way if I ever wished to store the country of origin of the Conductor, for example, I could just create [ConCountry] for that. 

If it was possible, I would then make [ComCountry] relational; One [ComCountry] for each [Composer]. But alas that isn't possible.


Anyway, I'm just rambling a bit since this question caught my attention. Carry on. 

[timwtheov]

@RoderickGI:  yes, basically what you wrote has been my dream, not only with country, but with other composer-based (or even conductor, orchestra, soloist, et al) information: maybe bios, work lists, whatever.  It would be nice if you could important work lists from ArchivMusic, for example, or from Presto, plus any other relevant meta-data and then as you say, store them in some sort of table. I already use the former site to fill [Composition] for composers' works which aren't in Musichi's database, but it's all done by hand (thank God for Chrome's "Copy Link Text" extension, since all the compositions in Archiv are hyperlinks). 

Anyway, yes, some kind of composer/artist table would be a lifesaver in terms of metadata.   

[ferday]

you could always build a big ifelse statement, but that would get pretty unwieldy after some amount of composer names. 

while i'd love more access to relation fields as well (+1 to RGI) i also think this specific issue would be helped if we could choose which tags update from the internet

[timwtheov]

@ferday

Yes, I thought about this big ifelse expression but quickly saw the problem of unwieldiness. That's actually the problem I was trying to solve when I originally wrote. I did manage to get a list built based on my main classical scheme, where I everything I want tagged is tagged how I want it.  I did a Listbuild that combined the [Composer] with the [Country] delimited with semicolons, e.g., "Beethoven, Ludwig van (1770-1827); Germany." Then it seemed like for new files I could use some kind of Regex expression to find [Composer] in the Listbuild expression ([Composer] always gets tagged first in Musichi), compare it/them to the new files I'm tagging, and then automatically fill in [Country] based on what I have in my main view scheme. Obviously if a composer isn't in that main view scheme yet, it wouldn't work, but I was mainly thinking of this for composers I have tons of discs of (Beethoven, Mahler, Schumann, et al) just to speed things up a bit.  Blgentry's suggestion works, too, but it takes longer than simply plugging an expression in the search bar or copying an expression-based custom field into [Country] automatically.

Another possible way, though I don't know the expression/syntax for it: search through a list of countries from my main classical view scheme, i.e., those albums/files that have a country tag, compare that [Composer] to the new set of files that have a matching [Composer] and then fill in the right country.  So search through Germany, and if a "Beethoven, Ludwig van (1770-1827)" is found, then put that/those file/files into the present, empty [Country] field. 

[blgentry]

This type of data processing isn't well suited to MC's built in capabilities.  I've seen these discussions before and haven't seen any way to do "lookup tables" using the native interface.

For a short time I was working on a project to process MC data externally and then reimport it using MPL playlists.  But I sort of lost interest and it wasn't very elegant.

Luckily the famous Mr.C has built MCUtils, which can do pretty much anything you want if you're willing to do a little bit of programming.

http://yabb.jriver.com/interact/index.php?topic=100449.0

I have no experience with it, other than reading through the documentation some. 

Brian.

[dtc]

A lookup table feature would be extremely useful for many things. I would like to have a short form/long form artist and composer lookup, for example Beethoven and Ludwig von Beethoven for sorting and displaying.  Related to that  it could also be used for the famous sort by swap/unswap problem, which is never perfect. Composer by genre and sub-genre would also be nice. And so on. It could be a simple two column table, but a multi-column table would be better. For a composer table, for example, Column 1 could be short name, followed by full name, genre, sub-genre, country, years etc., all user defined. Then you could do the lookup either by numerical column or by a specified column header. 

Although not trivial, it seems like it would be relatively simple to add that new functionality. You would need a little utility to create/delete tables and to add and delete entires. Then you need a simple function to do a lookup and return the associated column.

Lookup tables would be a nice addition.

[ferday]

I don't think that would be trivial or even kind of easy to add.  There are certainly things I'd love to see first (an album as an "object", for one)

Exporting into excel is very simple and fast, but I have not had much luck getting the excel back in (it would be a trivial macro to write in excel)

 I think MCUtils can do some of this, I've only superficially played with it though.  Maybe I should revisit!

[dtc]

I don't think that would be trivial or even kind of easy to add.  There are certainly things I'd love to see first (an album as an "object", for one)

Exporting into excel is very simple and fast, but I have not had much luck getting the excel back in (it would be a trivial macro to write in excel)

 I think MCUtils can do some of this, I've only superficially played with it though.  Maybe I should revisit!

Maybe I am missing something, but I think it should be pretty easy to implement. This is the type of project I used to assign to my students when I was teaching programming. It is a pretty standard assignment for a second year student. It is certainly more than a few hours, but it should not be very complicated to do.

The table creation and storage could be, I believe, pretty much independent of the current database design. It would interact with the current tagging system only through the result returned from a simple table lookup. So, I think it would be pretty easy to implement.

For example

GetFromTable(ComposerTable,  "Beethoven", Genre)

It would be very similar to an Excel vlookup function.

In place of the table creation utility, I suppose an import from Excel might be doable. It is pretty much the same logic but without the user interface development time.

The trouble with MCUtils is that it is not easily available to the average user.

[RoderickGI]

The concept of separate "databases" does exist in the Library. For example, you can select to view only fields from TV, Audio, Video etc. in the Manage Library Fields function.

All that would be needed is to give the user the ability to add another grouping of fields, and then the ability to look up value within that group, and include them in view, playlists, etc.

Populating those groups could be done using existing View functionality, by creating a custom view that shows and allows editing of the contents of the field group.

What doesn't exist for users explicitly is the ability to do table joins, for lookup purposes. That could get messy, unless the functionality was limited to very simple joins based on one field, such as for Composer we have been discussing.

But adding the above capabilities could have implications for the upgradeability of MC. Particularly for major upgrades that include standard Library field changes.

Still, not impossible, and not all that difficult.

[dtc]

The concept of separate "databases" does exist in the Library. For example, you can select to view only fields from TV, Audio, Video etc. in the Manage Library Fields function.

All that would be needed is to give the user the ability to add another grouping of fields, and then the ability to look up value within that group, and include them in view, playlists, etc.

Populating those groups could be done using existing View functionality, by creating a custom view that shows and allows editing of the contents of the field group.

What doesn't exist for users explicitly is the ability to do table joins, for lookup purposes. That could get messy, unless the functionality was limited to very simple joins based on one field, such as for Composer we have been discussing.

But adding the above capabilities could have implications for the upgradeability of MC. Particularly for major upgrades that include standard Library field changes.

Still, not impossible, and not all that difficult.

So interesting ideas, but much more complicated than a simple set of lookup tables with the capability to search those tables to populate existing tags. My thought is to just maintain an independent set of tables and then have a function to search that table in order to populate an existing tag. Once you start joining tables you will loose most of the users. The most I would want to do is to populate column 1 of the table from the existing values for a given tag, for example Composer or Genre. After that populating the table is manual. Sure, that can take a while, but most entries are only done once. If you try to populate all the other columns from existing tags, the process gets very complicated. I would start with a simple table build process and go from there.

The idea of a separate set of tables, independent of the existing tagging structure, is that these tables do no relate to individual albums or movies. They are a separate, independent set of tables which are maintained separately from the normal tagging structure. There is then a function that can search the lookup table to populate a field for a given album or movie. I do not see a lookup table as a grouping of existing fields, but a separate table. Maybe not elegant, but simple to implement and simple for most users to understand.

[timwtheov]

Some kind of table structure sounds like it would be great, but not being a programmer, I have no idea how hard such a thing would be to implement.

@blgentry: I'm working with the MCUtils right now: just got it up and running, but I haven't done any tag mapping/shifting/field-creating yet. The AMG/Discogs look-ups would be a lot more useful for popular types of music than with classical, I think, unless one was starting from the ground zero with tagging; but since I already have a system I like, and since particular classical albums often aren't on AMG or Discogs, I think it's use would be limited there.  Obviously one could develop one's own scripts, however.  Anyway, thanks for the suggestion, as I didn't even know this existed.  I'll definitely use it with some of my Rock/Alternative albums. 

[RoderickGI]

Maybe not elegant, but simple to implement and simple for most users to understand.

May also be quite slow to display views with large numbers of records, compared to MC's current speed.

The speed of MC in displaying views is one of the things lots of users like.

[dtc]

May also be quite slow to display views with large numbers of records, compared to MC's current speed.

The speed of MC in displaying views is one of the things lots of users like.

Certainly something to consider. But the tables are relatively small and could be locked into memory. With fast search routines on sorted data the lookups should be pretty fast. And, they would be an optional tool, for those who liked the results.

[timwtheov]

Sorry to re-open an old topic, but I've been playing around with the above issue I noted above via some custom fields + expressions. I'm almost where I can do a quick Move/Copy Fields with this into [Country] based on the Composers and Countries I have in the "fully tagged" part of the classical section of my database, but I need some regex syntax help (I think).

Here's what I did:
1. Made a semicolon delimited list (via Listbuild) that includes [Composer];[Country], all for custom field [Country/Composer List]. A populated example of this field is "Beethoven, Ludwig van (1770-1827);Germany"  This part works.
2. Made a Regex expression to search this custom field, find the semicolon, and then output what comes after it as the proper country, all for custom field [Country/Composer (Country]. Here's where the problem is, probably somewhere in the syntax. Right now I have

Code: [Select]

regex([Country/Composer List],/#(;)*(.+)#/, 2, 0)
Right now this expression is returning just the name of the searched field (i.e., [Country/Composer List]) as the result instead of the country name. What am I doing wrong?

Thanks!

[blgentry]

2. Made a Regex expression to search this custom field, find the semicolon, and then output what comes after it as the proper country, all for custom field [Country/Composer (Country]. Here's where the problem is, probably somewhere in the syntax. Right now I have

Code: [Select]

regex([Country/Composer List],/#(;)*(.+)#/, 2, 0)
Right now this expression is returning just the name of the searched field (i.e., [Country/Composer List]) as the result instead of the country name. What am I doing wrong?

I'd take out the star (*) leaving:

Code: [Select]

regex([Country/Composer List],/#(;)(.+)#/, 2, 0)

But I'm kind of unclear on exactly what will be in that field (just one pair, or many pairs?) and how you are "looking up" the country.  After all, if you already have the country in that field, it implies that you knew the country to start with at some point.

I still think MCUtils might be the way to go.  Or you could make it really ugly by cheating and writing external code that generates your giant ifelse statement for you.  With your table, I might even be able to quickly throw together the code to generate the giant ifelse.

Brian.

[timwtheov]

Yeah, I tried it without the * in there, and it didn't work.

What I'm trying to accomplish is to fill in [Country] for NEW files where [Composer] is already tagged correctly ([Composer] is tagged in Musichi); thus, with the first expression/custom field, I just scanned all my already tagged [Composer]/[Country] files and created the list. Then when I enter new Beethoven files, say, I can just quickly do a Move/Copy and fill in "Germany" for [Country]. I'm simply trying to automate a bit more, since I often import hundreds of new classical files at a time, many of which repeat composers already in the fully tagged Classical part of my library.

I've looked at MCUtils, and yes, it undoubtedly would work; it's just the learning curve + time: I haven't yet wanted to go down into that rabbit hole. Maybe this is my spur!

Regarding your question, if it's not clear from the above: what should output to the new tag ([Country/Composer (Country)]) would be simply the right [Country] for [Composer] as based on my already extant tags in the "fully tagged" part of my library. I was kind of going off what you wrote in a post earlier this summer (https://yabb.jriver.com/interact/index.php?topic=106045.0) where you were able to extract information from the person's [Comment] area via Regex. I'm just still new at/with Regex syntax. And again, a little intimidated by MCUtils.

[blgentry]

What I'm trying to accomplish is to fill in [Country] for NEW files where [Composer] is already tagged correctly ([Composer] is tagged in Musichi); thus, with the first expression/custom field, I just scanned all my already tagged [Composer]/[Country] files and created the list.

OK, so [Country/Composer List] contains *all* composer and country pairs?  Or does it only have the composer and country for the album you are working on?  If it's ALL, it's going to require some sort of filtering in order to isolate the Composer you want.

I just cut and pasted your Beethoven example into the [Comment] field for an album I have.  Then I put this expression into an expression column:

Code: [Select]

regex([Comment],/#(;)(.+)#/, 2,0)
It outputs "Germany" as expected.  I'm guessing that your field actually has ALL of the values in it, not just the one value for a particular composer.

It might be helpful if you could cut and paste the full value of that field into a CODE section here.  Then I could play with it and maybe help you with an expression.

Brian.

[timwtheov]

It looks like it's working by track, that is, when I look at an Erich Kleiber performance of Beethoven's Symphony No. 3, by individual tracks or by the whole album, that custom field shows as "Beethoven, Ludwig (1770-1827;Germany." It's like that for all of [Composer] "Beethoven, Ludwig van 91770-1827)" in my library (so long as I'm in the "fully tagged" part, that is). The expression for this field is just a simple Listbuild:

Code: [Select]

ListBuild(1, ;, [Composer], [Country])

Not sure how I'd cut and paste the "full value" of a field in here. I tried the expression

Code: [Select]

Field(Country/Composer List, 1)
and

Code: [Select]

Field(Country/Composer List, 0)
but only got the title of the custom field "Country/Composer List" as a result. But that isn't what shows up in the tag, which is what I wrote above.

Screenshot attached.

[blgentry]

I'm really confused now.  You already *have* [Country].  That's how you're building your list field.  But now you want to extract it from the list field?

I'm guessing you want to somehow use this data when you import new files.  But you won't be able to do that because you can't really search all of your existing files to populate new files.  But I'm just guessing.  What is it you are trying to do?  Make a big lookup table?

If so, you need more like a giant ifelse statement like I've been saying.

Brian.

[timwtheov]

Basically, I'm trying to do what you just said I can't do: search old files' tags to help populate new files' tags, since there's a lot of overlap with classical music (the composers, countries, and works are relatively constant; the new things are the different performers, orchestras, labels, et al). Since I already have a sizable chunk of my classical collection fully tagged (the primary view is by composer, by type of composition, e.g., orchestral, choral, opera, etc., and then by the composition itself: Symphony No. 5, Piano Concerto No. 1, and so forth), I thought I could speed some of the tagging of new files up by just copying what I already have for the stuff noted above that remains relatively constant. I suppose it is indeed like a table, as in MCUtils. Anyway, again, building off what you helped someone else do here (https://yabb.jriver.com/interact/index.php?topic=106045.0), I thought I could build a custom tag somewhat like his [Comments] section, but in my case a list with both [Composer] and [Country] delimited by a semicolon, and then pattern-match it with Regex, using the the semicolon as the marker, with the expression spitting out everything to the right of said semicolon, all in a new custom field that I could then use in a Move/Copy on a highlighted bunch of new files. In actually typing this out now, however, I see the error in logic that would still require a massive If/Then construction: missed that before in all the details. Stupid me!

I suppose I'll just have to jump into MCUtils. Or keep doing [Country] and a couple of other fields by hand.

Sorry for the confusion!
 

[Ferdi]

MCUtils is a great help, despite some short comings!

However, it requires you to have all the data in excel, or similar, to do the look-ups etc (era, nationality, type of work etc). Where do you get the data from? I started with Wikipedia, classical.net etc: but it's hours of copy/pasting, cleansing, merging,... of thousands of composers.
And it seems even harder to find such data for Jazz.
Any tips?

[blgentry]

Basically, I'm trying to do what you just said I can't do: search old files' tags to help populate new files' tags,

It's not strictly "impossible".  Just sort of hard and uncharted territory for me.  You *might* be able to do it with global variables.  They are strange and difficult to work with, but they definitely do work.

It might be possible to make an MPL export of your library and then post process that using either spreadsheet, database software, or a programming language.  The later is what I would try, but that's just because of my background.

With the post processing, you could build a table pretty easily and then use MCUtils for it.  Or, you might be able to construct a huge ifelse statement that has one clause per composer.  You could then make a NEW field in MC's database, with calculated data and this calculated data would be the ifselse() clause.  Because it's calculated, every single file would have this field, including brand new ones.  So lookups in this field should work correctly I would think.

Getting the post processing done is the "hard part" I think.

Brian.

[timwtheov]

@blgentry

Thanks for the suggestions. You know, I was thinking about it, and at this point, it's really only [Country] that I need to automate anyway. The other tags ([Conductor], [Soloists], et al) are so album-specific that without a reliable metadata lookup solution, it's probably safer, if not faster, to keep doing them manually. I think I'll just add [Country] to my main classical tagging window and then just highlight all composers from the same country and then fill the tag that way (e.g., Ravel, Debussy, Berlioz, etc. all are assigned "France"): it's probably simplest.

@Ferdi

My current workflow goes like this:

• Import files into Musichi's tag editor to fill in [Composer] and [Composition]. They have an excellent database of composers (in last name, first name, years format), and for the most popular composers, they have nearly complete catalogs of compositions, too. There are some tricky things with standardizing how the composition appears (i.e., you can set it up to display these in the tag as something like "Quartet for Piano and Strings," "Piano Quartet," etc., including different spots/punctuation/spacing for opus numbers, tonality, and the like), if for example you have to uninstall the software and re-install it. That caused me some issues a while back, with the Windows 10 upgrade. You can also look up individual albums via Musicbrainz, Amazon, freedb, and I think one other, too, though the metadata from these places is never in the form I want it in. You can also use their "Clean" feature, which will look up composers for you (so long as they're in last name, first name format), compositions, and performers, though the success rate here is spotty. Musichi's also nice in that you can group the main view by tag and even nest these; thus, I typically group by Composer and then either Album or Directory.
• If the odd, rare composer isn't in Musichi, I just create the tag manually via a Google search.
• If a composer is present but the compositions aren't, then my first, and usually only, choice for composition look up is ArkivMusic (http://www.arkivmusic.com/classical/main.jsp), which though it's a store selling CDs and downloads, has a standardized set of hyperlinks for compositions, performers, orchestras, ensembles, and so forth. I just use the Chrome add in "Copy Link Text" to copy and paste the info into either Musichi or MC. A couple of caveats with Arkiv, though: first, they like to minutely subdivide works, so if an album has a part of an opera on it (an aria, say), they'll make the composition the opera name + the individual aria, which is nice if you're searching their site if you want to see what recordings are available for that aria but not so great if like me you want all instances of an opera together (i.e., I just want one entry for "Tristan und Isolde" and not 30 for all the complete versions + one or five offs on aria discs. Second, Arkiv basically translates all orchestra/ensemble names into English, so if you have an album with the Orchestre Revolutionnaire et Romantique, it might translate that to Revolutionary and Romantic Orchestra (didn't look up this particular instance, but they generally do this). Thus, regarding MC, you have to be careful with consistency (is it the Concertgebeouw Orchestra, The Royal Concertgebeouw Orchestra, or . . . ?). Luckily MC remembers these tags for you! If a rare composition isn't on Arkiv somewhere, I usually use Presto Classical, which also has a hyperlinked set of compositions, et al. If a composition isn't on there, I usually just create it myself, but this situation has only happened a handful of times with nearly 30, 000 tracks.
• At this point, once I have [Composer] and [Composition] tagged, I move back to MC and fill in global box set tags, if that's what I'm tagging: [Disc #], [Album], [Soloist]/[Conductor]/[Orchestra], if the box is performer-specific, i.e., Sviatslav Richter's RCA recordings, or something, [Label], and usually [Country] of birth or common association. I do cover art here, too
• After that, it's album-by-album with all the [Soloist], [Orchestra], [Ensemble], [Label] (I like to see how many Decca vs. Sony vs. DG recordings I have, say), [Conductor], etc. tags. That goodness for Universal Classics's excellent online catalogs here, especially for box sets: they're a lifesaver!
• The final 2 steps are filling in [Grouping] (i.e., Orchestral, Solo Instrument, Opera, Choral, Concerto, Chamber, and so forth), which is one of my primary sorting fields in nearly every classical view scheme I use. Finally, with blgentry's help on here, I created a custom field called [Artist List] to fill in [Artist] via Move/Copy with an expression-created, semicolon-delimited list of [Soloists], [Ensemble], [Conductor], and [Orchestra], in that order. This latter is mostly for the sake of Poweramp and PlayerPro on my phone, which like most such apps, show only [Artist] and [Album] on the ticker under the artwork when playing a track.

I attached a couple of screen shots so you can see what it looks like when it's all done (and pretty).

[timwtheov]

Conductor view attached.

[Ferdi]

@timwtheov - thanks so much for your detailed reply!
It took me a while to take a closer look at Musichi. From what you describe, it has great features; I lack the patience to learn it and the willingness to invest into such an unintuitive tool though. Might be worth it if I'd not start with an already decent organization of my collection.
I guess that for now, I continue my mix of MusicBrainz Piccard, jmone's data fiddler http://yabb.jriver.com/interact/index.php/topic,102906.0.html for manual batch changes (using downloaded lists of composers, works, orchestras etc) and of course manual editing in MC (orchestras, soloists etc).

I'll keep in eye on Musichi - perhaps a miracle happens and they relaunch with a better UX (I keep on hoping the same for MC )