Topic: amg.pl: an A M G helper and dummy file creator (Read 87896 times)

MrC · « **on:** October 09, 2013, 08:52:46 pm »

Description

The following command script pulls metadata from AMG to tag your Media Center tracks. In addition, it can also create dummy files that can be imported and tagged to simulate collections for which physical media does not exist (e.g Vinyl, desired CDs, etc.). It is for personal use only, and it's sole purpose is to avoid the risk of carpal tunnel from the copying and pasting you are doing already. :-)

The script is written in Perl, and can be used on Windows, Mac OS X, or Linux. It basically works like this:

   1. You supply an album's AMG release URL
   2. You select the album's tracks in a view in MC
   3. You run the script

The script will do one or both of:

  - update MC fields directly via MCWS (version 19.0.54+, Media Server needs to be running with authentication enabled)
  - produce an MPL file which will update the MC fields when imported into MC via File > Import Playlist

Available Metadata

The following metadata can be retrieved (when available):

   AMG album, track, and artist IDs and URLs
   AMG album and track picks
   AMG album and track reviews
   Ratings (AMG rating, AMG-to-MC-mapped rating)
   Awards (Billboard Singles, Billboard Albums, Grammy Albums)
   Releases
   Album credits (with and without names)
   Styles, Moods, Themes
   Release date
   Album
   Genre
   Track name
   Track artist
   Track composer
   Track featuring
   Lyrics
   Disc and track numbers

Download the attached zip file, and unzip the items to a folder (right-click > Extract All). See the README.txt file inside the folder.

The original post containing the development and release history of this software is at:

   http://yabb.jriver.com/interact/index.php?topic=79942.0

MrC · « **Reply #1 on:** October 09, 2013, 08:57:12 pm »

Options

There are several options available to change the behavior of the script. You can view a list of the options with:

perl amg.pl --help

Although most should be self-explanatory, it is worth commenting on a couple of options here.

The script works by pulling the track info from the clipboard, matching the disc and track numbers from your selected file list with those from AMG. Valid track numbers or sequences are required for this matching, because to import the items and associate them with your existing MC entries, the track's Filename field must be used.

The --sequence and --fileregex options allow you to deal with situations where your tracks do not quite match up with the AMG tracks. Your release may contain bonus tracks, and the --sequence option instructs the script to use the list order of your tracks and match them against the track order in the AMG list. For any supplementary tracks you have (that are not listed in the AMG track list), album-related field values will still be output for your additional tracks. The --fileregex option allows you to grab track numbers contained in your file names as the basis for creating the sequence in which to match against the AMG track sequence.

In Step 3, you supplied the URL on the command line. If at least the first track of the album is tagged with an AMG URL, and the AMG URL field is in the view, the -u option is not necessary and the AMG URL from MC will used. This is useful if you want to re-scrape the data in the future (or us it as the target of an MC Link). The command would then just be:

perl amg.pl

Two options allow you to quickly include or exclude fields. By default, the script will attempt to produce values for all of the available fields. The --include option reverses this behavior such that only the listed fields are made available. The --exclude option allows you to specify a list of fields that should be excluded. Some fields require additional web page look-ups (e.g. credits, awards, track reviews, lyrics), so it is best to always use the --include option with the specific list of fields you want. Once you become familiar with how the script works, you can switch to using the --exclude option to ignore only those fields you don't want. Each of these options is supplied with a comma-separated list of field keys, which you can see in the sample config file; there is one key per Field.

The --configfile option allows you to specify options inside a configuration file. You examined the supplied file earlier. It would be worthwhile to copy the supplied configuration file and make your changes to your copy. This way, new releases of amg.pl will not overwrite your changes. Spend a few minutes reading the comments in the configuration file.

MrC · « **Reply #2 on:** October 09, 2013, 08:57:17 pm »

Usage Tips

Creating and using MC Links to jump to Album, Artist and Track sections at AMG
Version 1.25 added support for Album, Artist, Composer, and Track IDs. These are convenient for creating and using MC Links to jump to the correct location on the AMG site. Here are several expression-based Search URLs that can help you locate the correct entries on AMG when no IDs exist yet for your track(s), or find the exact item when your ID fields are populated. I found the Album search link particularly useful when initially searching for an Album ID, as the correct album entry almost always appeared in the search results, usually at or near the the top. For those difficult albums, a track-search or artist search helped. Simple select the tracks for an album, and click one of the Links.

The following searches will attempt to jump to the exact album, artist, composer, or track, respectively, based the corresponding ID value if it exists, or based on the corresponding MC field contents (Album, Artist, Composer, and Name):

Album
http:////www.allmusic.com//If(!isempty([AMG Album ID,0]), album//ifelse(isequal(left([AMG Album ID], 2), mr),release//)[AMG Album ID], search//albums//[album])

Artist
http:////www.allmusic.com//If(!isempty([AMG Artist ID,0]), artist//[AMG Artist ID], search//artist//[Artist])

Composer
http:////www.allmusic.com//If(!isempty([AMG Composer ID,0]), artist//[AMG Composer ID], search//artist//FirstNotEmpty([Composer],[Artist]))

Track Name
If(!isempty([AMG Track ID,0]), http:////www.allmusic.com//if(isequal([AMG Track ID],mq,7),performance,song)//[AMG Track ID], http:////www.allmusic.com//search//songs//[Name])

When you configure the links, as a Base URL of http://www.allmusic.com for each of these, and use the above search expressions for the Search URL. I configure mine to open in an external browser. You can limit the links to Media Type=Audio by selecting the Edit Search... button, and setting a rule for Media Type is Audio. More help on creating links.

marko · « **Reply #3 on:** October 10, 2013, 01:43:05 am »

Who remembers aTagger by al@n? It was brilliant, and continued to work for some years after the rug got pulled.
I'm not sure if they fall into the same category, it certainly feels like they do, in which case, get ready for a 'cease and desist'

MrC · « **Reply #4 on:** October 10, 2013, 02:46:20 am »

This is one reason the script essentially requires a visit to the AMG site via browser.

Arindelle · « **Reply #5 on:** October 11, 2013, 05:32:26 am »

Hey MrC!

Looking good, thanks again.

I'm having a couple of issues though ... I'm getting these errors often

Quote

C:\Users\S\Desktop>perl amg.pl -u http://www.allmusic.com/album/no-place-for-my
dream-mw0002541163
Ignored extra track in selection: 1.1 Z:\Temp\FEMI KUTI\No Place for My Dream\0
_Nothing to Show for It_No Place for My Dream_Femi Kuti_2013.flac
Ignored extra track in selection: 1.2 Z:\Temp\FEMI KUTI\No Place for My Dream\0
_The World Is Changing_No Place for My Dream_Femi Kuti_2013.flac
Ignored extra track in selection: 1.3 Z:\Temp\FEMI KUTI\No Place for My Dream\0
_No Place for My Dream_No Place for My Dream_Femi Kuti_2013.flac
Ignored extra track in selection: 1.4 Z:\Temp\FEMI KUTI\No Place for My Dream\0
_Action Time_No Place for My Dream_Femi Kuti_2013.flac
Ignored extra track in selection: 1.5 Z:\Temp\FEMI KUTI\No Place for My Dream\0
_No Work No Job No Money_No Place for My Dream_Femi Kuti_2013.flac
Ignored extra track in selection: 1.6 Z:\Temp\FEMI KUTI\No Place for My Dream\0
_Carry on Pushing on_No Place for My Dream_Femi Kuti_2013.flac
Ignored extra track in selection: 1.7 Z:\Temp\FEMI KUTI\No Place for My Dream\0
_Politics Na Big Business_No Place for My Dream_Femi Kuti_2013.flac
Ignored extra track in selection: 1.8 Z:\Temp\FEMI KUTI\No Place for My Dream\0
_Na So Wee See Am_No Place for My Dream_Femi Kuti_2013.flac
Ignored extra track in selection: 1.9 Z:\Temp\FEMI KUTI\No Place for My Dream\0
_One Man Show_No Place for My Dream_Femi Kuti_2013.flac
Ignored extra track in selection: 1.10 Z:\Temp\FEMI KUTI\No Place for My Dream\
0_Wey Our Money_No Place for My Dream_Femi Kuti_2013.flac
Ignored extra track in selection: 1.11 Z:\Temp\FEMI KUTI\No Place for My Dream\
1_This Is Only the Beginning_No Place for My Dream_Femi Kuti_2013.flac

C:\Users\S\Desktop>perl amg.pl -u http://www.allmusic.com/album/rossini-la-donna
-del-lago-mw0001857236
AMG HTML has changed. amg.pl needs to be updated for the new format.
Stopped at amg.pl line 335.

Questions

- Can this be run on a network drive, where the files do not reside locally?
- Can this be run on multiple CDs where some of the Trk info has been changed (eg. Disk one track 1-12 (no change) Disk 2 track 1 to 13 renamed to 13-25)

MrC · « **Reply #6 on:** October 11, 2013, 11:52:08 am »

I believe the problem you are having with the No Place for My Dream album is that you have Disc # set to 1, and typically Disc # would be left empty for single-album discs. No matter. Just use the --sequence (or the shorter -s) option with your tracks in track order in the file list.

Regarding your questions:

1. So long as the files are available to MC as windows file systems (mounted or local), it doesn't matter.

2. Use -s, with the tracks in sequence. amg.pl will then ignore the disc and track numbers, and construct its index from the sequence of your tracks.

Once you're comfortable, try using the --mcws method for direct updates (if you have Media Server running). You can have amg.pl also output MPL so you can examine it.

SkGe · « **Reply #7 on:** October 11, 2013, 01:20:15 pm »

One more thing about some error
Failed to get page: http://www.allmusic.com/song/where-you-are-club-version-mt0042906550/lyrics
404 Not Found
Use of uninitialized value $_ in pattern match (m//) at amg.pl line 863.

Arindelle · « **Reply #8 on:** October 11, 2013, 01:21:28 pm »

Quote from: sirgeo on October 11, 2013, 01:20:15 pm

One more thing about some error
Failed to get page: http://www.allmusic.com/song/where-you-are-club-version-mt0042906550/lyrics
404 Not Found
Use of uninitialized value $_ in pattern match (m//) at amg.pl line 863.

AMG is down for maintenance give; try again a bit later

SkGe · « **Reply #9 on:** October 11, 2013, 01:22:57 pm »

Now i see that there is not any lyrics

SkGe · « **Reply #10 on:** October 11, 2013, 01:24:04 pm »

Is not down now, it was earlier when i was browse the page there.

MrC · « **Reply #11 on:** October 11, 2013, 01:25:00 pm »

A 404 error means a temporary failure. I'll put in better error messages.

SkGe · « **Reply #12 on:** October 11, 2013, 01:32:11 pm »

And what by this message on some albums?
C:\Users\S\Desktop>perl amg.pl -u http://www.allmusic.com/album/rossini-la-donna
-del-lago-mw0001857236
AMG HTML has changed. amg.pl needs to be updated for the new format.
Stopped at amg.pl line 335.
If often happen when album is some movie soundtrack or related to this.

MrC · « **Reply #13 on:** October 11, 2013, 03:29:17 pm »

Quote from: sirgeo on October 11, 2013, 01:32:11 pm

And what by this message on some albums?
C:\Users\S\Desktop>perl amg.pl -u http://www.allmusic.com/album/rossini-la-donna
-del-lago-mw0001857236
AMG HTML has changed. amg.pl needs to be updated for the new format.
Stopped at amg.pl line 335.
If often happen when album is some movie soundtrack or related to this.

I see what the problem is, and I'm going to change how I'm parsing tracks. Back shortly...

marko · « **Reply #14 on:** October 12, 2013, 03:35:22 am »

Had a look at this...

Went for ActivePerl 5.18 thinking latest = best, but all of the 'ppm' instructions failed.
Removed and installed 5.16, ppm instructions all successful.

Followed steps for mpl, failed...

Quote

Failed to open \Users\Mark Wilkinson\Desktop\import.mpl: No such file or directory
Stopped at amg.pl line 903.

I tried this twice. The directory definitely exists, if you add C: to the front of the path. I created a blank import.mpl file there and tried again, same error.
I went to 'System Variables' and added an entry 'Desktop' using the full path and tried again... same error.

I have had some success with the -mcws option, but only via the command line, it does not appear to working via the config file.

-marko

MrC · « **Reply #15 on:** October 12, 2013, 04:07:10 am »

5.18 must be new, and the modules have not yet been released for that version. I'll note it.

~~You need either a relative path, or a full path. The path starting with \ is probably not what you want.~~

I've used the config file with mcws. Can you show the settings and cmd line you used? Obfuscate the password, of course.

Edit. You can use the -o option to specify an MPL file path. I'm not sure why the script can't create a file on your desktop. I'll check on this tomorrow.

Edit 2: sorry for the brief reply - I was replying on a tablet. I'd like to get to the bottom of the problem you're having, but I'll need to see your command line, and know the directory you're in (run command DIR). There is no need to set any system variables.

MrC · « **Reply #16 on:** October 12, 2013, 01:53:49 pm »

Version 1.12 is now posted:

Changes:
- Fix: occasional problems parsing amg tracks (parsing algorithm changed and simplified - please report
any problems).
- Fix: better error messages when page loads fail.

@sergio - this should resolve the problems with the URL you showed above.

Sapagrino · « **Reply #17 on:** October 15, 2013, 10:10:40 pm »

Thank you for developing this. I have it almost working. When I import playlist, nothing is imported for the first track, and the other tracks have the info disassociated with the music info--as if the filename had not been supplied to match things up. When I examine the import file, I see that the first track's filename is among the fields/data, but the filename field is missing for the other tracks. So, two things are surprising: 1) while track 1's filename is in the import file, none of the data from the input file is seen in MC for that track, and no disassociated track 1 data either; 2) Even though I highlighted all tracks in MC--with filename and track # included in view--before copying (tried both Ctrl-C and right click) only the first track's filename seems to have passed all the way through to the input file.

I tried copying from MC directly into Notepad, and indeed that demonstrates that all track's filenames and track #s have made it to the clipboard before hitting enter on the perl script command line; and the input file is filled with data for all tracks.

I am sure I am missing something, but I have tried over a dozen times, read some of your other posts on importing data, etc., so was wondering if the above fact pattern suggests to you what I might be doing wrong. Thank you for your help.

MrC · « **Reply #18 on:** October 15, 2013, 10:32:53 pm »

Can you do me a favor? Run the command and this time include at the beginning of the command line -v -d and at the end of the command line, redirect the output to a file on your desktop, like this:

perl amg.pl -d -v ... > amg-out.txt

where you replace ... with your current options. Then, after the command completes, zip both the import.mpl and amg-out.txt file and attach it here.

Sapagrino · « **Reply #19 on:** October 16, 2013, 06:54:03 am »

Here is the zip file. Thank you for looking into this.

MrC · « **Reply #20 on:** October 16, 2013, 12:06:06 pm »

I've sent you a PM requesting a few more details. We can continue via email. It appears something is wrong with the old (MC18) clipboard contents, or in splitting it into rows. But I'll need your help in reproducing it.

Sapagrino · « **Reply #21 on:** October 16, 2013, 12:24:52 pm »

Thank you. Just sent you an email.

MrC · « **Reply #22 on:** October 16, 2013, 05:24:41 pm »

I've updated the script to version 1.13. Everyone using the script should update. The changes are as follows:

Version 1.13
- Fix: clipboard parsing for MC18
- Fix: handle Unicode clipboard contents on Windows.
- Fix: output diagnostics in UTF8

Arindelle · « **Reply #23 on:** October 17, 2013, 06:01:47 am »

aha a new version ...

for info Mr C I got the 1.12 version to work using the -s and --configfile options in the command line. However I have not been able to use the --mcsw option when I put the server IP in nothing happened, and so I tried using just "localhost" which totally crashed my network

For info, I have an Apache web server running with some perlscripts loaded, should I be turning this off?

BTW -- running off a jr client with ip of lets say 1.192.168.12 with the jrserver at 1.192.168.11 am I right in assuming that in the config file I would uncomment and put in

MCWS_server: 1.192.168.11:52199:52199
MCWS_username: jriverusername
MCWS_password: jriveraccesspwd

or do I use my network username and password?

Anyway, I'll try with the new version before giving you the details ...

marko · « **Reply #24 on:** October 17, 2013, 11:58:09 am »

I've not forgotten about this, just have no time. I'm software testing on dayshift this week, then back to nightshift next week for holiday cover. One yukky fortnight!

MrC · « **Reply #25 on:** October 17, 2013, 12:27:47 pm »

Quote from: Arindelle on October 17, 2013, 06:01:47 am

for info Mr C I got the 1.12 version to work using the -s and --configfile options in the command line.

Thanks for the update.

Quote from: Arindelle on October 17, 2013, 06:01:47 am

However I have not been able to use the --mcsw option when I put the server IP in nothing happened, and so I tried using just "localhost" which totally crashed my network

Nice!

I'm not sure how this could have crashed a network. Did you include the port number, as per the script's example? Without it, you'd be connecting to the web server running on port 90 of the local machine, and that's probably not MC.

Quote from: Arindelle on October 17, 2013, 06:01:47 am

For info, I have an Apache web server running with some perlscripts loaded, should I be turning this off?

It is not necessary. MC uses a different port than the standard port 80 for web services.

Quote from: Arindelle on October 17, 2013, 06:01:47 am

BTW -- running off a jr client with ip of lets say 1.192.168.12 with the jrserver at 1.192.168.11 am I right in assuming that in the config file I would uncomment and put in

MCWS_server: 1.192.168.11:52199:52199
MCWS_username: jriverusername
MCWS_password: jriveraccesspwd

or do I use my network username and password?

There's an extra port number in MCWS_server. Use the username and password you configured in MC's Media Network settings.

Make sure you using at least MC 19.0.54.

SkGe · « **Reply #26 on:** October 22, 2013, 03:25:58 pm »

If i want to extract only this text from amg url what is the expression?
I did try several type and none help me, and did saw that you use regex() but nothing help me.
Example:
http://www.allmusic.com/album/get-wet-mw0002569538
Each url have a end of - mw(numbers)

MrC · « **Reply #27 on:** October 22, 2013, 03:35:51 pm »

You could use:

regex([AMG URL], /#-([^-]+)$#/, 1)

SkGe · « **Reply #28 on:** October 22, 2013, 03:44:38 pm »

Thank you!

SkGe · « **Reply #29 on:** October 30, 2013, 10:53:08 am »

MrC one question about this script?
Can this script cature the section with similar albums?

MrC · « **Reply #30 on:** October 30, 2013, 12:01:50 pm »

Quote from: sirgeo on October 30, 2013, 10:53:08 am

MrC one question about this script?
Can this script cature the section with similar albums?

It could. If others are interested, I'll add it.

Sapagrino · « **Reply #31 on:** November 01, 2013, 06:59:02 am »

+1 from me for Similar Albums

Most important missing info from my perspective is the "Recording Date(s)" (e.g., for Beatles Help!, would be February 15, 1965 - June 17, 1965). Can you consider adding?

Also would love to see the most universal Billboard charts have their own field containing the Peak Position--that would seem to be "The Billboard Hot 100" for songs, and "the Billboard 200" for albums. Being able to create smart lists with, say, songs that peaked at #1 would be great, or count downs from lowest to highest charted...

I had intended doing all three of the above fields manually over time until this script, so would love to see added.

I have a few other observations/ruminations (not necessessarily requests yet):

If you take my above suggestion, would giving more of the Billboard charts (e.g., R&B Singles/Albums) their own field be a good idea? I would say yes, but not sure how much extra work.

All of the Awards associated with a given song are populated to ALL songs on the album. I might prefer the song awards to just appear with the relevant song. However, maybe that is not easy, and it's still great the way it is.

It would be great to also copy/paste some of the data associated with the Album's Artist page. Artist Bio, Group Members, active, formed, disbanded, and the related tab would be my top field candidates, but the possibilities are many. I presume this too is not straightforward.

I note the Album and Track Picks take different approaches. Tracks are "pick" but Album is "1". Is there a reason they are not both "pick" or both "1"? Not a big deal, obviously.

I am wondering if there is any benefit to capturing the AMG Pop ID or Rovi Music ID for a given Album. Not currently sure what I'd use it for, and the URL seems to provide the link back to the page, so maybe nothing to be gained. Others might have ideas though, so thought I'd mention.

Thank you for considering these ideas, MrC.

Sapagrino

SkGe · « **Reply #32 on:** November 01, 2013, 11:10:46 am »

The id of album/artist or track is the end of each url on that page, and you can use what MrC show me to do, just to extract that id.
I also think about if it possible to get that info from artist page, especially relation tab.
Example of album id h.ttp://www.allmusic.com/album/the-lone-ranger-mw0002556628
And for artist id: h.ttp://www.allmusic.com/artist/hans-zimmer-mn0000961427
And for track pick and album pick isn't important, before was same as, 1, now is pick/1 but i think you can change on the code, didn't try yet.

MrC · « **Reply #33 on:** November 01, 2013, 08:18:12 pm »

Quote from: Sapagrino on November 01, 2013, 06:59:02 am

+1 from me for Similar Albums

OK, I'll add Similar Albums (and map it to the same field name). It will be a semicolon-separated list of artist/album pairs. For each item, how about the format: "Help!\The Beatles" or "Help! [The Beatles]" ?

Quote from: Sapagrino

Most important missing info from my perspective is the "Recording Date(s)" (e.g., for Beatles Help!, would be February 15, 1965 - June 17, 1965). Can you consider adding?

Sure, this is easy. The return will just be the string presented on the page, so it won't be a date type. I'll map it to the field Recording Date.

Quote from: Sapagrino

Also would love to see the most universal Billboard charts have their own field containing the Peak Position--that would seem to be "The Billboard Hot 100" for songs, and "the Billboard 200" for albums. Being able to create smart lists with, say, songs that peaked at #1 would be great, or count downs from lowest to highest charted...

I had intended doing all three of the above fields manually over time until this script, so would love to see added.

So you want, for example, a "Billboard Hot 100 (peak)" field, which contains the peak value? That's easy enough - all the parts are already there.

Quote from: Sapagrino

I have a few other observations/ruminations (not necessessarily requests yet):

If you take my above suggestion, would giving more of the Billboard charts (e.g., R&B Singles/Albums) their own field be a good idea? I would say yes, but not sure how much extra work.

The entire page is already pull/parsed, so adding the values to additional fields is no problem. I don't have a handle on how many different awards there are. Do you want each one, or just a subset (and if so, which ones)?

Quote from: Sapagrino

All of the Awards associated with a given song are populated to ALL songs on the album. I might prefer the song awards to just appear with the relevant song. However, maybe that is not easy, and it's still great the way it is.

Probably an oversight - I'll correct this.

Quote from: Sapagrino

It would be great to also copy/paste some of the data associated with the Album's Artist page. Artist Bio, Group Members, active, formed, disbanded, and the related tab would be my top field candidates, but the possibilities are many. I presume this too is not straightforward.

The implementation is straightforward - it's just another page pull/parse and field assignments. However, implementing this might create a problem. For single artist albums, it requires one more page pull/parse for the main Artist page, and then one more for the Biography, and then one more for the Related. For multi-artist albums, this would be those three pages multiplied by the number of tracks. That's a lot of pages to grab. I certainly would not enable this by default. With everything enabled, an album of 10 tracks would pull 5 + (3 * 10) = 35 pages! It gets much worse when several artists are listed per track. I don't want the script to be abusive to A M G.

In fact, I'm thinking of disabling by default all page pulls but the main album page; specifying options would be required to enable them.

Quote from: Sapagrino

I note the Album and Track Picks take different approaches. Tracks are "pick" but Album is "1". Is there a reason they are not both "pick" or both "1"? Not a big deal, obviously.

I'll make these consistent - it will be set to 1 when appropriate, and empty otherwise.

Quote from: Sapagrino

I am wondering if there is any benefit to capturing the AMG Pop ID or Rovi Music ID for a given Album. Not currently sure what I'd use it for, and the URL seems to provide the link back to the page, so maybe nothing to be gained. Others might have ideas though, so thought I'd mention.

If you have a need for it, then I can grab it. Otherwise, let's ignore it until its needed.

Sapagrino · « **Reply #34 on:** November 02, 2013, 11:27:37 am »

Quote from: MrC on November 01, 2013, 08:18:12 pm

OK, I'll add Similar Albums (and map it to the same field name). It will be a semicolon-separated list of artist/album pairs. For each item, how about the format: "Help!\The Beatles" or "Help! [The Beatles]" ?

Of those two, I'm a vote for [the brackets], but may not have considered all pros/cons, and would defer to any strong preference of yours/sergio's. Two other choices might be "The Beatles/help!" or "The Beatles\help!".

The following is a bit off-topic (or part of a broader topic), but…

...I bring up these two other choices especially because I noticed something interesting about how MC handles slashes, at least with respect to certain fields. That is, I am on my iPad right now, but from memory, take note of the following:

1. with respect to the Genre field...
2. dbpa ripped "Pop/Rock" for (only) one of the five file formats I tested as "Pop\Rock" (I.e. a reverse/)...
3. within MC Panes view, Genre pane...
4. Showed "pop/rock" for files with respect to four of the five file formats...
5. But showed "pop", and then a twisty that you could click to have it then show "rock" for files with respect to that one of the five file formats.

Thus, the reverse/ seems to be a very useful way to get MC to reflect hierarchical relationships that can be manipulated within a single Pane, a fairly sensible approach especially with respect to genres. I do not know if the same functionality applies by default for user-defined fields, or if such feature could be incorporated in future versions. I will assume yes.

So back to the question at hand. If the above is the case, I think I might change my vote to "the Beatles\Help!". That is, use the reverse/ not merely as a convention but actually as a way to build in a specific feature. Thus, if one were to say that an album's track is similar to several albums by the Beatles, the Beatles would show up first as the option in the pane and then a twisty could be clicked, where you would see the specific Beatles albums that were noted as being similar.

Of course, the above would be academic if there are not many similar albums populated in the field, which in practice may be the case. Still, if the functionality is there for user defined fields, and no one sees a downside to this convention, I think that would be my vote.

Having said all this, I am perfectly fine using the square brackets. I just thought the above was sort of interesting.

I will reply separately to the other points above.

Sapagrino · « **Reply #35 on:** November 02, 2013, 12:32:51 pm »

Quote from: MrC on November 01, 2013, 08:18:12 pm

Sure, this is easy. The return will just be the string presented on the page, so it won't be a date type. I'll map it to the field Recording Date.

Wonderful!

I recommend pluralizing (Recording Dates)–- to help distinguish it from the fields that are truly date format. And the fact that it is likely a range of dates.

Thank you so much, MrC.

MrC · « **Reply #36 on:** November 02, 2013, 12:40:23 pm »

Regarding brackets vs. forward- or backward-slashes:

- Brackets are more difficult to parse out with MC (if you have a need in the future to do so). Single-character separators are easy.
- Forward slash is ambiguous sometimes:
+ Dirty Deeds Done Dirt Cheap/AC/DC
+ Beethoven: Concerto No.5/Rachmaninov: Concerto No.2/Fritz Reiner
- Backslash allows easily creating hierarchical lists (as you noticed) in panes and categories views.

Sapagrino · « **Reply #37 on:** November 02, 2013, 03:14:24 pm »

Quote from: MrC on November 02, 2013, 12:40:23 pm

Regarding brackets vs. forward- or backward-slashes:

- Brackets are more difficult to parse out with MC (if you have a need in the future to do so). Single-character separators are easy.
- Forward slash is ambiguous sometimes:
+ Dirty Deeds Done Dirt Cheap/AC/DC
+ Beethoven: Concerto No.5/Rachmaninov: Concerto No.2/Fritz Reiner
- Backslash allows easily creating hierarchical lists (as you noticed) in panes and categories views.

Backslash (what I was calling reverse/ above), with Artist preceding Album "Beatles\Help!", definitely seems the way to go, for all these reasons. ...and probably use Sort Order Artist so not "The..."?

Could make populating this field more interesting and manageable. Nice solution. Thanks for the link.

I will use this backslash strategy. Trying to recall which of the other fields in the script are of a list type such that this hierarchical feature would be available or useful. One might be able to "group" various Moods, for example into four quadrants of high/low positive/negative energy were someone so inclined.

I will think some more about this. Thanks.

Sapagrino · « **Reply #38 on:** November 03, 2013, 06:06:06 pm »

MrC, Apologies that I am getting to these replies in pieces, as opposed to the nice logical complete reply you provided!

Quote from: MrC on November 01, 2013, 08:18:12 pm

So you want, for example, a "Billboard Hot 100 (peak)" field, which contains the peak value? That's easy enough - all the parts are already there.

Exactly. Having this for the Billboard Hot 100 and Billboard 200 is a high priority of mine. Thank you.

----------------

Quote from: MrC on November 01, 2013, 08:18:12 pm

The entire page is already pull/parsed, so adding the values to additional fields is no problem. I don't have a handle on how many different awards there are. Do you want each one, or just a subset (and if so, which ones)?

I have found the following interesting resources:

http://www.billboard.com/charts
http://en.wikipedia.org/wiki/Billboard_charts
http://en.wikipedia.org/wiki/Hot_R%26B/Hip-Hop_Songs

There are clearly too many charts to give devoted fields to, but from a quick look, I can see already that I could pull together a rational representation of the major Genres. I will look a bit more at this and report back.

One slight complication—if you look, say, at Stevie Wonder’s awards from the Artist page, you see that he was always climbing up or leading the “R&B Singles” charts, until 1984, when it became the "Hot R&B/Hip-Hop Singles & Tracks”. I can provide a proposed mapping and brief rationale. It wouldn’t be perfect, but could be tweaked for any glaring omissions. I assume you can have the script funnel both of the above into a field called, say, “ BB R&B/Hip-Hop Singles”?

For me, having the major Single and major Album charts is already just terrific.

I do think it would bring things to a great new level to be able to better “mine” high charting tracks for certain Genres at certain times. Say, Hot R&B/Hip-Hop Singles that peaked very high in that chart, and which have any value (i.e., charted at all) on the overall Hot 100.

That might help me curate a mix for a party that had a small contingent that especially appreciated a given Genre, for example, but knowing I wanted to try and keep things familiar for the rest of the guests (e.g., Lauryn Hill’s Doo Wop (That Thing) (#2/#1). Or maybe the other way around, charting Genre tracks that didn’t even chart on the overall (Zion (-/#77), from that same album)—more the after-hours party where that smaller contingent would appreciate the greater niche. Possibilities are endless.

Like I said, I will be looking at how things map out and report back; I have been wanting to do this anyway so I understand which are the truly major chart toppers in the various Genres.

Will finish responding to the other couple open points shortly...

MrC · « **Reply #39 on:** November 04, 2013, 07:16:31 pm »

Here's a 1.14 pre-release that includes some of the changes thus far requested:

   - Fix: function to capture Recording Date was not being called. Key 'rdate' maps to
   MC field "Recording Dates".
   - New: Similar Albums captured: key 'similar'.
   - Fix: Track Pick value set to 1 (instead of 'pick') for track picks.

I'm not sure how I'm going to handle the awards yet. There is some difficulty with creating the (peak) versions. For example, the Help! album has two entries for Billboard Albums: Top Soundtracks, one for the year 2013 and one for 2007. Each happen to have a peak of 1. But other the chart Top Pop Catalog has two entries also, one for 2013 which peaks at 4, and the other for 2009 which peaks at 7. How should this be handled? An XYZ (peak) field which shows only the peak value won't work, and a list field will contain each peak value, but doesn't include the date, so you'll have a list with numbers that have no relation to any dates.

Sapagrino · « **Reply #40 on:** November 04, 2013, 11:11:59 pm »

Thank you! I will try and test it this week.

Quote from: MrC on November 04, 2013, 07:16:31 pm

There is some difficulty with creating the (peak) versions. For example, the Help! album has two entries for Billboard Albums: Top Soundtracks, one for the year 2013 and one for 2007. Each happen to have a peak of 1. But other the chart Top Pop Catalog has two entries also, one for 2013 which peaks at 4, and the other for 2009 which peaks at 7. How should this be handled? An XYZ (peak) field which shows only the peak value won't work, and a list field will contain each peak value, but doesn't include the date, so you'll have a list with numbers that have no relation to any dates.

Yes. I see.

How about a default to use:
1) the highest it peaked on that chart in ANY year? ("4" in your 2nd example)
With a switch to instead use;
2) its peak in the earliest year it charted ("7"); or
3) it's peak in the latest year it charted ("4")?

Or maybe #2 is the better default, in the broader context (even if not here)? Likely, a SONG's original year of release, rather than some later year it may have re-emerged on a movie soundtrack?

I think we want a single sortable # in the field, and of course all the award details with each year's peak still resides in the original awards field. You could still go back to that were you to need to see which songs in the library charted in what years to what peak.

SkGe · « **Reply #41 on:** November 05, 2013, 03:30:38 am »

Something change in the last version when i want to grab information
C:\Users\...\Desktop>perl amg.pl -u http://www.allmusic.com/album/... -c amg.config
*** can't open config file amg.config
Stopped at amg.pl line 408.
I did look on the file at that section from last and this new version and nothing changed. But if i use the older one isn't a problem but with this one this error appear.

Sapagrino · « **Reply #42 on:** November 05, 2013, 06:01:17 am »

Quote from: MrC on November 01, 2013, 08:18:12 pm

The implementation is straightforward - it's just another page pull/parse and field assignments. However, implementing this might create a problem. For single artist albums, it requires one more page pull/parse for the main Artist page, and then one more for the Biography, and then one more for the Related. For multi-artist albums, this would be those three pages multiplied by the number of tracks. That's a lot of pages to grab. I certainly would not enable this by default. With everything enabled, an album of 10 tracks would pull 5 + (3 * 10) = 35 pages! It gets much worse when several artists are listed per track. I don't want the script to be abusive to A M G.

I would think to limit it to single artist albums, or perhaps leverage the Album Artist concept where a high enough percentage of tracks are from same Artist. Not that odd I think to have "missing" artist details on Various Artists albums.

Sapagrino · « **Reply #43 on:** November 05, 2013, 06:12:49 am »

MrC, I have been meaning to follow-up. In a PM, you said:

"There are two ratings you can use. MC's rating is 0-5, so the AMG rating (i.e. 'rating_mc') is scaled. You can use the 'rating' key which maps by default to AMG Rating, and configure that value to be either float or decimal. See the help in:

amg.pl --help

specifically looking at the --rating option."

I tried that, but must be misunderstanding the syntax as it did not work for me. Can you type out exactly what to type? I want albums that AMG rates 4.5 to retain the half star, so 9. Now, it rounds to 8 or 10 depending on file format. Thank you.

SkGe · « **Reply #44 on:** November 05, 2013, 06:30:50 am »

Quote from: Sapagrino on November 05, 2013, 06:12:49 am

MrC, I have been meaning to follow-up. In a PM, you said:

"There are two ratings you can use. MC's rating is 0-5, so the AMG rating (i.e. 'rating_mc') is scaled. You can use the 'rating' key which maps by default to AMG Rating, and configure that value to be either float or decimal. See the help in:

amg.pl --help

specifically looking at the --rating option."

I tried that, but must be misunderstanding the syntax as it did not work for me. Can you type out exactly what to type? I want albums that AMG rates 4.5 to retain the half star, so 9. Now, it rounds to 8 or 10 depending on file format. Thank you.

Find this lines in the code:
else {
$opts{'rating'} = 'd';
}
Change from "d" to "f" and it will be like in allmusic 0.0-5.0 (.5)

MrC · « **Reply #45 on:** November 05, 2013, 10:49:03 am »

Quote from: sirgeo on November 05, 2013, 03:30:38 am

Something change in the last version when i want to grab information
C:\Users\...\Desktop>perl amg.pl -u http://www.allmusic.com/album/... -c amg.config
*** can't open config file amg.config
Stopped at amg.pl line 408.
I did look on the file at that section from last and this new version and nothing changed. But if i use the older one isn't a problem but with this one this error appear.

I've added better diagnostics for this case. You can add it to your script. Change line 409 to:

die "can't open config file $configfile\n$!\nStopped";

and retry to see the specific problem. Are you sure the file is there, and by that name?

MrC · « **Reply #46 on:** November 05, 2013, 10:50:56 am »

Quote from: Sapagrino on November 05, 2013, 06:12:49 am

I tried that, but must be misunderstanding the syntax as it did not work for me. Can you type out exactly what to type? I want albums that AMG rates 4.5 to retain the half star, so 9. Now, it rounds to 8 or 10 depending on file format. Thank you.

Choose one of:

--rating d
-r d
--rating f
-r f

SkGe · « **Reply #47 on:** November 05, 2013, 12:27:36 pm »

C:\Users\\Desktop>perl amg.pl -u http://www.allmusic.com/album/ignition-mw0000193310 -c amg.config
*** can't open config file amg.config
No such file or directory
Stopped at amg.pl line 408.
Changed the script to that you write here but it seems is the same issues like the one i told you early.
My file are both where was the last files. When i changed this new one with the older i only modify the field to meet my field assigned.

MrC · « **Reply #48 on:** November 05, 2013, 12:29:28 pm »

I'm not sure I'm following you. The error "No such file or directory" really means that file does not exist by that name at that location. Probably it has a .txt suffix (and you have suffixes hidden in your system's Folder prefs so don't see it)

SkGe · « **Reply #49 on:** November 05, 2013, 12:36:33 pm »

Yeah, you have right, the file it seems saved in .txt file, dunno, maybe when i did redefine the fields.
Now is working.
Thanks.

INTERACT FORUM

Author Topic: amg.pl: an A M G helper and dummy file creator (Read 87896 times)