INTERACT FORUM
Windows => Third Party Plug-ins, Programs, and Skins => Topic started by: MrC on October 09, 2013, 08:52:46 pm
-
Description
The following command script pulls metadata from AMG to tag your Media Center tracks. In addition, it can also create dummy files that can be imported and tagged to simulate collections for which physical media does not exist (e.g Vinyl, desired CDs, etc.). It is for personal use only, and it's sole purpose is to avoid the risk of carpal tunnel from the copying and pasting you are doing already. :-)
The script is written in Perl, and can be used on Windows, Mac OS X, or Linux. It basically works like this:
1. You supply an album's AMG release URL
2. You select the album's tracks in a view in MC
3. You run the script
The script will do one or both of:
- update MC fields directly via MCWS (version 19.0.54+, Media Server needs to be running with authentication enabled)
- produce an MPL file which will update the MC fields when imported into MC via File > Import Playlist
Available Metadata
The following metadata can be retrieved (when available):
AMG album, track, and artist IDs and URLs
AMG album and track picks
AMG album and track reviews
Ratings (AMG rating, AMG-to-MC-mapped rating)
Awards (Billboard Singles, Billboard Albums, Grammy Albums)
Releases
Album credits (with and without names)
Styles, Moods, Themes
Release date
Album
Genre
Track name
Track artist
Track composer
Track featuring
Lyrics
Disc and track numbers
Download the attached zip file, and unzip the items to a folder (right-click > Extract All). See the README.txt file inside the folder.
The original post containing the development and release history of this software is at:
http://yabb.jriver.com/interact/index.php?topic=79942.0 (http://yabb.jriver.com/interact/index.php?topic=79942.0)
-
Options
There are several options available to change the behavior of the script. You can view a list of the options with:
perl amg.pl --help
Although most should be self-explanatory, it is worth commenting on a couple of options here.
The script works by pulling the track info from the clipboard, matching the disc and track numbers from your selected file list with those from AMG. Valid track numbers or sequences are required for this matching, because to import the items and associate them with your existing MC entries, the track's Filename field must be used.
The --sequence and --fileregex options allow you to deal with situations where your tracks do not quite match up with the AMG tracks. Your release may contain bonus tracks, and the --sequence option instructs the script to use the list order of your tracks and match them against the track order in the AMG list. For any supplementary tracks you have (that are not listed in the AMG track list), album-related field values will still be output for your additional tracks. The --fileregex option allows you to grab track numbers contained in your file names as the basis for creating the sequence in which to match against the AMG track sequence.
In Step 3, you supplied the URL on the command line. If at least the first track of the album is tagged with an AMG URL, and the AMG URL field is in the view, the -u option is not necessary and the AMG URL from MC will used. This is useful if you want to re-scrape the data in the future (or us it as the target of an MC Link). The command would then just be:
perl amg.pl
Two options allow you to quickly include or exclude fields. By default, the script will attempt to produce values for all of the available fields. The --include option reverses this behavior such that only the listed fields are made available. The --exclude option allows you to specify a list of fields that should be excluded. Some fields require additional web page look-ups (e.g. credits, awards, track reviews, lyrics), so it is best to always use the --include option with the specific list of fields you want. Once you become familiar with how the script works, you can switch to using the --exclude option to ignore only those fields you don't want. Each of these options is supplied with a comma-separated list of field keys, which you can see in the sample config file; there is one key per Field.
The --configfile option allows you to specify options inside a configuration file. You examined the supplied file earlier. It would be worthwhile to copy the supplied configuration file and make your changes to your copy. This way, new releases of amg.pl will not overwrite your changes. Spend a few minutes reading the comments in the configuration file.
-
Usage Tips
- Creating and using MC Links to jump to Album, Artist and Track sections at AMG
Version 1.25 added support for Album, Artist, Composer, and Track IDs. These are convenient for creating and using MC Links to jump to the correct location on the AMG site. Here are several expression-based Search URLs that can help you locate the correct entries on AMG when no IDs exist yet for your track(s), or find the exact item when your ID fields are populated. I found the Album search link particularly useful when initially searching for an Album ID, as the correct album entry almost always appeared in the search results, usually at or near the the top. For those difficult albums, a track-search or artist search helped. Simple select the tracks for an album, and click one of the Links.
(http://i1200.photobucket.com/albums/bb336/Mr__C/MC%20Screenies/AMGlinks_zpsa7812ab6.png)
The following searches will attempt to jump to the exact album, artist, composer, or track, respectively, based the corresponding ID value if it exists, or based on the corresponding MC field contents (Album, Artist, Composer, and Name):
Album
http:////www.allmusic.com//If(!isempty([AMG Album ID,0]), album//ifelse(isequal(left([AMG Album ID], 2), mr),release//)[AMG Album ID], search//albums//[album])
Artist
http:////www.allmusic.com//If(!isempty([AMG Artist ID,0]), artist//[AMG Artist ID], search//artist//[Artist])
Composer
http:////www.allmusic.com//If(!isempty([AMG Composer ID,0]), artist//[AMG Composer ID], search//artist//FirstNotEmpty([Composer],[Artist]))
Track Name
If(!isempty([AMG Track ID,0]), http:////www.allmusic.com//if(isequal([AMG Track ID],mq,7),performance,song)//[AMG Track ID], http:////www.allmusic.com//search//songs//[Name])
When you configure the links, as a Base URL of http://www.allmusic.com for each of these, and use the above search expressions for the Search URL. I configure mine to open in an external browser. You can limit the links to Media Type=Audio by selecting the Edit Search... button, and setting a rule for Media Type is Audio. More help on creating links (http://yabb.jriver.com/interact/index.php?topic=83371.0).
-
Who remembers aTagger by al@n? It was brilliant, and continued to work for some years after the rug got pulled.
I'm not sure if they fall into the same category, it certainly feels like they do, in which case, get ready for a 'cease and desist' (http://yabb.jriver.com/interact/index.php?topic=32276.0)
-
This is one reason the script essentially requires a visit to the AMG site via browser.
-
Hey MrC!
Looking good, thanks again.
I'm having a couple of issues though ... I'm getting these errors often
C:\Users\S\Desktop>perl amg.pl -u http://www.allmusic.com/album/no-place-for-my
dream-mw0002541163
Ignored extra track in selection: 1.1 Z:\Temp\FEMI KUTI\No Place for My Dream\0
_Nothing to Show for It_No Place for My Dream_Femi Kuti_2013.flac
Ignored extra track in selection: 1.2 Z:\Temp\FEMI KUTI\No Place for My Dream\0
_The World Is Changing_No Place for My Dream_Femi Kuti_2013.flac
Ignored extra track in selection: 1.3 Z:\Temp\FEMI KUTI\No Place for My Dream\0
_No Place for My Dream_No Place for My Dream_Femi Kuti_2013.flac
Ignored extra track in selection: 1.4 Z:\Temp\FEMI KUTI\No Place for My Dream\0
_Action Time_No Place for My Dream_Femi Kuti_2013.flac
Ignored extra track in selection: 1.5 Z:\Temp\FEMI KUTI\No Place for My Dream\0
_No Work No Job No Money_No Place for My Dream_Femi Kuti_2013.flac
Ignored extra track in selection: 1.6 Z:\Temp\FEMI KUTI\No Place for My Dream\0
_Carry on Pushing on_No Place for My Dream_Femi Kuti_2013.flac
Ignored extra track in selection: 1.7 Z:\Temp\FEMI KUTI\No Place for My Dream\0
_Politics Na Big Business_No Place for My Dream_Femi Kuti_2013.flac
Ignored extra track in selection: 1.8 Z:\Temp\FEMI KUTI\No Place for My Dream\0
_Na So Wee See Am_No Place for My Dream_Femi Kuti_2013.flac
Ignored extra track in selection: 1.9 Z:\Temp\FEMI KUTI\No Place for My Dream\0
_One Man Show_No Place for My Dream_Femi Kuti_2013.flac
Ignored extra track in selection: 1.10 Z:\Temp\FEMI KUTI\No Place for My Dream\
0_Wey Our Money_No Place for My Dream_Femi Kuti_2013.flac
Ignored extra track in selection: 1.11 Z:\Temp\FEMI KUTI\No Place for My Dream\
1_This Is Only the Beginning_No Place for My Dream_Femi Kuti_2013.flac
C:\Users\S\Desktop>perl amg.pl -u http://www.allmusic.com/album/rossini-la-donna
-del-lago-mw0001857236
AMG HTML has changed. amg.pl needs to be updated for the new format.
Stopped at amg.pl line 335.
Questions
- Can this be run on a network drive, where the files do not reside locally?
- Can this be run on multiple CDs where some of the Trk info has been changed (eg. Disk one track 1-12 (no change) Disk 2 track 1 to 13 renamed to 13-25)
-
I believe the problem you are having with the No Place for My Dream album is that you have Disc # set to 1, and typically Disc # would be left empty for single-album discs. No matter. Just use the --sequence (or the shorter -s) option with your tracks in track order in the file list.
Regarding your questions:
1. So long as the files are available to MC as windows file systems (mounted or local), it doesn't matter.
2. Use -s, with the tracks in sequence. amg.pl will then ignore the disc and track numbers, and construct its index from the sequence of your tracks.
Once you're comfortable, try using the --mcws method for direct updates (if you have Media Server running). You can have amg.pl also output MPL so you can examine it.
-
One more thing about some error
Failed to get page: http://www.allmusic.com/song/where-you-are-club-version-mt0042906550/lyrics
404 Not Found
Use of uninitialized value $_ in pattern match (m//) at amg.pl line 863.
-
One more thing about some error
Failed to get page: http://www.allmusic.com/song/where-you-are-club-version-mt0042906550/lyrics
404 Not Found
Use of uninitialized value $_ in pattern match (m//) at amg.pl line 863.
AMG is down for maintenance give; try again a bit later
-
Now i see that there is not any lyrics :D
-
Is not down now, it was earlier when i was browse the page there.
-
A 404 error means a temporary failure. I'll put in better error messages.
-
And what by this message on some albums?
C:\Users\S\Desktop>perl amg.pl -u http://www.allmusic.com/album/rossini-la-donna
-del-lago-mw0001857236
AMG HTML has changed. amg.pl needs to be updated for the new format.
Stopped at amg.pl line 335.
If often happen when album is some movie soundtrack or related to this.
-
And what by this message on some albums?
C:\Users\S\Desktop>perl amg.pl -u http://www.allmusic.com/album/rossini-la-donna
-del-lago-mw0001857236
AMG HTML has changed. amg.pl needs to be updated for the new format.
Stopped at amg.pl line 335.
If often happen when album is some movie soundtrack or related to this.
I see what the problem is, and I'm going to change how I'm parsing tracks. Back shortly...
-
Had a look at this...
Went for ActivePerl 5.18 thinking latest = best, but all of the 'ppm' instructions failed.
Removed and installed 5.16, ppm instructions all successful.
Followed steps for mpl, failed...
Failed to open \Users\Mark Wilkinson\Desktop\import.mpl: No such file or directory
Stopped at amg.pl line 903.
I tried this twice. The directory definitely exists, if you add C: to the front of the path. I created a blank import.mpl file there and tried again, same error.
I went to 'System Variables' and added an entry 'Desktop' using the full path and tried again... same error.
I have had some success with the -mcws option, but only via the command line, it does not appear to working via the config file.
-marko
-
5.18 must be new, and the modules have not yet been released for that version. I'll note it.
You need either a relative path, or a full path. The path starting with \ is probably not what you want.
I've used the config file with mcws. Can you show the settings and cmd line you used? Obfuscate the password, of course.
Edit. You can use the -o option to specify an MPL file path. I'm not sure why the script can't create a file on your desktop. I'll check on this tomorrow.
Edit 2: sorry for the brief reply - I was replying on a tablet. I'd like to get to the bottom of the problem you're having, but I'll need to see your command line, and know the directory you're in (run command DIR). There is no need to set any system variables.
-
Version 1.12 is now posted:
Changes:
- Fix: occasional problems parsing amg tracks (parsing algorithm changed and simplified - please report
any problems).
- Fix: better error messages when page loads fail.
@sergio - this should resolve the problems with the URL you showed above.
-
Thank you for developing this. I have it almost working. When I import playlist, nothing is imported for the first track, and the other tracks have the info disassociated with the music info--as if the filename had not been supplied to match things up. When I examine the import file, I see that the first track's filename is among the fields/data, but the filename field is missing for the other tracks. So, two things are surprising: 1) while track 1's filename is in the import file, none of the data from the input file is seen in JMC for that track, and no disassociated track 1 data either; 2) Even though I highlighted all tracks in JMC--with filename and track # included in view--before copying (tried both Ctrl-C and right click) only the first track's filename seems to have passed all the way through to the input file.
I tried copying from JMC directly into Notepad, and indeed that demonstrates that all track's filenames and track #s have made it to the clipboard before hitting enter on the perl script command line; and the input file is filled with data for all tracks.
I am sure I am missing something, but I have tried over a dozen times, read some of your other posts on importing data, etc., so was wondering if the above fact pattern suggests to you what I might be doing wrong. Thank you for your help.
-
Can you do me a favor? Run the command and this time include at the beginning of the command line -v -d and at the end of the command line, redirect the output to a file on your desktop, like this:
perl amg.pl -d -v ... > amg-out.txt
where you replace ... with your current options. Then, after the command completes, zip both the import.mpl and amg-out.txt file and attach it here.
-
Here is the zip file. Thank you for looking into this.
-
I've sent you a PM requesting a few more details. We can continue via email. It appears something is wrong with the old (MC18) clipboard contents, or in splitting it into rows. But I'll need your help in reproducing it.
-
Thank you. Just sent you an email.
-
I've updated the script to version 1.13. Everyone using the script should update. The changes are as follows:
Version 1.13
- Fix: clipboard parsing for MC18
- Fix: handle Unicode clipboard contents on Windows.
- Fix: output diagnostics in UTF8
-
aha a new version ...
for info Mr C I got the 1.12 version to work using the -s and --configfile options in the command line. However I have not been able to use the --mcsw option when I put the server IP in nothing happened, and so I tried using just "localhost" which totally crashed my network ;D
For info, I have an Apache web server running with some perlscripts loaded, should I be turning this off?
BTW -- running off a jr client with ip of lets say 1.192.168.12 with the jrserver at 1.192.168.11 am I right in assuming that in the config file I would uncomment and put in
MCWS_server: 1.192.168.11:52199:52199
MCWS_username: jriverusername
MCWS_password: jriveraccesspwd
or do I use my network username and password?
Anyway, I'll try with the new version before giving you the details ...
-
I've not forgotten about this, just have no time. I'm software testing on dayshift this week, then back to nightshift next week for holiday cover. One yukky fortnight!
-
for info Mr C I got the 1.12 version to work using the -s and --configfile options in the command line.
Thanks for the update.
However I have not been able to use the --mcsw option when I put the server IP in nothing happened, and so I tried using just "localhost" which totally crashed my network ;D
Nice!
I'm not sure how this could have crashed a network. Did you include the port number, as per the script's example? Without it, you'd be connecting to the web server running on port 90 of the local machine, and that's probably not MC.
For info, I have an Apache web server running with some perlscripts loaded, should I be turning this off?
It is not necessary. MC uses a different port than the standard port 80 for web services.
BTW -- running off a jr client with ip of lets say 1.192.168.12 with the jrserver at 1.192.168.11 am I right in assuming that in the config file I would uncomment and put in
MCWS_server: 1.192.168.11:52199:52199
MCWS_username: jriverusername
MCWS_password: jriveraccesspwd
or do I use my network username and password?
There's an extra port number in MCWS_server. Use the username and password you configured in MC's Media Network settings.
Make sure you using at least MC 19.0.54.
-
If i want to extract only this text from amg url what is the expression?
I did try several type and none help me, and did saw that you use regex() but nothing help me.
Example:
http://www.allmusic.com/album/get-wet-mw0002569538
Each url have a end of - mw(numbers)
-
You could use:
regex([AMG URL], /#-([^-]+)$#/, 1)
-
Thank you!
-
MrC one question about this script?
Can this script cature the section with similar albums?
-
MrC one question about this script?
Can this script cature the section with similar albums?
It could. If others are interested, I'll add it.
-
+1 from me for Similar Albums
Most important missing info from my perspective is the "Recording Date(s)" (e.g., for Beatles Help!, would be February 15, 1965 - June 17, 1965). Can you consider adding?
Also would love to see the most universal Billboard charts have their own field containing the Peak Position--that would seem to be "The Billboard Hot 100" for songs, and "the Billboard 200" for albums. Being able to create smart lists with, say, songs that peaked at #1 would be great, or count downs from lowest to highest charted...
I had intended doing all three of the above fields manually over time until this script, so would love to see added.
I have a few other observations/ruminations (not necessessarily requests yet):
If you take my above suggestion, would giving more of the Billboard charts (e.g., R&B Singles/Albums) their own field be a good idea? I would say yes, but not sure how much extra work.
All of the Awards associated with a given song are populated to ALL songs on the album. I might prefer the song awards to just appear with the relevant song. However, maybe that is not easy, and it's still great the way it is.
It would be great to also copy/paste some of the data associated with the Album's Artist page. Artist Bio, Group Members, active, formed, disbanded, and the related tab would be my top field candidates, but the possibilities are many. I presume this too is not straightforward.
I note the Album and Track Picks take different approaches. Tracks are "pick" but Album is "1". Is there a reason they are not both "pick" or both "1"? Not a big deal, obviously.
I am wondering if there is any benefit to capturing the AMG Pop ID or Rovi Music ID for a given Album. Not currently sure what I'd use it for, and the URL seems to provide the link back to the page, so maybe nothing to be gained. Others might have ideas though, so thought I'd mention.
Thank you for considering these ideas, MrC.
Sapagrino
-
The id of album/artist or track is the end of each url on that page, and you can use what MrC show me to do, just to extract that id.
I also think about if it possible to get that info from artist page, especially relation tab.
Example of album id h.ttp://www.allmusic.com/album/the-lone-ranger-mw0002556628
And for artist id: h.ttp://www.allmusic.com/artist/hans-zimmer-mn0000961427
And for track pick and album pick isn't important, before was same as, 1, now is pick/1 but i think you can change on the code, didn't try yet.
-
+1 from me for Similar Albums
OK, I'll add Similar Albums (and map it to the same field name). It will be a semicolon-separated list of artist/album pairs. For each item, how about the format: "Help!\The Beatles" or "Help! [The Beatles]" ?
Most important missing info from my perspective is the "Recording Date(s)" (e.g., for Beatles Help!, would be February 15, 1965 - June 17, 1965). Can you consider adding?
Sure, this is easy. The return will just be the string presented on the page, so it won't be a date type. I'll map it to the field Recording Date.
Also would love to see the most universal Billboard charts have their own field containing the Peak Position--that would seem to be "The Billboard Hot 100" for songs, and "the Billboard 200" for albums. Being able to create smart lists with, say, songs that peaked at #1 would be great, or count downs from lowest to highest charted...
I had intended doing all three of the above fields manually over time until this script, so would love to see added.
So you want, for example, a "Billboard Hot 100 (peak)" field, which contains the peak value? That's easy enough - all the parts are already there.
I have a few other observations/ruminations (not necessessarily requests yet):
If you take my above suggestion, would giving more of the Billboard charts (e.g., R&B Singles/Albums) their own field be a good idea? I would say yes, but not sure how much extra work.
The entire page is already pull/parsed, so adding the values to additional fields is no problem. I don't have a handle on how many different awards there are. Do you want each one, or just a subset (and if so, which ones)?
All of the Awards associated with a given song are populated to ALL songs on the album. I might prefer the song awards to just appear with the relevant song. However, maybe that is not easy, and it's still great the way it is.
Probably an oversight - I'll correct this.
It would be great to also copy/paste some of the data associated with the Album's Artist page. Artist Bio, Group Members, active, formed, disbanded, and the related tab would be my top field candidates, but the possibilities are many. I presume this too is not straightforward.
The implementation is straightforward - it's just another page pull/parse and field assignments. However, implementing this might create a problem. For single artist albums, it requires one more page pull/parse for the main Artist page, and then one more for the Biography, and then one more for the Related. For multi-artist albums, this would be those three pages multiplied by the number of tracks. That's a lot of pages to grab. I certainly would not enable this by default. With everything enabled, an album of 10 tracks would pull 5 + (3 * 10) = 35 pages! It gets much worse when several artists are listed per track. I don't want the script to be abusive to A M G.
In fact, I'm thinking of disabling by default all page pulls but the main album page; specifying options would be required to enable them.
I note the Album and Track Picks take different approaches. Tracks are "pick" but Album is "1". Is there a reason they are not both "pick" or both "1"? Not a big deal, obviously.
I'll make these consistent - it will be set to 1 when appropriate, and empty otherwise.
I am wondering if there is any benefit to capturing the AMG Pop ID or Rovi Music ID for a given Album. Not currently sure what I'd use it for, and the URL seems to provide the link back to the page, so maybe nothing to be gained. Others might have ideas though, so thought I'd mention.
If you have a need for it, then I can grab it. Otherwise, let's ignore it until its needed.
-
OK, I'll add Similar Albums (and map it to the same field name). It will be a semicolon-separated list of artist/album pairs. For each item, how about the format: "Help!\The Beatles" or "Help! [The Beatles]" ?
Of those two, I'm a vote for [the brackets], but may not have considered all pros/cons, and would defer to any strong preference of yours/sergio's. Two other choices might be "The Beatles/help!" or "The Beatles\help!".
The following is a bit off-topic (or part of a broader topic), but…
...I bring up these two other choices especially because I noticed something interesting about how JMC handles slashes, at least with respect to certain fields. That is, I am on my iPad right now, but from memory, take note of the following:
1. with respect to the Genre field...
2. dbpa ripped "Pop/Rock" for (only) one of the five file formats I tested as "Pop\Rock" (I.e. a reverse/)...
3. within JMC Panes view, Genre pane...
4. Showed "pop/rock" for files with respect to four of the five file formats...
5. But showed "pop", and then a twisty that you could click to have it then show "rock" for files with respect to that one of the five file formats.
Thus, the reverse/ seems to be a very useful way to get JMC to reflect hierarchical relationships that can be manipulated within a single Pane, a fairly sensible approach especially with respect to genres. I do not know if the same functionality applies by default for user-defined fields, or if such feature could be incorporated in future versions. I will assume yes.
So back to the question at hand. If the above is the case, I think I might change my vote to "the Beatles\Help!". That is, use the reverse/ not merely as a convention but actually as a way to build in a specific feature. Thus, if one were to say that an album's track is similar to several albums by the Beatles, the Beatles would show up first as the option in the pane and then a twisty could be clicked, where you would see the specific Beatles albums that were noted as being similar.
Of course, the above would be academic if there are not many similar albums populated in the field, which in practice may be the case. Still, if the functionality is there for user defined fields, and no one sees a downside to this convention, I think that would be my vote.
Having said all this, I am perfectly fine using the square brackets. I just thought the above was sort of interesting.
I will reply separately to the other points above.
-
Sure, this is easy. The return will just be the string presented on the page, so it won't be a date type. I'll map it to the field Recording Date.
Wonderful!
I recommend pluralizing (Recording Dates)–- to help distinguish it from the fields that are truly date format. And the fact that it is likely a range of dates.
Thank you so much, MrC.
-
Regarding brackets vs. forward- or backward-slashes:
- Brackets are more difficult to parse out with MC (if you have a need in the future to do so). Single-character separators are easy.
- Forward slash is ambiguous sometimes:
+ Dirty Deeds Done Dirt Cheap/AC/DC
+ Beethoven: Concerto No.5/Rachmaninov: Concerto No.2/Fritz Reiner
- Backslash allows easily creating hierarchical lists (http://wiki.jriver.com/index.php/Expression_Language#Lists_and_Trees) (as you noticed) in panes and categories views.
-
Regarding brackets vs. forward- or backward-slashes:
- Brackets are more difficult to parse out with MC (if you have a need in the future to do so). Single-character separators are easy.
- Forward slash is ambiguous sometimes:
+ Dirty Deeds Done Dirt Cheap/AC/DC
+ Beethoven: Concerto No.5/Rachmaninov: Concerto No.2/Fritz Reiner
- Backslash allows easily creating hierarchical lists (http://wiki.jriver.com/index.php/Expression_Language#Lists_and_Trees) (as you noticed) in panes and categories views.
Backslash (what I was calling reverse/ above), with Artist preceding Album "Beatles\Help!", definitely seems the way to go, for all these reasons. ...and probably use Sort Order Artist so not "The..."?
Could make populating this field more interesting and manageable. Nice solution. Thanks for the link.
I will use this backslash strategy. Trying to recall which of the other fields in the script are of a list type such that this hierarchical feature would be available or useful. One might be able to "group" various Moods, for example into four quadrants of high/low positive/negative energy were someone so inclined.
I will think some more about this. Thanks.
-
MrC, Apologies that I am getting to these replies in pieces, as opposed to the nice logical complete reply you provided!
So you want, for example, a "Billboard Hot 100 (peak)" field, which contains the peak value? That's easy enough - all the parts are already there.
Exactly. Having this for the Billboard Hot 100 and Billboard 200 is a high priority of mine. Thank you.
----------------
The entire page is already pull/parsed, so adding the values to additional fields is no problem. I don't have a handle on how many different awards there are. Do you want each one, or just a subset (and if so, which ones)?
I have found the following interesting resources:
http://www.billboard.com/charts
http://en.wikipedia.org/wiki/Billboard_charts
http://en.wikipedia.org/wiki/Hot_R%26B/Hip-Hop_Songs
There are clearly too many charts to give devoted fields to, but from a quick look, I can see already that I could pull together a rational representation of the major Genres. I will look a bit more at this and report back.
One slight complication—if you look, say, at Stevie Wonder’s awards from the Artist page, you see that he was always climbing up or leading the “R&B Singles” charts, until 1984, when it became the "Hot R&B/Hip-Hop Singles & Tracks”. I can provide a proposed mapping and brief rationale. It wouldn’t be perfect, but could be tweaked for any glaring omissions. I assume you can have the script funnel both of the above into a field called, say, “ BB R&B/Hip-Hop Singles”?
For me, having the major Single and major Album charts is already just terrific.
I do think it would bring things to a great new level to be able to better “mine” high charting tracks for certain Genres at certain times. Say, Hot R&B/Hip-Hop Singles that peaked very high in that chart, and which have any value (i.e., charted at all) on the overall Hot 100.
That might help me curate a mix for a party that had a small contingent that especially appreciated a given Genre, for example, but knowing I wanted to try and keep things familiar for the rest of the guests (e.g., Lauryn Hill’s Doo Wop (That Thing) (#2/#1). Or maybe the other way around, charting Genre tracks that didn’t even chart on the overall (Zion (-/#77), from that same album)—more the after-hours party where that smaller contingent would appreciate the greater niche. Possibilities are endless.
Like I said, I will be looking at how things map out and report back; I have been wanting to do this anyway so I understand which are the truly major chart toppers in the various Genres.
Will finish responding to the other couple open points shortly...
-
Here's a 1.14 pre-release that includes some of the changes thus far requested:
- Fix: function to capture Recording Date was not being called. Key 'rdate' maps to
MC field "Recording Dates".
- New: Similar Albums captured: key 'similar'.
- Fix: Track Pick value set to 1 (instead of 'pick') for track picks.
I'm not sure how I'm going to handle the awards yet. There is some difficulty with creating the (peak) versions. For example, the Help! album has two entries for Billboard Albums: Top Soundtracks, one for the year 2013 and one for 2007. Each happen to have a peak of 1. But other the chart Top Pop Catalog has two entries also, one for 2013 which peaks at 4, and the other for 2009 which peaks at 7. How should this be handled? An XYZ (peak) field which shows only the peak value won't work, and a list field will contain each peak value, but doesn't include the date, so you'll have a list with numbers that have no relation to any dates.
-
Thank you! I will try and test it this week.
There is some difficulty with creating the (peak) versions. For example, the Help! album has two entries for Billboard Albums: Top Soundtracks, one for the year 2013 and one for 2007. Each happen to have a peak of 1. But other the chart Top Pop Catalog has two entries also, one for 2013 which peaks at 4, and the other for 2009 which peaks at 7. How should this be handled? An XYZ (peak) field which shows only the peak value won't work, and a list field will contain each peak value, but doesn't include the date, so you'll have a list with numbers that have no relation to any dates.
Yes. I see.
How about a default to use:
1) the highest it peaked on that chart in ANY year? ("4" in your 2nd example)
With a switch to instead use;
2) its peak in the earliest year it charted ("7"); or
3) it's peak in the latest year it charted ("4")?
Or maybe #2 is the better default, in the broader context (even if not here)? Likely, a SONG's original year of release, rather than some later year it may have re-emerged on a movie soundtrack?
I think we want a single sortable # in the field, and of course all the award details with each year's peak still resides in the original awards field. You could still go back to that were you to need to see which songs in the library charted in what years to what peak.
-
Something change in the last version when i want to grab information
C:\Users\...\Desktop>perl amg.pl -u http://www.allmusic.com/album/... -c amg.config
*** can't open config file amg.config
Stopped at amg.pl line 408.
I did look on the file at that section from last and this new version and nothing changed. But if i use the older one isn't a problem but with this one this error appear.
-
The implementation is straightforward - it's just another page pull/parse and field assignments. However, implementing this might create a problem. For single artist albums, it requires one more page pull/parse for the main Artist page, and then one more for the Biography, and then one more for the Related. For multi-artist albums, this would be those three pages multiplied by the number of tracks. That's a lot of pages to grab. I certainly would not enable this by default. With everything enabled, an album of 10 tracks would pull 5 + (3 * 10) = 35 pages! It gets much worse when several artists are listed per track. I don't want the script to be abusive to A M G.
I would think to limit it to single artist albums, or perhaps leverage the Album Artist concept where a high enough percentage of tracks are from same Artist. Not that odd I think to have "missing" artist details on Various Artists albums.
-
MrC, I have been meaning to follow-up. In a PM, you said:
"There are two ratings you can use. MC's rating is 0-5, so the AMG rating (i.e. 'rating_mc') is scaled. You can use the 'rating' key which maps by default to AMG Rating, and configure that value to be either float or decimal. See the help in:
amg.pl --help
specifically looking at the --rating option."
I tried that, but must be misunderstanding the syntax as it did not work for me. Can you type out exactly what to type? I want albums that AMG rates 4.5 to retain the half star, so 9. Now, it rounds to 8 or 10 depending on file format. Thank you.
-
MrC, I have been meaning to follow-up. In a PM, you said:
"There are two ratings you can use. MC's rating is 0-5, so the AMG rating (i.e. 'rating_mc') is scaled. You can use the 'rating' key which maps by default to AMG Rating, and configure that value to be either float or decimal. See the help in:
amg.pl --help
specifically looking at the --rating option."
I tried that, but must be misunderstanding the syntax as it did not work for me. Can you type out exactly what to type? I want albums that AMG rates 4.5 to retain the half star, so 9. Now, it rounds to 8 or 10 depending on file format. Thank you.
Find this lines in the code:
else {
$opts{'rating'} = 'd';
}
Change from "d" to "f" and it will be like in allmusic 0.0-5.0 (.5)
-
Something change in the last version when i want to grab information
C:\Users\...\Desktop>perl amg.pl -u http://www.allmusic.com/album/... -c amg.config
*** can't open config file amg.config
Stopped at amg.pl line 408.
I did look on the file at that section from last and this new version and nothing changed. But if i use the older one isn't a problem but with this one this error appear.
I've added better diagnostics for this case. You can add it to your script. Change line 409 to:
die "can't open config file $configfile\n$!\nStopped";
and retry to see the specific problem. Are you sure the file is there, and by that name?
-
I tried that, but must be misunderstanding the syntax as it did not work for me. Can you type out exactly what to type? I want albums that AMG rates 4.5 to retain the half star, so 9. Now, it rounds to 8 or 10 depending on file format. Thank you.
Choose one of:
--rating d
-r d
--rating f
-r f
-
C:\Users\\Desktop>perl amg.pl -u http://www.allmusic.com/album/ignition-mw0000193310 -c amg.config
*** can't open config file amg.config
No such file or directory
Stopped at amg.pl line 408.
Changed the script to that you write here but it seems is the same issues like the one i told you early.
My file are both where was the last files. When i changed this new one with the older i only modify the field to meet my field assigned.
-
I'm not sure I'm following you. The error "No such file or directory" really means that file does not exist by that name at that location. Probably it has a .txt suffix (and you have suffixes hidden in your system's Folder prefs so don't see it)
-
Yeah, you have right, the file it seems saved in .txt file, dunno, maybe when i did redefine the fields.
Now is working.
Thanks.
-
One question about the new similar albums.
If i want to create a playlist with this field and to be associated to the specific name of albums it can be possible?
-
I don't see any easy way to relate the various entries in this list field with the entries on your library. Playlists are lists of files (tracks, in the case of Audio), and the entries in Similar Albums are Artist\Album pairs, where Album may or may not match the entries in your library. I can't think of away to expand each of those Albums into the possibly corresponding tracks in your library.
-
Hey MrC, i didn't come here in the last time, because this piece of ART work very well, but something is rather annoying.
The diacritics when are pulled in the mpl is messed up. Can be done something with this problem?!
-
Sample URL please.
-
This is a few site witch where i meet that problem:
http://www.allmusic.com/album/in-search-of-sunrise-vol-3-panama-mw0000228136
http://www.allmusic.com/album/in-search-of-sunrise-vol-11-las-vegas-mw0002549913
http://www.allmusic.com/album/in-search-of-sunrise-vol-6-ibiza-mw0000488883
-
Please give this pre-release version a try:
Changes in 1.14:
- Fix: function to capture Recording Date was not being called. Key 'rdate' maps to
MC field "Recording Dates".
- New: new field Similar Albums: key 'similar'.
- Fix: the Track Pick value is now set to 1 (instead of 'pick') for track picks.
- New: better diagnostics when config file cannot be opened.
- Fix: AMG entries with no tracks were failing to produce output even with -A.
- Fix: output MPL was not correctly UTF-8 encoded.
Edit: pre-release removed in favor of version 1.14 (see first post in this thread).
-
I tested the new file and it seems to work fine.
As always MrC, thank you.
-
I've updated the script to version 1.14. The most notable change since the pre-release posted above is that when using --mcws, you no longer have to manually Copy the track data. The selected track's metadata will automatically be placed on the clipboard.
Version 1.14
- New: manual Copy of clipboard not required when using --mcws
- Fix: function to capture Recording Date was not being called. Key 'rdate' maps to
MC field "Recording Dates".
- New: new field Similar Albums: key 'similar'.
- Fix: the Track Pick value is now set to 1 (instead of 'pick') for track picks.
- New: better diagnostics when config file cannot be opened.
- Fix: AMG entries with no tracks were failing to produce output even with -A.
- Fix: output MPL was not correctly UTF-8 encoded.
- Fix: remove extra comma in options argument list (might have caused problems setting mcws password
from the command line).
-
Trouble with the last track being left out of the script.
Here is the URL: http://www.allmusic.com/album/to-the-sea-mw0001985381
Command line: perl amg.pl -u http://www.allmusic.com/album/to-the-sea-mw0001985381
-
Thanks. I just tested the same and received all 13 tracks, but via MPL import and MCWS.
Your clipboard does indeed have 13 tracks on it.
Let's work offline, I'll send you my email via PM. I'd like to get a copy of the debug output and the created mpl file.
Add to your command line:
-d -v -o - --mpl > debug.txt
and email me the debug.txt file if you could.
-
Ok, we figured out the problem.
stevem87's view had an empty column at the end of the view, and this confounded the detection of old-style clipboard contents vs. the newer contents available in more recent MCs. For now, don't have an empty column at the end of the view's file list.
I found another bug too - if the Filename (name) or Filename (path) field was in the view, but not the Filename field, the script didn't notice this, and would add new (non-file backed) entries into MC rather output the error indicating that Filename was missing.
I'll fix these for the next release.
-
Sorry Mike, one more thing...
Where can I put the credit mapping commands in the config file? I want to map the 'Producer' credit to the Producer tag field. Just don't know where to put it.
Thanks
-
First you'll need the update I posted above (v1.15) which corrects the issues we worked out today, plus fixes an issue mapping remapped credits when they were not specifically asked for with --include.
Next, you'll need to use the config file, so copy and edit it. You'll find a line that is commented out in the Credits section that shows some mappings:
#Credits: Technical As.*:ignore, Photo.*:ignore, Saxophone:ignore, Producer:Producer, Composer:Composer
You'll want an uncommented version specifying your mapping(s):
Credits: Producer:Producer
This says to map the pattern "Producer" (left side) to the field named Producer (right side) in MC. See the comments in the config file.
Finally, you'll want to add the config file to your command line:
-c name_of_your_config_file
-
I've updated the script to version 1.15:
Changes:
- Fix: remove extraneous debug output when getting similar artists.
- Fix: a missing Filename column was sometimes not detected.
- Fix: ensure inclusion of mapped credits when no exclusion is present.
- Fix: problem mis-detecting old clipboard data when last column in view was empty.
-
I got another one..or two. Is there a way to stop the script from changing my cover art. I have the artwork set and embedded inside the file. After running the script it changes it to external location.
Also, I tried running it in mcws and it gave me an error and only populated the first few tracks. It did populate some, so the authentication stuff must be ok.
-
The script doesn't touch the Image File field (and hence your cover art).
When we ran it on your system, where you didn't have a Filename field, you were getting new entries. Are you still doing the same, and is this the cause?
I suppose when you import fields, MC might be re-calculating where artwork comes from, as based on your preferences. I've not seen this happen. What is your preference setting for cover art location in Options > File Locations for Cover Art?
I'd need to see the error you get w/MCWS. Can you post it? Was it MC that gave the error, or the script?
-
I posted 1.16 which fixes a problem introduced in 1.15.
-
I got another one..or two. Is there a way to stop the script from changing my cover art. I have the artwork set and embedded inside the file. After running the script it changes it to external location.
I have reproduced this here. This is an MC bug. It appears that when an MPL is imported, MC is re-calculating where cover art is located. And it seems to prefer external cover art over embedded cover art, so immediately after doing the import, the Image File field is changed from "Inside File" to the name of the external cover art location/file. You can correct this by running a Cover Art > Quick Find In File / Cover Art Directory.
Reported: http://yabb.jriver.com/interact/index.php?topic=87247.msg597343#msg597343 (http://yabb.jriver.com/interact/index.php?topic=87247.msg597343#msg597343)
-
I was just going to say that is how I fixed it, but it would be nice if it didn't happen. I attached a text file of the error from the script.
Thanks
-
OK, there are several problems.
The first was correctly in the 1.16 release I just posted. Please download and use that release. Sorry about that.
The second is that amg is timing out, but this shouldn't be catastrophic - just no lyrics.
The third you can fix. Its this:
*** can't open config file amg-config.txt--mcws
Add a space between the two arguments, before the --mcws.
The fourth is that MC is balking about the data being presented in the URL for doing the updates. I'll have to decompose the data to see what's going on, and what MC finds offensive. Back ASAP...
-
Yeah, I had fixed the others already. It's the last one that was the real problem.
-
I tested the URL you were using (10,000 Maniacs, MTV Unplugged), and it worked for me here with MCWS.
I don't see anything wrong with the URL being sent, so we'll have to dig deeper.
To your command line, add:
-d -v -o - --mpl > debug.txt
so we can capture the entire session. Send the debug.txt and your config file. I'm probably going to have to reproduce everything exactly.
-
No need, I used the update that you posted and it works fine now. I tried it on a few albums and works great. I have been playing around with the credits field and mapping different credits to different field, man that is awesome! This script does everything I wanted it too and more. Thanks so much!
BTW, Using mcsw fixed the cover art issue.
Also, the AMG rating field always comes in as '5' regardless of what they rated it. I tried it as an integer and decimal, same result.
-
Excellent news, thanks for the follow-up.
There are two rating-related keywords: rating and ratingmc. The keyword 'rating' is the 0 to 9 point number amg uses to encode the 0 to 5 rating with .5 increments. The 'ratingmc' keyword is mapped to fit MC's 0 to 5 integer. The --rating <format> option affects the former, not the latter.
With my test last night on the 10,000 Maniacs album, Rating was correctly set to 2 (keyword 'ratingmc', by default mapped to MC's Rating field), and the AMG Rating field (keyword 'rating') was set to 3 (which is shown as 2-stars on their site).
-
OK, that same error keeps popping up. When I run it .mpl it imports fine. Happens with certain albums consistently, but other albums are fine.
Thanks
-
I'll try to reproduce it here. If you can narrow down the problem a bit, to certain tracks, and certain fields, that would help. I'm suspecting that the MCWS service's new support for packaged field/value pairs is having trouble with something.
I don't see a server error in the debug.txt file, so I'm not sure what I'm looking for.
In your config file, you have an error:
Optins: -o image file
Options is misspelled, and you currently can't use whitespace inside an option argument such as the one you specified with -o ("image file"), so use image_file, or imagefile,or something with no whitespace instead. I'll have to add some quoting mechanisms here to support whitespace.
-
Depends on the album, with that one it did pull in anything. With Aerosmith's Greatest Hits it pulled in everything but the last two tracks. With 10,000 Maniacs it was pulled in the first two. I thought it might be a timeout thing or something, but I tried the same album a few times and it stopped at the same places. I moved on to the next album, and it worked fine, went back again and it stopped in the same place.
As for the typo, I'll just take it out, I don't think it helps anything anyway.
Thanks
-
Do these tracks already have track or disc numbers which may be incorrect? If so, use the sequence option -s or --sequence. This tells the script to ignore your disc and track numbers, and just use the sequence of files in the view.
Whenever you get an errors like this, feel free to email me the debug.txt output for that particular run (you may have to re-run the command to capture the output). That will give me the most data regarding what might be going wrong.
Timeouts should be seen in the output.
Please re-download version 1.16. Somehow I goofed in creating the zip archive, and your version is the old 1.15 version. Sorry.
And thanks for your help and patience.
-
Double checked the track numbers on this, no problems here.
BTW, the option of having the mpl spit out the separate files (when the 'filename' field was absent did have some merit. It worked on things like box sets, where all of the tracks or discs are not present/renamed or whatever. That way you could just copy the tags track by track and not write over the names and track numbers. I used this for the Genesis box sets that have like 4 studio albums 5 DVDs and a bonus disc. I just needed the bonus disc because the studio albums were named accordingly. The file that came in the mpl had all of the tracks including the DVDs.
-
This one got up to track 5 on disc two... All fields populated on track 4, none on track 5. Tried it twice.
-
I've been able to reproduce there error here; it was your credits mapping that triggered the problem.
My current thinking is that the URL being generated to post the updates is too long (the most recent one is 6685 chars), so I'll try breaking it up into smaller chunks to see if that helps.
By the way, your server's password is in the config file. I'll remove the file from your post above, but you should probably strip it before posting. If you care, you might want to change it in MC.
[Edit: Nope, not length. Keep trying...]
[Edit: Got it! Grab the new 1.17 version.]
-
I just posted version 1.17 which should fix your issues (and maybe those reported by others in the past, that went undiagnosed).
Changes:
- Fix: content being returned from web pages was not being decoded from external UTF8 to internal form.
I was failing to decode the UTF8 from the web pages, and when it was later being re-encoded for output to MC, some characters became invalid (and this is why things failed sometimes, but always on the same tracks). Unfortunately, MC does not indicate this to be an invalid UTF8 problem, but instead claimed it was a CSV formatting problem with the CSV-encoded content passed to MCWS. Now we know.
-
Brilliant!!!! I think that's got it.
-
Great, thanks for the confirmation.
I have a overarching question for all...
Currently the script does not populate fields when there are no values reported via amg. This means stale, perhaps incorrect values can remain for your tracks. This might happen if you've used the wrong URL, or accidentally placed the wrong value in some field. So if you have a bunch of tracks, that have a field, say Track Pick, set to 1, and run the script, but only Tracks 2, 3, and 10 are Track Picks, the script will only set a value of 1 for those tracks, leaving the others untouched. But the tracks already had bad data; so the Track Pick field remains clouded.
Alternatively, certain fields can be assumed to always have values for each track, and for Boolean (True/False) types, they are set to 1 for TRUE, and could be set to 0 for FALSE when there is no data present. When the field types are Decimal and Integer fields, setting a 0 value seems the right thing to do (but I don't know your field types), and a 0 value is tantamount to an empty value. But for String and Large value fields, a 0 is a 0 and there's no way to clear a field via MPL or the method I'm using via MCWS. So if I write a 0, and your Track Pick field is a String field, you'll get a 0 in your columns (this could be a good indication to you to change your data type).
However, for a field like Lyrics, amg may not have any lyrics for some tracks, but you may. So clearing your Lyrics fields for these tracks would not be good.
But if you had the wrong lyrics for some reason, and you run the script, there's way to tell the script to clear the field, and no way currently to force a clearing of such a field anyway (the value could be set to "N/A" or some other string to indicate emptiness).
My feeling is that if you've asked for the data to be pulled, it should be correctly reflected in your tracks. And in some cases, missing data means FALSE and in other cases missing data means DUNNO (and you might want LEAVE ALONE or OVERWRITE).
What to do?
-
I would opt for leaving it the way it is. Let the user decide that the data is wrong. I'm still more concerned about it overwriting data that I don't want it to, like names and track numbers. Or getting the box set info like I discussed earlier.
-
Just some question.
Sometimes when grab info from amg to my album, some album are not in the same line as the track listed on that page so if i don't check first the tracks to be as the link it will write on my album the track listed on that link, so that it will broke my album and need to redo back to the previous and to look and fix the track list and to grab again. Now the problem is after i grab info the script already write the info on my track list and i can't redo with ctrl+z so i need to look manually in the track list and change to be exact as the link where i grab the info, is somehow a better solution to redo the info back to that state or i do by hand. Not always i look at the track list but just when i see they change after i import the script. Sometimes is pretty annoying, to see that and need to check every single track of change and redo.
Thx!
-
Got the error again...
-
@sirgeo -Some tips:
- first check that the AMG URL you are using is the correct one for your album release. Sometimes the primary listing is correct, other times you'll want to use another one listed under the Releases tab. Compare your number of tracks with their listing. The status window at the bottom of the MC window will show you how many items you have selected.
- be sure your tracks either have valid and correct Track #/Disc # entries OR use the -s or --fileregex options. The script needs some way to correlate your (possibly ordered) tracks in the view with the always ordered tracks on the web page. The script tries to use your disc/track numbers, and if those don't exist, will use the order in the view. Or you can override that behavior by using the --sequence (i.e. -s) option (which means use the order of the items in the view), or --fileregex option to grab a track number from your file names.
- you can make a backup of your track data using File > Export Playlist, and use the MPL format, selecting All Fields and The Selection. Do this before you use the script on an album, and if you don't like how it turned out, you can just File > Import Playlist that MPL file to restore the entries (mostly, it won't CLEAR data to empty).
-
Got the error again...
Drats. I'll have a look, thanks for the report.
-
Thanks, here's another. Happened a few time now.
-
Edit: This is resolved in MC 19.0.116.
I'm now pretty sure this is a bug with MC. It appears that MC is getting confused during the deconstruction of the URL being sent to the MCWS service into its constituent components (decode UTF8 URL encoding, and then decode CSV data).
The failure is occurring on the character é, which is Unicode: U+00E9 and UTF-8: C3 A9.
The URL we're passing is using UTF8, and that character encoded as %C3%A9 occurs in two places. This causes the MCWS write to fail. If I change one or both them to Unicode representation of %E9, then the write succeeds.
I'm going to have to get some JRiver help on this.
Edit: if this isn't addressed within a couple of days, I may implement a workaround; after an error is encountered, the script would fall-back to single-field writes for the track. This will make the updates slower, but at least should work reliably until the problem is resolved.
-
FYI, I tried a different release page on both of those albums and it worked fine.
-
BTW, the option of having the mpl spit out the separate files (when the 'filename' field was absent did have some merit. It worked on things like box sets, where all of the tracks or discs are not present/renamed or whatever. That way you could just copy the tags track by track and not write over the names and track numbers. I used this for the Genesis box sets that have like 4 studio albums 5 DVDs and a bonus disc. I just needed the bonus disc because the studio albums were named accordingly. The file that came in the mpl had all of the tracks including the DVDs.
I'd like to understand your usage scenario here a bit more:
a) which AMG URL are you referring to?
b) what discs/tracks do you have?
The goal of the script is to match your selection when the correct entries, so I'd like to continue to refine ways to do this if necessary, so that you don't have to copy / paste.
Also, if you only want specific fields, you should use --include (or -i) to specify the list of desired fields. This is not only much faster, because it avoids unnecessary page look-ups (there can be MANY per album), but also can help you avoid overwriting fields you want unmodified.
-
@stevem87
Regarding your Credits mappings from your last posted config file:
Credits: Producer:Producer, Composer:ignore, Artwork:Artwork, Photo.*:Artwork, Engineer:Engineer, Assistant Engineer:Engineer, Coordination:ignore, Art Direction:Artwork, Mixing.*:Mixing, Mastering.*:Engineer, Live Technician:Engineer, Live Sound Engineer:Engineer, Design:Artwork, Digital.*:Engineer, Member of Attributed Artist:ignore, Remaster.*:Engineer, Liner Notes:ignore, Original Photography:Artwork, Primary Artist:ignore, Cover.*:Artwork, Illust.*:Artwork, Pre-Production:Engineer, Equip.*:Ignore, Audio.*:Engineer, Manage.*:ignore, Group Member:ignore, Unknown.*:ignore, Special Ass.*:ignore, Drawing:Artwork, Original.*:ignore, Sleeve.*:Artwork, Memor.*:ignore, Retouch.*:ignore, Pre-Produc.*:ignore, Songwriter:ignore, Project.*:ignore, Assistant Producer:Producer, Executive Producer:Producer, Remix.*:Mixer, Compilation Producer:Producer, Produc.*:Producer, Assistant:ignore, Vocal Engineer:Engineer, Pro-Tools:ignore, Additional Pers.*:ignore, Editing:ignore, String Engineer:ignore
There were a couple of problems:
1. the word "ignore" must be in lowercase - you had one as "Ignore". This is just to help make it clear it is not an MC field.
2. You have Pre-Production:Engineer and later Pre-Produc.*:ignore. It just happens that the code uses a first-wins implementation, so Pre-Production would be mapped to Engineer, and not ignore. If that's what you intended, it works.
Since the left-hand side of the mappings are regular expressions, you can make the credits shorter by using the OR | character, and broken down, you basically have these 5 credit groupings (ignore, Artwork, Engineer, Mixer and Producer):
(Additional Pers.*|Assistant|Composer|Coordination|Editing|Equip.*|Group Member|Liner Notes|Manage.*|Member of Attributed Artist|Memor.*|Original.*|Primary Artist|Pro-Tools|Project.*|Retouch.*|Songwriter|Special Ass.*|String Engineer|Unknown.*):ignore
(Art Direction|Artwork|Cover.*|Design|Drawing|Illust.*|.*Photo.*|Sleeve.*):Artwork
(.*Engineer|Audio.*|Digital.*|Live Technician|Mastering.*|Pre-Production|Remaster.*):Engineer
(Remix.*|Mixing.*):Mixer
[^-]*Produc.*:Producer
These can now be combined for a shorter and grouped by mapping final Credits line in the config file :
Credits: (Additional Pers.*|Assistant|Composer|Coordination|Editing|Equip.*|Group Member|Liner Notes|Manage.*|Member of Attributed Artist|Memor.*|Original.*|Primary Artist|Pro-Tools|Project.*|Retouch.*|Songwriter|Special Ass.*|String Engineer|Unknown.*):ignore,(Art Direction|Artwork|Cover.*|Design|Drawing|Illust.*|.*Photo.*|Sleeve.*):Artwork,(.*Engineer|Audio.*|Digital.*|Live Technician|Mastering.*|Pre-Production|Remaster.*):Engineer,(Remix.*|Mixing.*):Mixer,[^-]*Produc.*:Producer
or the easier to read multiple Credits line version:
Credits: (Additional Pers.*|Assistant|Composer|Coordination|Editing|Equip.*|Group Member|Liner Notes|Manage.*|Member of Attributed Artist|Memor.*|Original.*|Primary Artist|Pro-Tools|Project.*|Retouch.*|Songwriter|Special Ass.*|String Engineer|Unknown.*):ignore
Credits: (Art Direction|Artwork|Cover.*|Design|Drawing|Illust.*|.*Photo.*|Sleeve.*):Artwork
Credits: (.*Engineer|Audio.*|Digital.*|Live Technician|Mastering.*|Pre-Production|Remaster.*):Engineer
Credits: (Remix.*|Mixing.*):Mixer
Credits: [^-]*Produc.*:Producer
Note: in the next release, I've reversed the MC field/regular expression, so lines will look like this:
Credits: ignore:(Additional Pers.*|Assistant|Composer|Coordination|Editing|Equip.*|Group Member|Liner Notes|Manage.*|Member of Attributed Artist|Memor.*|Original.*|Primary Artist|Pro-Tools|Project.*|Retouch.*|Songwriter|Special Ass.*|String Engineer|Unknown.*)
Credits: Engineer:(.*Engineer|Audio.*|Digital.*|Live Technician|Mastering.*|Pre-Production|Remaster.*)
-
@stevem87
Note: in the next release, I've reversed the MC field/regular expression, so lines will look like this:
Thanks for showing me that simpler way of writing the Credit mapping, but does this mean I have to rewrite my config file?
-
BTW, there is some issue with character mapping going on. When certain characters are on the page, they get imorted like this:
Vanda &
Young
and this:
Robert John "
Mutt"
Lange
-
In regards to my copy and paste thing, here are some examples. I emailed you mpl files of what I have in my library. With Genesis, the set contains 5 studio albums that are in essence the individual albums. When you rip them they come in as the regular individual release. There is a bonus disc of unreleased tracks that still needs to be accounted for as you see in my mpl file. I guess there is a way to get your script to map to those, but I don't know that is really easier than the copy and paste method.
The Police release is a little different, It too contains all of the full studio albums and bonus tracks, but they are thrown on four CDs rather than separate disc that would retain the proper track numbers. For this I manually changed the track numbers and album names, so in my collection I have the individual albums and an extra disc of bonus tracks which were originally spread out on the set.
http://www.allmusic.com/album/genesis-1976-1982-mw0000779142
http://www.allmusic.com/album/message-in-a-box-the-complete-recordings-mw0000102060
-
http://www.allmusic.com/album/i-never-loved-a-man-the-way-i-love-you-mw0000651820
On this one it loaded tracks 1-8 and I got the same error. I then selected tracks 10-14 in the library and re-ran the script. It pulled in those except for the last track, which is the same name as track 9.
-
Replying to your last 4 posts...
1. about the & and " characters (these are HTML entities, used because & and " are reserved characters in HTML) - which fields are you seeing these in, and which AMG URL. I'll check to see where all the conversions might be going wrong. I thought these were all addressed, but obviously not.
2. I replied to your email regarding your MPL samples, and offered some suggestions for dealing with box sets. I see the URLs in the post above too, so that's great, thanks.
3. I'll test out Aretha Franklin album, but I'm pretty sure its the same problem. Track 9 has an track review, and it contains some UTF-8 characters which may be triggering the same issue. Today, I'll see how difficult it is to try the fall-back workaround I suggested above so at least you might proceed w/out having to wait JRiver's and my conversations.
Edit: added number 4...
4. Yes, you'll need to change how your Credits entry is formatted starting with the next release. Sorry, its just not worth trying to have to code fix it for you, and I think the benefit will outweigh the pain. If you've have not updated your Credits mapping since your last posting of the config file, the samples above are easily reformatted, or send me your current one and I'll return a valid one for you.
-
1. http://www.allmusic.com/album/back-in-black-mw0000188862 Producer credit (mapped from Credits).
4. No worries, I can do it myself, just wanted to be sure before I did it.
Thanks,
Steve
-
1. about the & and " characters (these are HTML entities, used because & and " are reserved characters in HTML) - which fields are you seeing these in, and which AMG URL. I'll check to see where all the conversions might be going wrong. I thought these were all addressed, but obviously not.
There were two bugs here. One, I've addressed. In MCWS mode, I was not decoding the HTML entities before sending them off to MCWS. The MPL mode didn't have this problem, because the HTML entities are required.
[Edit: this bug is now resolved in MC 19.0.116]
The second bug is a bummer. It may be the same, or another, MC bug. MC is not correctly handling the deconstruction of the URL into CSV, and then into text for the field. So even after I've applied my fix above and convert the HTML entities, in the case of double quotes, MC ends up removing them entirely. So an entry:
Robert John "Mutt" Lange
gets placed into the field as:
Robert John Mutt Lange
I've posted in the beta thread, but am not sure it will get addressed for a while. The workaround I've proposed will solve this issue, but will slow down the MCWS writes substantially since I'll have to check every value for double quotes and then fall back.
I'd say, for now, use MPL...
-
Thanks again, it would seem that the per track pulls (lyrics and song reviews) are going to add the most mileage. I have about 3,000 CDs in my library, should I be concerned? Would spreading them out over time help?
-
So others can follow, your comment is in reply to our email exchange about disabling fields you don't need, reducing the number of pages pulled.
You're correct - lyrics and track reviews are expensive. Pull them if you want, and just be reasonable and I think everything will be fine.
I hope you're importing the URL so in the future you can more easily get to the release's page.
-
For the next release, I've implemented page caching:
- New: pulled pages are now cached to avoid excessive server hits. The cached directory named '_lwp_cache',
created in the current working director, may be safely deleted at your convenience. One sample run for 10 tracks,
requiring 20 pages which initially took 17.6 seconds, is reduced to 1/5 second for subsequent runs. Page cache
expires as per AMG page cache settings, which is about 8 hours.
-
Question?
If an allmusic link don't have any track listed and i want to grab info from it the scrip will ignore all the track and don't do nothing. Do i need to have track there or what?!
-
URL please...
-
Here is an example
http://www.allmusic.com/album/guilty-mw0001632154
-
Include the -all option on your command line to force non-track specific (i.e. album-specific) items to be output.
-
I've posted version 1.18. There is one incompatible change if you are using Credits mappings. Be sure to see the Credits section in the amg-config.txt file.
I think you're going to like the new page caching and --range options. Try running the script with the same URL a second time.
Version 1.18
- Change: Credits mappings are swapped: the old pattern:field is reversed to field:pattern. This makes it much
easier to see your field mappings up front, espeically when using multiple Credits lines, and patterns are long.
- New: pulled pages are now cached to avoid excessive server hits. The cached directory named '_lwp_cache',
created in the current working director, may be safely deleted at your convenience. One sample run for 10 tracks,
requiring 20 pages which initially took 17.6 seconds, is reduced to 1/5 second for subsequent runs. Page cache
expires as per AMG page cache settings, which is about 8 hours.
- New: option --range <range list> allows selecting the AMG discs and tracks to be used. This is useful for
box sets, etc., to select only those items that map to your collection. The --range option implies the --sequence
option. The <range list> is one or more comma-separated <disc spec><track spec>, where either one may be omitted,
and <disc spec> is a 'D' followed by a <range>, and <track spec> is a 'T' followed by a <range>. A <range> is
a number followed optionally by a dash an another higher number (e.g. 3, or 4-9). An omitted <disc spec> or
<track spec> implies ALL. Examples:
--range d2 all tracks from disc 2
--range d2-4 all tracks from discs 2 through 4, inclusive
--range d2T1-9 tracks 1 through 9 from disc 2
--range T1-4 tracks 1 through 4 from all discs
--range D1,D3,D5T1-10 all tracks from discs 1, 3, but only tracks 1 through 10 for disc 5
- Fix: in MCWS mode, HTML entities were not being unencoded (note: this revealed in MC bug, which is unresolved
currently).
I've updated the instructions on the main page to indicate that you should change directories into the amg folder you created when you unzip the archive.
-
MrC., Getting an error with the new script. "can't locate LWP/useragent/Cached.pm" ....
-
Did you move the amg.pl script out of the folder that contains the LWP folder? The page caching code needs that, so keep the amg.pl, config file, and LWP folder all in the same folder, and execute by CD'ing into the folder (you're probably on the Desktop). In the command shell, instead of:
cd Desktop
do
cd Desktop\amg_1.18
and run as normal.
-
Thanks, that sounds like it is it.
-
On this URL http://www.allmusic.com/album/genesis-1983-1998-mw0000490100
I am trying to pull tracks 1, 3, 5, 7, 9, 11-13 off of disc 9. I tried just pulling track 1 from disc 9 (comes in as track 2)
I wrote it like this --R D9T1, D9T3, D9T5, D9T7, D9T9, D9T11-13
What came back was disc 9 tracks 2-9.
tried it lots of different ways and I keep getting the same result.
-
Two problems I see...
1. Use -R (single dash for letter options) or --range (double dash with long option names).
2. If you do this:
--R D9T1, D9T3, D9T5, D9T7, D9T9, D9T11-13
then you need quotes around the argument to protect the white space from the command shell:
-R "D9T1, D9T3, D9T5, D9T7, D9T9, D9T11-13"
Instead, do this...
-R D9T1,D9T3,D9T5,D9T7,D9T9,D9T11-13
-
Thanks, but it pulls in 1-8 now, not the track specified.
-
Bah, my algorithm didn't consider these individual tracks in a single disc (I assumed a range). I'll fix it tonight. Thanks.
-
Version 1.19 is now posted to fix the --range problems you had.
If it will help, I can modify the --range parsing code to allow you to more easily express:
-R D9T1,D9T3,D9T5,D9T7,D9T9,D9T11-13
as this:
-R D9T1,3,5,7,9,11-13
So you'll use comma to separate items within a disc or track range, and semicolon to separate multiple ranges, as in:
-R "D9T1,3,5,7,9,11-13; D10T1-3,5-7"
-
Thank you. Sorry for being such a pain. I know I have been blowing up this board. The comma looks good, less key strokes.
-
It appears that the -e option is not working now
-
I'm not seeing a problem w/-e (aka --exclude), and that code hasn't changed in a while. If you have problems, I'll almost always need to see your command line, and config (strip out MCWS_username and MCWS_password lines) to reproduce the problem, or see where there might be a problem with the command line itself.
I have the new range code done and working.
-
Sorry, I figured out the -e thing. Did you do another release with the range update. I don't see it.
-
I had not posted the update, as I was waiting for an MC update that fixes the two issues we discussed earlier (MCWS failures and problems with Unicode characters). The beta release that came out last night appears to have resolved the issue. Once that's made available, you should be able to test the cases where you were getting failures.
-
Version 1.20 is now posted.
Version 1.20 Changes:
- Change: --range specs are now a semicolon separated list of individual disc/track pairs, where within a disc track pair,
disc or tracks may be specified as a comma-separated list. Examples:
--range "D9T1,3,5,7,9,11-13; D10T1-3,5-7" tracks 1,3,5,7,9, 11-13 for disc 9, tracks 1-3 and 5-7 for disc 10*
--range D1-3T1-9,11-13,15;T1-3,5-7;D6T1-5 tracks 1-9, 11-13, and 15 for discs 1-3, tracks 1-5 for disc 6,
and tracks 1-3 and 5-7 for other discs
* note the double quotes surrounding the entire --range argument, due to use of (optional) white space
- New: enable connection caching for better performance.
Note: Be sure to update to at least MC 19.0.116, as it fixes the two issues mentioned earlier in this thread (MCWS failures, and loss of double quotes).
-
Thanks MrC, but I can't find 19.0.116, only 19.0.108.
-
The latest build is always at the top of the MC19 board.
-
Got it thanks. BTW, I can't seem to get the -R to work anymore. I am also getting some extra lines coming through the script in the command shell. Check the attached file.
Thanks
Steve
-
Sorry about that. I'll send you via email a pre-release 1.21 version to see if it resolves the issue.
-
That did it, thanks again!
-
Thanks for testing and the feedback. Version 1.21 posted.
Version 1.21 Changes:
- Fix: fix some error messages when disc number was empty (single-disc albums).
- Fix: set Disc # / Track Pick values to 0 to reflect no disc value (i.e. single disc) / non-track pick. These
will be shown as empty by default in MC for Integer fields.
- Fix: broken logic testing --range values when no disc was specified.
-
Hey MrC, I keep getting a error on this URL: http://www.allmusic.com/album/try-this-mw0000325851
The script say that there is no main HTML page available.
-
I'm not having any problems with that link. I wonder if your cached page is corrupt? You could delete the cached pages, which are under _lwp_cache.
-
I've just posted a large update to version 1.22. Most of the code has changed, to make it as much the same as possible with discogs.pl script. There are also some nifty new features here. Enjoy.
Version 1.22
- New: option --create will create dummy files (useful for tracking vinyl).
- New: amg.pl now iterates over all urls on the clipboard so multiple albums can be updated, or on the command line when
creating dummy tracks.
- Change: remove obsolete debug option --clipfile.
- Change: fieldname 'url' changed to 'rid' (to align with discogs.pl). Option --rid, --url, and -u are equivalent
(but not -r, which is reserved for --rating).
- Fix: a disc-only --range spec (e.g. D2) not working.
- Internal: amg.pl, discogs.pl, and pscriptor.pl now share common modules (under directory MCUtils).
- New: be tolerant of uppcase letters in key names used on the command line
-
Clearing the cache worked, thanks!
-
I saw an occurrence of this last night too, and clearing the cached fixed the issue. I'll dig deeper next time it happens.
By the way, I just implemented:
- New: key 'totaltracks' mapped by default to Total Tracks returns the total number of tracks across
all discs.
-
I've posted version 1.23:
Version 1.23
- New: key 'totaltracks' mapped by default to Total Tracks returns the total number of tracks across
all discs.
- New: warn when an --include or --exclude key does not exist.
- Fix: do lazy pull-parsing of per-track info.
- New: debug and verbose messages are easier to read.
- Internal: more cleanups and bug fixes.
-
This script become better with each release thx you for spending the time doing this for us.
-
Failed to get page: http://www.allmusic.com/album/razorblade-suitcase-mw0000080628/releases
500 read timeout
Use of uninitialized value $_ in pattern match (m//) at amg.pl line 876 (#1)
(W uninitialized) An undefined value was used as if it were already
defined. It was interpreted as a "" or a 0, but maybe it was a mistake.
To suppress this warning assign a defined value to your variables.
To help you figure out what was undefined, perl will try to tell you
the name of the variable (if any) that was undefined. In some cases
it cannot do this, so it also tells you what operation you used the
undefined value in. Note, however, that perl optimizes your program
anid the operation displayed in the warning may not necessarily appear
literally in your program. For example, "that $foo" is usually
optimized into "that " . $foo, and the warning will refer to the
concatenation (.) operator, even though there is no . in
your program.
This error appears on some albums after i try to get info, but if i try to retry the information again, maybe the site isn't responde and reach that 500 read timeout it show me the same info. I suppose that info what he got from that link is stored somewhere and wont do again a check, like before.
I checked that link with release and seems to have info there but the script don't want to recheck again and instead it pop up this message. How to deal with that?! If i close the cmd and open again the same message pop up appear.
-
I recheck and read the readme so i deleted the cache file from inside of folder and seems that have solve the problem.
But that mean every time when the site doesn't respond in that 500 read timeout i will need to delete the cache? or can be done in another way?
-
I think the module I'm using for caching has a bug, and I'll dig into that later. For now and until I find it, clearing the cache works.
Edit: I may have a fix for this. Tonight I tagged almost 2400 files before I finally hit the cache error. I applied my fix, and the the problem did not reappear.
Coincidentally, when the cache issue hit, I was tagging a Buckwheat Zydeco album 100% Fortified Zydeco, and this triggered an MC bug with a filename containing a % character. I've posted the problem in the beta thread.
-
Another issue, on some albums some tracks have [cd-rom track] as name, and if you didn't check your tracks with the link you get info, that track is rewrite as [cd-rom track] can this be avoided?
-
As always, I need a URL to work out the details.
-
Here is the link with the thing i told you and did something change on date type because it only get as year and nothing more.
I use the same cmd as i did before and with the new changes.
http://www.allmusic.com/album/from-the-roots-up-mw0002377235
-
I've posted version 1.24:
Version 1.24
- Fix: don't cache any 5xx responses. This may help the problem users were seeing with 500 replies to
good page requests.
- Fix: the Date field was returning year only (cause by an incomplete debugging cleanup from 1.23).
Regarding the [CD-ROM Track] track name, I think that's pretty rare, and esoteric, so unless we find a bunch of these, I'd prefer not to special case this one this single name. What you should do is use the --range option, to indicate you want tracks 1-12:
--range T1-12
-
Thx for the response, i suppose is pretty rare, it occur to me 2 or 3 times all this time so i will do a check of tags before i grab the data. (Did you do something with my request on the other thread?).
-
(Did you do something with my request on the other thread?).
Which one? You leave me guessing. :-)
- The cache error might be resolved.
- The date problem is fixed.
The release notes above for 1.24 say this.
-
Seems we have a miss communication :)
I did ask you if you can help me with the request in the other thread about albums playcount, you didn't answer so i presume or you don't read or don't understand my thoughts :)
The last version is checked and works ok so far. The date is ok.
-
I have a question, why does the 'Date' field only populate with the year. Even though the full date is shown under 'Release Date' on the web page. I seem to remember that the script used to populate the full date, but recently it is not.
-
See reply #142.
-
I want to give you folks a heads up. I've added a couple of new fields, as I've learned a little bit more about AMG track, artist, and album IDs.
Currently amg.pl wants an entire AMG URL, which is a lengthy string that includes the base URL, the album's name, and an album identifier:
http://www.allmusic.com/album/morning-phase-mw0002607867
However, the last part, the mw0002607867, is sufficient to generate a URL to the same page. By combining the base with the Album ID:
http://www.allmusic.com/album/ + mw0002607867
you get a simple, unique Album URL. As such, I've added the capability to save the Album ID portion alone. In addition, I'm also providing the Artist ID and the Track ID (so you can create Links to go directly to the Artist or Track page directly). The new fields that will be available are:
url => AMG Album URL
artistid => AMG Artist ID
artisturl => AMG Artist URL
tid => AMG Track ID
The existing fields will still be available:
rid => AMG Album ID
turl => AMG Track URL
However, and here's the Heads Up. The script will only read it's 'rid' value from the 'rid' field, which currently contains a full URL. And from now on, it will always store the shorter album ID into the 'rid' field and the full URL into the 'url' field. So if you want to save your full URLs, you'll need to create a new field and copy the URLs to that field. The script won't care if the value in the 'rid' field is a full URL or an ID - either will work.
Example: My 'rid' field is currently named AMG URL. Since the script will now store an ID into that field, and I want to keep the URLs (for the time being anyway), I'll create a new MC field called AMG ID. I'll copy the AMG URL values for all my files into AMG ID (using MC's Move/Copy fields tool). And I'll edit my amg-config.txt file to assign 'rid' to the field name AMG ID, and the new 'url' key will be assigned to AMG URL. New album IDs will be stored in 'rid' and the full url will be stored into the 'url' field.
Got it?
Side note: I've personally processed over 14k files.
Edit: I've now processed all my files. Today I added composer id/url to the script. With all these ID or URL fields available, you can create some nifty direct album, track, artist, and composer Links in MC to take you directly to the relevant page. I have, and am routinely using, MC Links for AMG Album, Track, Artist, and Composer data, which use the stored IDs, and if they don't exist, do a search on AMG.
-
I got another one..or two. Is there a way to stop the script from changing my cover art. I have the artwork set and embedded inside the file. After running the script it changes it to external location.
Just to follow-up on this. This thumbnail deletion problem is an MC bug which is fixed in an upcoming MC19 beta release. Stay tuned.
Edit: MC version 19.0.120 is posted. It solves the cover art problem.
-
I've posted version 1.25. There are many new fields, and one large change that will affect your AMG URL field. See the notes below, and the important post above (http://yabb.jriver.com/interact/index.php?topic=84268.msg601216#msg601216). I'll provide an expression to allow you to convert your existing URLs into IDs if you want.
Version 1.25
- Change: WARNING: values stored into existing key 'rid' (field: 'AMG Album ID') will now be short album ID
values. Old style URLs will be automatically converted to album IDs in this field. See this important
post: http://yabb.jriver.com/interact/index.php?topic=84268.msg601216#msg601216
- New: key 'url' (field: 'AMG Album URL') will contain the short AMG album URL.
- New: per-track keys 'artistid', 'artisturl', 'composerid', 'composerurl', capture the AMG artist ID and URL,
and AMG composer ID and URL. These can be used with Links to get to the artist's or composer's landing page.
The fields are mapped to AMG Artist ID, AMG Artist URL, AMG Composer ID, and AMG Composer URL, respectively.
The AMG Artist ID or AMG Composer ID is sufficient to generate the AMG Artist URL or AMG Composer URL, so
capturing both are not necessary, however you can choose either or both.
- New: per-track key 'tid' captures the AMG track ID. This is ID-only portion of the 'turl' (track URL) field.
You can use either or both of these; the 'turl' value caries with it descriptive text reflecting the track's
name, so this might be your preferred value; however, it is a lenthier value that the simpler 'tid' value.
- Change: move the MCWS section in the config file to the top of the file, and change comments to reflect that
the MCWS service is the default now. The 'tid' value can be used to generate a Link to the track's entry
at AMG.
- Fix: debugging messages on Windows cmd console now output in Unicode correctly. Must use a Unicode font
such as Consolas for the Console (right-click the console's titlebar, select the Font tab, and select the
Consolas font).
- Change: allow compressed page gets.
- Change: increase page timeout to 15 seconds.
-
I've added a tip on creating Links using the new IDs, or to help find album IDs, in reply #2 (http://yabb.jriver.com/interact/index.php?topic=84268.msg575808#msg575808).
-
I didn't try the last version but i have a question on the addition.
If you get the last identifier id, and with combination with URL and album it get the information like the previous one?!
However if you do that, how do you save the url in the other field, because you only get that identifier and nothing more, the combination of the all this 3 things get together and get inserted in the new field (URL)?
For the moment i have this expression:
removeleft(fixcase(regex([AMG URL], /#-([^-]+)$#/, 1), 3), 4) witch get the identifier to another field show me only that. I suppose it will be something like this isn't?!
-
I may be a little uncertain of your question. But here's how it works:
If you already have a URL in your current 'rid' field, that URL is used for the lookup. There are new IDs being employed on AMG, and there are older ones. These are easily distinguished.
If you have a newer AMG album ID in the 'rid' field, it is used to construct the URL.
In any case, the ID returned in the 'rid' field will be the newer form of AMG ID, and the URL stored in the 'url' field will be the standardized album URL plus the ID. The friendly but superfluous album text will not part of the URL returned. You don't need it in MC, since you already know the Album name.
-
I have another question is not quite related to the script but instead of the performer and the album artist when you get the info.
Let say you have this album:
http://www.allmusic.com/album/watch-the-throne-mw0002107551
Where there you have 2 performer and featuring. That mean you have 2 artist per album but in the field of album artist you can have only one type of it, how can this be deal and for who i shall reserve this album either A or B, or both!? If i choose the artist A on album artist (auto) the artist B remain only at the artist level, but if you look deeper they are listed as be the album artist and also as an artist.
Till now how i deal with this is, either i delete on of artist and put into one. But i think that way they deal in the allmusic with artist/featuring and the main album artist.
-
I'm gone try the last version to see how it works before i put more question, but i understand the part where you have the new id caller.
-
I have another question is not quite related to the script but instead of the performer and the album artist when you get the info.
Let say you have this album:
http://www.allmusic.com/album/watch-the-throne-mw0002107551
Where there you have 2 performer and featuring. That mean you have 2 artist per album but in the field of album artist you can have only one type of it, how can this be deal and for who i shall reserve this album either A or B, or both!? If i choose the artist A on album artist (auto) the artist B remain only at the artist level, but if you look deeper they are listed as be the album artist and also as an artist.
Till now how i deal with this is, either i delete on of artist and put into one. But i think that way they deal in the allmusic with artist/featuring and the main album artist.
How you deal with Album Artist is entirely a personal choice. You should use an Album Artist that helps you organize and find music for your needs.
For example, I have many Pat Metheny albums. And his albums are listed under any of "Pat Metheny", "The Pat Metheny Group", "The Pat Metheny Unity Group", "Pat Metheny Quartet". I really don't care about all those variations when I search for music. So the Album Artist is set to "Pat Metheny", and I'll note in another field the actual "official" album artist name.
In your example, if I had only one Jay-Z album and 10 Kanye West albums, I'd probably set the Album Artist to Kanye West, and include Jay-Z as a featured artist. If I only had this one album from either, I'd list the Album Artist as "Jay-Z & Kanye West".
-
I'm gone try the last version to see how it works before i put more question, but i understand the part where you have the new id caller.
OK. But take my advice and do the COPY of the existing ID field (which contains your URLs). See the link I posted.
-
For the moment i have this expression:
removeleft(fixcase(regex([AMG URL], /#-([^-]+)$#/, 1), 3), 4) witch get the identifier to another field show me only that. I suppose it will be something like this isn't?!
I wouldn't use this expression, because I don't think it distinguishes between old and new IDs. I used:
regex([AMG ID],/#http:.*?-(m[wr]\d+)#/, 1)
to do the conversions. Old IDs do not start with -mw or -mr.
-
I've posted a minor update.
Version 1.26
- Internal: update to reflect API change to create_mpl_item() function.
- Fix: incorrect letter option for --rid shown in usage
- Fix: warning about uninitialized rating value when release had no rating.
-
Hey MrC
Sometimes when you have a url where album don't have tracks included but have the rest of information, it can't be trigger not a data because all the time the script say "unable get releases for" i try it every way possible adding that data but all the time that announce come. Have something changes in the script in last period on that matter?
Here is a link where that problem appear. http://www.allmusic.com/album/drive-mw0002676684 .
Thank you!
-
This is a case where there is no Overview page so there are no tracks listed. The default entry for this album is the Credits page. Since everything is keyed of the existence of the Overview page, amg.pl basically aborts doing more work if the page doesn't exist.
Although this new album album has very little information yet (it will probably soon be updated), if it feels important to you to try to grab the little information that exists, I could see about grabbing the per-album values (if they exist).
-
Sometimes happen for older albums who don't have any track available in the list but i did try it your action to do -all but it seems now that action don't work anymore, yet i looked on the script and used -A and the same thing happen, nothing is added. Well is not a big deal because i forgot that in the past all the info was added manually and now im easy with this tool :) so i forgot how to do the hard work...
If you can have an look on it i will be glad if not i say don't butter you already did a great job so thanks.
-
The --all / -A option can't override the fact that there is no Overview page. The script simply aborts trying to process that album further at that point. I think I should be able to get the script to still allow processing any additional information. If you happen to know of another entry that contains no Overview page, but has some data on the other pages such as Credits, etc., that would be helpful.
-
Thanks MrC but am I missing a file as I don't have a LibXML.pm in either of the directories mentioned below
c:\Users\Nathan\Desktop\amg>perl amg.pl --help
Can't locate XML/LibXML.pm in @INC (@INC contains: C:/Perl64/site/lib C:/Perl64/
lib .) at amg.pl line 21.
BEGIN failed--compilation aborted at amg.pl line 21.
Thanks
Nathan
-
No special privs for you. You need to read and follow the README instructions. :-)
Step 4 in README.install.txt indicates the modules you need to install. But no cheating... be sure you've followed all the steps.
Scratch that. I need to update the instructions. Add the additional module:
ppm install XML-LibXML
and then you should be good to go.
-
;D - I always like spec privs! Anyway, once the other model installed, it will runs! As you can see from the screen shot, it is pulling all the data correctly but then changed One Track (the last one I created the new particle for) and updated it's the Name (but to the last in the data stream pulled) and also changed Media Type from Video to Audio. The rest were left as is (see Pic)
From what I read, does the script rely on unique filenames (as particles all have the same one)?
-
Ok, good to know.
I hadn't thought about the single Filename issue - that was dumb of me. However, we can use FileKey() instead. I'll have to update the code to handle a FileKey value instead of a Filename value. Then, you can use it for anything. You can add an expression column to your view with the value FileKey() to see the unique file key entry.
I'll also create something to allow you to override Media Type being automatically set to Audio.
-
Got it. Makes sense. Added the FileKey() Col
Thanks
Nathan
-
Nathan,
Attached is a pre-release ready for you to test. It includes two new options:
--mediatype
--filekey
The --mediatype option allows you to set the media type, and --filekey allows you to tell amg.pl to use a Filekey column instead of the Filename column (you can leave the Filename column present and you'll get better debug/verbose output). You'll have to add an expression column called Filekey with the value:
FileKey()
So your new command line should include:
--mediatype Video --filekey
I'd been meaning to add Filekey support anyway, so thanks for the prompting.
-
...and we have a Winner! :D Well done again MrC
Now to think about how to best to use this new found power 8)
-
Nice!
-
....mmmm this could speed up particle tagging for Music Videos esp if (when) MC can auto create particles based on the Chapters but it seems the info on All Music is not that reliable for BD Music Videos as I found that unfortunately for the bulk of my Music BDs they either
- don't exist as a separate Release at all, or
- it it does exist they do not have a track listing just a description.
No to to be deterred I figured just use another Release and the good news is I've found that if I use say a CD listing of the same Album even though the tracks may be broken up over 2 CD (eg D1T1...D1T6, then D2T1...D2T5) the Helper will still map the Names across in the correct order (eg Track 1-11 which is great)! ... but I then found on the first disk I tested that track listing on my BD is different to all the ones listed (an intro inserted and a track deleted).
For example I have this one:
Blu-ray Alchemy Live 23rd July 1983, Hammersmith Odeon, London
Intro (Saturday Night At The Movies)
Once Upon a Time in the West
Expresso Love
Romeo & Juliet
Private Investigations
Sultans of Swing
Two Young Lovers
Tunnel of Love
Telegraph Road
Solid Rock
Going Home - Theme from 'Local Hero
All music's listing is here - http://www.allmusic.com/album/alchemy-dire-straits-live-mw0000189678/releases
Also, I was then thinking that your helper & a calculation field could maybe derive the playback range based on the "Length" field from where it is exists. Any idea on how to do this?, eg:
Track 1: Time = "13:01" --> Playback Range = "0.00-13:01"
Track 2: Time = "5:45" --> Playback Range = "13:01-18:46"
etc
(note: the format for Playback Range is HH:MM:SS and some albums go over the hour mark)
Thanks
Nathan
-
Discogs usually has better entries, and many more of them. I can update the Discogs.pl script tomorrow.
I can give you a playback range option.
-
wooooo woooo - How exciting as these are the slowest bits in creating particles, entering the name and calculating and typing in the Playback Range (which you then get wrong!)
Thanks mate
Nathan
-
FYI - I've got a BD "Pure Audio" Video (The Rolling Stones - Exile on Main Street) ready to go with a blank Particle created and numbered for each of the 18 tracks but sans any other meta data (except Artist/Album) so I'm happy to pull all available meta data (not just "Name"). Both ALLMUSIC (http://www.allmusic.com/album/release/exile-on-main-st-mr0004025078) and DiscOrgs (http://www.discogs.com/Rolling-Stones-Exile-On-Main-St/release/5397834)have good track listings with DiscOrgs also have times that seem to line up with the Chapter Marks. It will be interesting to see what data it pulls!
-
See the most recent discogs.pl thread:
http://yabb.jriver.com/interact/index.php?topic=87032.msg613495#msg613495 (http://yabb.jriver.com/interact/index.php?topic=87032.msg613495#msg613495)
-
On a fresh install of windows, i encounter this problem
Can't locate XML/LibXML.pm in @INC (@INC contains: C:/Perl64/site/lib C:/Perl64/lib .) at amg.pl line 21.
BEGIN failed--compilation aborted at amg.pl line 21.
I search on the file at line 21 and it is this
XML::LibXML '1.70';
I searched on internet for a solution but i can't find one to help me, do you know what is the problem MrC?!
Thank you!
-
You need to install the XML-LibXML module. In the command shell, run:
ppm install XML-LibXML
-
That was?! darn, and i looked a whole day for a solution over internet... Thx what to say.
-
You can use the better Google-based search:
http://jriver.com/search.html
Try searching for XML/LibXML.pm using the link above.
-
I noticed in the past few weeks with my new albums that I am tagging that the individual tracks are being tagged with amg trackid and track URL fine but the fields amg albumid, releases, similar albums, billboard albums or singles is being filled at all. I even tried an album already tagged, cleared all the tags and used amg 1.26. It filled the trackid track URL but none of the other tags. Any help would be great. By the way MrC this little app is a godsend and it really helps to bring out MC on a 55" screen in a loft. Hoping to go to 110" projection with kinect motion capability and this media software. Thanks!
-
One quick question: would the dummy files/MPL route work with MC 18?
-
Hi MrC,
My album art file's media type is being changed from IMAGE to AUDIO. I guess the --mediatype parameter won't help in this situation where I have tracks and art selected? No big deal to modify them afterwards though.
-
I was having some issues with certain tracks causing an unhandled exception when dealing with certain characters and was able to fix it by modifying line 63 of the MCWS.pm script to use uri_escape_utf8 instead of just uri_escape.
-
Hey Mr. C, just noticed that the last few albums that I have tagged are missing the lyrics. I checked AMG's site and the lyrics are there, they just aren't getting pulled in by the script. NIckleback -Nofixed Address and Kings of Leon - Only By the Night. There was more, but that should get the point. Still using 1.24 and Media Center 19. Just noticed the problem goes back a few weeks, no more.
Thanks,
Steve
-
Hi Steve, FYI MrC has not been online for some time - more -> http://yabb.jriver.com/interact/index.php?topic=90825.0
-
WOW, that sucks. Wonder what happened.
Is anyone else have trouble running the script? Did AMG change something on their end?
-
running fine here, although i dont know about lyrics, dont use em
-
Yeah, everything works except the lyrics...strange.
-
Lyrics fix...
replace line 1119:
if (m#<div class="lyric SCREENONLY">.*?<p id="hidden_without_js" style="display: none;">\s+(.*?)\s*<img src="/images/transparent.png">\s*</p>\s+<p><br /><br />(.+?)<script>#ms) {
with line:
if (m#<div class="lyric SCREENONLY">.*?<p id="hidden_without_js" style="display: none;">\s+(.*?)\s*<img src="http://assets.allmusic.com/images/transparent.png">\s*</p>\s+<p><br /><br />(.+?)<script>#ms) {
-
A small correction of the person above, the line is 1079 not the 1119.
-
Thank you guys so much for this, but how do I actually edit the script. When I right click and click edit, it opens in notepad, but there are no line numbers or anything to find the script. Also, any ideas on how it got messed up in the first place. It was working a few weeks age.
Thanks!
-
Dissregard, I just used Find and Replace...THANKS! It worked.
-
I know that MrC is kinda gone, but maybe can anyone who meet this problem can help me.
I install a newly windows and i installed activeperl but something is wrong.
Each time it give me an error that it say "ppm install failed: Can't find any package that provides XML-LibXML". I did install the xml-libxml and still can't do nothing with it. I got the last vers. of activeperl.
-
Any Perl/Regex knowledge here? Review doesn't always parse anymore and therefore the tag doesn't get the value in such cases. I have identified the faulty line but haven't been able to fix the issue.
sub parse_review {
local $_ = shift;
my $ret = undef;
if (m#<h4 class="review-author headline">\s*Review\s+(.*?)</h4>\s*<div [^>]+>\s*<p>\s*(.+?)\s*</div>#ms) {
If I remove <h4 class="review-author headline"> from the beginning of the if parsing succeeds with the ones that didn't get the Review parsed earlie but it isn't a correct solution for this.
I used Elvis Is Back! album as an example for this.
http://www.regexe.com/ was used to test regex line against page source
-
I have the same problem with the review not getting recorded. No idea how to fix it. Hopefully someone smarter than me will respond.
-
This issue is after they added custom review in the album page. It seems they change something there and the parse can't take anymore the review.
-
It works with my changes but I just don't know if it is foolproof as it will make lookup more generic
-
so it is broken I take it? damnit. this was a handy thing. I get the XML LibXML error too.
-
This thread has been locked and while it contains useful additional info, the script utility has been removed and is now superseded with the single MCUtils script package (http://yabb.jriver.com/interact/index.php?topic=100449.0)